Task scheduling method and device, electronic equipment, storage medium and product

Abstract

The invention discloses a task scheduling method and device, electronic equipment, a storage medium and a product, and relates to the technical field of distributed task scheduling, the task scheduling method and device are applied to a distributed system, the distributed system comprises all distributed equipment, and the task scheduling method comprises the steps that the task dependency relationship of all distributed tasks on the distributed system is acquired and analyzed, and an initial task execution sequence is obtained; the distributed tasks in the initial task execution sequence are subjected to priority distribution, time windows of the distributed tasks are adjusted based on the distributed priorities, a target task execution sequence is obtained, and the time windows are used for limiting the execution time of the distributed tasks; the resource state of each distributed device is monitored, at least one target device meeting the resource requirement of each distributed task is searched, each distributed task is distributed to the at least one target device, and the distributed tasks are executed according to the execution sequence of the target tasks. The technical problem that the task scheduling effect is poor is solved.

Description

Technical Field

[0001] The present application relates to the technical field of distributed task scheduling, and in particular to a task scheduling method, device, electronic device, storage medium and computer program product. Background Art

[0002] In the current distributed task execution process, each task needs to manually configure dependencies, which is not only complicated and error-prone, but also, due to resource differences and network communication delays between different distributed devices, tasks may be advanced or delayed when executing distributed tasks on multiple distributed devices, causing confusion in the order of task execution and affecting the synchronization of the overall system. Therefore, the current distributed task execution process has the problem of poor task scheduling effect. Summary of the invention

[0003] The main purpose of the present application is to provide a task scheduling method, device, electronic device, storage medium and computer program product, aiming to solve the technical problem of poor task scheduling effect.

[0004] To achieve the above object, the present application proposes a task scheduling method, which is applied to a distributed system, wherein the distributed system includes distributed devices, and the task scheduling method includes: Obtaining and analyzing the task dependencies of each distributed task on the distributed system to obtain an initial task execution order; Assigning priorities to the distributed tasks in the initial task execution order, and adjusting the time windows of the distributed tasks based on the assigned priorities to obtain a target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; Monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, assign each distributed task to the at least one target device, and execute them in the execution order of the target tasks.

[0005] In one embodiment, the step of acquiring and analyzing the task dependencies of the distributed tasks on the distributed system to obtain the initial task execution order includes: Parsing the task description files of each distributed task on the distributed system to determine the input-output relationship of each distributed task; Determine the dependency relationship of the distributed tasks based on the input-output relationship, and construct a task dependency graph, wherein each node in the task dependency graph represents each distributed task, and each edge in the task dependency graph represents the dependency relationship of the distributed tasks; The task dependency graph is sorted to generate an initial task execution order.

[0006] In one embodiment, the step of assigning priorities to the distributed tasks in the initial task execution order, and adjusting the time windows of the distributed tasks based on the assigned priorities to obtain the target task execution order includes: Calculate the priority of each distributed task, sort them according to the priority, and generate a priority queue; Allocating a time window for each distributed task to adjust the execution time of each distributed task, wherein the time window includes an earliest start time and a latest completion time of the corresponding distributed task, and the execution time is later than the earliest start time and earlier than the latest completion time; When the time window is met, the priority queue is optimized to obtain the target task execution order.

[0007] In one embodiment, the step of executing each distributed task according to the target task execution order includes: For any one of the multiple target devices, monitoring the execution status of each distributed task on the target device; If the execution status of each distributed task on the target device is completed, then searching for the next task to be executed of the distributed task and the execution device corresponding to the task to be executed according to the execution order of the target task, and calculating the network delay between the target device and the execution device; The execution information is sent to the execution device, and the network delay is compensated to the start time of the task to be executed.

[0008] In one embodiment, after the step of allocating each distributed task to the target device, the step further includes: For any one of the target devices, monitoring the resource status of the target device; The resource status is compared with a preset resource threshold, and when the resource status exceeds the preset resource threshold, each distributed task on the target device is scheduled to be executed in an idle device.

[0009] In one embodiment, after the step of executing each distributed task according to the target task execution order, the step further includes: For any one of the distributed tasks, determine whether the execution status of the distributed task satisfies the execution status in the task exception library, wherein the task exception library contains the execution status of the distributed task when an exception occurs; If the execution status of the distributed task satisfies the execution status in the task exception library, the target task execution order and the target device for executing the distributed task are adjusted.

[0010] In addition, to achieve the above purpose, the present application also proposes a task scheduling device, which is applied to a distributed system, wherein the distributed system includes distributed devices, and the task scheduling device includes: A dependency acquisition module is used to acquire and analyze the task dependency relationship of each distributed task on the distributed system to obtain an initial task execution order; An order determination module, used to assign priorities to the distributed tasks in the initial task execution order, and adjust the time windows of the distributed tasks based on the assigned priorities to obtain a target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; The resource allocation module is used to monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, allocate each distributed task to the at least one target device, and execute it in the execution order of the target tasks.

[0011] In addition, to achieve the above objectives, the present application also proposes an electronic device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the task scheduling method described above.

[0012] In addition, to achieve the above objectives, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the task scheduling method described above are implemented.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method described above are implemented.

[0014] The present application provides a task scheduling method, which is applied to a distributed system, wherein the distributed system includes various distributed devices, and the task scheduling method includes: obtaining and analyzing the task dependency relationship of each distributed task on the distributed system to obtain an initial task execution order; assigning priorities to each distributed task in the initial task execution order, and adjusting the time window of each distributed task based on the assigned priority to obtain a target task execution order, wherein the time window is used to limit the execution time of each distributed task; monitoring the resource status of each distributed device, searching for at least one target device that meets the resource requirements of each distributed task, assigning each distributed task to at least one target device, and executing them in accordance with the target task execution order.

[0015] This application automatically obtains and analyzes the dependencies of tasks, accurately and quickly determines the order of tasks, and combines priority allocation and time window adjustment to optimize the execution order of distributed tasks, ensure that tasks can be completed within a strict time window, maintain the synchronization of the system, and ensure that tasks are executed on appropriate devices by real-time monitoring of resource status and dynamic allocation of tasks, thereby improving resource utilization and task success rate. Compared with related solutions that require manual configuration of dependencies, the order of task execution may be confused due to resource differences and network communication delays. This application automatically analyzes task dependencies, assigns priorities to tasks, and adjusts time windows to limit the execution time of tasks, ensuring that tasks can be completed within the time window, and executing tasks on appropriate devices according to the target task execution order, achieving precise control and synchronization of task execution, and improving the stability and reliability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of the first embodiment of the task scheduling method of the present application is provided; Figure 2 A flowchart of the second embodiment of the task scheduling method of the present application is provided; Figure 3 A flowchart of the third embodiment of the task scheduling method of the present application is provided; Figure 4 This is a schematic diagram of the module structure of the task scheduling device according to an embodiment of the present application; Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the task scheduling method in the embodiment of the present application.

[0019] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0022] The embodiments of the present application are applied to a distributed system, which includes various distributed devices. The main solution is: obtaining and analyzing the task dependencies of various distributed tasks on the distributed system to obtain an initial task execution order; assigning priorities to various distributed tasks in the initial task execution order, and adjusting the time windows of various distributed tasks based on the assigned priorities to obtain a target task execution order, wherein the time window is used to limit the execution time of each distributed task; monitoring the resource status of each distributed device, searching for at least one target device that meets the resource requirements of each distributed task, assigning each distributed task to at least one target device, and executing them in accordance with the target task execution order.

[0023] In this embodiment, for the convenience of description, the following description is made with the task scheduling system as the execution subject.

[0024] Since the existing technology requires manual configuration of task dependencies and delay requirements, it is complex and error-prone. In addition, due to resource differences and network communication delays between different distributed devices, tasks may be advanced or delayed when executing distributed tasks on multiple distributed devices. The task execution order is chaotic, affecting the synchronization of the overall system, resulting in poor task scheduling effects and reducing system efficiency and reliability.

[0025] The present application provides a solution that automatically analyzes task dependencies, assigns priorities to tasks, and adjusts time windows, thereby limiting the execution time of tasks, ensuring that tasks can be completed within the time window, and executing tasks on appropriate devices in the target task execution order, thereby achieving precise control and synchronization of task execution and effectively solving the problem of poor task scheduling in a distributed environment.

[0026] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a task scheduling system, etc. The following takes the task scheduling system as an example to illustrate this embodiment and the following embodiments.

[0027] Based on this, the present application embodiment provides a task scheduling method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the task scheduling method of the present application.

[0028] In this embodiment, the method is applied to a distributed system, which includes distributed devices. The task scheduling method includes steps S01 to S03: Step S01, obtaining and analyzing the task dependencies of each distributed task on the distributed system to obtain the initial task execution order; It should be noted that a distributed system refers to a computing system formed by multiple computing devices (such as servers, workstations, embedded devices, etc.) connected through a network. Distributed devices are computing units in a distributed system, which can be physical servers, virtual machines, containers, or any devices with computing capabilities. These devices are connected through a network and can communicate with each other and share data. Distributed tasks refer to computing tasks that are decomposed and assigned to different distributed devices for execution in a distributed system. Task dependencies refer to the sequence relationship between tasks during the execution of distributed tasks. The initial task execution order is generated based on the task dependency graph, which indicates the order in which tasks should be executed without considering resource constraints and priorities.

[0029] In addition, it should be noted that the system comprehensively collects information on all distributed tasks, including but not limited to the input and output of tasks, and the dependencies between them, and automatically analyzes the task dependencies to construct a directed graph, namely the task dependency graph. The nodes in the graph represent tasks, and the edges represent the dependencies between tasks. The task dependency graph is topologically sorted to obtain an acyclic initial task execution order to ensure that all dependencies are met.

[0030] It is understandable that since the existing solution relies on manual configuration of dependencies, which is both complex and error-prone, step S01 is performed to greatly simplify the complexity of task scheduling by automatically parsing dependencies, accurately express the complex dependencies between tasks, obtain the initial task execution order, ensure the correctness of task execution, and improve the efficiency and accuracy of task scheduling.

[0031] Step S02, assigning priorities to the distributed tasks in the initial task execution order, and adjusting the time windows of the distributed tasks based on the assigned priorities to obtain the target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; It should be noted that priority allocation is to assign different priorities to tasks based on factors such as the urgency of the task and real-time requirements. The time window refers to the time range in which the task can be executed. It defines the start time and end time of the task to ensure that the task can be completed within the specified time. The target task execution order is the final task execution order formulated after considering task dependencies, priority allocation and resource constraints. This order can ensure that the tasks are executed in sequence according to the dependencies within the specified time, while maximizing the use of device resources. The execution time refers to the time period in which the task is actually executed, which is within the time window.

[0032] In addition, it should be noted that according to factors such as the urgency, importance, and resource requirements of the task, a priority is assigned to each distributed task in the initial task execution order. On the premise of satisfying the task dependencies, a time window is assigned to each task to ensure that the task starts and ends within this window. The initial task execution order is adjusted based on the priority and the time window of the task to further optimize the task execution order and obtain the target task execution order.

[0033] It is understandable that due to the existing solutions, the resource differences of different devices and network communication delays may easily lead to confusion in the order of task execution, resulting in tasks being advanced or delayed. Therefore, step S02 is performed to assign priorities to tasks to ensure that critical tasks are executed with top priority, and to ensure that each task can be executed within the specified execution time through a time window, so as to avoid asynchrony problems due to device differences, thereby improving the corresponding synchronization of the overall system.

[0034] Step S03, monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, assign each distributed task to at least one target device, and execute it in the target task execution order.

[0035] It should be noted that resource status refers to the current resource usage of distributed devices, including CPU, memory, network bandwidth, etc. Resource demand refers to the amount of resources required during task execution, including CPU, memory, storage space, etc. Target device refers to the device selected to perform a specific task.

[0036] In addition, it should be noted that the system monitors the resource status of each device in the distributed environment in real time, including CPU, memory, storage, etc., searches for target devices that meet the resource requirements of each distributed task, assigns tasks to the target devices, and executes tasks according to the target task execution order.

[0037] It is understandable that, since existing solutions are often based on static strategies or fixed rules and cannot be dynamically adjusted according to real-time resource status and task requirements, step S03 is performed to dynamically monitor resource status and task requirements, reasonably allocate system resources, and ensure that tasks can be executed on appropriate devices, thereby improving the success rate of task execution and the overall performance of the system.

[0038] In a feasible implementation, in step S01, the task dependency relationship of each distributed task on the distributed system is obtained and analyzed, and the step of obtaining the initial task execution order includes steps A01 to A03: Step A01, parsing the task description files of each distributed task on the distributed system to determine the input and output relationship of each distributed task; It should be noted that the system reads the task description file of each distributed task on the distributed system and extracts the input-output relationship of each distributed task from it. The task description file refers to the detailed description of each task in the distributed system, which is usually defined in a standard format (such as XML, JSON, etc.), including task attributes (such as task name, type, priority, etc.), resource requirements (such as CPU, memory, storage, etc.), event trigger conditions (such as specific time, sensor data changes, user operations, etc.) and other dependent tasks. In a distributed system, each task has its specific input data and output data. The input data is the data source required before the task is executed, and the output data is the result generated after the task is executed. The dependency relationship between tasks is established through these input and output data. For example, if the output data of task A is the input data of task B, then task A is the predecessor task of task B, that is, task B depends on task A.

[0039] Step A02, determining the dependency relationship of each distributed task based on the input-output relationship, and constructing a task dependency graph, wherein each node in the task dependency graph represents each distributed task, and each edge in the task dependency graph represents the dependency relationship of each distributed task; It should be noted that the task dependency graph (DAG) is a directed acyclic graph used to represent the dependency relationship between tasks. The task dependency graph is expressed as:

[0040] Where G is the task dependency graph, V is the task set, and E is the dependency relationship between tasks. In the task dependency graph, each node represents a task, and each edge represents the dependency relationship between tasks. Must be on task completed before, there is a arrive The directed edge of .

[0041] In addition, it should be noted that the system traverses all tasks, and for each task, checks whether its input data comes from the output data of other tasks. If so, it means that there is a dependency relationship between the task and other tasks, and the corresponding directed edges are added to the task dependency graph.

[0042] Step A03, sorting the task dependency graph to generate an initial task execution order.

[0043] It should be noted that after constructing the task dependency graph, the module uses a topological sorting algorithm (such as Kahn algorithm or DFS) to sort the task dependency graph, verify the directed acyclicity of the DAG, and ensure that there is no closed loop, that is, for any task sequence ( , , ..., ), must satisfy → → ... → , and will not return to The result of sorting is the initial task execution order. Topological sorting is an algorithm for sorting DAG, which can ensure that a linear execution order is generated according to the dependency relationship of tasks. In this order, each task is executed after its dependent tasks.

[0044] In this implementation, by parsing the task description file, the input-output relationship of each task is accurately extracted. In the subsequent task scheduling process, the execution order and dependency of the tasks are determined based on these input-output relationships, thereby avoiding conflicts and repeated executions between tasks and improving the accuracy and efficiency of task scheduling. By constructing a task dependency graph, the dependency relationship and execution order between tasks are clearly displayed, providing strong support for subsequent task scheduling. At the same time, the directed acyclic nature of the task dependency graph also ensures the correctness and conflict-free nature of task scheduling.

[0045] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 2 In step S02, priorities are assigned to the distributed tasks in the initial task execution sequence, and the time windows of the distributed tasks are adjusted based on the assigned priorities. The steps of obtaining the target task execution sequence include steps S11 to S13: Step S11, calculating the priority of each distributed task, and sorting them according to the priority to generate a priority queue; It should be noted that for each task in the initial task execution order, the system calculates a priority value based on its urgency, the task's demand for resources, and the complexity of task dependencies. The calculation formula for task priority is:

[0046] in, Representing distributed tasks The priority of Indicates the urgency of the task. Indicates the resource requirements of the task. Indicates the task dependency complexity, parameter , , They are the weights of urgency, task demand for resources, and task dependency complexity, which can be adjusted according to actual needs to ensure that critical tasks have higher scheduling priority. All tasks are sorted according to their priority values ​​and placed in different priority queues. The scheduler executes the tasks in the highest priority queue first. The priority queue is a special data structure in which each element has a priority. During the scheduling process, the system always selects the highest priority task for execution. For example, critical tasks (such as real-time response tasks) have the highest priority, while background tasks have a lower priority. The system gives priority to critical tasks for execution.

[0047] Step S12, allocating a time window for each distributed task to adjust the execution time of each distributed task, wherein the time window includes the earliest start time and the latest completion time of the corresponding distributed task, and the execution time is later than the earliest start time and earlier than the latest completion time; It should be noted that the system assigns a time window to each task, including the earliest start time (EST) and the latest finish time (LFT). The earliest start time refers to the earliest time point at which the task can start execution, while the latest finish time refers to the time point before which the task must be completed. By analyzing the upstream tasks and communication delays that the task depends on, the earliest start time of each task is calculated. The calculation formula for the earliest start time is:

[0048] in, Representation Task The earliest start time, For upstream tasks The latest completion time, For the task and tasks The communication delay between tasks is taken into account, and the latest completion time is derived by reverse deduction of the execution time limit of the task. The system maintains a time window table to record the time window information of each task.

[0049] Step S13, when the time window is met, optimize the priority queue to obtain the target task execution order.

[0050] It should be noted that the system adjusts the execution time of the task in real time according to the priority queue and the time window table in step S12. During scheduling, the task will not be scheduled to be executed before its earliest start time, nor will it exceed its latest completion time. High-priority tasks will be executed first. For example, high-priority tasks are executed as soon as possible under the premise that the time window allows, while low-priority tasks may be postponed to the edge of their latest completion time. When a new task arrives, check whether its time window conflicts with the execution time of an existing task. If there is no conflict, insert it into the priority queue according to its priority. If there is a conflict, adjust its execution time through the delay compensation mechanism to ensure that it can be completed within the specified time.

[0051] In this implementation, through the priority queue, the system can schedule according to the task priority, give priority to the execution of key tasks, and improve the efficiency and accuracy of task execution. Through the allocation of time windows and the delay compensation mechanism, the system can ensure that the task is completed within the specified time, improving the certainty and reliability of task execution. Through dynamic resource scheduling and optimization of priority queues, the system can make full use of equipment resources and improve the efficiency and accuracy of task execution. At the same time, the system can perform intelligent scheduling according to the importance of the task and the status of resources, ensuring the fairness and efficiency of task execution.

[0052] In a feasible implementation manner, in step S02, the steps performed in the target task execution order include steps B01 to B03: Step B01, for any one of the multiple target devices, monitoring the execution status of each distributed task on the target device; It should be noted that the system obtains the task status through periodic query or event-driven methods, synchronizes the status information between devices through distributed communication protocols (such as DSoftBus), and each device is configured with a task manager, which is responsible for maintaining and updating the task status.

[0053] Step B02, if the execution status of each distributed task on the target device is completed, then search for the next task to be executed of the distributed task and the execution device corresponding to the task to be executed according to the execution order of the target task, and calculate the network delay between the target device and the execution device; It should be noted that when the distributed task on the target device is completed, the next task to be executed is found according to the execution order of the target tasks and the current system status, and it is determined on which device the task should be executed, that is, the execution device. At the same time, the monitored data is used to predict the network delay between the target device and the execution device through the Bayesian network model.

[0054] Step B03, sending the execution information to the execution device, and compensating the network delay to the start time of the task to be executed.

[0055] It should be noted that the execution information of the task to be executed is sent to the corresponding execution device to ensure that the task execution progress between devices is notified and synchronized with each other. The expression of the synchronous communication function is:

[0056] in, Refers to the synchronous communication function, which is used to send execution information and notify the task on device D2 Can be executed, D1 is the task The device where it is located, D2 is the dependent task Mission The device where you are located, Representation Task With the task Dependencies between tasks After completing on device D1, send the task that depends on it to The device D2 where it is located sends the execution information. At the same time, the calculated network delay is compensated to the start time of the task to be executed, so that even in the case of network fluctuations, the task can be completed within the expected time. The network delay compensation formula is:

[0057] in, Representation Task The start time of It is a prerequisite task The end time of is the network delay between device D1 and device D2.

[0058] In this implementation, through real-time status monitoring, the system can more effectively manage distributed tasks and improve the transparency and controllability of task execution. By clarifying task dependencies and accurately calculating network delays, the system can more accurately formulate task scheduling plans and improve the synchronization and efficiency of task execution. Through delay compensation, the reliability and synchronization of task execution are improved, avoiding the problem of poor task scheduling due to network delays. At the same time, this also enables the system to better adapt to uncertain factors such as network fluctuations.

[0059] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those of the first and second embodiments can be referred to the above description, and will not be repeated in the following. Figure 3 In step S03, the steps executed in the target task execution order also include steps S21-S22: Step S21, for any one of the target devices, monitoring the resource status of the target device; It should be noted that the resource monitoring agent deployed on the target device collects the resource status of each target device in real time or periodically, including key indicators such as CPU usage, memory usage, and network bandwidth usage. The resource collection formula is:

[0060] in, Indicates the device Resource usage at time t, including CPU usage , memory usage and network bandwidth usage , encapsulates the resource status in a specific format and sends it to the system for resource scheduling through distributed communication protocols (such as HTTP, TCP / IP, etc.).

[0061] Step S22: compare the resource status with a preset resource threshold, and when the resource status exceeds the preset resource threshold, schedule each distributed task on the target device to be executed on an idle device.

[0062] It should be noted that the preset resource threshold refers to the resource usage upper limit set by the system for each device. When the resource status of the device exceeds this upper limit, the system considers that the device is in an overloaded state and needs to take measures to optimize resources or schedule tasks. These thresholds can be dynamically adjusted according to the performance of the device, task requirements and the overall configuration of the system. When the device is overloaded, the system reallocates resources and, based on the current resource status and task requirements, schedules some or all tasks on the overloaded device to other idle or resource-rich devices for execution, thereby balancing the load of each device and improving resource utilization efficiency and the overall performance of the system. Idle devices refer to devices with low current resource utilization that have not yet taken on too many tasks. These devices have sufficient resources and capacity to receive and process tasks scheduled from overloaded devices.

[0063] In addition, it should be noted that after receiving the resource status, the system will compare it with the preset resource threshold through the resource warning formula. The resource warning formula is:

[0064] in, Indicates that the device resources are exceeded. is the resource status, is the preset resource threshold. >0 (i.e. the resource status of the device exceeds the preset threshold), the system determines that the device is in an overloaded state. For example, when the CPU usage of a device is continuously higher than 80%, the system can consider that the device is in a high-load state and needs to pay close attention to its resource usage and perform task scheduling or resource optimization in a timely manner. When the device is overloaded, resource reallocation is immediately triggered to reduce the device load. The resource reallocation formula is:

[0065] Where N is the total number of devices, is the resource status, The resource thresholds are preset to ensure load balancing by minimizing the differences.

[0066] In this implementation, through the deployment of resource monitoring agents and real-time data collection, the system can timely grasp the resource usage of the equipment, provide data support for subsequent task scheduling and resource optimization, balance the load of each device through the resource reallocation mechanism, and improve resource utilization efficiency and the overall performance of the system.

[0067] In a feasible implementation manner, in step S03, the steps executed in the target task execution order further include steps E01 to E02: Step E01, for any distributed task among the distributed tasks, determine whether the execution status of the distributed task satisfies the execution status in the task exception library, wherein the task exception library contains the execution status of the distributed task when an exception occurs; It should be noted that the system automatically collects the execution logs of tasks from the device side where each distributed task is executed according to the preset cycle in advance, conducts in-depth analysis of the execution logs, and pays attention to the execution status of tasks when abnormalities occur (such as execution delays, abnormally high resource usage, equipment failure, etc.). Once an abnormality is identified, the execution status when the abnormality occurs is stored in the task abnormality library. Among them, the preset cycle refers to the time interval at which the system regularly collects task execution logs, such as every hour, every minute, etc. The specific cycle is set according to system requirements and task characteristics. The execution log records detailed information during the task execution process, including execution progress, resource consumption, device status, etc. The execution status refers to the specific performance of the task when an abnormality occurs, such as execution delays, abnormal resource usage, etc. The task abnormality library refers to a database that stores the abnormal execution status of historical tasks, which is used for subsequent task abnormality identification and rescheduling decisions.

[0068] In addition, it should be noted that the system checks the execution status of each distributed task in real time or periodically, and compares it with the abnormal execution status in the task abnormality library. The execution status comparison formula is:

[0069] in, Representation Task The state difference, It is the abnormal execution status in the task exception library. is the execution status of each distributed task. When ≤0, it indicates that the execution status of a task matches an abnormal situation in the task abnormality library, and the task has the risk of abnormality.

[0070] Step E02: If the execution status of the distributed task meets the execution status in the task exception library, the target task execution order and the target device for executing the distributed task are adjusted.

[0071] It should be noted that when a distributed task is judged to have an abnormal risk, the resource rescheduling mechanism is triggered. According to the current system resource situation and task dependencies, the task is reallocated to other idle or resource-rich devices for execution, and the execution order of related tasks is adjusted to ensure that the task can be completed efficiently and stably, while avoiding device overload and performance degradation. The resource rescheduling formula is:

[0072] in, Representation Task The latest completion time, Representation Task Actual completion time,The goal of resource rescheduling is to minimize the difference between the latest completion time and the actual completion time of each task, thereby reducing the overall impact of delays on the system.

[0073] In this implementation, through log collection and exception analysis at preset periods, the system can grasp the execution status of the task in real time, provide key data support for subsequent task scheduling and resource optimization, and by comparing the task execution status with the abnormal conditions in the task exception library, the system can promptly discover potential task abnormality risks. Through the resource rescheduling mechanism, the system can respond before the task exception occurs or in the early stage of the exception, thereby avoiding the impact of task failure or delay on system performance.

[0074] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the task scheduling method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0075] The present application also provides a task scheduling device, which is applied to a distributed environment, in which distributed tasks are run, and distributed tasks are run on distributed devices. Please refer to Figure 4 , the task scheduling device comprises: The dependency acquisition module 10 is used to acquire and analyze the task dependency relationship of each distributed task on the distributed system to obtain the initial task execution order; The order determination module 20 is used to assign priorities to the distributed tasks in the initial task execution order, and adjust the time windows of the distributed tasks based on the assigned priorities to obtain the target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; The resource allocation module 30 is used to monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, allocate each distributed task to at least one target device, and execute it in the target task execution order.

[0076] Optionally, the dependency acquisition module 10 is further used for: Parse the task description files of each distributed task on the distributed system and determine the input and output relationship of each distributed task; Determine the dependency relationship of each distributed task based on the input-output relationship, and construct a task dependency graph, wherein each node in the task dependency graph represents each distributed task, and each edge in the task dependency graph represents the dependency relationship of each distributed task; Sort the task dependency graph and generate the initial task execution order.

[0077] Optionally, the sequence determination module 20 is further configured to: Calculate the priority of each distributed task, sort them according to the priority, and generate a priority queue; Allocate a time window for each distributed task to adjust the execution time of each distributed task, wherein the time window includes the earliest start time and the latest completion time of the corresponding distributed task, and the execution time is later than the earliest start time and earlier than the latest completion time; When the time window is met, the priority queue is optimized to obtain the target task execution order.

[0078] Optionally, the resource allocation module 30 is further configured to: For any one of the multiple target devices, monitoring the execution status of each distributed task on the target device; If the execution status of each distributed task on the target device is completed, then find the next task to be executed of the distributed task and the execution device corresponding to the task to be executed according to the execution order of the target task, and calculate the network delay between the target device and the execution device; The execution information is sent to the execution device, and the network delay is compensated to the start time of the task to be executed.

[0079] Optionally, the resource allocation module 30 is further configured to: For any one of the target devices, monitoring the resource status of the target device; The resource status is compared with a preset resource threshold, and when the resource status exceeds the preset resource threshold, each distributed task on the target device is scheduled to be executed on an idle device.

[0080] Optionally, the task scheduling device includes a resource scheduling module 40, and the resource scheduling module 40 is further used for: For any distributed task among the distributed tasks, determine whether the execution status of the distributed task satisfies the execution status in the task exception library, wherein the task exception library contains the execution status of the distributed task when an exception occurs; If the execution status of the distributed task meets the execution status in the task exception library, the target task execution order and the target device for executing the distributed task are adjusted.

[0081] The task scheduling device provided by the present application adopts the task scheduling method in the above embodiment, which can solve the technical problem of poor task scheduling effect. Compared with the prior art, the beneficial effects of the task scheduling device provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, and other technical features in the task scheduling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0082] The present application provides an electronic device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the task scheduling method in the above-mentioned embodiment 1.

[0083] Reference below Figure 5 , which shows a schematic diagram of the structure of an electronic device suitable for implementing an embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, notebook computers, PADs (Portable Application Description: tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0084] like Figure 5As shown, the electronic device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the electronic device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or have alternatively.

[0085] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0086] The electronic device provided by the present application adopts the task scheduling method in the above embodiment to solve the technical problem of poor task scheduling effect. Compared with the prior art, the beneficial effects of the electronic device provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, and other technical features in the electronic device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0087] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0088] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0089] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the task scheduling method in the above-mentioned embodiment.

[0090] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0091] The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0092] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the task scheduling device is applied to a distributed system. The distributed system includes various distributed devices. The task scheduling device can obtain and analyze the task dependencies of each distributed task on the distributed system to obtain an initial task execution order; assign priorities to each distributed task in the initial task execution order, and adjust the time window of each distributed task based on the assigned priority to obtain a target task execution order, wherein the time window is used to limit the execution time of each distributed task; monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, assign each distributed task to at least one target device, and execute it according to the target task execution order.

[0093] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0095] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0096] The readable storage medium provided in the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned task scheduling method, and can solve the technical problem of poor task scheduling effect. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in the present application are the same as the beneficial effects of the task scheduling method provided in the above-mentioned embodiment, and will not be described in detail here.

[0097] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned task scheduling method when executed by a processor.

[0098] The computer program product provided by the present application can solve the technical problem of poor task scheduling effect. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as the beneficial effects of the task scheduling method provided by the above embodiment, which will not be repeated here.

[0099] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A task scheduling method, characterized in that: Applied to a distributed system, the distributed system includes distributed devices, and the task scheduling method includes: Obtaining and analyzing the task dependencies of each distributed task on the distributed system to obtain an initial task execution order; Assigning priorities to the distributed tasks in the initial task execution order, and adjusting the time windows of the distributed tasks based on the assigned priorities to obtain a target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; Monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, assign each distributed task to the at least one target device, and execute them in the execution order of the target tasks.

2. The task scheduling method according to claim 1, characterized in that: The step of acquiring and analyzing the task dependencies of the distributed tasks on the distributed system to obtain the initial task execution order comprises: Parsing the task description files of each distributed task on the distributed system to determine the input-output relationship of each distributed task; Determine the dependency relationship of the distributed tasks based on the input-output relationship, and construct a task dependency graph, wherein each node in the task dependency graph represents each distributed task, and each edge in the task dependency graph represents the dependency relationship of the distributed tasks; The task dependency graph is sorted to generate an initial task execution order.

3. The task scheduling method according to claim 1, characterized in that: The step of assigning priorities to the distributed tasks in the initial task execution order, and adjusting the time windows of the distributed tasks based on the assigned priorities to obtain the target task execution order includes: Calculate the priority of each distributed task, sort them according to the priority, and generate a priority queue; Allocating a time window for each distributed task to adjust the execution time of each distributed task, wherein the time window includes an earliest start time and a latest completion time of the corresponding distributed task, and the execution time is later than the earliest start time and earlier than the latest completion time; When the time window is met, the priority queue is optimized to obtain the target task execution order.

4. The task scheduling method according to claim 1, characterized in that: The steps of executing in the target task execution order include: For any one of the multiple target devices, monitoring the execution status of each distributed task on the target device; If the execution status of each distributed task on the target device is completed, then searching for the next task to be executed of the distributed task and the execution device corresponding to the task to be executed according to the execution order of the target task, and calculating the network delay between the target device and the execution device; The execution information is sent to the execution device, and the network delay is compensated to the start time of the task to be executed.

5. The task scheduling method according to claim 1, characterized in that: The step of executing in the target task execution order further includes: For any one of the target devices, monitoring the resource status of the target device; The resource status is compared with a preset resource threshold, and when the resource status exceeds the preset resource threshold, each distributed task on the target device is scheduled to be executed in an idle device.

6. The task scheduling method according to claim 1, characterized in that: The step of executing in the target task execution order further includes: For any one of the distributed tasks, determine whether the execution status of the distributed task satisfies the execution status in the task exception library, wherein the task exception library contains the execution status of the distributed task when an exception occurs; If the execution status of the distributed task satisfies the execution status in the task exception library, the target task execution order and the target device for executing the distributed task are adjusted.

7. A task scheduling device, characterized in that: Applied to a distributed system, the distributed system includes various distributed devices, and the task scheduling device includes: A dependency acquisition module is used to acquire and analyze the task dependency relationship of each distributed task on the distributed system to obtain an initial task execution order; An order determination module, used to assign priorities to the distributed tasks in the initial task execution order, and adjust the time windows of the distributed tasks based on the assigned priorities to obtain a target task execution order, wherein the time windows are used to limit the execution time of the distributed tasks; The resource allocation module is used to monitor the resource status of each distributed device, find at least one target device that meets the resource requirements of each distributed task, allocate each distributed task to the at least one target device, and execute it in the execution order of the target tasks.

8. An electronic device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the task scheduling method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 6 are implemented.

Images