Task scheduling method and device, electronic equipment and storage medium

Abstract

This invention provides a task scheduling method, apparatus, electronic device, and storage medium, relating to the field of task scheduling technology. The method includes: determining resource requirement data for a target task; dividing the target task into multiple subtasks based on the resource requirement data; determining real-time resource status data for each task scheduling system; and determining candidate task scheduling systems within the task scheduling system based on the real-time resource status data and resource constraint data; and allocating each subtask to a corresponding candidate task scheduling system based on a load balancing algorithm, with the candidate task scheduling system executing the corresponding subtask. This invention, through task partitioning, can decompose large-scale tasks and allocate them to multiple task scheduling systems for parallel execution, effectively shortening the total task completion time and avoiding blocking due to waiting for a single resource, thereby effectively improving task execution efficiency.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, and in particular to a task scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of cloud computing, edge computing, and distributed system technologies, computing environments are increasingly exhibiting heterogeneous and complex characteristics. Against this backdrop, efficiently and rationally scheduling large-scale computing tasks (such as scientific computing, AI model training, and real-time rendering) across geographically distributed computing resources has become crucial for improving overall system performance. Task scheduling technology, as the core of distributed system management, aims to maximize resource utilization, shorten task completion time, and ensure overall system stability while meeting specific task requirements.

[0003] Existing task scheduling methods typically treat a single task as an indivisible scheduling unit, directly assigning it to a resource-rich task scheduling system for execution. However, when faced with large and complex target tasks, the resources of a single scheduling system may not be sufficient to meet the overall requirements of the target task, causing tasks to be blocked for extended periods while waiting for resources, resulting in low task execution efficiency. Summary of the Invention

[0004] This invention provides a task scheduling method, apparatus, electronic device, and storage medium to solve the technical problem that the resources of a single scheduling system in the prior art may not be able to meet the overall needs of the target task, resulting in the task being blocked for a long time while waiting for resources and the task execution efficiency being low.

[0005] This invention provides a task scheduling method, comprising: Determine the resource requirements data for the target task; Based on the resource requirement data, the target task is divided into multiple sub-tasks; Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. Each of the subtasks is assigned to a corresponding candidate task scheduling system, and the candidate task scheduling system executes the corresponding subtask.

[0006] According to a task scheduling method provided by the present invention, determining the resource requirement data of the target task includes: Extract the task metrics for the target task; Based on historical task operation data and the task indicators, a regression analysis model is used to predict the resource requirements of the target task during execution.

[0007] According to a task scheduling method provided by the present invention, the step of dividing the target task into multiple sub-tasks based on the resource demand data includes: The target task is divided into multiple functional modules, and the resource requirement level of each functional module is determined based on the resource requirement data. The functional module with the highest resource requirement level is identified as the core sub-task, and the remaining functional modules are identified as auxiliary sub-tasks.

[0008] According to a task scheduling method provided by the present invention, after allocating each subtask to a corresponding candidate task scheduling system and executing the corresponding subtask by the candidate task scheduling system, the method further includes: Obtain real-time running data of the subtask during execution; Based on a unified time reference, the real-time running data is time-aligned.

[0009] According to a task scheduling method provided by the present invention, the real-time running data includes real-time task execution progress; After obtaining the real-time running data of the subtask during execution, the method further includes: If the deviation between the real-time task execution progress and the preset task execution progress at the current moment exceeds the preset deviation, the task execution is determined to be abnormal; If the task execution error is due to insufficient resources, redundant resources are allocated from the idle task scheduling system to the current task scheduling system. If the task execution anomaly is due to data transmission delay, the data channel of the current subtask will be set to the highest priority through a unified message passing mechanism.

[0010] According to a task scheduling method provided by the present invention, after allocating each subtask to a corresponding candidate task scheduling system and executing the corresponding subtask by the candidate task scheduling system, the method further includes: A multi-dimensional evaluation indicator system is constructed based on real-time operational data; The weights of each indicator in the multi-dimensional evaluation index system are determined using the analytic hierarchy process (AHP), and the comprehensive score of each sub-task is calculated based on the weights. For each subtask, a task execution evaluation result is generated based on the comparison between the overall score and the historical score.

[0011] The present invention also provides a task scheduling device, comprising: The resource requirement data determination module is used to determine the resource requirement data for the target task. The subtask division module is used to divide the target task into multiple subtasks based on the resource requirement data; The candidate task scheduling system determination module is used to determine the real-time resource status data of each task scheduling system, and determine the candidate task scheduling system based on the real-time resource status data and resource constraint data. The task execution module is used to assign each of the subtasks to the corresponding candidate task scheduling system, so that the candidate task scheduling system can execute the corresponding subtasks.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the task scheduling method described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task scheduling method as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the task scheduling method described above.

[0015] This invention divides the target task into multiple subtasks based on the resource requirement data of the target task. Using a load balancing algorithm, each subtask is assigned to a corresponding candidate task scheduling system, which then executes the corresponding subtask. This task division allows large-scale tasks to be broken down and distributed across multiple task scheduling systems for parallel execution, effectively shortening the total task completion time and avoiding blocking due to waiting for a single resource. Furthermore, the load balancing algorithm ensures that each subtask is assigned to the currently suitable task scheduling system, fully utilizing the computing power of the task scheduling system and thus effectively improving task execution efficiency.

[0016] Furthermore, by breaking down the target task and accurately identifying the functional module with the highest resource demand level as the core sub-task, the present invention enables the heterogeneous resources of the scheduling system to prioritize the execution of the most critical path, effectively avoiding the barrel effect caused by the average allocation of resources, thereby ensuring that the core links in the entire task can operate at the highest efficiency, directly shortening the overall task time, and improving the completion efficiency and success rate of large and complex tasks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the task scheduling method provided by the present invention; Figure 2 This is a schematic diagram of the task scheduling device provided by the present invention; Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] Figure 1 This is one of the flowcharts illustrating the task scheduling method provided by the present invention, such as... Figure 1 As shown, the method includes the following: S1. Determine the resource requirements data for the target task; In this embodiment of the invention, the types of target tasks include teaching and training tasks, experimental simulation tasks, and combat simulation tasks. This embodiment of the invention can use natural language processing technology to interpret and extract task indicators from the task description text of the current target task, and then further determine resource requirement data based on the task indicators. These task indicators may include task completion time limits, data transmission latency thresholds, computational accuracy requirements, reliability and fault tolerance indicators, and security and confidentiality levels.

[0021] In this embodiment of the invention, the task completion time limit is the time requirement that the entire task or key stages must be completed, such as the simulation needing to be completed within 30 minutes or the real-time response delay being less than 1 second. Data transmission latency threshold: Applicable to tasks that rely on data streams or collaborative interactions, such as multi-node simulation synchronization and real-time audio and video transmission, it refers to the maximum allowable latency for data transmission between systems, for example, inter-node communication latency ≤50ms; Computational precision requirements: For example, the numerical precision required in simulation tasks, such as double precision for floating-point calculations; or the accuracy threshold in AI inference tasks, such as target recognition accuracy ≥ 95%; Reliability and fault tolerance metrics: such as task execution success rate requirements. For example, a task success rate of ≥99.99%, or the maximum allowable recovery time in the event of a failure; Security and confidentiality levels: such as data encryption strength, access control level, such as intranet transmission only and compliance with Level-4 security standards.

[0022] Resource requirement data further maps the aforementioned task indicators into specific data processing and resource scheduling parameters. Resource requirement data can include computing power requirements, dedicated hardware requirements, and software environment dependencies. Among them, computing power requirements are derived from the task time limit and computational complexity, which derives the required CPU / GPU computing power, memory capacity, and storage I / O performance. Dedicated hardware requirements include whether specific accelerator cards, sensors, or special peripherals are needed. Software environment dependencies include operating system type, specific algorithm library, or simulation engine version.

[0023] S2. Based on the resource requirement data, the target task is divided into multiple sub-tasks; In this embodiment of the invention, the target task can be divided into multiple functional modules. Based on the resource requirement data level of these functional modules, the sub-tasks corresponding to each functional module are determined, including core sub-tasks and auxiliary sub-tasks.

[0024] In this embodiment of the invention, a greedy algorithm can also be used to decompose the target task. A greedy algorithm is a heuristic algorithm that seeks a globally approximate optimum through local optima, and is suitable for task decomposition in heterogeneous systems.

[0025] In this embodiment of the invention, the execution steps of the greedy algorithm may include: S201. Target and Mission Analysis: The original target and mission, such as "joint air defense combat simulation", is decomposed into multiple functional modules, such as "target detection", "threat assessment", "firepower allocation" and "damage assessment" modules. The resource requirements priority of each module is clarified to ensure the computing power of the core module.

[0026] S202. Core Subtask Extraction: Identify modules in the task that play a decisive role in the overall goal, define them as core subtasks, and allocate resources to them with priority.

[0027] In this embodiment of the invention, the criteria for determining core sub-tasks may include task criticality and resource demand intensity. Among them, tasks with high task criticality may be target detection tasks, which are the basis for all subsequent decisions and are thus core sub-tasks; tasks with high task criticality may also be firepower allocation tasks, and these core sub-tasks require priority in ensuring computing power.

[0028] For example, the target task is battlefield situation generation, multi-sensor data fusion is the core sub-task, which requires priority allocation of GPU resources, while situation result visualization is an auxiliary sub-task, which can be allocated CPU resources.

[0029] S203. Auxiliary Subtask Splitting: Non-core modules are split into auxiliary subtasks, which are usually supportive and parallelizable tasks, such as data preprocessing, log recording, and result verification. Their resource requirements are low or can be adjusted flexibly.

[0030] S204. Subtask resource matching: Based on the capability data of the task scheduling system, assign an execution entity to each subtask, i.e., the most suitable task scheduling system.

[0031] In this embodiment of the invention, the task scheduling system can be a heterogeneous system, including computing systems, storage systems, and network systems, etc., and can be used for task scheduling. This embodiment of the invention can employ a greedy strategy to select the system with the best performance and remaining resources to meet the needs of the current subtask. For example, the core subtask is target recognition, requiring GPU computing power ≥ 8 TFLOPS, and is matched with task scheduling system A, which has the most available GPU resources; the auxiliary subtask is data transmission, requiring bandwidth ≥ 500 Mbps, and is matched with task scheduling system B, which has sufficient network resources.

[0032] In this embodiment of the invention, the three elements of a subtask can be further clarified, including the executing entity, dependencies, and time window. The executing entity is which system executes each subtask, such as system A executing a target detection task, system B executing a threat assessment task, etc. In this embodiment of the invention, the determining criteria for the executing entity can be whether the system's resource capabilities (such as computing power, bandwidth, etc.) meet the subtask requirements, and historical execution efficiency (such as a system having a shorter average processing time for similar tasks).

[0033] Dependencies refer to the sequence or data transfer logic between subtasks, typically represented by a directed acyclic graph (DAG). Dependencies include data dependencies and execution dependencies. Data dependencies occur when the output of task A serves as the input for task B, such as the result of a target detection task being the input data for a threat assessment task. Execution dependencies occur when task B can only be started after task A is completed, such as the firepower allocation task needing to be executed after the threat assessment is finished.

[0034] The time window is the planned start time and the latest end time for each subtask, and must meet the following requirements: (1) it does not exceed the total time limit of the target task; (2) it conforms to the dependency relationship, such as the start time of subtask B ≥ the end time of subtask A; (3) it reserves resource scheduling buffer time, such as subsystem switching and data transmission delay. Example: The total time of the target task must be ≤100 seconds, the time window of subtask A (data acquisition) is [0,20s], subtask B (data preprocessing) is [20s,50s], subtask C (target identification) is [50s,80s], and subtask D (result output) is [80s,100s].

[0035] S3. Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. In this embodiment of the invention, the real-time resource status data of the task scheduling system may include system capability data and task resource requirement data. The system capability data includes system available resource data and hardware computing power data, and the task resource requirement data includes computing resource requirement data, time requirement data, and dependent resource data.

[0036] In this embodiment of the invention, the available resource data of the system refers to the hardware / software resources currently available for allocation in each heterogeneous subsystem. For example, hardware resources include the number of CPU cores, memory capacity, GPU computing power, storage space, network bandwidth, etc.; software resources include callable algorithm libraries, such as deep learning frameworks and simulation modules, supported interface protocol types, and historical task processing performance (such as average response time and success rate). For instance, a certain UAV simulation system currently has 10 TFLOPS of available GPU computing power and 20 GB of memory; a certain ground command and control system (heterogeneous subsystem B) has 8 available CPU cores and 1 Gbps of network bandwidth.

[0037] Hardware computing power data is more refined computing power data used to evaluate the efficiency of a subsystem in processing specific tasks. It includes computing power type and computing power value. The computing power type is general computing, parallel computing, and special computing (such as for encryption or signal processing). The computing power value is measured in floating-point operations per second or trillion operations per second. For example, the computing power of an edge device is 2 TOPS, which is suitable for real-time data preprocessing.

[0038] In this embodiment of the invention, the computational resource requirements are as follows: for example, battlefield situation rendering tasks require GPU computing power ≥ 5 TFLOPS and memory ≥ 8 GB; data encryption tasks require FPGA computing power ≥ 1 TOPS. Time requirements include: a maximum total task time, such as completing target identification within 30 seconds; and deadlines for critical nodes, such as completing data acquisition within the first 10 seconds. Dependent resources include: whether specific software interfaces are required, such as calling AI models; and data input formats such as video streams and sensor data.

[0039] In this embodiment of the invention, resource constraint data includes the maximum load threshold of hardware devices and the maximum concurrent access limit of software authorization, etc.

[0040] S4. Assign each of the subtasks to the corresponding candidate task scheduling system, and have the candidate task scheduling system execute the corresponding subtask.

[0041] In this embodiment of the invention, a task-resource mapping model can be constructed based on the sub-tasks and their corresponding resource requirement data to quantitatively associate the task indicators of the sub-tasks with the resource requirements.

[0042] In this embodiment of the invention, each subtask can be assigned to a corresponding candidate task scheduling system based on a load balancing algorithm, and the candidate task scheduling system executes the corresponding subtask. The load balancing algorithm includes a weighted round-robin algorithm, whose weight coefficients can be determined by the resource utilization fluctuations in historical operating data. This algorithm allocates the resource requirements of each subtask to the corresponding candidate task scheduler, thereby achieving task scheduling and avoiding overload of a single system.

[0043] This invention, based on the resource requirement data of the target task, divides the target task into multiple subtasks. Using a load balancing algorithm, each subtask is assigned to a corresponding candidate task scheduling system, which then executes the corresponding subtask. By dividing the task, large-scale tasks can be broken down and distributed across multiple task scheduling systems for parallel execution, effectively shortening the total task completion time and avoiding blocking due to waiting for a single resource. Furthermore, the load balancing algorithm ensures that each subtask is assigned to the currently suitable task scheduling system, fully utilizing the computing power of the task scheduling system and thus effectively improving task execution efficiency.

[0044] In one embodiment, step S1, determining the resource requirement data for the target task, includes: S11. Extract the task metrics for the target task; In this embodiment of the invention, the user can input the target task via voice, text file, form, or interactive interface, and then use natural language processing technology to extract task metrics from the text description file of the target task, including: Segment the entire text into word or character sequences, filter out function words with no actual meaning such as "的", "了", "在", etc., and retain the key content words; train or use a pre-trained NER model to identify specific types of entities in the text. Among them, entities include time indicators, quantity indicators, performance indicator task types, resource types, and constraints, etc.; adopt dependency syntactic analysis or relation extraction models to construct the relationships between entities, form a triple of subject, relation, and object, and map the extracted triples to a predefined task indicator knowledge base. The system converts a long text description into a structured JSON object, that is, a task indicator. Among them, the knowledge base stores the standard indicator systems of various tasks and their corresponding machine-readable formats.

[0045] S12. Based on the operation data of historical tasks and the task indicators, use a regression analysis model to predict the resource requirement data during the execution of the target task.

[0046] In an embodiment of the present invention, the historical task can be a history of the same type as the current target task, and the operation data is the historical task resource requirement data and historical task indicators generated during the actual execution of the historical task. The historical task resource requirement data includes computing resource consumption, network resource consumption, and timing performance data.

[0047] In an embodiment of the present invention, the historical task indicators can be used as features, and the historical task resource requirement data can be used as labels to train a regression analysis model. Input the task indicators of the current target task into the trained regression analysis model, and output a predicted value of the resource requirement data.

[0048] In an embodiment of the present invention, the operation data further includes bottleneck records. The bottleneck points in the bottleneck records are the actual bottleneck points that occur during the task operation, such as "subsystem A has a surge in response due to insufficient memory at time t", "network channel B is congested", etc. An embodiment of the present invention can also predict the bottleneck points during the execution of the target task based on the regression analysis model and the historical task bottleneck points.

[0049] In an embodiment of the present invention, the regression analysis model is suitable for predicting continuous numerical output variables. That is, in an embodiment of the present invention, the resource requirement data can be accurately analyzed through the regression analysis model. An embodiment of the present invention can adopt a linear regression model or a multiple regression model, a gradient boosting regression tree, and a random forest regression model. Among them, the linear regression model or the multiple regression model is suitable for scenarios where there is an obvious linear relationship between features and target values; the gradient boosting regression tree can effectively capture complex non-linear relationships and data interaction effects, and has a high prediction accuracy for tabular data; the gradient boosting regression tree can effectively suppress overfitting through regularization and randomness strategies, and has a high training efficiency in most practices.

[0050] This invention utilizes a regression analysis model to predict resource requirements based on historical data, enabling a fully automated conversion from fuzzy task requirements to precise quantitative resource planning. This effectively reduces reliance on manual evaluation by domain experts, avoids the subjectivity and errors of human estimation, and allows the system to perform reasonable resource reservation and scheduling planning before task execution, thereby improving the reliability of task scheduling.

[0051] In one embodiment, step S2, based on the resource requirement data, divides the target task into multiple sub-tasks, including: S21. Divide the target task into multiple functional modules, and determine the resource requirement level of each functional module based on the resource requirement data; In this embodiment of the invention, functional modules can be divided according to the inherent logic and business workflow of the target task. Taking a combat simulation task as an example, it can be divided into the following functional modules: Functional Module A: Battlefield environment loading and rendering, responsible for the generation and graphics drawing of terrain, weather, and infrastructure; Functional Module B: Entity behavior simulation, responsible for AI behavior calculations such as movement, perception, and decision-making of troop units and weapons; Functional Module C: Physical effect calculation, responsible for high-precision calculations such as collision detection, ballistic calculation, and damage assessment; Functional Module D: Data communication and synchronization, responsible for state synchronization and message transmission between multiple subsystems or nodes; Functional Module E: Monitoring and log recording, responsible for monitoring operational status, recording data, and generating post-event review files.

[0052] In this embodiment of the invention, based on the division of functional modules, the resource requirement data of each functional module can be further determined, and the functional module with the highest resource requirement level can be determined according to the resource intensity and resource criticality of the resource requirement data. Resource intensity refers to the absolute demand of a module for one or more resources (CPU, GPU, memory, bandwidth). For example, a module requiring GPU computing power >10 TFLOPS has an extremely high resource intensity level. Resource criticality refers to the degree to which the performance of a module directly affects the overall task objective. Even if a module has low resource requirements, if it becomes stuck, causing the entire task to stall, it becomes a node on the critical path, and its criticality is also extremely high.

[0053] S22. Identify the functional module with the highest resource requirement level as the core sub-task, and identify the remaining functional modules as auxiliary sub-tasks.

[0054] In this embodiment of the invention, the core subtask is characterized by: performance sensitivity: the execution efficiency of the core subtask directly determines the time consumption and success or failure of the entire target task; resource greed: the core subtask requires a large amount of the highest quality resources in the system, such as high-performance GPUs, high-frequency CPUs, and low-latency networks; and scheduling priority: the core subtask has the highest priority in resource allocation and scheduling order. The system must prioritize and make every effort to match and guarantee the necessary resources for it. In subsequent resource allocation and task execution, the priority of the core subtask is higher than that of the auxiliary subtasks.

[0055] In this embodiment of the invention, the execution subject and dependencies of each subtask can be further specified, such as the output of subtask A being the input of subtask B and the time window.

[0056] This invention breaks down the target task and accurately identifies the functional module with the highest resource demand level as the core sub-task. This enables the heterogeneous resources of the scheduling system to prioritize the execution of the most critical path, effectively avoiding the "weakest link" effect caused by the average allocation of resources. This ensures that the core links in the entire task can operate at the highest efficiency, directly shortening the overall task time and improving the completion efficiency and success rate of large and complex tasks.

[0057] In one embodiment, after step S4, which involves assigning each subtask to a corresponding candidate task scheduling system based on the task scheduling scheme, and having the candidate task scheduling system execute the corresponding subtask, the method further includes: S5. Obtain the real-time running data of the subtask during the execution process; In this embodiment of the invention, real-time running data includes task execution progress, real-time resource usage data, signal interaction data, and data transmission rate, etc.

[0058] S6. Based on a unified time reference, perform time alignment processing on the real-time running data.

[0059] In this embodiment of the invention, real-time running data can be time-aligned based on a unified time base to ensure the consistency of system actions in time sequence. For example, if system A sends data at timestamp T1, system B needs to receive and process it within T1+10ms.

[0060] In this embodiment of the invention, real-time resource usage data can be compared with early warning thresholds, and a sliding window algorithm can be used to monitor data fluctuation trends. For example, if the CPU usage rate increases by more than or equal to 10% for three consecutive sampling periods, the risk of resource overload can be predicted.

[0061] In this embodiment of the invention, the core operating parameters of the task scheduling system, such as data cache size, model inference batches, and signal sampling frequency, can be dynamically adjusted based on real-time operating data. The adjustment strategy is determined based on the comparison results between real-time operating data and historical best data. For example, when the data transmission rate is lower than 100Mbps, the cache size is adjusted from 512MB to 1GB.

[0062] This invention, by acquiring multi-dimensional real-time operational data and performing alignment processing based on a unified timestamp, ensures the temporal consistency of system actions, which is beneficial to improving the efficiency of task execution.

[0063] In one embodiment, the real-time runtime data includes real-time task execution progress; After obtaining the real-time running data of the subtask during execution, the method further includes: S51. If the deviation between the real-time task execution progress and the current preset task execution progress exceeds the preset deviation, determine that the task execution is abnormal. In this embodiment of the invention, the amount of work completed or the simulation time advanced is continuously collected and calculated. For example, in a simulation task with a total duration of 1 hour, ideally, 50% of the progress should be completed by the 30th minute.

[0064] The current preset task execution progress is the baseline progress that should theoretically be achieved at this moment, derived from the task plan and timeline. This baseline is usually derived from the plan generated during task initialization.

[0065] The preset deviation is a configurable threshold, typically set as a percentage (e.g., ±5%) or an absolute time value (e.g., ±60 seconds), depending on the criticality of the task. The preset deviation defines the acceptable range of normal fluctuation between the plan and reality.

[0066] The logic for determining whether task execution is abnormal in this embodiment of the invention is as follows: the system periodically compares the real-time progress with the preset progress. Once |real-time progress - preset progress| > preset deviation, an alarm is immediately triggered, confirming that the task execution is abnormal.

[0067] S52. If the task execution abnormality is due to insufficient resources, allocate redundant resources from the idle task scheduling system to the current task scheduling system. In this embodiment of the invention, if the CPU utilization rate of the task scheduling system where the current subtask is located is continuously higher than the preset utilization rate threshold and the system load (Load Average) is much higher than the number of CPU cores, and the task progress is slow, then it is determined that there are insufficient computing resources.

[0068] In this embodiment of the invention, the criteria for determining insufficient computing resources also include queue indicators and correlation analysis data. For example, if there is a large backlog of tasks or requests waiting to be processed in the task queue, it can be determined that computing resources are insufficient; if there is a global performance degradation of the system, it can be determined that computing resources are insufficient.

[0069] In this embodiment of the invention, when it is determined that computing resources are insufficient, the resource manager can query the global resources for idle task scheduling systems with similar surplus resources, and use the idle task scheduling system to allocate redundant resources to the current scheduling system. The scheduling method can include vertical expansion and horizontal expansion. Vertical expansion is to dynamically add resources to the currently overloaded task scheduling system, such as adding CPU and memory online. Horizontal expansion is to migrate some subtasks of the current system or directly schedule them to the newly discovered idle task scheduling system for execution, thereby achieving load balancing.

[0070] S53. In the event that the task execution abnormality is due to data transmission delay, the data channel of the current subtask is set to the highest priority through a unified message passing mechanism.

[0071] In this embodiment of the invention, the unified messaging mechanism adopts a standardized communication middleware that supports quality of service policies and priority queues. Based on the unified messaging mechanism, the data channel of the current subtask can be set to the highest priority, and messages of the channel can be scheduled and processed first, bandwidth resources can be allocated first, and routing and forwarding can be performed first.

[0072] In this embodiment of the invention, if resource utilization is normal but network indicators deteriorate, message queues accumulate, or tasks are blocked while waiting for I / O, it is determined to be a data transmission delay.

[0073] In this embodiment of the invention, instructions can be sent to the middleware through a unified messaging mechanism to raise the priority of a specific topic or communication channel used by the affected subtask to the highest level. This allows for priority scheduling, routing, and delivery of data packets on the high-priority channel, allocating sufficient bandwidth resources to it, effectively reducing its end-to-end transmission latency, breaking communication bottlenecks, and enabling downstream tasks to obtain data and continue execution in a timely manner.

[0074] This invention, through the introduction of multi-dimensional monitoring indicators such as CPU utilization, system load, and task progress, enables accurate and automated diagnosis of anomalies. This avoids resource waste caused by misjudgments, and resource allocation is only performed when computing resources are truly needed, preventing the erroneous expansion of computing resources during network congestion and thus exacerbating the congestion.

[0075] In one embodiment, after assigning each subtask to a corresponding candidate task scheduling system and executing the corresponding subtask using the candidate task scheduling system, the method further includes: A multi-dimensional evaluation indicator system is constructed based on real-time operational data; In this embodiment of the invention, the indicators of the multi-dimensional evaluation indicator system may include collaborative efficiency indicators, resource utilization indicators, and task quality indicators. Among them, collaborative efficiency indicators include total task completion time, average cross-system data interaction delay, etc.; resource utilization indicators may include deviation rate between actual resource occupancy and predicted value, idle resource ratio, etc.; task quality indicators include sub-task completion rate, collaborative accuracy error, and result accuracy, etc.

[0076] The weights of each indicator in the multi-dimensional evaluation index system are determined using the analytic hierarchy process (AHP), and the comprehensive score of each sub-task is calculated based on the weights. In this embodiment of the invention, the weights of the collaborative efficiency index (0.4), resource utilization index (0.3), and task quality index (0.3) are determined using the analytic hierarchy process (AHP), and the comprehensive score is calculated using a fuzzy comprehensive evaluation model.

[0077] For each subtask, a task execution evaluation result is generated based on the comparison between the overall score and the historical score.

[0078] In this embodiment of the invention, if the overall score is lower than that of similar historical tasks, the lower-scoring indicators can be identified, and the difference analysis method can be used to locate optimization points. For example, if the resource utilization rate indicator is found to have a low score, the task allocation strategy needs to be adjusted.

[0079] To improve the accuracy of dynamic adjustments, embodiments of the present invention can execute the same target task multiple times, and make corresponding adjustments if the overall score of each execution is lower than the historical score.

[0080] In this embodiment of the invention, the task execution evaluation results may include experimental data statistics, detailed indicator scores, a list of problems, and targeted optimization suggestions.

[0081] This invention introduces a quantitative evaluation and feedback mechanism after task execution. By constructing a multi-dimensional indicator system and calculating a comprehensive score, and by comparing it with historical data, it can accurately locate indicators with low scores, thereby providing a clear direction for optimizing the task scheduling system and improving the efficiency of task execution.

[0082] Implementing the embodiments of the present invention has the following beneficial effects: This invention, based on the resource requirement data of the target task, divides the target task into multiple subtasks. Using a load balancing algorithm, each subtask is assigned to a corresponding candidate task scheduling system, which then executes the corresponding subtask. By dividing the task, large-scale tasks can be broken down and distributed across multiple task scheduling systems for parallel execution, effectively shortening the total task completion time and avoiding blocking due to waiting for a single resource. Furthermore, the load balancing algorithm ensures that each subtask is assigned to the currently suitable task scheduling system, fully utilizing the computing power of the task scheduling system and thus effectively improving task execution efficiency.

[0083] Furthermore, by breaking down the target task and accurately identifying the functional module with the highest resource demand level as the core sub-task, the heterogeneous resources of the scheduling system can prioritize the execution of the most critical path, effectively avoiding the barrel effect caused by the average allocation of resources. This ensures that the core links in the entire task can operate at the highest efficiency, directly shortening the overall task time and improving the completion efficiency and success rate of large and complex tasks.

[0084] The task scheduling device provided by the present invention is described below. The task scheduling device described below and the task scheduling method described above can be referred to in correspondence.

[0085] Please see Figure 2 The above is a schematic diagram of a task scheduling device provided in an embodiment of the present invention. The device includes: Resource requirement data determination module 210 is used to determine the resource requirement data of the target task; The subtask division module 220 is used to divide the target task into multiple subtasks based on the resource requirement data. The candidate task scheduling system determination module 230 is used to determine the real-time resource status data of each task scheduling system, and determine the candidate task scheduling system in the task scheduling system based on the real-time resource status data and resource constraint data. The task execution module 240 is used to assign each of the subtasks to the corresponding candidate task scheduling system, so that the candidate task scheduling system can execute the corresponding subtasks.

[0086] In one embodiment, the determination of the resource requirement data for the target task includes: Extract the task metrics for the target task; Based on historical task operation data and the task indicators, a regression analysis model is used to predict the resource requirements of the target task during execution.

[0087] In one embodiment, dividing the target task into multiple sub-tasks based on the resource requirement data includes: The target task is divided into multiple functional modules, and the resource requirement level of each functional module is determined based on the resource requirement data. The functional module with the highest resource requirement level is identified as the core sub-task, and the remaining functional modules are identified as auxiliary sub-tasks.

[0088] In one embodiment, after assigning each subtask to a corresponding candidate task scheduling system and executing the corresponding subtask using the candidate task scheduling system, the method further includes: Obtain real-time running data of the subtask during execution; Based on a unified time reference, the real-time running data is time-aligned.

[0089] In one embodiment, the real-time runtime data includes real-time task execution progress; After obtaining the real-time running data of the subtask during execution, the method further includes: If the deviation between the real-time task execution progress and the preset task execution progress at the current moment exceeds the preset deviation, the task execution is determined to be abnormal; If the task execution error is due to insufficient resources, redundant resources are allocated from the idle task scheduling system to the current task scheduling system. If the task execution anomaly is due to data transmission delay, the data channel of the current subtask will be set to the highest priority through a unified message passing mechanism.

[0090] In one embodiment, after assigning each subtask to a corresponding candidate task scheduling system and executing the corresponding subtask using the candidate task scheduling system, the method further includes: A multi-dimensional evaluation indicator system is constructed based on real-time operational data; The weights of each indicator in the multi-dimensional evaluation index system are determined using the analytic hierarchy process (AHP), and the comprehensive score of each sub-task is calculated based on the weights. For each subtask, a task execution evaluation result is generated based on the comparison between the overall score and the historical score.

[0091] Implementing the embodiments of the present invention has the following beneficial effects: This invention, based on the resource requirement data of the target task, divides the target task into multiple subtasks. Using a load balancing algorithm, each subtask is assigned to a corresponding candidate task scheduling system, which then executes the corresponding subtask. By dividing the task, large-scale tasks can be broken down and distributed across multiple task scheduling systems for parallel execution, effectively shortening the total task completion time and avoiding blocking due to waiting for a single resource. Furthermore, the load balancing algorithm ensures that each subtask is assigned to the currently suitable task scheduling system, fully utilizing the computing power of the task scheduling system and thus effectively improving task execution efficiency.

[0092] Furthermore, by breaking down the target task and accurately identifying the functional module with the highest resource demand level as the core sub-task, the heterogeneous resources of the scheduling system can prioritize the execution of the most critical path, effectively avoiding the barrel effect caused by the average allocation of resources. This ensures that the core links in the entire task can operate at the highest efficiency, directly shortening the overall task time and improving the completion efficiency and success rate of large and complex tasks.

[0093] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include: a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a task scheduling method, including: Determine the resource requirements data for the target task; Based on the resource requirement data, the target task is divided into multiple sub-tasks; Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. Each of the subtasks is assigned to a corresponding candidate task scheduling system, and the candidate task scheduling system executes the corresponding subtask.

[0094] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer being able to execute a task scheduling method provided by the above methods, comprising: Determine the resource requirements data for the target task; Based on the resource requirement data, the target task is divided into multiple sub-tasks; Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. Each of the subtasks is assigned to a corresponding candidate task scheduling system, and the candidate task scheduling system executes the corresponding subtask.

[0096] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a task scheduling method provided by the methods described above, comprising: Determine the resource requirements data for the target task; Based on the resource requirement data, the target task is divided into multiple sub-tasks; Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. Each of the subtasks is assigned to a corresponding candidate task scheduling system, and the candidate task scheduling system executes the corresponding subtask.

[0097] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task scheduling method, characterized in that, include: Determine the resource requirements data for the target task; Based on the resource requirement data, the target task is divided into multiple sub-tasks; Determine the real-time resource status data of each task scheduling system, and determine candidate task scheduling systems based on the real-time resource status data and resource constraint data. Each of the subtasks is assigned to a corresponding candidate task scheduling system, and the candidate task scheduling system executes the corresponding subtask.

2. The task scheduling method as described in claim 1, characterized in that, The resource requirement data for determining the target task includes: Extract the task metrics for the target task; Based on historical task operation data and the task indicators, a regression analysis model is used to predict the resource requirements of the target task during execution.

3. The task scheduling method as described in claim 1, characterized in that, Based on the resource requirement data, the target task is divided into multiple sub-tasks, including: The target task is divided into multiple functional modules, and the resource requirement level of each functional module is determined based on the resource requirement data. The functional module with the highest resource requirement level is identified as the core sub-task, and the remaining functional modules are identified as auxiliary sub-tasks.

4. The task scheduling method as described in claim 1, characterized in that, After assigning each subtask to the corresponding candidate task scheduling system and executing the corresponding subtask using the candidate task scheduling system, the process further includes: Obtain real-time running data of the subtask during execution; Based on a unified time reference, the real-time running data is time-aligned.

5. The task scheduling method as described in claim 4, characterized in that, The real-time operational data includes the real-time task execution progress; After obtaining the real-time running data of the subtask during execution, the method further includes: If the deviation between the real-time task execution progress and the preset task execution progress at the current moment exceeds the preset deviation, the task execution is determined to be abnormal; If the task execution error is due to insufficient resources, redundant resources are allocated from the idle task scheduling system to the current task scheduling system. If the task execution anomaly is due to data transmission delay, the data channel of the current subtask will be set to the highest priority through a unified message passing mechanism.

6. The task scheduling method according to any one of claims 1-5, characterized in that, After assigning each subtask to the corresponding candidate task scheduling system and executing the corresponding subtask using the candidate task scheduling system, the process further includes: A multi-dimensional evaluation indicator system is constructed based on real-time operational data; The weights of each indicator in the multi-dimensional evaluation index system are determined using the analytic hierarchy process (AHP), and the comprehensive score of each sub-task is calculated based on the weights. For each subtask, a task execution evaluation result is generated based on the comparison between the overall score and the historical score.

7. A task scheduling device, characterized in that, include: The resource requirement data determination module is used to determine the resource requirement data for the target task. The subtask division module is used to divide the target task into multiple subtasks based on the resource requirement data; The candidate task scheduling system determination module is used to determine the real-time resource status data of each task scheduling system, and determine the candidate task scheduling system based on the real-time resource status data and resource constraint data. The task execution module is used to assign each of the subtasks to the corresponding candidate task scheduling system, so that the candidate task scheduling system can execute the corresponding subtasks.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the task scheduling method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the task scheduling method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the task scheduling method as described in any one of claims 1 to 6.

Images