DAG task scheduling control method based on neural network scoring

Abstract

The invention provides a DAG (Directed Acyclic Graph) task scheduling control method based on neural network scoring, which belongs to the technical field of task scheduling control and comprises the following steps of: 1, constructing a DAG task dependency graph according to tasks and dependency relationships thereof; 2, extracting multi-dimensional task features from the DAG task dependency graph and system historical data; 3, inputting core task characteristics, and outputting task scheduling priority scores; 4, performing resource availability judgment on the task; step 5, dynamically adjusting the maximum degree of parallelism of the task; step 6, when the scheduling task is executed, performing task execution monitoring and state updating at the same time; and 7, cyclically scheduling and repeatedly executing the processes from the step 2 to the step 6 until all tasks in the DAG task dependency graph are completed. By constructing a multi-dimensional task feature system, designing a hybrid neural network scoring model and dynamically optimizing a scheduling strategy and the degree of parallelism, efficient, intelligent and adaptive task scheduling oriented to a complex distributed environment is realized.

Description

Technical Field

[0001] This invention belongs to the field of task scheduling and control technology, specifically relating to a DAG task scheduling and control method based on neural network scoring. Background Technology

[0002] In the interdisciplinary field of computer science, artificial intelligence, and distributed computing, DAG (Directed Acyclic Graph) task scheduling is a key step in achieving efficient execution of complex tasks. With the rapid development of deep learning and reinforcement learning technologies, researchers are gradually trying to introduce intelligent scheduling mechanisms, hoping to use neural networks to model and learn task and system states to achieve adaptive scheduling strategies to cope with the challenges of increased task complexity, diversified resource types, and dynamic load changes in distributed systems.

[0003] However, existing DAG task scheduling methods have significant bottlenecks: Feature modeling is limited: Most models focus only on single-dimensional feature representation, such as task type and resource load, and lack joint modeling of multi-dimensional information such as task graph structure features (such as dependency complexity), historical execution behavior (such as execution time and success rate), and global resource utilization. This results in scheduling decisions failing to fully reflect the matching relationship between task requirements and system status.

[0004] Weak resource adaptability: In heterogeneous resource environments, existing methods are unable to dynamically adapt to the differences in resource requirements of different tasks, which can easily lead to resource idleness or overload, reducing the overall resource utilization efficiency of the system.

[0005] Imbalance between execution efficiency and global optimality: Traditional static rule scheduling strategies cannot adjust in real time according to system load fluctuations and task execution status, which may lead to problems such as excessively long waiting time for high-priority tasks, low task throughput, or slow average response speed, making it difficult to achieve global scheduling optimality.

[0006] Therefore, there is an urgent need for an intelligent DAG task scheduling method that integrates a multi-feature neural network scoring mechanism, can dynamically evaluate task execution priority, respond to changes in system resource status in real time, and has autonomous learning capabilities, so as to overcome the limitations of existing technologies in terms of heterogeneous resource adaptation, execution efficiency, and global optimality. Summary of the Invention

[0007] The purpose of this invention is to provide a DAG task scheduling and control method based on neural network scoring. By constructing a multi-dimensional task feature system, designing a hybrid neural network scoring model, and dynamically optimizing the scheduling strategy and parallelism, this method achieves efficient, intelligent, and adaptive task scheduling for complex distributed environments.

[0008] The present invention employs the following technical solution.

[0009] A DAG task scheduling and control method based on neural network scoring includes: Step 1: Construct a DAG task dependency graph based on the tasks and their dependencies; Step 2: Extract multi-dimensional task features from the DAG task dependency graph and system historical data; Step 3: Design a hybrid neural network model combining CNN and MLP, inputting core task features and outputting task scheduling priority scores; Step 4: Determine the resource availability of tasks based on their task scheduling priority scores, from highest to lowest. Step 5: Dynamically adjust the maximum parallelism of tasks during task execution: Step 6: While scheduling task execution, simultaneously monitor task execution and update its status; Step 7: Repeat the process from Step 2 to Step 6 in a loop until all tasks in the DAG task dependency graph are completed.

[0010] Preferably, step 1 specifically includes: Based on the tasks and their dependencies, a directed acyclic graph (DAG) is generated to clarify the task execution order and constraint relationships. The graph structure of a directed acyclic graph (DAG) is defined as follows: Let the task set be... Each task For a node in a directed acyclic graph (DAG), where , The total number of tasks; edges To represent a dependency relationship, that is yes The prerequisite task.

[0011] Preferably, in step 1, each task It includes three basic attributes: Task Priority ; Resource requirements ; Dependency relationships: Records the set of prerequisite tasks .

[0012] Preferably, step 2 specifically includes: Four core task features were extracted from the DAG graph and historical system data; these four core task features include: Task Standardized priority ; Task resource requirements ; Task Historical execution data ; Task Dependency complexity .

[0013] Preferably, in step 2, the standardized priority is obtained. The methods include: Tasks can be set manually or automatically assigned by the system. Task priority The task priority value is 1 indicates the highest priority. By standardizing, the value is transformed into a continuous value between 0 and 1. The formula for standardization is: ; in This is the standardized priority.

[0014] Preferably, in step 2, the resource requirements are obtained. The methods include: The formula for calculating the comprehensive resource demand score is as follows: ; in The quantity of different types of resources required for the task; For the task For the The demand for similar resources ; For the first Importance weights for resource categories; For the task Resource requirements.

[0015] Preferably, in step 2, historical execution data is obtained. The methods include: The formula for calculating the historical efficiency confidence score is as follows: ; Among them, weight Calculated using the exponential decay function: , This refers to the size of the historical scheduling window. For the first Whether the execution was successful; The z-score standardized version is the first z-score obtained by applying the z-score standardization method. The execution time; The attenuation coefficient; For the first Weighting of historical data; For the task Historical execution data.

[0016] Preferably, in step 2, the dependency complexity is obtained. The methods include: The complexity of quantifiable task dependencies is calculated using the following formula: ; Attention value Calculated using the following formula: , For the Softmax function, For the task Depth in the DAG task dependency graph; For the task in-degree; For the task The set of prerequisite dependent tasks; and Tasks Query vectors and tasks The key vector; The vector dimension normalization coefficient; , and All are weighting coefficients that are greater than zero; For the task The dependency complexity.

[0017] Preferably, in step 3, the model structure of the hybrid CNN and MLP neural network specifically includes: The CNN module contains at least one convolutional layer, one global average pooling layer, and one fully connected layer for input tasks. Dependency complexity Convolutional layers are based on the task Dependency complexity Local dependency patterns are extracted as convolutional features, and global average pooling layers compress these features into fixed-dimensional vectors. Fully connected layers then process these fixed-dimensional vectors and output a scalar. feature; The MLP module contains 2-3 hidden layers for inputting and processing tasks. Standardized priority Resource requirements Historical execution data Such structured features learn the nonlinear relationships between features. The ReLU activation function processes these nonlinear relationships and outputs a scalar. Features, range mapped to [0,1]; The output layer, connected to both the CNN and MLP modules, is used to fuse the scalar outputs of the CNN and MLP modules using a weighted linear combination. Features and scalars Features are used to generate task scheduling priority scores.

[0018] Preferably, in step 3, a weighted linear combination is used to fuse the scalar outputs from CNN and MLP respectively. Features and scalars The feature is that the formula for calculating the task scheduling priority score is: ; in For the task Task scheduling priority score, These are the weight coefficients output by the CNN module; These are the weighting coefficients output by the MLP module; For bias terms; constraints: , and .

[0019] Preferably, step 4 specifically includes: Based on the resource contention model, the system determines whether current resources meet task requirements by ranking tasks according to their scheduling priority scores from highest to lowest. The calculation formula for the resource contention model is as follows: ; in For the task In the The amount of resources that can be allocated to a resource class; For the task For the The amount of resources required for this type of resource; For all currently pending tasks, the first Total demand for class resources; For the current system number The total available amount of this type of resource; Next, the judgment logic is executed: only if the task... The amount of resources that can be allocated across all resource classes. All are not less than When, schedule the task. Execute; otherwise, leave the task Add to the waiting queue until resources are released to satisfy the task. The amount of resources that can be allocated across all resource classes. All are not less than Only then will tasks be scheduled. implement.

[0020] Preferably, step 5 specifically includes: The formula for calculating the parallelism of dynamically adjusted tasks is as follows: ; in For the task The maximum number of instances that can be parallelized; Therefore, in scheduling tasks During execution, limit the task. The maximum parallelism of the task is no higher than .

[0021] Preferably, step 6 specifically includes: Implement an asynchronous monitoring mechanism: Track the task execution status in real time through the system's task execution monitoring module; Fault-tolerant rescheduling: If a task times out or fails to execute, a rescheduling mechanism is triggered, which means the task is added back to the waiting queue; Execution status update: After the task is completed, update the task status in the DAG task dependency graph and wake up its subsequent dependent tasks.

[0022] The beneficial effects of the present invention are as follows: Compared with the prior art, the technical effects of the present invention include: (i) Resource utilization rate has been significantly improved By employing a resource contention model and dynamic parallelism adjustment, the system accurately matches task resource requirements with system resource status, avoiding resource idleness (e.g., increasing parallelism under low load) or overload (e.g., decreasing parallelism under high load). Experimental data shows that compared to traditional static scheduling methods, system resource utilization is improved by 25% to 40%, especially in heterogeneous resource environments (e.g., CPU+GPU hybrid clusters), where the improvement in resource adaptability is even more significant.

[0023] (II) Optimization of Task Execution Efficiency Task latency reduction: Priority scheduling based on neural network scoring ensures that high-priority and high-urgency tasks are executed first, while incorporating historical execution data to avoid inefficient tasks consuming resources. Compared to traditional FCFS (First-Come, First-Served) scheduling, the average task completion latency is reduced by 30%~50%, and the latency of critical tasks (such as P1 priority) is reduced by more than 60%. Improved task throughput: Dynamic parallelism adjustment and dependency bottleneck optimization (by extracting dependency features through CNN) reduce task blocking time, resulting in a 20%~35% increase in system task throughput.

[0024] (III) Enhanced Adaptive Capability of Scheduling Strategies Online learning adapts to dynamic loads: It supports model parameter updates based on real-time execution data, and can adapt to system load fluctuations (such as prioritizing scheduling tasks with low resource requirements during peak loads) and changes in task characteristics (such as feature learning for new task types), improving scheduling strategy stability by 45%; Enhanced fault tolerance: Asynchronous monitoring and rescheduling mechanisms ensure that tasks can be quickly rescheduled after timeout or failure, improving task execution success rate by 15%~25%, with even more prominent reliability advantages in distributed system node failure scenarios.

[0025] (iv) Advantages of scalability and interpretability Highly scalable: Supports DAG task dependency graphs with arbitrarily complex structures (such as multi-layer dependency tasks with a depth > 10), adapts to distributed systems of different sizes (from small clusters of 10 nodes to large clusters of 1000+ nodes), without requiring modification of the core logic. Enhanced decision interpretability: The weighted linear combination of the neural network output layer can clearly quantify the influence weight of each feature (priority, resource requirements, etc.) on scheduling decisions. Combined with the attention mechanism, it can explain the impact of dependencies on priority, making it easier for operators to optimize scheduling strategies.

[0026] (v) Demonstration of Business Value In practical applications (such as big data processing, AI model training, and cloud computing task scheduling), this invention can: Reduce enterprise IT resource investment costs (due to improved resource utilization); Improve business response speed (e.g., reduce latency of real-time data analysis tasks); It reduces business losses caused by task failures (such as the high reliability of financial transaction tasks) and has significant practical application value. Attached Figure Description

[0027] Figure 1 This is a flowchart of the DAG task scheduling and control method based on neural network scoring described in this invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, any other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0029] like Figure 1 As shown, the present invention provides a DAG task scheduling and control method based on neural network scoring, comprising: Step 1: Construct a DAG task dependency graph based on the tasks and their dependencies; In a preferred but non-limiting embodiment of the present invention, step 1 specifically includes: Based on the tasks and their dependencies, a directed acyclic graph (DAG) is generated to clarify the task execution order and constraint relationships. The graph structure of a directed acyclic graph (DAG) is defined as follows: Let the task set be... Each task For a node in a directed acyclic graph (DAG), where , The total number of tasks; edges To represent a dependency relationship, that is yes Prerequisite tasks ( Need to be Execute after completion).

[0030] In a preferred but non-limiting embodiment of the present invention, in step 1, the core attribute of the task is: each task It includes three basic attributes: Task Priority Task Priority It is set according to specific requirements and reflects the importance of the task. Resource requirements : Describes the task's resource consumption requirements, such as CPU, memory, and bandwidth; Dependency relationships: Records the set of prerequisite tasks .

[0031] Step 2: Extract multi-dimensional task features from the DAG task dependency graph and system historical data. The system specifically refers to the distributed computing system that executes the DAG task. In a preferred but non-limiting embodiment of the present invention, step 2 specifically includes: Four core task features were extracted from the DAG graph and historical system data to provide input for the neural network scoring; the four core task features include: Prioritizing tasks Standardization is used to achieve the task. Standardized priority ; In a preferred but non-limiting embodiment of the present invention, in step 2, the standardized priority is obtained. The methods include: Tasks can be set manually or automatically assigned by the system. Task priority The task priority value can be: 1 indicates the highest priority, which is... The larger the value, the lower the priority. Standardization transforms the data into continuous values ​​between 0 and 1, making it easier for neural networks to process. The formula for standardization is: ; in This represents the standardized priority (the larger the value, the higher the priority).

[0032] Task resource requirements ; In a preferred but non-limiting embodiment of the present invention, in step 2, the resource requirements are obtained. The methods include: Perform a weighted comprehensive calculation: Considering the differences in the task's demand for various types of resources and the weight of resource importance, calculate the comprehensive resource demand score. The calculation formula is as follows: ; in The quantity of resource types required by the task (such as CPU, memory, bandwidth, etc.). For the task For the Requirements for class resources (such as the number of CPU cores, the number of MB of memory, and the number of bandwidth after standardization using the z-score standardization method). ; For the first The importance weight of resource types (set through experience or specific requirements, to meet) and ); For the task The comprehensive resource requirement score, which is the task Resource requirements.

[0033] Task Historical execution data Historical execution data refers to key indicators recorded by a distributed computing system when it has performed similar or identical tasks in the past. It is an important basis for evaluating the stability, efficiency and resource requirements of task execution. In a preferred but non-limiting embodiment of the present invention, in step 2, historical execution data is obtained. The methods include: Execution time decay weighted model: considering the most recent Sub-task Based on historical performance data, a historical efficiency confidence score is calculated to reflect the stability and efficiency of task execution. The calculation formula is as follows: ; Among them, weight Calculated using the exponential decay function: , Size of the historical scheduling window (selecting the most recent one) (Records of the historical execution of this task) For the first Whether the execution was successful (1 = success, 0 = failure). =1 corresponds to the most recent execution. = The earliest execution; The z-score standardized version is the first z-score obtained by applying the z-score standardization method. The execution time; The attenuation coefficient ( It can be set according to specific requirements; it is used to control the rate of weight decay. >0 (the larger the value, the higher the weight of recent data relative to older data, and the more it highlights the impact of recent performance on the current score). For the first Weighting of historical data; For the task The historical efficiency confidence score, which is the task Historical execution data (the larger the value, the more stable and efficient the historical execution).

[0034] Task Dependency complexity .

[0035] In a preferred but non-limiting embodiment of the present invention, in step 2, the dependency complexity is obtained. The methods include: Execution structure and attention fusion model: combining structural information (depth, ingress, egress) of the DAG task dependency graph. The degree of complexity of task dependencies is quantified by the attention mechanism, and its calculation formula is as follows: ; Attention value Calculated using the following formula: , For the Softmax function, For the task Depth in the DAG task dependency graph (from the starting point of the DAG task dependency graph to...) The longest path length (number of edges) reflects the length of the dependency chain. For the task in-degree (task) The number of prerequisite tasks reflects the number of dependent tasks. For the task The set of prerequisite dependent tasks; and Tasks Query vectors and tasks key vector ( and Generated from the embedding layer of the Transformer model, with dimensions of ); The normalization coefficient for the vector dimension (i.e.) or (The dimension of the vector should be kept small to avoid excessively large values). , and All weight coefficients are greater than zero (satisfying) (Controlling the degree of influence of each factor on complexity). For the task The dependency complexity score, which is the task The dependency complexity (the larger the value, the more complex the dependency relationship).

[0036] Step 3: Construct and execute the neural network scoring model, that is, design a hybrid neural network model of CNN and MLP, input the core task features, and output the task scheduling priority score; In a preferred but non-limiting embodiment of the present invention, the model structure of the CNN and MLP hybrid neural network in step 3 specifically includes: The CNN module contains at least one convolutional layer, one global average pooling layer, and one fully connected layer for input tasks. Dependency complexity Convolutional layers are based on the task Dependency complexity Local dependency patterns (such as parallel dependencies of multiple preceding tasks and long chain serial dependencies) are extracted as convolutional features. Global average pooling layers compress the convolutional features into fixed-dimensional vectors. Fully connected layers process the fixed-dimensional vectors and output a scalar. Features, range mapped to [0,1] (via sigmoid activation function); The MLP module contains 2-3 hidden layers (using ReLU activation function) for inputting and processing tasks. Standardized priority Resource requirements Historical execution data Structured features like these are used to learn non-linear relationships between features (such as high-priority, low-resource-demand task features). The ReLU activation function processes these non-linear relationships and outputs a scalar. Features, range mapped to [0,1] (via sigmoid activation function); The output layer, connected to both the CNN and MLP modules, is used to fuse the scalar outputs of the CNN and MLP modules using a weighted linear combination. Features and scalars Features are used to generate task scheduling priority scores.

[0037] In a preferred but non-limiting embodiment of the present invention, in step 3, a weighted linear combination is used to fuse the scalar outputs of CNN and MLP respectively. Features and scalars The feature is that the formula for calculating the task scheduling priority score is: ; in For the task Task scheduling priority score ( The larger the value, the higher the scheduling priority. The weight coefficients (learnable parameters that reflect the importance of dependent structural features) are output by the CNN module. The weight coefficients output by the MLP module (learnable parameters that reflect the importance of task attribute features). Bias term (learnable parameter, adjusts the score baseline); Constraints: , and (Ensure that the weights are relative in importance and have clear physical meaning).

[0038] The technical effects of this invention in generating task scheduling priority scores are as follows: Feature contributions can be explained: through and The numerical value can directly determine the proportion of influence of structural features and attribute features on scheduling decisions (e.g., =0.6) indicates that 60% of the priority is determined by the dependency structure. Module decoupling: CNN modules and MLP modules can be optimized independently (e.g., adjusting the kernel size of the CNN module to optimize structural feature extraction, or increasing the number of layers in the MLP module to optimize attribute feature learning), thereby improving model scalability; Adapting to dynamic scenarios: Weights are dynamically learned according to the system state (e.g., when the task type changes from parallel-intensive to serial-intensive). (Automatic increase), enhancing the adaptive capability of the scheduling strategy.

[0039] Experiments show that, compared with existing technologies, this method improves the prediction accuracy of scheduling priorities by 12% to 18% and further reduces the average task completion time by 8% to 15%.

[0040] Step 4: Determine the resource availability of tasks based on their task scheduling priority scores, from highest to lowest. In a preferred but non-limiting embodiment of the present invention, step 4 specifically includes: Based on the resource contention model, the system determines whether current resources meet the task requirements by ranking tasks according to their scheduling priority scores from highest to lowest, thereby deciding whether to schedule or wait for the task. The calculation formula for the resource contention model is as follows: ; in For the task In the The amount of resources that can be allocated to a resource class (and) (Use consistent units, such as CPU cores, memory (MB), bandwidth). For the task For the The amount of resources required for this type of resource; For all currently pending tasks (total) (the first) to the first Total demand for class resources; For the current system number The total available amount of resources (such as available CPU cores, remaining memory in MB, and available bandwidth).

[0041] Next, the judgment logic is executed: only if the task... The amount of resources that can be allocated across all resource classes. All are not less than When the allocated resources meet the task requirements, the task is scheduled. Execute; otherwise, leave the task Add to the waiting queue until resources are released to satisfy the task. The amount of resources that can be allocated across all resource classes. All are not less than Only then will tasks be scheduled. implement.

[0042] Step 5: When scheduling task execution, dynamically adjust the maximum parallelism of the task to avoid resource waste or overload: In a preferred but non-limiting embodiment of the present invention, step 5 specifically includes: The formula for calculating the parallelism of dynamically adjusted tasks is as follows: ; in For the task The maximum number of parallelizable instances (the maximum number of parallelizable instances is the maximum parallelism of the task, which is the task's maximum parallelism). The number of task instances that can be executed simultaneously. This refers to the same task under the current system resource constraints. The maximum number of replicas that can be split or copied and run simultaneously. Therefore, in scheduling tasks During execution, limit the task. The maximum parallelism of the task is no higher than .

[0043] Step 6: While scheduling task execution, simultaneously monitor task execution and update its status; In a preferred but non-limiting embodiment of the present invention, step 6 specifically includes: Implement an asynchronous monitoring mechanism: Track the task execution status (such as execution progress and time consumption) in real time through the system's task execution monitoring module. Fault-tolerant rescheduling: If a task times out (exceeds a preset time threshold) or fails to execute, a rescheduling mechanism is triggered, which means the task is added back to the waiting queue; Execution status update: After a task is completed, update the task status in the DAG task dependency graph (e.g., "completed") and wake up its subsequent dependent tasks (transfer the subsequent tasks from the dependent state to the scheduling state).

[0044] Step 7: Repeat steps 2 through 6 in a loop until all tasks in the DAG task dependency graph are completed. Simultaneously, online learning is supported: based on actual data during task execution (such as newly added historical execution records and resource status changes), the neural network model parameters are optimized in real time to improve the accuracy and stability of the scheduling strategy.

[0045] The beneficial effects of the present invention are as follows: Compared with the prior art, the technical effects of the present invention include: (i) Resource utilization rate has been significantly improved By employing a resource contention model and dynamic parallelism adjustment, the system accurately matches task resource requirements with system resource status, avoiding resource idleness (e.g., increasing parallelism under low load) or overload (e.g., decreasing parallelism under high load). Experimental data shows that compared to traditional static scheduling methods, system resource utilization is improved by 25% to 40%, especially in heterogeneous resource environments (e.g., CPU+GPU hybrid clusters), where the improvement in resource adaptability is even more significant.

[0046] (II) Optimization of Task Execution Efficiency Task latency reduction: Priority scheduling based on neural network scoring ensures that high-priority and high-urgency tasks are executed first, while incorporating historical execution data to avoid inefficient tasks consuming resources. Compared to traditional FCFS (First-Come, First-Served) scheduling, the average task completion latency is reduced by 30%~50%, and the latency of critical tasks (such as P1 priority) is reduced by more than 60%. Improved task throughput: Dynamic parallelism adjustment and dependency bottleneck optimization (by extracting dependency features through CNN) reduce task blocking time, resulting in a 20%~35% increase in system task throughput.

[0047] (III) Enhanced Adaptive Capability of Scheduling Strategies Online learning adapts to dynamic loads: It supports model parameter updates based on real-time execution data, and can adapt to system load fluctuations (such as prioritizing scheduling tasks with low resource requirements during peak loads) and changes in task characteristics (such as feature learning for new task types), improving scheduling strategy stability by 45%; Enhanced fault tolerance: Asynchronous monitoring and rescheduling mechanisms ensure that tasks can be quickly rescheduled after timeout or failure, improving task execution success rate by 15%~25%, with even more prominent reliability advantages in distributed system node failure scenarios.

[0048] (iv) Advantages of scalability and interpretability Highly scalable: Supports DAG task dependency graphs with arbitrarily complex structures (such as multi-layer dependency tasks with a depth > 10), adapts to distributed systems of different sizes (from small clusters of 10 nodes to large clusters of 1000+ nodes), without requiring modification of the core logic. Enhanced decision interpretability: The weighted linear combination of the neural network output layer can clearly quantify the influence weight of each feature (priority, resource requirements, etc.) on scheduling decisions. Combined with the attention mechanism, it can explain the impact of dependencies on priority, making it easier for operators to optimize scheduling strategies.

[0049] (v) Demonstration of Business Value In practical applications (such as big data processing, AI model training, and cloud computing task scheduling), this invention can: Reduce enterprise IT resource investment costs (due to improved resource utilization); Improve business response speed (e.g., reduce latency of real-time data analysis tasks); It reduces business losses caused by task failures (such as the high reliability of financial transaction tasks) and has significant practical application value.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention without departing from the spirit and scope of the present invention. Any modifications or equivalent substitutions should be covered within the protection scope of the claims of the present invention.

Claims

1. A DAG task scheduling and control method based on neural network scoring, characterized in that, include: Step 1: Construct a DAG task dependency graph based on the tasks and their dependencies; Step 2: Extract multi-dimensional task features from the DAG task dependency graph and system historical data; Step 3: Design a hybrid neural network model combining CNN and MLP, inputting core task features and outputting task scheduling priority scores; Step 4: Determine the resource availability of tasks based on their task scheduling priority scores, from highest to lowest. Step 5: Dynamically adjust the maximum parallelism of tasks during task execution: Step 6: While scheduling task execution, simultaneously monitor task execution and update its status; Step 7: Repeat the process from Step 2 to Step 6 in a loop until all tasks in the DAG task dependency graph are completed.

2. The DAG task scheduling and control method based on neural network scoring according to claim 1, characterized in that, Step 1 specifically includes: Based on the tasks and their dependencies, a directed acyclic graph (DAG) is generated to clarify the task execution order and constraint relationships. The graph structure of a directed acyclic graph (DAG) is defined as follows: Let the task set be... Each task For a node in a directed acyclic graph (DAG), where , The total number of tasks; edges To indicate a dependency relationship, i.e. yes The prerequisite task.

3. The DAG task scheduling and control method based on neural network scoring according to claim 2, characterized in that, In step 1, each task It includes three basic attributes: Task Priority ; Resource requirements ; Dependency relationships: Records the set of prerequisite tasks .

4. The DAG task scheduling and control method based on neural network scoring according to claim 3, characterized in that, Step 2 specifically includes: Four core task features were extracted from the DAG graph and historical system data; these four core task features include: Task Standardized priority ; Task resource requirements ; Task Historical execution data ; Task Dependency complexity .

5. The DAG task scheduling and control method based on neural network scoring according to claim 4, characterized in that, In step 2, the standardized priority is obtained. The methods include: Tasks can be set manually or automatically assigned by the system. Task priority The task priority value is 1 indicates the highest priority. By standardizing, the value is transformed into a continuous value between 0 and 1. The formula for standardization is: ; in The standardized priority; In step 2, obtain resource requirements. The methods include: The formula for calculating the comprehensive resource demand score is as follows: ； in The quantity of different types of resources required for the task; For the task For the The demand for similar resources ; For the first Importance weights for resource categories; For the task Resource requirements; In step 2, historical execution data is obtained. The methods include: The formula for calculating the historical efficiency confidence score is as follows: ； Among them, weight Calculated using the exponential decay function: , This refers to the size of the historical scheduling window. For the first Whether the execution was successful; The z-score standardized version is the first z-score obtained by applying the z-score standardization method. The execution time; The attenuation coefficient; For the first Weighting of historical data; For the task Historical execution data; In step 2, the dependency complexity is obtained. The methods include: The complexity of quantifiable task dependencies is calculated using the following formula: ; Attention value Calculated using the following formula: , For the Softmax function, For the task Depth in the DAG task dependency graph; For the task in-degree; For the task The set of prerequisite dependent tasks; and Tasks Query vectors and tasks The key vector; The vector dimension normalization coefficient; , and All are weighting coefficients that are greater than zero; For the task The dependency complexity.

6. The DAG task scheduling and control method based on neural network scoring according to claim 5, characterized in that, In step 3, the specific model structure of the hybrid CNN and MLP neural network includes: The CNN module contains at least one convolutional layer, one global average pooling layer, and one fully connected layer for input tasks. Dependency complexity Convolutional layers are based on the task Dependency complexity Local dependency patterns are extracted as convolutional features, and global average pooling layers compress these features into fixed-dimensional vectors. Fully connected layers then process these fixed-dimensional vectors and output a scalar. feature; The MLP module contains 2-3 hidden layers for inputting and processing tasks. Standardized priority Resource requirements Historical execution data Such structured features learn the nonlinear relationships between features. The ReLU activation function processes these nonlinear relationships and outputs a scalar. Features, range mapped to [0,1]; The output layer, connected to both the CNN and MLP modules, is used to fuse the scalar outputs of the CNN and MLP modules using a weighted linear combination. Features and scalars Features are used to generate task scheduling priority scores.

7. The DAG task scheduling and control method based on neural network scoring according to claim 6, characterized in that, In step 3, a weighted linear combination is used to fuse the scalar outputs from the CNN and MLP respectively. Features and scalars The feature is that the formula for calculating the task scheduling priority score is: ； in For the task Task scheduling priority score, These are the weight coefficients output by the CNN module; These are the weighting coefficients output by the MLP module; Bias term; Constraints: , and .

8. The DAG task scheduling and control method based on neural network scoring according to claim 7, characterized in that, Step 4 specifically includes: Based on the resource contention model, the system determines whether current resources meet task requirements by ranking tasks according to their scheduling priority scores from highest to lowest. The calculation formula for the resource contention model is as follows: ； in For the task In the The amount of resources that can be allocated to a resource class; For the task For the The amount of resources required for this type of resource; For all currently pending tasks, the first Total demand for class resources; For the current system number The total available amount of this type of resource; Next, the judgment logic is executed: only if the task... The amount of resources that can be allocated across all resource classes. All are not less than When, schedule the task. Execute; otherwise, leave the task Add to the waiting queue until resources are released to satisfy the task. The amount of resources that can be allocated across all resource classes. All are not less than Tasks are scheduled only at that time. implement.

9. The DAG task scheduling and control method based on neural network scoring according to claim 8, characterized in that, Step 5 specifically includes: The formula for calculating the parallelism of dynamically adjusted tasks is as follows: ； in For the task The maximum number of instances that can be parallelized; Therefore, in scheduling tasks During execution, limit the task. The maximum parallelism of the task is no higher than .

10. The DAG task scheduling and control method based on neural network scoring according to claim 9, characterized in that, Step 6 specifically includes: Implement an asynchronous monitoring mechanism: Track the task execution status in real time through the system's task execution monitoring module; Fault-tolerant rescheduling: If a task times out or fails to execute, a rescheduling mechanism is triggered, which means the task is added back to the waiting queue; Execution status update: After the task is completed, update the task status in the DAG task dependency graph and wake up its subsequent dependent tasks.

Images