Computing-storage stream joint scheduling optimization method and system based on DDQN and heuristic strategy

Abstract

The invention relates to a calculation-storage stream joint scheduling optimization method and system based on DDQN and a heuristic strategy, and belongs to the technical field of distributed calculation and intelligent scheduling. The calculation-storage stream joint scheduling optimization method comprises the following steps: (1) defining a calculation stream and a storage stream; (2) workflow scheduling and execution: when the workflow is triggered, a scheduler evaluates the executable states of the tasks one by one according to the dependency relationship between the tasks and a scheduling strategy, and submits the executable tasks to a task executor; the task executor is responsible for executing the tasks on the working nodes and ensuring that the tasks are completed according to a preset sequence and priority; (3) collecting a DAG structure, queue information and an operation number; (4) task operation time and resource use prediction based on historical operation data; (5) scheduling optimization based on DDQN and a heuristic strategy; and (6) acquiring and storing scheduling information. The task execution efficiency and the resource utilization rate of the system are remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of distributed computing and intelligent scheduling, and particularly relates to a calculation-storage flow joint scheduling optimization method and system based on DDQN and heuristic strategies. Background Art

[0002] In a large-scale distributed computing environment, task scheduling is the core of the efficient operation of the system, and its performance directly determines the computing efficiency, resource utilization rate, and service quality guarantee ability. With the explosive growth of data scale and the continuous improvement of computing density, the scheduling system is facing challenges such as high concurrency of task requests, complex dependency relationships, and frequent fluctuations in resource status. In this context, traditional static scheduling strategies are difficult to meet the dual requirements of modern systems in terms of high performance and high elasticity due to the lack of the ability to perceive and respond to dynamic changes in the environment.

[0003] Taking Airflow as an example, although it supports flexible DAG workflow orchestration and task dependency management, its scheduling strategy is still mainly based on static rules, making it difficult to dynamically perceive changes in resource load and task execution status, resulting in difficulty in achieving optimal scheduling performance in complex environments. Therefore, introducing an intelligent scheduling mechanism with adaptive capabilities has become the key path to improving the system's resource scheduling efficiency and task execution quality, and is the core direction for supporting efficient task scheduling in heterogeneous resource environments in the future.

[0004] Especially in the new "data-centric" computing paradigm, factors such as the location of data, storage medium, access path, and transmission delay are gradually becoming key variables affecting scheduling efficiency. The optimization goal of the scheduling system has shifted from the traditional computing resource orientation to the dynamic perception and intelligent modeling of the entire life cycle of the data flow. At this time, data is not only the input for task execution but also the core driver for scheduling decisions, and its state changes directly affect the rationality of scheduling and the overall performance of the system.

[0005] The scheduling mechanism of Airflow is mainly based on static strategies and lacks the ability to adapt to dynamic changes in the operating environment, making it difficult to meet the scheduling requirements of high performance and high elasticity in large-scale distributed systems. The specific problems are mainly reflected in the following three aspects:

[0006] 1. Lack of dynamic perception ability: The existing scheduling mechanism of Airflow cannot perceive changes in resource load, the evolution of task dependency structures, and real-time fluctuations in the scheduling environment, resulting in low resource utilization rate and high task latency in high-concurrency and variable task scenarios, affecting the overall scheduling efficiency.

[0007] 2. Separation of data and computing: Traditional scheduling strategies generally treat data as static resources, ignoring key attributes such as its location, access frequency, storage media, and migration cost, making it difficult to achieve coordination between data flow and computing scheduling. This separation can easily lead to data access bottlenecks and scheduling imbalances in data-driven tasks, seriously restricting system performance.

[0008] 3. Single dimension of state modeling: Some existing scheduling methods based on reinforcement learning only use shallow features such as task length and CPU usage in state representation, ignoring the complex dependencies between tasks, critical path structure and resource topology information, resulting in weak strategy generalization ability and low decision-making accuracy, making it difficult to adapt to heterogeneous resource environments and dynamic load scenarios. Summary of the invention

[0009] In view of the shortcomings of the prior art, the present invention proposes a computing-storage flow joint scheduling optimization method based on DDQN and heuristic strategy.

[0010] In this invention, the traditional workflow is decoupled and explicitly divided into computing flow and storage flow to achieve collaborative optimization of task execution and data management. At the same time, the state representation and action space of DDQN are optimized, and the task dependency structure, resource status and historical scheduling records are integrated to achieve deep modeling and perception of the environment. Finally, by integrating DDQN with heuristic strategies, combining Q value evaluation and rule guidance, the optimal queue is dynamically selected for the task, achieving the goal of task load balancing and minimizing execution time.

[0011] The method of the present invention optimizes the state representation and action space by jointly modeling the computation flow and storage flow, comprehensively considering the task dependency structure, queue resource status and historical scheduling information. On this basis, the scheduling decision is made based on DDQN and heuristic strategy, and the optimal execution queue is selected for each task by integrating Q value evaluation and heuristic rules, thereby achieving task load balancing and minimizing execution time.

[0012] The present invention also proposes a computing-storage flow joint scheduling optimization system based on DDQN and heuristic strategies.

[0013] The technical solution of the present invention is: The computation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy includes: (1) Definition of computing flow and storage flow; (2) Workflow scheduling and execution: When a workflow is triggered, the scheduler evaluates the executable status of each task based on the dependencies between tasks and the scheduling strategy, and submits the executable tasks to the task executor; the task executor is responsible for executing these tasks on the work nodes to ensure that the tasks are completed in the predetermined order and priority. (3) DAG structure, queue information, and running data collection; (4) Prediction of task running time and resource usage based on historical running data; (5) Scheduling optimization based on DDQN and heuristic strategies; (6) Acquisition and preservation of scheduling information.

[0014] Preferably according to the present invention, in step (1), the definition of the computation flow and the storage flow includes: First, use the DAG model to uniformly model the entire task process; in the DAG, each node represents an independent task, and the directed edges between the nodes represent the dependency relationships between tasks; Logically divide tasks into two types of sub-flows based on the functional attributes of the tasks and the types of resource usage: Computation flow: including the execution of operator logic and the scheduling and allocation of computing power resources; Storage flow: including the storage, reading, transmission, and management of the data on which the tasks depend; All task nodes are declared and parameterized through Python script invention files; A linkage trigger mechanism is implemented between the computation flow and the storage flow through the TriggerDagRunOperator: when the computation task is completed, the corresponding storage task is automatically triggered to ensure the sequential connection of the processing flow; at the same time, the ExternalTaskSensor is introduced to achieve cross-flow dependency synchronization: the computation flow task listens to the state changes of specific storage tasks to determine its start time.

[0015] Preferably according to the present invention, in step (2), workflow scheduling and execution; specifically include: When the Airflow workflow is triggered, the scheduler evaluates the state of each task according to the task dependency relationships and scheduling policies defined in the DAG; the scheduler checks which tasks meet the dependency conditions and determines whether the scheduling time window has been reached; the eligible tasks are marked as schedulable and then placed in the task queue, waiting to be processed by the task executor; The executor obtains tasks from the task queue and assigns the tasks to the corresponding worker nodes or containers for running; During the execution process, the task updates its state in real time. If the task execution fails, according to the retry policy of the task, the scheduler reschedules the task; throughout the process, the scheduler ensures that the tasks in the DAG are executed strictly in the defined dependency order and priority order until all tasks are completed, thus completing the closed-loop of the entire workflow scheduling and execution.

[0016] Preferably according to the present invention, in step (3), the collection of the DAG structure, queue information, and running data includes: In terms of the DAG structure, collect and organize the following information for each task node: dag_id, task_id, whether it is a critical path task, whether it belongs to a computation flow task, whether it is a TriggerDagRunOperator task that triggers a storage flow, the target DAG it triggers, whether it is an ExternalTaskSensor type task, the DAG and task list it depends on, and the DAG adjacency list structure between the overall computation flow and storage flow; dag_id refers to the id of the DAG, and task_id refers to the id of the task; In terms of task running data, i.e., running data, record the running status of historical tasks, including: task_id, dag_id, run_id, job_id, pid, avg_cpu, min_cpu, max_cpu, avg_mem, min_mem, max_mem, rss, file_size, and duration; run_id refers to the DAG instance running number, job_id refers to the task running number generated by the scheduler, pid refers to the task process number, avg_cpu refers to the average CPU usage rate, min_cpu refers to the minimum CPU usage rate, max_cpu refers to the maximum CPU usage rate, avg_mem refers to the average memory occupancy, min_mem refers to the minimum memory occupancy, max_mem refers to the maximum memory occupancy, rss refers to the resident memory set size, file_size refers to the file size, and duration refers to the task execution duration; In terms of queue resources, i.e., queue information, collect for each execution queue, including: queue_id, queue_code, currently available CPU, memory, storage, and I / O bandwidth, and build a real-time mapping relationship between tasks and execution resources; queue_id represents the queue name, and queue_code represents the queue number; Save the above-collected DAG structure and queue resources to a JSON file, and save the task running data to the airflow_task_stats table in the MySQL database.

[0017] Preferably according to the present invention, in step (4), the prediction of task running time and resource usage based on historical running data includes: (4-1) Set the basic environment for system operation, including the model storage path and operation log configuration; obtain historical task data from the airflow_task_stats table in the MySQL database, including the fields: task_id, dag_id, run_id, avg_cpu, avg_mem, rss, file_size, and duration; Perform data preprocessing on the historical data, including type conversion and missing value filling; forcibly convert the historical task data to numerical type and save it in the MySQL database, as shown in formula (1), and fill the missing values with 0, as shown in formula (2): ; ; Among them, represents the original input data, represents the converted numerical data, and illegal values are converted to NaN; represents the function provided by Pandas for converting data to numerical type; represents that when the data cannot be converted to a numerical value, it is forcibly converted to , that is, missing values; represents the value after filling the missing values, represents filling all missing values with 0; (4-2) Adopt the XGBoost regression model, train three models independently for each task, and deploy 3 XGBoost regression models independently for each task_id to predict the running time, CPU usage rate, and memory occupancy respectively, as shown in formula (3), and persistently save the generated regression model as a pkl file; ; Among them, represents the XGBoost regression model trained for predicting the running time of the task; represents the XGBoost regression model trained for predicting the CPU usage rate; represents the XGBoost regression model trained for predicting the memory occupancy; represents the predicted running time of the task, represents the predicted CPU usage rate, represents the predicted memory occupancy; Use the Optuna framework for Bayesian optimization to search for the optimal hyperparameter combination, with the goal of minimizing the mean absolute error. When training the XGBoost regression model, minimize the squared error, as shown in formula (4): ; Among them, represents the true value of the th sample, the th sample's predicted value, represents the L1 regularization coefficient, represents the L2 regularization coefficient, represents the minimum loss descent threshold for controlling tree splitting, represents the number of leaf nodes, represents the leaf node weight vector; represents the L1 norm, that is, the sum of the absolute values of all leaf node weights; represents the square of the L2 norm, that is, the sum of the squares of all leaf node weights; represents the total loss function; (4 - 3) Load the latest prediction model according to the model storage path. The data in the airflow_task_stats table of the MySQL database is divided into datasets by task_id. The input features are , and the output target is , , ; Predict the latest record for each task, as shown in formula (5): ; Among them, represents the latest task feature vector, represents the predicted value, which is time, CPU, or memory; represents the trained XGBoost regression model function; (4 - 4) Scan the configuration file directory and write the predicted values to the predicted_time, predicted_cpu, and predicted_memory fields of the JSON file.

[0018] Preferably according to the present invention, in step (5), based on DDQN and heuristic policy scheduling optimization, it includes: (5 - 1) Construct a four - dimensional state tensor , as shown in formula (6): ; Among them, represents the batch size, represents the total number of tasks, represents the total number of queues, represents the feature dimension, is ; Task characteristics are , as shown in formula (7): ; Among them, represents the task weight, represents the assigned queue number, represents whether it is a critical path task, represents whether it is a computational flow task, represents whether it is a trigger task, represents whether it is a sensor task, represents the predicted running time, CPU, and memory, represents the one-hot encoding of dependent tasks; Queue characteristics: , as shown in formula (8): ; Among them, represents the available CPU resources of the queue, represents the total CPU resources of the queue, represents the available memory of the queue, represents the total memory of the queue, represents the available storage of the queue, represents the total storage of the queue, represents the number of tasks already assigned in the queue, represents the end timestamp of the current task in the queue; (5 - 2) Conduct multiple rounds of Episode training until the termination condition is met; each Episode includes the following steps: Environment reset: Restore the simulation environment to the initial state and clear the previous interaction history data; Task completion determination: Detect whether all preset task goals have been completed in the current Episode. If the tasks have been completed, skip the subsequent steps and directly enter the next Episode; (5 - 3) In the current Episode, perform the following operations at each time step: Available action detection: Obtain the set of legal actions based on the current state of the environment; Obtain a legal action from the set of legal actions. The key function for verifying the legality of the action , as shown in formula (9): ; Among them, represents the specific operation on task , represents the currently schedulable task, Represents the dependency set, Represents the dependent task, Represents the task status; According to the hybrid epsilon - greedy and heuristic rules , as shown in formula (10), select the optimal action; ; Among them, Represents the exploration rate, Represents the online network parameters, Represents the task 's legal queue set; Represents an action in the legal queue set, Represents the Q - value function in reinforcement learning, Represents the heuristic policy function; The heuristic rule preferentially assigns tasks to queues with high current load but low demand matching degree to avoid resource overload, as shown in formula (11): ; For each queue , calculate its load score , as shown in formula (12): ; Represents the CPU importance weight, Represents the memory importance weight; CPU utilization rate , as shown in formula (13): ; Represents the queue 's current available CPU cores, Represents the queue 's total CPU quota; Memory utilization rate , as shown in formula (14): ; Represents the queue 's current remaining available memory, Represents the queue 's total memory quota; By means of multi - objective weighting, comprehensively evaluate the advantages and disadvantages of scheduling actions, as shown in formula (15): ; Among them, Represents the weight of the resource load balancing score; Represents the resource load balancing score; Represents the weight of the task quantity balancing score; Represents the task quantity balancing score; Represents the weight of the dependency completion reward; Represents the dependency completion reward; Represents the weight of the local utilization reward; Represents the local utilization reward; Represents the penalty coefficient of the global difference penalty; Represents the global difference penalty; The specific scoring items are as follows: Load balancing score Represents the balance of CPU / memory utilization of each queue, as shown in formula (16): ; Among them, Represents the CPU utilization of each queue, Represents the memory utilization of each queue, Standard deviation, measuring the volatility of resource utilization; Task quantity balancing score Represents the balance of the number of tasks in each queue, as shown in formula (17): ; Among them, Each represents the number of tasks in the queue, Represents the standard deviation of the number of tasks, Represents the maximum number of tasks in all queues; Dependency completion reward Represents the penalty for the number of uncompleted dependent tasks, as shown in formula (18): ; Among them, Represents the dependent task of the current task, Represents the indicator function, with a value of 1 if the dependent task is not completed, otherwise 0; Local utilization reward Represents the CPU / memory utilization of the current queue, as shown in formula (19): ; Among them, Represents the current queue 's CPU utilization, Represents the current queue 's memory utilization; The global difference penalty is Indicates the difference between the maximum and minimum CPU utilization rates of each queue, as shown in Equation (20): ; Among them, Indicates the maximum CPU utilization rate among all queues, Indicates the minimum CPU utilization rate among all queues; (5 - 4) Continuously collects experience data through a continuous interaction process and stores it in the experience replay buffer. During each task scheduling process, an interaction sample in the form of a five-tuple is recorded, as shown in Equation (21): ; Among them, Indicates the state of the current scheduling environment, Indicates the scheduling decision made by the agent in the current state, Indicates the immediate feedback brought by the current scheduling action, Indicates the next state of the environment after the action is executed, Indicates whether the current scheduling process has ended, which is a boolean value; By continuously executing scheduling actions, five-tuples are gradually collected and stored in the experience replay pool; When the number of samples reaches a certain scale, a training set is sampled from them for optimizing and training the Q-value function.

[0019] Further preferably, the scheduling policy model includes an online network and a target network ; The online network and the target network , The parameter update formula, as shown in Equation (22): ; Among them, Indicates the online network parameters, responsible for real-time updating and training through experience replay; Indicates the target network parameters, synchronously obtained from the online network periodically, used for stable Q-value calculation; Indicates the synchronization coefficient, ; The target Q-value calculation, as shown in Equation (23): ; Among them, Indicates the discount factor, Indicates the next state, the new state obtained by environmental transition after the action is executed; Indicates the immediate reward, calculated by the reward function; The online network and the target network The neural network structures are the same. The neural network includes a first fully connected layer, a first activation function, a second fully connected layer, a second activation function, and a third fully connected layer, as shown in Formula (24): ; Among them, represents the dimension of the input feature vector, which is composed of the task feature and the queue feature concatenated together; represents the first fully connected layer, which contains 128 neurons, performs a linear transformation on the input features, and the output is ; Subsequently, through represents the activation function, the rectified linear unit, which introduces non-linear expression ability, and the output is still Next is the second fully connected layer , which contains 64 neurons, further extracts high-order features, and the output is , and is activated by ReLU again; finally, the feature is mapped to a scalar , that is, the Q value, which represents the expected reward of the current task-queue pair in the given state.

[0020] Further preferably, the loss function for batch training of the scheduling policy model adopts the mean squared error, as shown in Formula (25): ; Among them,[[]] Indicate batch size, the number of state-action pairs sampled in a single training; represents the state vector of the th sample; represents the task index of the th sample; represents the queue index of the th sample; represents the Q value predicted by the online network, and the predicted reward for the task and the queue after inputting ; represents the Q value calculated by the target network.

[0021] According to the preference of the present invention, in step (6), the acquisition and storage of scheduling information includes: Obtaining the scheduling optimization result of each workflow through step (5); at the same time, saving the workflow scheduling optimization result to a specific table in the MySQL database. The specific table includes three columns: "dag_id", "task_id", and "queue_id".

[0022] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy are realized.

[0023] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the steps of the above-mentioned calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy are realized.

[0024] The calculation-storage flow joint scheduling optimization system based on DDQN and heuristic strategy includes: A calculation flow and storage flow definition module, configured to: define a calculation flow and a storage flow; A workflow scheduling and execution module, configured to: when a workflow is triggered, the scheduler evaluates the executable status of tasks one by one according to the dependency relationship and scheduling strategy between tasks, and submits the executable tasks to the task executor; the task executor is responsible for executing these tasks on the working nodes to ensure that the tasks are completed in a predetermined order and priority; A data collection module, configured to: collect DAG structure, queue information, and running numbers; A prediction module, configured to: predict task running time and resource usage based on historical running data; A scheduling optimization module, configured to: perform scheduling optimization based on DDQN and heuristic strategy; An information acquisition and saving module, configured to: acquire and save scheduling information.

[0025] Generally speaking, compared with the prior art by the above technical solutions conceived by the present invention, the following beneficial effects can be achieved: To address the above challenges, the present invention proposes a method for decoupling and modeling "computation flow" and "storage flow", and structurally reconstructs the traditional hybrid scheduling process. On this basis, an intelligent task scheduling method based on "double-flow collaboration" is further designed. This method combines the Double DQN reinforcement learning algorithm and heuristic strategy, and through the joint perception and collaborative scheduling of the computation flow and storage flow, realizes the precise matching of tasks among computing power, data, and storage resources, thereby significantly improving the task execution efficiency and resource utilization rate of the system.

[0026] 1. Realize the refinement of task scheduling: By separating and modeling the computation flow and storage flow and collaborative scheduling, it is possible to optimize the task execution logic and data access path respectively, realize the efficient coupling and allocation of tasks between computing power resources and data resources, and thus effectively improve the accuracy level of task scheduling.

[0027] 2. Fusion of Heuristic Strategy and Reinforcement Learning to Improve Policy Quality: Introduce a heuristic action guidance mechanism into the ε-greedy policy, combine task dependency relationships, resource status, and scheduling history information to reduce ineffective exploration behavior and accelerate policy convergence.

[0028] 3. More Expressive State Space Modeling: The state space not only includes the current characteristics of tasks, resource requirements, and queue status, but also integrates task history scheduling information and critical path characteristics to more comprehensively reflect the scheduling environment and improve the generalization ability and perception ability of the model.

[0029] 4. Multi-objective Scheduling Optimization: On the basis of taking the shortest execution time (Makespan) as the core objective, further consider system load balancing, energy consumption control, and storage resource utilization rate to achieve joint optimization of multiple objectives and improve the overall performance of the system. Description of the Drawings

[0030] Figure 1 It is a schematic flow diagram of the joint scheduling optimization method for compute-storage flow based on DDQN and heuristic strategy.

[0031] Figure 2 It is a schematic diagram of compute flow and storage flow.

[0032] Figure 3 It is a schematic diagram of queue information and running data collection.

[0033] Figure 4 It is a schematic flow diagram of running time and resource prediction.

[0034] Figure 5 It is a schematic flow diagram of scheduling optimization based on DDQN and heuristic strategy. Detailed Implementation Modes

[0035] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0036] Term Explanation: 1. Apache Airflow: Apache Airflow is an open-source workflow scheduling and management platform used to define and orchestrate the task execution process through Directed Acyclic Graphs (DAGs). It supports task scheduling at specific times, dependency control, and failure retry, has good scalability and a visualization interface, and is widely used in automated processes such as data processing, ETL, and machine learning. Airflow can integrate various computing and storage systems and is suitable for building complex, monitorable, and maintainable data-driven workflows.

[0037] 2. Double DQN: Double DQN, also known as DDQN (Double Deep Q Network), is an improved deep reinforcement learning algorithm used to solve the problem of policy instability caused by overestimating the action value (Q-value) during the training of traditional DQN. Double DQN effectively reduces the Q-value estimation bias and improves learning stability and decision-making accuracy by introducing a dual-network structure that uses the main network to select actions and the target network to evaluate actions. DDQN (Double Deep Q Network) is an improved deep reinforcement learning algorithm used to solve the problem of policy instability caused by overestimating the action value (Q-value) during the training of traditional DQN. DDQN effectively reduces the Q-value estimation bias and improves learning stability and decision-making accuracy by introducing a dual-network structure that uses the main network to select actions and the target network to evaluate actions.

[0038] 3. CeleryExecutor: CeleryExecutor is a distributed task executor provided by Apache Airflow and is implemented based on the Celery framework. It asynchronously distributes tasks to multiple independent worker nodes and is suitable for large-scale, multi-task scheduling scenarios. When used in combination with a message middleware (such as Redis), CeleryExecutor can significantly improve the scalability and throughput of Airflow and is an important component for building a highly available scheduling system.

[0039] 4. Scheduler: Scheduler is one of the core components of Apache Airflow, responsible for periodically parsing DAG files, determining whether tasks meet the scheduling conditions, and submitting executable tasks to the executor (such as CeleryExecutor) for processing. It runs continuously to ensure that tasks are triggered and executed according to the scheduled time or dependencies. It is the control center for the automated scheduling of the entire workflow and is crucial for the orderly execution of tasks and the stable operation of the system.

[0040] 5.Webserver: The Webserver is a visualization interface component provided by Apache Airflow. Built on Flask, it is used to display DAG status, task execution status, log information, scheduling configurations, etc. Through the web interface, users can intuitively manage the workflow, including triggering tasks, modifying parameters, viewing dependency graphs, and monitoring the running status. It is an important entry point for using Airflow for task orchestration and operation and maintenance management.

[0041] 6.Flower: Flower is a visualization monitoring tool for Apache Airflow when using CeleryExecutor. It is used to view the task execution status, queue information, and worker running status in real time. It provides intuitive distributed task monitoring capabilities through the Web UI, which helps improve the observability and operation and maintenance efficiency of the scheduling system.

[0042] 7.Worker: The Worker is the computing node that actually executes tasks in Apache Airflow. It is usually enabled when using distributed executors such as CeleryExecutor. The Worker receives tasks distributed by the Scheduler from the message queue and independently completes the execution according to the defined logic.

[0043] 8.Redis: Redis (Remote Dictionary Server) is an open-source high-performance key-value database that supports various data structures such as strings, hashes, lists, sets, and sorted sets. It is commonly used in scenarios such as caching, message queues, and real-time data processing.

[0044] 9.TriggerDagRunOperator: It is an operator in Airflow used to trigger the execution of another DAG in a DAG. It is commonly used to achieve cross-workflow collaboration and process decoupling. It supports dynamically starting a specified sub-DAG during task execution and can pass runtime configurations through the conf parameter. It is suitable for scenarios such as triggering model training and modular orchestration after data processing is completed. When used in conjunction with ExternalTaskSensor, it can also achieve cross-DAG dependency synchronization and execution control.

[0045] 10. ExternalTaskSensor: It is a sensor used for cross-DAG task dependency synchronization in Airflow. It is used to monitor the status of a task in another DAG until it meets certain conditions (such as successful completion) and then continue the execution of the current task. It is suitable for scenarios where there are logical dependencies between multiple DAGs. It can achieve orderly connection and scheduling coordination between different DAGs and is an important tool for building complex distributed workflows.

[0046] 11. airflow_task_stats table: used to record the resource usage information of Airflow tasks, including task identifiers (such as dag_id, task_id, run_id, job_id, pid), average, minimum and maximum values of CPU and memory (avg_cpu, min_cpu, max_cpu, avg_mem, min_mem, max_mem), and process-related information such as vsz, rss, tty and stat, to facilitate subsequent resource analysis and scheduling optimization.

[0047] 12. XGBoost regression model: It is an integrated learning method based on gradient boosting. It has the advantages of high efficiency, accuracy, and resistance to overfitting. It is suitable for regression tasks that process structured data. It fits the residual by gradually building a decision tree, has a built-in regularization mechanism, supports automatic processing of missing values, and optimizes parallel computing performance.

[0048] Example 1 The computation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy, such as Figure 1 As shown, including: (1) Definition of computing flow and storage flow; (2) Workflow scheduling and execution: When a workflow is triggered, the scheduler evaluates the executable status of each task based on the dependencies between tasks and the scheduling strategy, and submits the executable tasks to the task executor; the task executor is responsible for executing these tasks on the work nodes to ensure that the tasks are completed in the predetermined order and priority. (3) DAG structure, queue information, and run number collection; (4) Task running time and resource usage prediction based on historical running data; (5) Scheduling optimization based on DDQN and heuristic strategies; (6) Obtaining and saving scheduling information.

[0049] Example 2 The computing-storage flow joint scheduling optimization method based on DDQN and heuristic strategy described in Example 1 is different in that: In step (1), the computation flow and the storage flow are defined as follows: Figure 2 including: First, a DAG (Directed Acyclic Graph) model is used to uniformly model the entire task process. In the DAG, each node represents an independent task, and the directed edges between nodes represent the dependencies between tasks. That is, a certain task can only start execution after all its upstream tasks have been successfully completed. Since it is an "acyclic" graph structure, the DAG naturally avoids circular dependencies and ensures the logical closed-loop and terminability of task scheduling.

[0050] Logically, tasks are divided into two types of sub-flows based on the functional attributes of the tasks and the types of resources used: Compute Flow: It includes the execution of operator logic and the scheduling and allocation of computing power resources. Among them, the execution of operator logic is, for example, running a Python script, executing a Shell command, triggering an SQL query, and calling an API. The computing power resource scheduling is to distribute tasks to the corresponding execution queues on different machines. Storage Flow: It includes the storage, reading, transmission, and management of the data relied on by the tasks. All task nodes are declared and parameterized through Python script files. All task nodes (Tasks) are explicitly declared and parameterized through the DAG files in Python scripts. Each DAG file is essentially a Python script, and developers define the structure, task content, and scheduling logic of the entire workflow by writing code. In the script, tasks are usually represented by instances of the Operator class, such as PythonOperator, BashOperator, HttpOperator, etc. Each task node realizes fine control of task behavior through configuration parameters (such as task_id, executable function, dependency settings, number of retries, resource requirements, etc.), thereby ensuring the consistency and maintainability of the task structure within the global scope. This centralized configuration method facilitates the unified management, version control, and flexible evolution of task scheduling strategies.

[0051] In terms of the control mechanism of the task flow, a linkage trigger mechanism is implemented between the computing flow and the storage flow through the TriggerDagRunOperator: when the computing task is completed, the corresponding storage task is automatically triggered to ensure the sequential connection of the processing flow; at the same time, the ExternalTaskSensor is introduced to achieve cross-flow dependency synchronization: the computing flow task listens to the status changes of specific storage tasks to determine its start time. Vice versa, thus ensuring the coordination consistency and dependency integrity of the computing flow and the storage flow in scheduling.

[0052] As Figure 2 shown, in this distributed system architecture, the computing flow and the storage flow work together to achieve two-way decoupling and dynamic linkage of tasks and data. The computing flow starts from the "Start" node and triggers the storage flow through the TriggerDagRunOperator, forming three parallel paths: C1 listens to the storage flow S1 and connects to C4 after completion, and C4 converges to C6 after listening to the status of S3; C2 and C3 simultaneously listen to the storage flow S2 and aggregate to C5 after completion, and finally the results of C4 and C5 are uniformly processed at C6.

[0053] The storage flow also starts from the "Start" node and branches into S1 and S2: S1 listens to the computing nodes C1 and C2, and S2 listens to C2 and C3. Both flows are aggregated into S3 through the ExternalTaskSensor to dynamically sense the status of C4 and C5.

[0054] This architecture realizes the efficient decoupling and collaborative drive of the computing flow and the storage flow through the two-way sensing mechanism of "computing depends on storage status, and storage tracks computing progress", thus supporting the high-performance scheduling of large-scale distributed tasks and real-time data status synchronization.

[0055] In step (2), the workflow is scheduled and executed; specifically including: When the Airflow workflow (DAG) is triggered, the scheduler evaluates the status of each task according to the task dependency relationships and scheduling policies defined in the DAG; the scheduler checks which tasks meet the dependency conditions (such as whether the upstream tasks have been successfully completed) and determines whether the scheduling time window has been reached; the eligible tasks are marked as schedulable (queued) and then placed in the task queue, waiting to be processed by the task executor (Executor); The type of task executor (such as LocalExecutor, CeleryExecutor, or KubernetesExecutor) determines how tasks are distributed and executed. The executor fetches tasks from the task queue and assigns the task to the corresponding worker node or container for execution; the execution of each task occurs in an isolated process or environment to ensure resource isolation and task independence.

[0056] During the execution process, the task updates its status in real-time (such as running, success, failed, etc.), and these status changes are monitored by the scheduler and the database is updated. If the task execution fails, according to the task's retry policy (such as the number of retries and retry intervals), the scheduler reschedules the task; throughout the process, the scheduler ensures that the tasks in the DAG are executed strictly in the defined dependency order and priority order until all tasks are completed, thus completing the entire workflow scheduling and execution closed-loop.

[0057] In step (3), the DAG structure, queue information, and running data collection, as Figure 3 shown, include: Build a multi-dimensional data foundation required for intelligent task scheduling decision-making, including DAG structure modeling, dynamic collection of task running data and queue resource status perception; through unified modeling, build a task scheduling state space; provide high-dimensional and interpretable input features for subsequent scheduling optimization strategies based on deep reinforcement learning; In terms of the DAG structure, collect and organize the following information for each task node: dag_id, task_id, whether it is a critical path task, whether it belongs to a computational flow task, whether it is a TriggerDagRunOperator task that triggers a storage flow, its triggered target DAG, whether it is an ExternalTaskSensor type task, its dependent DAG and task list, and the DAG adjacency list structure between the overall computational flow and storage flow; thus comprehensively describing the dependency topology and triggering logic between tasks; dag_id refers to the id of the DAG, and task_id refers to the id of the task; the above DAG structure and task node information are mainly automatically obtained by parsing the Airflow DAG file, combined with task type judgment (such as TriggerDagRunOperator, ExternalTaskSensor), task attribute extraction (such as dag_id, task_id, upstream_list, etc.), and based on the construction of the adjacency matrix and cross-DAG dependency completion.

[0058] Regarding the task running data, i.e., the running data, record the running status of historical tasks, including: task_id, dag_id, run_id, job_id, pid, avg_cpu, min_cpu, max_cpu, avg_mem, min_mem, max_mem, rss, file_size, and duration; run_id refers to the DAG instance running number, job_id refers to the task running number generated by the scheduler, pid refers to the task process ID, avg_cpu refers to the average CPU usage rate, min_cpu refers to the minimum CPU usage rate, max_cpu refers to the maximum CPU usage rate, avg_mem refers to the average memory occupancy, min_mem refers to the minimum memory occupancy, max_mem refers to the maximum memory occupancy, rss refers to the resident set size, file_size refers to the file size, and duration refers to the task execution duration; the above running data is read from the airflow_task_stats table and can be used to construct the resource consumption profile of the task; Regarding the queue resources, i.e., the queue information, collect for each execution queue, including: queue_id, queue_code, currently available CPU, memory (kB), storage (kB), and I / O bandwidth (kB), to build a real-time mapping relationship between the task and the execution resources; queue_id represents the queue name, and queue_code represents the queue number; the above queue resource information is obtained by connecting to each Celery Worker host via SSH and calling standard Linux commands to obtain the CPU idle rate, available memory, available disk space, and IO bandwidth in real time, and is uniformly encapsulated as a scheduling queue object in combination with the Celery queue name to provide data support for scheduling optimization and resource-aware decision-making.

[0059] Save the above-collected DAG structure and queue resources to a JSON file, and save the task running data to the airflow_task_stats table in the MySQL database.

[0060] In step (4), the prediction of the task running time and resource usage based on historical running data, as Figure 4 shown, includes: (4-1) Set the basic environment for the system to run, including the model storage path and operation log configuration; obtain historical task data from the airflow_task_stats table in the MySQL database, including the fields: task_id, dag_id, run_id, avg_cpu, avg_mem, rss, file_size, and duration; Preprocess the historical data, including type conversion and missing value filling; force the conversion of historical task data to numerical type and save it to the MySQL database as shown in formula (1), and fill the missing values with 0 as shown in formula (2): ; ; Among them, represents the original input data (which may contain non-numerical fields), represents the converted numerical data, and illegal values are converted to NaN; represents the function provided by Pandas for converting data to numerical type; represents that when the data cannot be converted to a numerical value, it is forced to be converted to , that is, missing values; represents the value after filling the missing values, represents filling all missing values ( ) with 0; (4 - 2) Adopt the XGBoost regression model. Three models are independently trained for each task, and three XGBoost regression models are independently deployed for each task_id to predict the running time (duration), CPU usage rate (avg_cpu), and memory occupancy (rss) respectively, as shown in formula (3), and the generated regression models are persistently saved as pkl files; ; Among them, represents the XGBoost regression model trained for predicting the running time of the task; represents the XGBoost regression model trained for predicting the CPU usage rate; represents the XGBoost regression model trained for predicting the memory occupancy; represents the predicted running time of the task, represents the predicted CPU usage rate, represents the predicted memory occupancy; Use the Optuna framework for Bayesian optimization to search for the optimal hyperparameter combination, with the goal of minimizing the mean absolute error (MAE). When training the XGBoost regression model, minimize the squared error, as shown in formula (4): ; Among them, represents the true value of the th sample, The predicted value of the first sample, represents the L1 regularization coefficient (controlling sparsity), represents the L2 regularization coefficient (preventing overfitting), represents the minimum loss decrease threshold for controlling tree splitting, represents the number of leaf nodes, represents the leaf node weight vector; represents the L1 norm, that is, the sum of the absolute values of all leaf node weights; represents the square of the L2 norm, that is, the sum of the squares of all leaf node weights; represents the total loss function (objective function); composed of prediction error and model complexity; (4 - 3) Load the latest prediction model according to the model storage path. The data in the airflow_task_stats table of the MySQL database is divided into datasets by task_id to ensure task independence. The input features are and the output target is , , ;

[0061] Predict the latest record (the maximum value of run_id) for each task, as shown in formula (5): ; Among them, represents the latest task feature vector, represents the predicted value, which is time, CPU, or memory; represents the trained XGBoost regression model function; (4 - 4) Scan the configuration file directory (config_dir) and write the predicted values to the predicted_time, predicted_cpu, and predicted_memory fields of the JSON file. As shown in Table 1:

[0062] Table 1 Example of predicted values;

[0063] In step (5), based on DDQN and heuristic policy scheduling optimization, as Figure 5 shown, it includes: (5 - 1) Create an experience replay buffer (Replay Buffer) and set the upper limit of the storage capacity to store the experience samples of the agent's interaction.

[0064] Build a double - network architecture, including: Policy Network (Online Network): Updates in real-time and is used to estimate the Q-values of each action in the current state; Target Network (Target Network): Synchronizes with a delay and updates the policy network parameters periodically to improve training stability.

[0065] Set the initial exploration rate and configure its decay strategy.

[0066] Construct a four-dimensional state tensor , as shown in Equation (6): ; where, represents the batch size, represents the total number of tasks, represents the total number of queues, represents the feature dimension (task feature + queue feature), is ; Task feature is , as shown in Equation (7): ; where, represents the task weight (priority), represents the assigned queue number, represents whether it is a critical path task (0 / 1), represents whether it is a computational flow task (0 / 1), represents whether it is a trigger task (0 / 1), represents whether it is a sensor task (0 / 1), represents the predicted running time, CPU, and memory, represents the one-hot encoding of dependent tasks; Queue feature: , as shown in Equation (8): ; where, represents the available CPU resources of the queue, represents the total CPU resources of the queue, represents the available memory (KB) of the queue, represents the total memory (KB) of the queue, represents the available storage (KB) of the queue, represents the total storage (KB) of the queue, represents the number of tasks already assigned in the queue, represents the end timestamp of the current task in the queue; Perform multiple rounds of Episode training until the termination condition is met; each Episode includes the following steps: Environment reset: Restore the simulation environment to its initial state and clear the previous interaction history data; ensure the independence and fairness of training.

[0067] Task completion determination: Detect whether all preset task goals have been completed in the current Episode. If the tasks have been completed, skip the subsequent steps and directly proceed to the next Episode; (5-3)In the current Episode, perform the following operations at each time step: Available action detection: Obtain the set of legal actions based on the current state of the environment; avoid selecting illegal actions.

[0068] Obtain a legal action from the set of legal actions, the key function for verifying action legality , as shown in Equation (9): ; where, represents the specific operation on task (such as assigning to a certain queue, starting execution, etc.), represents the task to be scheduled currently, represents the dependency set, represents the dependent task, represents the task status; According to the hybrid epsilon-greedy and heuristic rules , as shown in Equation (10), select the optimal action; ; where, represents the exploration rate (decaying with the number of training rounds), represents the online network parameters, represents the task 's set of legal queues; represents an action in the set of legal queues, represents the Q-value function in reinforcement learning, represents the heuristic policy function; The heuristic rule preferentially assigns tasks to queues with high current load but low demand matching to avoid resource overload, as shown in Equation (11): ; For each queue , calculate its load score , as shown in Equation (12): ; Indicates the importance weight of the CPU, which is 0.5. Indicates the importance weight of the memory; it is 0.5.

[0069] CPU utilization rate , as shown in formula (13): ; Indicates the queue The current number of available CPU cores, Indicates the queue The total CPU quota of; Memory utilization rate , as shown in formula (14): ; Indicates the queue The currently remaining available memory, Indicates the queue The total memory quota of; By means of multi-objective weighting, comprehensively evaluate the advantages and disadvantages of the scheduling action, as shown in formula (15): ; Among them, Indicates the weight of the resource load balancing score; Indicates the resource load balancing score; Indicates the weight of the task quantity balancing score; Indicates the task quantity balancing score; Indicates the weight of the dependency completion reward; Indicates the dependency completion reward; Indicates the weight of the local utilization reward; Indicates the local utilization reward; Indicates the penalty coefficient of the global difference penalty; Indicates the global difference penalty; The specific scoring items are as follows: Load balancing score Indicates the balance of CPU / memory utilization of each queue, as shown in formula (16): ; Among them, Indicates the CPU utilization rate of each queue, Indicates the memory utilization rate of each queue, Standard deviation, measuring the volatility of resource utilization; Task quantity balancing score Indicates the balance of the number of tasks in each queue, as shown in Equation (17): ; Among them, Each represents the number of tasks in the queue, Represents the standard deviation of the number of tasks, Represents the maximum number of tasks in all queues; Dependency completion reward Represents the penalty for the number of uncompleted dependent tasks, as shown in Equation (18): ; Among them, Represents the dependent task of the current task, Represents the indicator function, with a value of 1 if the dependent task is not completed, otherwise 0; Local utilization reward Represents the CPU / memory utilization of the current queue, as shown in Equation (19): ; Among them, Represents the current queue 's CPU utilization, Represents the current queue 's memory utilization; The global difference penalty is Represents the difference between the maximum and minimum CPU utilization of each queue, as shown in Equation (20): ; Among them, Represents the maximum CPU utilization among all queues, Represents the minimum CPU utilization among all queues; Execute the action , and the environment returns the next state and the immediate reward . Store the experience sample five-tuple in the experience replay buffer for subsequent training.

[0070] (5-4) To train an efficient task scheduling strategy, it is necessary to continuously collect experience data through a continuous interaction process and save it in the experience replay buffer. In each task scheduling process, an interaction sample in the form of a five-tuple is recorded, as shown in Equation (21): ; Among them, Represents the state of the current scheduling environment, Represents the scheduling decision made by the agent in the current state, Represents the immediate feedback brought by the current scheduling action, Represents the next state of the environment after the action is executed. Indicates whether the current scheduling process has ended, which is a boolean value. By continuously executing scheduling actions (task → queue allocation), gradually collect five-tuples and store the five-tuples in the experience replay pool (Replay Buffer). When the number of samples reaches a certain scale (forming a batch), sample from it to form a training set (the batch size is set to BATCH_SIZE = 256) for optimizing and training the Q-value function.

[0071] Use the training set to train the scheduling policy model, optimize the Q-value function, so as to gradually approach the optimal task scheduling policy and achieve the efficient allocation of tasks and the maximization of resource utilization in a multi-queue environment.

[0072] The scheduling policy model includes an online network and a target network ; The online network and the target network , the parameter update formula is as shown in formula (22): ; Among them, represents the online network parameters, which are responsible for real-time updating and training through experience replay; represents the target network parameters, which are periodically synchronized from the online network for stable Q-value calculation; represents the synchronization coefficient, ; Periodic hard update; The calculation of the target Q-value is as shown in formula (23): ; Among them, represents the discount factor, which controls the decay weight of future rewards. represents the next state, the new state obtained by the environmental transition after executing the action; represents the immediate reward, which is calculated by the reward function;

[0073] The online network and the target network have the same neural network structure. The neural network includes the first fully connected layer, the first activation function, the second fully connected layer, the second activation function, and the third fully connected layer; as shown in formula (24): ; Among them, represents the dimension of the input feature vector, which is composed of the task feature and the queue feature Stitched together; Represents the first fully connected layer, containing 128 neurons, which performs a linear transformation on the input features and outputs ; Represents the intermediate dimension, the output dimension of the first hidden layer (128 dimensions). Subsequently, it passes through Represents the activation function, the rectified linear unit, which introduces non-linear expressive ability, and the output is still Followed by the second fully connected layer , containing 64 neurons, further extracting high-order features, and the output is , Represents the intermediate dimension, the output dimension of the second hidden layer (64 dimensions); and passes through the ReLU activation again; finally, the features are mapped to a scalar through the output layer (fully connected layer) , that is, the Q value, representing the expected reward of the current task-queue pair in the given state.

[0074] Loss function for batch training of the scheduling policy model Adopts the mean squared error, as shown in formula (25): ; Where, Indicate Batch size, the number of state-action pairs sampled in a single training; Represents the th state vector of the sample; Represents the th task index of the sample; Represents the th queue index of the sample; Represents the Q value predicted by the online network, and the predicted reward for the task after inputting and the queue ; Represents the Q value calculated by the target network.

[0075] The core output of the scheduling policy model is the Q value function , which is used to evaluate: in the current state , if the task is assigned to the queue , the expected reward that can be obtained. Based on the Q value function, the agent can adopt a certain action selection strategy to generate a specific scheduling decision.

[0076] In step (6), the scheduling information is obtained and saved, including: Each workflow scheduling optimization result is obtained through step (5); meanwhile, the workflow scheduling optimization results are saved into a specific table in the MySQL database. The specific table includes three columns: "dag_id", "task_id", and "queue_id". As shown in Table 2.

[0077] Table 2 Scheduling Optimization Results;

[0078] Example 3 A computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of the calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy described in Example 1 or 2 are implemented.

[0079] Example 4 A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the steps of the calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy described in Example 1 or 2 are implemented.

[0080] Example 5 The calculation-storage flow joint scheduling optimization system based on DDQN and heuristic strategy includes: A calculation flow and storage flow definition module, configured to: define the calculation flow and storage flow; A workflow scheduling and execution module, configured to: when the workflow is triggered, the scheduler evaluates the executable status of tasks one by one according to the dependency relationship and scheduling strategy between tasks, and submits the executable tasks to the task executor; the task executor is responsible for executing these tasks on the working nodes to ensure that the tasks are completed in the predetermined order and priority; A data collection module, configured to: collect DAG structure, queue information, and running numbers; A prediction module, configured to: predict task running time and resource usage based on historical running data; A scheduling optimization module, configured to: perform scheduling optimization based on DDQN and heuristic strategy; An information acquisition and saving module, configured to: acquire and save scheduling information.

Claims

1. A joint scheduling optimization method for compute-storage flow based on DDQN and heuristic strategy, characterized in that include: (1) Definition of computing flow and storage flow; (2) Workflow scheduling and execution: When a workflow is triggered, the scheduler evaluates the executable status of each task based on the dependencies between tasks and the scheduling strategy, and submits the executable tasks to the task executor. The task executor is responsible for executing these tasks on the worker nodes, ensuring that the tasks are completed in the predetermined order and priority; (3) DAG structure, queue information, and run number collection; (4) Task running time and resource usage prediction based on historical running data; (5) Scheduling optimization based on DDQN and heuristic strategies; (6) Obtaining and saving scheduling information.

2. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 1, characterized in that In step (1), the calculation flow and storage flow definition include: First, the DAG model is used to uniformly model the entire task process. In DAG, each node represents an independent task, and the directed edges between nodes represent the dependencies between tasks. Based on the functional attributes and resource usage types of tasks, tasks are logically divided into two sub-flows: Computation flow: includes the execution of operator logic and the scheduling and allocation of computing resources; Storage flow: including the storage, reading, transmission and management of data that tasks depend on; All task nodes are declared and parameterized through Python scripting; The computation flow and the storage flow are linked by a trigger mechanism called TriggerDagRunOperator: when the computation task is completed, the corresponding storage task is automatically triggered to ensure the timing connection of the processing flow; at the same time, ExternalTaskSensor is introduced to achieve cross-flow dependency synchronization: the computation flow task monitors the status changes of a specific storage task to determine when to start it.

3. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 1, characterized in that, In step (2), workflow scheduling and execution; specifically including: When an Airflow workflow is triggered, the scheduler evaluates the status of each task according to the task dependencies and scheduling policies defined in the DAG. The scheduler checks which tasks meet the dependency conditions and determines whether the scheduling time window has been reached. Tasks that meet the conditions are marked as schedulable and then placed in the task queue, waiting to be processed by the task executor. The executor obtains tasks from the task queue and assigns them to the corresponding working nodes or containers for execution; During the execution process, the task status is updated in real time. If the task fails to execute, the scheduler reschedules the task according to the task retry strategy. During the whole process, the scheduler ensures that the tasks in the DAG are strictly executed according to the defined dependency order and priority order until all tasks are completed, thus completing the entire workflow scheduling and execution closed loop.

4. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 1, characterized in that In step (3), DAG structure, queue information and operation data collection include: In terms of the DAG structure, the following information is collected and organized for each task node: dag_id, task_id, whether it is a critical path task, whether it belongs to a computation flow task, whether it is a TriggerDagRunOperator task that triggers a storage flow, the target DAG it triggers, whether it is an ExternalTaskSensor type task, the DAG and task list it depends on, and the DAG adjacency list structure between the overall computation flow and storage flow; dag_id refers to the id of the DAG, and task_id refers to the id of the task; In terms of task running data, i.e., running data, record the running status of historical tasks, including: task_id, dag_id, run_id, job_id, pid, avg_cpu, min_cpu, max_cpu, avg_mem, min_mem, max_mem, rss, file_size, and duration; run_id refers to the DAG instance running number, job_id refers to the task running number generated by the scheduler, pid refers to the task process number, avg_cpu refers to the average CPU usage rate, min_cpu refers to the minimum CPU usage rate, max_cpu refers to the maximum CPU usage rate, avg_mem refers to the average memory occupancy, min_mem refers to the minimum memory occupancy, max_mem refers to the maximum memory occupancy, rss refers to the resident memory set size, file_size refers to the file size, and duration refers to the task execution duration; In terms of queue resources, i.e., queue information, collect for each execution queue, including: queue_id, queue_code, currently available CPU, memory, storage, and I / O bandwidth, and build a real-time mapping relationship between tasks and execution resources; queue_id represents the queue name, and queue_code represents the queue number; Save the above-collected DAG structure and queue resources to a JSON file, and save the task running data to the airflow_task_stats table in the MySQL database.

5. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 1, characterized in that In step (4), the prediction of task running time and resource usage based on historical running data includes: (4-1) Set the basic environment for system operation, including the model storage path and operation log configuration; obtain historical task data from the airflow_task_stats table in the MySQL database, including the fields: task_id, dag_id, run_id, avg_cpu, avg_mem, rss, file_size, and duration; Perform data preprocessing on the historical data, including type conversion and missing value filling; forcibly convert the historical task data to a numerical type and save it to the MySQL database, as shown in formula (1), and fill the missing values with 0, as shown in formula (2): ； ； Among them, represents the original input data, represents the converted numerical data, and illegal values are converted to NaN; represents a function provided by Pandas to convert data to a numerical type; represents that when the data cannot be converted to a number, it is forcibly converted to , that is, a missing value; represents the value after filling in the missing values, represents filling all missing values with 0; (4-2) Using the XGBoost regression model, three models are trained independently for each task, and three XGBoost regression models are deployed independently for each task_id to predict: running time, CPU usage, and memory usage, respectively, as shown in formula (3). The generated regression model is persisted and saved as a pkl file; ； Among them, represents the XGBoost regression model trained for predicting the running time of the task; represents the XGBoost regression model trained for predicting the CPU usage rate; represents the XGBoost regression model trained for predicting the memory occupancy; represents the predicted running time of the task, represents the predicted CPU usage rate, represents the predicted memory occupancy; The Optuna framework is used for Bayesian optimization to search for the optimal hyperparameter combination. The goal is to minimize the mean absolute error and minimize the square error when training the XGBoost regression model, as shown in formula (4): ； Among them, represents the true value of the th sample, the th sample's predicted value, represents the L1 regularization coefficient, represents the L2 regularization coefficient, represents the minimum loss decrease threshold for controlling tree splitting, represents the number of leaf nodes, represents the leaf node weight vector; represents the L1 norm, that is, the sum of the absolute values of all leaf node weights; represents the square of the L2 norm, that is, the sum of the squares of all leaf node weights; represents the total loss function; (4-3) Load the latest prediction model according to the model storage path. The data in the airflow_task_stats table of the MySQL database is divided into datasets by task_id, and the input features are , and the output target is , , ; The latest record prediction for each task is shown in formula (5): ； Among them, represents the latest task feature vector, represents the predicted value, which is time, CPU or memory; represents the trained XGBoost regression model function; (4-4) Scan the configuration file directory and write the predicted values to the predicted_time, predicted_cpu, and predicted_memory fields of the JSON file.

6. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 1, characterized in that In step (5), scheduling optimization based on DDQN and heuristic strategy includes: Construct a four-dimensional state tensor as shown in Equation (6): , as shown in Equation (6): ； Among them, represents the batch size, represents the total number of tasks, represents the total number of queues, represents the feature dimension, is ; Task characteristics is , as shown in formula (7): ； Among them, represents the task weight, represents the assigned queue number, represents whether it is a critical path task, represents whether it is a computational flow task, represents whether it is a trigger task, represents whether it is a sensor task, represents the predicted running time, CPU, and memory, represents the one-hot encoding of dependent tasks; Queue characteristics: , as shown in formula (8): ； Among them, represents the available CPU resources of the queue, represents the total CPU resources of the queue, represents the available memory of the queue, represents the total memory of the queue, represents the available storage of the queue, represents the total storage of the queue, represents the number of allocated tasks in the queue, represents the timestamp when the current task in the queue ends; (5-2) Perform multiple rounds of Episode training until the termination condition is met; each Episode includes the following steps: Environment reset: restore the simulation environment to its initial state and clear previous interaction history data; Task completion judgment: Check whether the current Episode has completed all preset task objectives. If the task has been completed, skip the subsequent steps and go directly to the next Episode; (5-3) In the current Episode, perform the following operations according to the time step: Available action detection: Get the legal action set based on the current state of the environment; Obtain legal actions from the set of legal actions, which is the key function for verifying the legality of actions , as shown in Equation (9): ； Among them, represents the specific operation on the task ; represents the task to be scheduled currently, represents the dependency set, represents the dependent task, represents the task status; According to the hybrid epsilon-greedy and heuristic rules , select the optimal action as shown in Equation (10); ； Among them, represents the exploration rate, represents the online network parameters, represents the task of the set of legal queues; Represents an action in the set of legal queues Represents the Q-value function in reinforcement learning Represents the heuristic policy function The heuristic rule preferentially assigns tasks to queues with high current load but low demand matching to avoid resource overload, as shown in formula (11): ； For each queue calculate its load score as shown in formula (12): ； Indicates the CPU importance weight, Indicates the memory importance weight; CPU usage , as shown in formula (13): ； Indicates the queue The number of currently available CPU cores, Indicates the queue The total CPU quota; Memory usage , as shown in formula (14): ； Indicates the queue The currently remaining available memory, Indicates the queue The total memory quota; The pros and cons of scheduling actions are comprehensively evaluated by multi-objective weighting, as shown in formula (15): ； Among them, represents the weight of the resource load balancing score; represents the resource load balancing score; represents the weight of the task quantity balancing score; represents the task quantity balancing score; represents the weight of the dependency completion reward; represents the dependency completion reward; represents the weight of the local utilization reward; represents the local utilization reward; represents the penalty coefficient of the global difference penalty; represents the global difference penalty; The specific scoring items are as follows: Load balancing score Indicates the balance of CPU / memory utilization of each queue, as shown in formula (16): ； Among them, represents the CPU utilization rate of each queue, represents the memory utilization rate of each queue, standard deviation, which measures the volatility of resource utilization; Task quantity balance score Indicates the balance of the number of tasks in each queue, as shown in formula (17): ； wherein, each represents the number of tasks in the queue, represents the standard deviation of the number of tasks, represents the maximum number of tasks in all queues; Dependency Completion Reward Indicates the penalty for the number of incomplete dependency tasks, as shown in formula (18): ； Among them, represents the dependent task of the current task, represents the indicator function, with a value of 1 if the dependent task is not completed, otherwise 0; Local utilization reward Indicates the CPU / memory utilization of the current queue, as shown in formula (19): ； Among them, represents the CPU utilization rate of the current queue , and represents the memory utilization rate of the current queue . The global difference penalty is which represents the difference between the maximum and minimum CPU utilization rates of each queue, as shown in Equation (20): ； Among them, represents the maximum CPU utilization rate among all queues, represents the minimum CPU utilization rate among all queues; (5-4) Through the continuous interaction process, experience data is continuously collected and saved in the experience playback buffer. In each task scheduling process, an interaction sample in the form of a five-tuple is recorded, as shown in formula (21): ； Among them, represents the state of the current scheduling environment, represents the scheduling decision made by the agent in the current state, represents the immediate feedback brought by the current scheduling action, represents the next state of the environment after the action is executed, represents whether the current scheduling process has ended, which is a boolean value; By continuously executing scheduling actions, five-tuples are gradually collected and stored in the experience replay pool; When the number of samples reaches a certain scale, they are sampled to form a training set for optimizing the Q-value function.

7. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 6, characterized in that The scheduling policy model includes an online network and a target network ; Online network and the target network , the parameter update formula is as shown in formula (22): ； Among them, represents the online network parameters, which are responsible for real-time update and training through experience replay; represents the target network parameters, which are periodically synchronized from the online network and are used for stable Q-value calculation; represents the synchronization coefficient, ; The target Q value is calculated as shown in formula (23): ； Among them, represents the discount factor, represents the next state, the new state obtained by the environment transition after executing the action; represents the immediate reward, calculated by the reward function; Online network and the target network have the same neural network structure. The neural network includes a first fully connected layer, a first activation function, a second fully connected layer, a second activation function, and a third fully connected layer, as shown in formula (24): ； Among them, represents the dimension of the input feature vector, which is composed of the task feature and the queue feature concatenated together; represents the first fully connected layer, which contains 128 neurons, performs a linear transformation on the input features, and the output is ; Subsequently, through represents the activation function, the rectified linear unit, which introduces non-linear expressive ability, and the output is still Then comes the second fully connected layer , which contains 64 neurons, further extracts high-order features, and the output is , and goes through the ReLU activation again; finally, the feature is mapped to a scalar , that is, the Q value, representing the expected reward of the current task-queue pair in the given state.

8. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to claim 6, characterized in that Loss function for batch training of scheduling policy model The mean squared error is adopted, as shown in formula (25): ； Among them, express Batch size, the number of state-action pairs sampled in a single training; Denote the state vector of the th sample; Denote the task index of the th sample; Denote the queue index of the th sample; Denote the Q value predicted by the online network, the predicted reward for task after input and queue ; ; Denote the Q value calculated by the target network.

9. The calculation-storage flow joint scheduling optimization method based on DDQN and heuristic strategy according to any one of claims 1-8, characterized in that, In step (6), the scheduling information is obtained and saved, including: Through step (5), each workflow scheduling optimization result is obtained; at the same time, the workflow scheduling optimization result is saved in a specific table of the MySQL database, and the specific table includes three columns: "dag_id", "task_id" and "queue_id".

10. A joint scheduling optimization system for compute-storage flow based on DDQN and heuristic strategy, characterized in that, include: The computing flow and storage flow definition module is configured to: define computing flow and storage flow; The workflow scheduling and execution module is configured as follows: when the workflow is triggered, the scheduler evaluates the executable status of tasks one by one according to the dependency relationship and scheduling strategy between tasks, and submits the executable tasks to the task executor; The task executor is responsible for executing these tasks on the worker nodes, ensuring that the tasks are completed in the predetermined order and priority; A data collection module, configured to: collect DAG structures, queue information, and the number of runs; A prediction module, configured to: predict task running time and resource usage based on historical running data; A scheduling optimization module, configured to: perform scheduling optimization based on DDQN and heuristic policies; An information acquisition and saving module, configured to: acquire and save scheduling information.

Images