Deep reinforcement learning driven multi-task cooperative scheduling method and system

Abstract

The invention relates to a deep reinforcement learning driven multi-task cooperative scheduling method and system, and the method comprises the steps: collecting operation data of a calculation cluster, carrying out the normalization processing, and converting the operation data into a three-dimensional state tensor; calculating a comprehensive priority score according to the information of the cloud computing tasks to be scheduled, and selecting the cloud computing task with the highest priority score as a current scheduling object; traversing the computing nodes in the computing cluster, and generating a mask vector according to the current scheduling object; constructing a deep reinforcement learning network, inputting the three-dimensional state tensor into the deep reinforcement learning network for training, and updating weight parameters by using an optimization algorithm to obtain a trained deep reinforcement learning network; and selecting an optimal computing node by using the trained deep reinforcement learning network. According to the invention, long-term hunger of low-priority tasks and overtime risks of emergency tasks are effectively prevented, and the overall service quality of the cloud platform is improved.

Description

Technical Field

[0001] This invention relates to a deep reinforcement learning-driven multi-task collaborative scheduling method and system, belonging to the field of cloud computing resource management technology. Background Technology

[0002] In the context of rapid development in information technology, distributed cloud computing has become a crucial driver of digital transformation for enterprises and organizations. With increasing user demands and the decentralization of computing resources, effective task scheduling has become an urgent issue. Traditional task scheduling methods are often based on centralized computing architectures, failing to fully consider factors such as resource availability, network latency, and dynamic load in a distributed environment. Therefore, there is a pressing need to explore novel scheduling strategies to optimize resource utilization and improve overall system performance.

[0003] In distributed cloud computing environments, task scheduling faces a series of complex and multi-dimensional problems and challenges. First, due to the distributed and heterogeneous nature of computing resources, achieving efficient load balancing is a critical issue. The computing power, storage capacity, and network bandwidth of each node vary, requiring scheduling algorithms to dynamically evaluate and rationally allocate tasks to improve resource utilization efficiency and reduce response time. Second, network latency and bandwidth instability pose a significant threat to task execution efficiency. In practical applications, data transmission delays can lead to unpredictable task execution, impacting overall system performance. Therefore, designing adaptive scheduling strategies that can adapt to changes in network conditions is crucial for improving system robustness. Furthermore, user needs in cloud environments are typically dynamic and diverse, encompassing different scenarios such as real-time data processing and batch computing. Scheduling algorithms must not only be flexible and adaptable but also effectively balance task priority, resource utilization, and scheduling costs. Ensuring efficient resource allocation while meeting user QoS (Quality of Service) requirements is a significant challenge in the scheduling process. Therefore, constructing a task scheduling framework that combines efficiency, flexibility, and reliability is of profound importance for the efficient management and optimization of distributed cloud computing resources.

[0004] To achieve efficient task scheduling, researchers have proposed various novel methods. Existing cloud computing task scheduling methods are generally classified into the following categories:

[0005] Constant policy scheduling: Constant policy scheduling determines the allocation of tasks and resources upon task submission, without considering dynamic changes during runtime. This strategy mainly includes heuristic algorithms and optimization algorithms, allocating tasks according to preset rules or models. Its advantages lie in its simplicity, ease of understanding and deployment, and suitability for environments with relatively stable resource requirements. It can quickly and effectively find optimal solutions and has low computational resource requirements, without relying on real-time data monitoring. However, this strategy has disadvantages, including a lack of flexible adjustment capabilities to dynamically changing loads and data flows, potentially leading to resource waste or task delays. Furthermore, its adaptability is poor, and it cannot effectively handle sudden demands or node failures.

[0006] Flexible response scheduling: During task execution, flexible response scheduling adjusts the allocation of tasks and resources in real time based on the current system state and resource usage. This type of strategy typically includes load balancing algorithms and event-based scheduling. Its advantages lie in its ability to respond to changes in system state in real time, rationally allocating resources, thereby improving flexibility and efficiency, significantly reducing resource idleness and waste, and enhancing resource utilization and system performance. However, the implementation and maintenance of this type of strategy are complex, requiring continuous monitoring and data analysis, and may lead to frequent context switching, thus increasing system overhead and latency.

[0007] Multi-faceted collaborative scheduling combines the advantages of the previous two strategies, employing a combination of multiple scheduling tactics to achieve more efficient task scheduling. This type of approach may involve container scheduling and hierarchical scheduling. Its advantages lie in its good flexibility and adaptability, capable of handling both static and dynamic scenarios simultaneously, and effectively combining the strengths of different strategies, potentially achieving better overall performance. However, the design and implementation of this strategy are relatively complex, requiring reasonable coordination between different strategies. Furthermore, the time and resources required for testing and optimization are also significant, potentially increasing system overhead.

[0008] With the rapid development of cloud computing and the increasing complexity of application scenarios, the demands for diverse and dynamically changing needs in distributed cloud environments are rising. Against this backdrop, Deep Reinforcement Learning (DRL), as an advanced intelligent optimization technique, has demonstrated enormous potential. Its adaptability allows it to adjust scheduling strategies in real time, effectively responding to dynamic load and resource demand changes. Furthermore, DRL, with its efficient exploration and optimization capabilities, can discover optimal scheduling decisions in complex state spaces, thereby improving system performance and resource utilization. Simultaneously, DRL possesses long-term decision-making capabilities, comprehensively considering multiple factors such as task execution time and priority, enhancing the scientific rigor and accuracy of scheduling. Facing the increasing complexity and dynamism of modern computing environments, DRL provides a flexible and intelligent solution for optimizing resource allocation and meeting future needs. Summary of the Invention

[0009] To address the shortcomings of existing technologies, this invention provides a deep reinforcement learning-driven multi-task collaborative scheduling method and system. It constructs a task scheduling model based on deep reinforcement learning, aiming to improve scheduling efficiency, reduce latency, and increase resource utilization, achieving real-time and efficient scheduling in complex heterogeneous environments. A deep reinforcement learning architecture combining concurrent evaluation and asynchronous feedback updates is proposed: the concurrent mechanism is used for synchronous evaluation of multi-task decisions, while the asynchronous mechanism supports non-blocking model updates under real-time feedback, thereby achieving parallelization of task execution and policy optimization.

[0010] Terminology Explanation:

[0011] 1. Min-Max: This is a data preprocessing technique that scales attribute data to a specific range. It is often used in data analysis to eliminate differences in units. Its core objective is to make the data fall into a limited range such as [-1,1] or [0,1] through linear or nonlinear transformations, so as to facilitate subsequent unified analysis.

[0012] 2. Deep Q Network (DQN): A deep reinforcement learning algorithm that combines deep learning neural networks with the classic Q-Learning algorithm. Q-Learning: A reinforcement learning method that evaluates the quality of an action performed in a given state by learning a function called the "Q-value". Deep Learning Neural Network: When the state space is very large or continuous, storing the Q-values ​​of all state-action pairs in a table becomes impractical. DQN uses a deep neural network to approximate this Q-function, taking the state as input and outputting the Q-value for each possible action.

[0013] The technical solution of the present invention is as follows:

[0014] The first aspect of this invention provides a deep reinforcement learning-driven multi-task cooperative scheduling method, comprising:

[0015] Step 1: Collect the running data of the computing cluster, normalize it, and convert it into a three-dimensional state tensor;

[0016] Step 2: Based on the information of the cloud computing tasks to be scheduled, calculate the comprehensive priority score, select the cloud computing task with the highest priority score as the current scheduling object;

[0017] Step 3: Traverse the compute nodes in the compute cluster and generate a mask vector based on the current scheduling object;

[0018] A deep reinforcement learning network is constructed by inputting a 3D state tensor into the network for training, and an optimization algorithm is used to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are then selected using the trained deep reinforcement learning network, including:

[0019] The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy.

[0020] The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

[0021] According to a preferred embodiment of the present invention, step 1 is implemented as follows:

[0022] The monitoring agent deployed on the cloud control node collects real-time operational data from the heterogeneous cluster, including resource status data. and task attribute data ;

[0023] Among them, resource status data Indicates data collection in heterogeneous clusters Real-time status of each computing node, including: CPU utilization Memory usage Disk I / O throughput and network bandwidth margin ;

[0024] Task attribute data Static attributes representing the head task of the acquisition task receive queue include: instruction length. Data size Arrival time and deadline ;

[0025] The collected runtime data is normalized as follows:

[0026] ;

[0027] in, This represents the normalized runtime data. Indicates runtime data, This represents the minimum value. Indicates the maximum value;

[0028] The normalized runtime data is converted into a three-dimensional state tensor. ,in Indicates the number of compute nodes. Indicates the time step in history. Indicates the number of feature channels.

[0029] According to a preferred embodiment of the present invention, step 2 is implemented as follows:

[0030] For each task in the task queue that is to be scheduled in the cloud computing task queue Obtain task information and calculate the lag ratio, as shown below:

[0031] ;

[0032] in, Indicates the lag ratio, Indicates the waiting time. Indicates the current time stamp. Indicates the absolute deadline timestamp for the task;

[0033] The task urgency index is calculated as follows:

[0034] ;

[0035] in, Indicates the urgency of the task. The estimated completion time is... The deadline is [date / time]. For the remaining workload;

[0036] Calculate the potential latency of the task as follows:

[0037] ;

[0038] in, Indicates the potential latency rate. This indicates the estimated total workload of the task. This indicates the amount of work that has been completed so far.

[0039] The calculated lag ratio, task urgency, and potential delay rate are normalized, including using Min-Max normalization for the lag ratio and potential delay rate.

[0040] To determine task urgency, inverse normalization is used.

[0041] ;

[0042] in Indicates the urgency of the task after processing. This represents the maximum value indicating the urgency of the task. The minimum value representing the urgency of the task;

[0043] Based on preset weights, the normalized indicators are weighted and summed to obtain a comprehensive priority score for each task. As shown below:

[0044] ;

[0045] in, , , For the preset weights, This represents the normalized lag ratio. This represents the normalized potential delay rate;

[0046] Scoring based on overall priority The task queue is sorted from highest to lowest score, and the task with the highest score is selected as the current scheduling target.

[0047] According to a preferred embodiment of the present invention, traversing the computing nodes in the computing cluster and generating a mask vector based on the currently scheduled object includes:

[0048] Iterate through all computing nodes in the computing cluster to generate a binary mask vector. ;in, This represents the validity mask value of the j-th computing node. If the task, i.e., the selected scheduling object, times out on the j-th computing node, then... If the j-th computing node has insufficient remaining resources, then ;otherwise, 1.

[0049] According to a preferred embodiment of the present invention, a three-dimensional state tensor is input into a deep reinforcement learning network to obtain the original output, and a mask vector is used to correct the original output to obtain the optimal computation node; including:

[0050] Deep reinforcement learning networks are deep Q-networks, which use three-dimensional state tensors. Inputting the deep Q-network yields the raw output, i.e., the raw Q-value. And perform a correction operation using a mask vector to adjust the original Q value. The correction is as follows:

[0051] ;

[0052] in, This represents the original Q value. This represents the corrected Q-value vector;

[0053] Get the current optimal computing node: ,in This represents the index of the optimal computing node.

[0054] According to a preferred embodiment of the present invention, a unique optimal computing node is selected through a resource selection strategy; including:

[0055] The resource selection strategy is one of the completion priority strategy, availability priority strategy, and high utilization priority strategy. Based on the scheduling goals of the cloud computing platform, the resource selection strategy is used to select a unique optimal computing node from multiple optimal computing nodes.

[0056] If the cloud computing platform needs to complete the entire batch of tasks quickly, the completion priority strategy is used to select the computing node with the earliest completion time as the sole optimal computing node; as shown below;

[0057] ;

[0058] in, Indicates the completion time. This is the end time of the preceding task. The estimated execution time for the current task. Indicates the earliest execution time of the current task. The earliest available time for the compute node; select the completion time. The smallest computing node is the optimal computing node;

[0059] If the cloud computing platform prioritizes response speed, then an availability-first strategy is used to select the compute node with the earliest available time as the sole optimal compute node; as shown below:

[0060] ;

[0061] in, Indicates available time; select available time. The smallest computing node is the optimal computing node;

[0062] If the goal of a cloud computing platform is to save costs and improve resource utilization, then a high utilization priority strategy is used to select the optimal computing node; as shown below:

[0063] ;

[0064] in, Indicates utilization rate. This refers to the time that the machine, i.e., the computing node, has been used. For machine availability time, Indicates all that satisfy A set of tasks; selecting utilization rates The largest computing node is the optimal computing node.

[0065] According to a preferred embodiment of the present invention, the weight parameters are updated using an optimization algorithm; including:

[0066] An algorithm combining a progressive acceleration mechanism and a gradient update mechanism is used to update the weights of the deep reinforcement learning network. The update formula is as follows:

[0067] ;

[0068] in, Indicates the updated weights. This indicates the network parameters before the update. For the asymptotic acceleration term through the momentum coefficient Dynamically adjusted update stride, This represents the temporary target weight calculated based on the current batch of data. This indicates an update to the stride control factor; For gradient update terms, The learning rate decays over time. This represents the gradient vector of the loss function with respect to the current weights.

[0069] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program implementing steps of a deep reinforcement learning-driven multi-task cooperative scheduling method.

[0070] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of a deep reinforcement learning-driven multi-task cooperative scheduling method.

[0071] A second aspect of the present invention provides a deep reinforcement learning-driven multi-task cooperative scheduling system, comprising:

[0072] The data acquisition module is configured to: collect the running data of the computing cluster, perform normalization processing, and convert it into a three-dimensional state tensor;

[0073] The priority scoring module is configured to: calculate a comprehensive priority score based on the information of the cloud computing tasks to be scheduled, and select the cloud computing task with the highest priority score as the current scheduling object;

[0074] The optimal computing node acquisition module is configured to: traverse the computing nodes in the computing cluster and generate a mask vector based on the current scheduling object;

[0075] A deep reinforcement learning network is constructed by inputting a 3D state tensor into the network for training, and an optimization algorithm is used to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are then selected using the trained deep reinforcement learning network, including:

[0076] The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy.

[0077] The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

[0078] The beneficial effects of this invention are as follows:

[0079] 1. Significantly improved the convergence speed and decision effectiveness of deep reinforcement learning models in heterogeneous environments:

[0080] This invention introduces an action masking mechanism into the DQN decision-making process. Unlike traditional DQN algorithms that blindly explore the entire action space, this invention pre-generates masks using rules related to urgent deadlines and resource scarcity, forcibly pruning and eliminating invalid nodes. This not only ensures that scheduling decisions strictly comply with SLA (Service Level Agreement) constraints, avoiding deadlocks or timeouts, but also significantly reduces the model's exploration space, solving the problems of slow cold start and numerous invalid interactions in the early stages of traditional reinforcement learning under complex constraints.

[0081] 2. Enhanced the ability to perceive the spatiotemporal load characteristics of heterogeneous resources:

[0082] This invention constructs an N×T×F three-dimensional spatiotemporal state tensor, replacing the traditional approach of simply using resource values ​​at the current moment. By introducing a time sliding window, the model can capture the fluctuation trend of computing node load in the time dimension (such as predicting nodes that are about to be overloaded) and the topological distribution characteristics in the spatial dimension, thereby making more forward-looking load balancing decisions and avoiding the generation of resource hotspots.

[0083] 3. Real-time response and continuous model evolution were achieved in high-concurrency scenarios:

[0084] This invention employs an asynchronous decoupled architecture that separates decision-making and training. The main thread performs only lightweight forward inference, ensuring low-latency response when facing massive concurrent tasks; the background thread uses an improved weight update algorithm (combining progressive acceleration and gradient updates) for asynchronous training. This architecture leverages momentum mechanisms to accelerate model convergence and ensures stability in the later stages of training through gradient decay, achieving smooth and continuous adaptive evolution of the scheduling strategy during task execution without blocking business processes.

[0085] 4. Effectively solves the conflict and long-tail delay problems in multi-objective scheduling:

[0086] This invention proposes a weighted evaluation mechanism based on multidimensional principles during the task selection phase. By comprehensively calculating the lag ratio, urgency index, and latency rate, and applying normalized weights, the system can achieve a dynamic balance between ensuring task deadlines and maintaining scheduling fairness. This effectively prevents the long-term starvation of low-priority tasks and the risk of timeouts for urgent tasks, thereby improving the overall service quality of the cloud platform. Attached Figure Description

[0087] Figure 1 This is a diagram of the deep reinforcement learning-driven multi-task collaborative scheduling system architecture of the present invention.

[0088] Figure 2 This is a diagram of the deep reinforcement learning model architecture of the present invention;

[0089] Figure 3 This is a comparison chart of the standard deviation of completion time between the method of the present invention and the comparison algorithm under different task scales;

[0090] Figure 4 This is a comparison chart of the standard deviation of resource utilization between the method of this invention and the comparison algorithm under different task scales;

[0091] Figure 5 This is a comparison chart of the average task delay standard deviation of the method of the present invention and the comparison algorithm under different task scales;

[0092] Figure 6 This is a performance comparison chart of the method of this invention and the comparison algorithm on the multi-objective optimization index GD;

[0093] Figure 7 This is a performance comparison chart of the method of this invention and the comparison algorithm on the multi-objective optimization index IGD. Detailed Implementation

[0094] The present invention will be further described below with reference to the embodiments and accompanying drawings, but is not limited thereto.

[0095] Example 1

[0096] A deep reinforcement learning-driven multi-task cooperative scheduling method, such as Figure 1 As shown, it includes:

[0097] Step 1: Collect the running data of the computing cluster, normalize it, and convert it into a three-dimensional state tensor;

[0098] Step 2: Based on the information of the cloud computing tasks to be scheduled, calculate the comprehensive priority score, select the cloud computing task with the highest priority score as the current scheduling object;

[0099] Step 3: Traverse the compute nodes in the compute cluster and generate a mask vector based on the current scheduling object;

[0100] A deep reinforcement learning network is constructed by inputting a 3D state tensor into the network for training, and an optimization algorithm is used to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are then selected using the trained deep reinforcement learning network, including:

[0101] The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy.

[0102] The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

[0103] Example 2

[0104] The difference between the deep reinforcement learning-driven multi-task cooperative scheduling method described in Example 1 and the one in Example 1 is as follows:

[0105] The implementation process of step 1 is as follows:

[0106] Monitoring agents deployed on cloud control nodes (such as the Master node of a Kubernetes cluster) (such as Prometheus data acquisition components that integrate cAdvisor container monitoring and Node Exporter node monitoring) collect real-time runtime data from heterogeneous clusters (hybrid computing clusters composed of CPU servers and GPU-accelerated servers with different configurations), including resource status data. and task attribute data ;

[0107] Among them, resource status data Indicates data collection in heterogeneous clusters Real-time status of each computing node, including: CPU utilization. Memory usage Disk I / O throughput and network bandwidth margin ;

[0108] Task attribute data Static attributes representing the head task of the acquisition task receive queue include: instruction length. Data size Arrival time and deadline ;

[0109] To capture the spatial topology distribution characteristics of resource load (such as a rack overheating) and its temporal trends (such as load volatility), the system introduces a length of... A time-sliding window is used; the collected runtime data is normalized as follows:

[0110] ;

[0111] in, This represents the normalized runtime data. Indicates runtime data, This represents the minimum value. This represents the maximum value; for resource status data... CPU utilization and memory usage minimum value Set to 0, maximum value Set to 100; Disk I / O throughput and network bandwidth margin. minimum value Set to 0, maximum value To calculate the maximum physical bandwidth limit (e.g., 10Gbps) of the node's network interface card or disk; for task attribute data. Instruction length and data size minimum value Set to 0, maximum value The maximum allowed task specification threshold or the maximum value observed historically; for arrival time and deadline Instead of directly normalizing the absolute timestamp, we calculate its offset relative to the start time of the current window and then map it to the interval ([0,1]).

[0112] The normalized runtime data is converted into a three-dimensional state tensor. ,in Indicates the number of computing nodes (spatial dimension). Indicates the historical time step (time dimension). It represents the number of feature channels (including CPU, memory, bandwidth, task features, etc.); this tensor is directly used as the input of the subsequent deep neural network, enabling it to perceive local resource hotspots.

[0113] Step 2 is implemented as follows:

[0114] The system employs a concurrent processing mechanism, simultaneously evaluating the priorities of candidate tasks in the task pool and selecting the optimal task to be scheduled. Specifically, it calculates based on the following three principles: priority of delayed tasks, optimization of urgent deadlines, and minimization of latency.

[0115] For each task in the task queue that is to be scheduled in the cloud computing task queue Obtain task information and calculate the lag ratio, as shown below:

[0116] ;

[0117] in, Indicates the lag ratio, Indicates the waiting time. Indicates the current time stamp. Indicates the absolute deadline timestamp for the task; preferred selection The task with the highest value;

[0118] The task urgency index is calculated as follows:

[0119] ;

[0120] in, Indicates the urgency of the task. The estimated completion time is... The deadline is [date / time]. For the remaining workload; prioritize tasks with the narrowest time windows. The smallest task refers to the task with the least remaining time.

[0121] Calculate the potential latency of the task as follows:

[0122] ;

[0123] in, Indicates the potential latency rate. This indicates the estimated total workload of the task (obtained by reading the number of metadata commands when the task is submitted). This indicates the amount of work that has been completed so far; prioritize tasks with the highest latency rate.

[0124] The calculated lag ratio, task urgency, and potential delay rate are normalized to eliminate dimensional differences. Specifically, for the lag ratio and potential delay rate, Min-Max normalization is used.

[0125] To determine task urgency, inverse normalization is used.

[0126] ;

[0127] in Indicates the urgency of the task after processing. This represents the maximum value indicating the urgency of the task. The minimum value representing the urgency of the task;

[0128] Based on preset weights (such as prioritizing deadlines or fairness), the normalized indicators are weighted and summed to obtain a comprehensive priority score for each task. As shown below:

[0129] ;

[0130] in, , , These are preset weights, and the sum of the weights equals 1. If it is necessary to prioritize ensuring the deadline, then you can set... , , The values ​​are 0.1, 0.7, and 0.2 respectively. If fair scheduling is required, then... , , The values ​​are 0.6, 0.2, and 0.2 respectively. If a balanced design is used, then... = = ; This represents the normalized lag ratio. This represents the normalized potential delay rate;

[0131] Scoring based on overall priority The task queue is sorted from highest to lowest score, and the task with the highest score is selected as the current scheduling target.

[0132] Traverse the compute nodes in the compute cluster and generate a mask vector based on the currently scheduled object; including:

[0133] Iterate through all compute nodes in the computing cluster (attempting to assign the task to each compute node in the cluster for preliminary screening; this is a preliminary screening of compute nodes before DQN to determine their suitability), and generate a binary mask vector. ;in, This represents the validity mask value (only 0 and 1) for the j-th computing node. If the task, i.e., the selected scheduling object, times out on the j-th computing node, then... If the j-th computing node has insufficient remaining resources (e.g., the number of remaining CPU cores is less than the task requirements, or the remaining memory capacity is less than the task requirements), then Otherwise (the computing node simultaneously satisfies the above constraints, i.e., the computation does not time out and there are sufficient remaining resources). 1.

[0134] The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. A mask vector is then used to correct the original output to obtain the optimal computation node; for example... Figure 2 As shown, it includes:

[0135] Deep reinforcement learning networks are deep Q-networks, which use three-dimensional state tensors. Inputting the data into a deep Q-network (DQN) yields the raw output, i.e., the raw Q-value. And perform a correction operation using a mask vector to adjust the original Q value. The correction is as follows:

[0136] ;

[0137] in, This represents the original Q value. This represents the corrected Q-value vector; it forces the Q-values ​​of invalid nodes to be reduced to negative infinity, ensuring that the model only explores within the legal space.

[0138] Get the current optimal computing node: ,in This represents the index of the optimal computing node (the index of the action with the largest Q value, corresponding to the ID of the optimal computing node).

[0139] Selecting a unique optimal computing node through a resource selection strategy; including:

[0140] The resource selection strategy is one of the completion priority strategy, availability priority strategy, and high utilization priority strategy. Based on the scheduling goals of the cloud computing platform, the resource selection strategy is used to select a unique optimal computing node from multiple optimal computing nodes.

[0141] If the cloud computing platform needs to complete the entire batch of tasks quickly, it uses a completion priority strategy to select the computing node with the earliest completion time as the only optimal computing node (for example, in a workflow task containing multiple interdependent tasks, the critical path of the workflow needs to be completed as soon as possible. The system will calculate and predict the completion time of multiple optimal computing nodes, such as nodes 1 and 3, and select the one that completes earlier); as shown below.

[0142] ;

[0143] in, Indicates the completion time. This is the end time of the preceding task. The estimated execution time for the current task. Indicates the earliest execution time of the current task. The earliest available time for the compute node; select the completion time. The smallest computing node is the optimal computing node;

[0144] If the cloud computing platform prioritizes response speed (reducing the average waiting time of tasks), it uses an availability-first strategy to select the compute node with the earliest available time as the sole optimal compute node (for example, in a service requiring immediate response, the system will choose one from 1 and 3; if node 1 has strong machine performance but a small task takes 10 seconds to run, while node 3 is available immediately, the system will choose node 3); as shown below:

[0145] ;

[0146] in, Indicates available time; select available time. The smallest computing node is the optimal computing node;

[0147] If the cloud computing platform aims to save costs and improve resource utilization, it uses a high-utilization-first strategy to select the optimal computing node. (The goal here is to minimize idle time fragmentation for each machine (for example, processing an offline data processing task). When choosing between nodes 1 and 3, if node 1 has 30 minutes of idle time and node 3 has 10 minutes, and the task takes exactly 10 minutes to run, the task will be assigned to node 3 to avoid the 20-minute time fragmentation on node 1.) As shown below:

[0148] ;

[0149] in, Indicates utilization rate. This represents the time a machine, or compute node, has been used (the total time a compute node has been occupied by assigned tasks within the current scheduling window). This refers to the machine's available time (the total length of the current scheduling time window for the compute node, such as the length of a scheduling cycle set by the system). Indicates all that satisfy A set of tasks; selecting utilization rates The largest computing node is the optimal computing node.

[0150] Update the weight parameters using an optimization algorithm; including:

[0151] To address the real-time scheduling issue, this invention designs an asynchronous decoupled architecture that separates decision-making and training.

[0152] Asynchronous dual-thread operation: Main thread (decision): Only performs forward inference, with response time controlled in milliseconds, ensuring that tasks are distributed as soon as they arrive, without queuing or blocking; Background thread (training): Runs independently, responsible for extracting data from the Experience Replay Buffer and updating network parameters;

[0153] An algorithm combining a progressive acceleration mechanism and a gradient update mechanism is used to update the weights of the deep reinforcement learning network. The update formula is as follows:

[0154] ;

[0155] in, Indicates the updated weights. This indicates the network parameters before the update. For the asymptotic acceleration term through the momentum coefficient Dynamically adjusted update stride, This represents the temporary target weights calculated based on the current batch of data (only the weights after gradient descent are applied). This represents the step size adjustment factor, which is used to control the contribution ratio of the gradual acceleration term in the weight update, and its value ranges from 0 to 1. For gradient update terms, The learning rate decays over time. This represents the gradient vector of the loss function with respect to the current weights (loss function L with respect to network parameters). The first-order partial derivatives are used to guide the parameters to move in the direction of loss reduction;

[0156] After updating the network weights, a deep Q-network with optimized parameters is obtained, which enables the system to make better decisions when processing subsequent tasks, thereby reducing overall task latency and improving resource utilization. Through this asynchronous and non-blocking update mechanism, the scheduling strategy can continuously and adaptively evolve during task execution.

[0157] In this section, to fully demonstrate the advantages of the algorithm proposed in reducing task latency, shortening completion time (makespan), and improving resource utilization, it is compared and analyzed with classic genetic algorithms (GA), ant colony optimization (ACO), and high-dimensional deep reinforcement learning algorithms (HDMDDQN). Figure 3-5 As shown, these comparative experiments provide a more intuitive way to verify the superiority of the proposed algorithm.

[0158] To comprehensively evaluate the generalization ability of the proposed algorithm in cloud computing environments of different scales, a test set covering three different task scales (l, m, and s) was designed, and corresponding virtual machine resources were configured for each task scale. During the experiments, multi-dimensional variables were set to account for the diversity of task complexity and resource allocation, including the number of tasks, the number of subtasks, and the number of available machines, to ensure that the experiments cover various real-world application scenarios and fully reveal the algorithm's performance. Specific experimental parameters are shown in Table 1. These parameter settings not only reflect the typical configuration of cloud computing systems in real-world environments but also provide a reliable basis for in-depth analysis of the algorithm's performance in complex systems.

[0159] Table 1 Experimental Parameters

[0160] parameter scope Number of tasks [20,60] Number of subtasks (E_ave) [40,160] Number of machines (vm) [10,50] Batch size (Batch_size) 16 Buffer size 2000 Attenuation rate (gamma) 0.95 Initial value of epsilon (e_greedy) 0.6 Number of training rounds (L) 20 Learning rate decay 0.01 shows exponential decay Momentum 0.9 is dynamically adjusted.

[0161] Evaluation Metrics: In heterogeneous cloud environments, task scheduling faces trade-offs between multiple optimization objectives, such as execution time, energy consumption, and resource utilization. To comprehensively evaluate the performance of scheduling algorithms on these objectives, multiple evaluation metrics are typically employed. These metrics not only measure the convergence and diversity of the solution set but also assess the overall quality of the solution set and its approximation to the ideal optimal solution. This paper selects standard deviation, generator distance (GD), and index distance (IGD) as the primary evaluation metrics, analyzing the performance of multi-objective task scheduling algorithms from different dimensions. Standard deviation is used to evaluate the balance of each algorithm, while generator distance and index distance measure the convergence and coverage of the solution set relative to the ideal optimal solution, respectively. The combination of these metrics comprehensively evaluates the performance of multi-objective scheduling algorithms, ensuring their effectiveness in complex heterogeneous environments.

[0162] 1) Standard deviation:

[0163] In the context of task scheduling, load balancing is one of the key indicators for ensuring the rational utilization of resources. By calculating the standard deviation of the load on each resource, the degree of balance in the resource load distribution can be quantitatively assessed. A smaller standard deviation means that the load distribution is closer to a uniform state, thus indicating that the scheduling algorithm has better load balancing performance.

[0164] ;

[0165] in, Standard deviation For the first Data points, This represents the mean of the data.

[0166] 2) Generation distance:

[0167] Generative distance measures how close the non-dominated solution set obtained by the algorithm is to the ideal optimal solution set. By calculating the distance between each non-dominated solution and the ideal point, the generative distance reflects the convergence of the solution set. By quantifying the distance between the solution and the ideal solution, Generative Distance (GD) can reflect the approximation of the scheduling scheme to each objective, ensuring that the scheduling algorithm can find a solution close to the optimum.

[0168] ;

[0169] in Is it a dominant solution set? It is the ideal solution. It represents Euclidean distance.

[0170] 3) Indicator Distance:

[0171] IGD is an improved generation distance algorithm used to evaluate the distance of a non-dominated solution set relative to a target front. It measures the approximation and coverage of the solution set by calculating the distance from points on the target front to the non-dominated solution set. By evaluating the approximation and coverage of the solution set to the target front, IGD can effectively measure the overall performance of scheduling algorithms in heterogeneous environments.

[0172] ;

[0173] in It is a set of points on the front line of the target. Is it a dominant solution set? It represents Euclidean distance.

[0174] The results are shown in the table below:

[0175] Table 2. Standard Deviation-Makespan Results Analysis Table

[0176] GA ACO HDMDDQN ICSDQN sss 7.77E+01 6.35E+01 7.91E+01 6.25E+01 ssm 9.07E+01 8.52E+01 8.84E+01 3.42E+01 ssl 1.37E+02 2.37E+02 8.89E+01 7.74E+01 sms 9.04E+01 7.44E+01 1.06E+02 3.83E+01 smm 1.00E+02 9.88E+01 7.47E+01 6.63E+01 sml 9.70E+01 6.27E+01 1.36E+02 4.48E+01 sls 7.00E+01 9.28E+01 8.10E+01 3.92E+01 slm 8.92E+01 1.21E+02 1.01E+02 5.56E+01 s-l-l 9.39E+01 1.77E+02 1.34E+02 4.91E+01 mls 4.71E+01 7.77E+01 6.27E+01 4.52E+01 mlm 6.34E+01 5.96E+01 5.28E+01 2.89E+01 m-l-l 5.39E+01 9.51E+01 9.09E+01 4.36E+01 mms 4.60E+01 8.83E+01 8.45E+01 4.22E+01 mmm 7.69E+01 8.17E+01 8.54E+01 7.42E+01 mml 7.85E+01 8.69E+01 1.51E+02 5.42E+01 mls 9.22E+01 8.78E+01 1.11E+02 8.61E+01 mlm 1.43E+02 1.47E+02 1.16E+02 5.88E+01 m-l-l 1.07E+02 1.02E+02 1.15E+02 9.86E+01 lss 5.23E+01 9.35E+01 6.01E+01 4.67E+01 lsm 6.32E+01 5.03E+01 5.90E+01 4.89E+01 l-s-l 9.67E+01 5.72E+01 1.21E+02 2.57E+01 lms 7.19E+01 8.57E+01 5.80E+01 3.36E+01 lmm 8.16E+01 1.03E+02 7.79E+01 6.16E+01 l-m-l 1.05E+02 8.23E+01 5.32E+01 5.23E+01 l-l-s 6.17E+01 1.01E+02 6.91E+01 5.91E+01 l-l-m 1.26E+02 9.33E+01 6.01E+01 8.90E+01 l-l-l 7.77E+01 1.33E+02 1.08E+02 7.70E+01

[0177] Table 3. Analysis of Standard Deviation-Utilization Results

[0178] GA ACO HDMDDQN ICSDQN sss 6.88E-02 5.50E-02 8.87E-02 4.28E-02 ssm 5.84E-02 5.70E-02 4.42E-02 4.29E-02 ssl 9.09E-02 7.66E-02 3.93E-02 3.20E-02 sms 6.22E-02 4.47E-02 7.71E-02 4.37E-02 smm 5.06E-02 5.87E-02 7.38E-02 3.76E-02 sml 6.35E-02 4.61E-02 5.95E-02 4.10E-02 sls 1.01E-01 6.16E-02 7.51E-02 5.03E-02 slm 7.97E-02 8.14E-02 5.64E-02 5.39E-02 s-l-l 5.42E-02 5.01E-02 6.81E-02 4.28E-02 mls 4.59E-02 3.71E-02 4.38E-02 3.61E-02 mlm 5.49E-02 5.46E-02 4.73E-02 3.66E-02 m-l-l 3.07E-02 3.66E-02 5.44E-02 2.97E-02 mms 6.23E-02 4.76E-02 3.10E-02 5.38E-02 mmm 4.34E-02 6.55E-02 4.17E-02 3.26E-02 mml 4.30E-02 4.17E-02 5.66E-02 2.97E-02 mls 6.02E-02 6.16E-02 6.42E-02 5.26E-02 mlm 5.52E-02 4.56E-02 4.37E-02 3.92E-02 m-l-l 6.37E-02 5.03E-02 2.21E-02 2.03E-02 lss 2.60E-02 4.69E-02 2.20E-02 2.51E-02 lsm 5.80E-02 4.94E-02 4.11E-02 3.20E-02 l-s-l 3.70E-02 3.12E-02 4.66E-02 3.35E-02 lms 3.31E-02 3.28E-02 3.22E-02 3.22E-02 lmm 3.14E-02 5.26E-02 4.14E-02 2.47E-02 l-m-l 4.74E-02 2.65E-02 4.67E-02 2.03E-02 l-l-s 4.66E-02 4.22E-02 4.00E-02 3.18E-02 l-l-m 3.40E-02 5.14E-02 3.79E-02 6.52E-02 l-l-l 4.27E-02 7.17E-02 5.24E-02 3.57E-02

[0179] Table 4. Standard Deviation-Delay Results Analysis Table

[0180] GA ACO HDMDDQN ICSDQN sss 2.87E+02 1.67E+02 3.94E+02 1.60E+02 ssm 5.73E+02 8.70E+02 5.47E+02 5.34E+02 ssl 2.61E+03 2.69E+03 1.92E+03 1.40E+03 sms 3.80E+02 3.24E+02 1.63E+02 1.16E+02 smm 5.70E+02 8.07E+02 5.31E+02 6.98E+02 sml 2.46E+03 2.35E+03 1.85E+03 1.73E+03 sls 1.92E+02 3.50E+02 2.33E+02 1.80E+02 slm 7.11E+02 6.58E+02 1.17E+03 4.51E+02 s-l-l 1.44E+03 2.01E+03 3.05E+03 1.21E+03 mls 2.58E+02 2.85E+02 2.45E+02 1.85E+02 mlm 4.35E+02 4.77E+02 3.88E+02 1.95E+02 m-l-l 6.50E+02 1.15E+03 1.73E+03 6.29E+02 mms 2.14E+02 3.14E+02 2.17E+02 1.50E+02 mmm 4.94E+02 3.60E+02 4.58E+02 2.81E+02 mml 1.18E+03 8.70E+02 1.59E+03 8.22E+02 mls 1.66E+02 3.56E+02 3.45E+02 1.42E+02 mlm 6.21E+02 7.06E+02 2.70E+02 1.59E+02 m-l-l 1.29E+03 8.81E+02 1.01E+03 3.13E+02 lss 1.35E+02 2.65E+02 1.75E+02 1.30E+02 lsm 3.67E+02 2.86E+02 5.29E+02 2.42E+02 l-s-l 8.07E+02 6.54E+02 8.89E+02 3.27E+02 lms 1.85E+02 2.31E+02 2.40E+02 1.10E+02 lmm 3.92E+02 3.75E+02 5.90E+02 1.93E+02 l-m-l 9.51E+02 8.57E+02 6.18E+02 3.60E+02 l-l-s 1.75E+02 1.60E+02 2.50E+02 1.45E+02 l-l-m 4.82E+02 3.61E+02 3.71E+02 1.64E+02 l-l-l 8.59E+02 1.15E+03 7.87E+02 2.66E+02

[0181] In this context, l, m, and s are the first letters of large, medium, and small, respectively. For example, sss (small-small-small): represents a simple scenario with few tasks, few subtasks, and few machines. lll (large-large-large): represents a high-concurrency, complex scenario with many tasks, many subtasks, and many machines. slm (small-large-medium): represents a scenario with a small number of tasks, but each task has very complex (large) subtasks, and a moderate number of machines.

[0182] like Figure 6-7As shown, the deep reinforcement learning-driven multi-task collaborative scheduling method ICSDQN proposed in this invention exhibits significantly lower GD values ​​than other algorithms in most test scenarios, especially demonstrating strong superiority in complex and large-scale task allocation. For example, in the `lml` scenario, the GD value of ICSDQN is only 188.78, a significant reduction compared to GA's 1090.69 and HDMDDQN's 793.92. The low GD value indicates that ICSDQN can get closer to the ideal solution, ensuring that the generated solution set maintains high accuracy and approximation in the target space. The reason behind this lies in ICSDQN's intelligent selection and dynamic scheduling strategy for heterogeneous resources, enabling it to converge to high-quality solutions more quickly in complex environments, laying the foundation for efficiency and accuracy in the scheduling process. IGD (Inverted Generational Distance): In terms of IGD, ICSDQN also demonstrates excellent solution set diversity and approximation. For example, in the `mll` scenario, ICSDQN's IGD is only 39.96, far lower than GA's 516.04 and HDMDDQN's 395.49, fully demonstrating ICSDQN's ability to cover the solution space and its continuous ability to approximate the optimal solution. The low IGD value not only demonstrates the breadth and uniformity of the solution set generated by ICSDQN, but also reflects its robustness in heterogeneous task scheduling. Through refined resource scheduling and in-depth resource utilization, ICSDQN can dynamically adapt to changing task loads and resource requirements, ensuring that the solution set always stays close to the ideal solution. In summary, the excellent performance of the ICSDQN algorithm in both GD and IGD metrics highlights its competitiveness and reliability in complex task scheduling scenarios. Its unique scheduling mechanism and intelligent resource allocation strategy enable ICSDQN to effectively reduce the risks of distance deviation and uneven solution distribution while maintaining the quality and diversity of the solution set. These advantages enable ICSDQN to demonstrate outstanding optimization performance and stable convergence characteristics in multi-objective optimization scheduling problems, especially in heterogeneous resource environments, and it has great potential for practical applications.

[0183] Example 3

[0184] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the deep reinforcement learning-driven multi-task cooperative scheduling method described in Embodiment 1 or 2.

[0185] Example 4

[0186] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the deep reinforcement learning-driven multi-task cooperative scheduling method described in Embodiment 1 or 2.

[0187] Example 5

[0188] A deep reinforcement learning-driven multi-task cooperative scheduling system includes:

[0189] The data acquisition module is configured to: collect the running data of the computing cluster, perform normalization processing, and convert it into a three-dimensional state tensor;

[0190] The priority scoring module is configured to: calculate a comprehensive priority score based on the information of the cloud computing tasks to be scheduled, and select the cloud computing task with the highest priority score as the current scheduling object;

[0191] The optimal computing node acquisition module is configured to: traverse the computing nodes in the computing cluster and generate a mask vector based on the current scheduling object;

[0192] A deep reinforcement learning network is constructed by inputting a 3D state tensor into the network for training, and an optimization algorithm is used to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are then selected using the trained deep reinforcement learning network, including:

[0193] The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy.

[0194] The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

Claims

1. A deep reinforcement learning-driven multi-task cooperative scheduling method, characterized in that, include: Step 1: Collect the running data of the computing cluster, normalize it, and convert it into a three-dimensional state tensor; Step 2: Based on the information of the cloud computing tasks to be scheduled, calculate the comprehensive priority score, select the cloud computing task with the highest priority score as the current scheduling object; Step 3: Traverse the compute nodes in the compute cluster and generate a mask vector based on the current scheduling object; Construct a deep reinforcement learning network by inputting a three-dimensional state tensor into the deep reinforcement learning network for training, and use an optimization algorithm to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are selected using a pre-trained deep reinforcement learning network, including: The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy. The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

2. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 1, characterized in that, Step 1 is implemented as follows: The monitoring agent deployed on the cloud control node collects real-time operational data from the heterogeneous cluster, including resource status data. and task attribute data ; Among them, resource status data Indicates data collection in heterogeneous clusters Real-time status of each computing node, including: CPU utilization Memory usage Disk I / O throughput and network bandwidth margin ; Task attribute data Static attributes representing the head task of the acquisition task receive queue include: instruction length. Data size Arrival time and deadline ; The collected runtime data is normalized as follows: ； in, This represents the normalized runtime data. Indicates runtime data, This represents the minimum value. Indicates the maximum value; The normalized runtime data is converted into a three-dimensional state tensor. ,in Indicates the number of compute nodes. Indicates the time step in history. Indicates the number of feature channels.

3. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 2, characterized in that, Step 2 is implemented as follows: For each task in the task queue that is to be scheduled in the cloud computing task queue Obtain task information and calculate the lag ratio, as shown below: ； in, Indicates the lag ratio, Indicates the waiting time. Indicates the current time stamp. Indicates the absolute deadline timestamp for the task; The task urgency index is calculated as follows: ； in, Indicates the urgency of the task. The estimated completion time is... The deadline is [date / time]. For the remaining workload; Calculate the potential latency of the task as follows: ； in, Indicates the potential latency rate. This indicates the estimated total workload of the task. This indicates the amount of work that has been completed so far. The calculated lag ratio, task urgency, and potential delay rate are normalized, including using Min-Max normalization for the lag ratio and potential delay rate. To determine task urgency, inverse normalization is used. ； in Indicates the urgency of the task after processing. This represents the maximum value indicating the urgency of the task. The minimum value representing the urgency of the task; Based on preset weights, the normalized indicators are weighted and summed to obtain a comprehensive priority score for each task. As shown below: ； in, , , For the preset weights, This represents the normalized lag ratio. This represents the normalized potential delay rate; Scoring based on overall priority The task queue is sorted from highest to lowest score, and the task with the highest score is selected as the current scheduling target.

4. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 3, characterized in that, Traverse the compute nodes in the compute cluster and generate a mask vector based on the currently scheduled object; including: Iterate through all computing nodes in the computing cluster to generate a binary mask vector. ;in, This represents the validity mask value of the j-th computing node. If the task, i.e., the selected scheduling object, times out on the j-th computing node, then... If the j-th computing node has insufficient remaining resources, then ;otherwise, 1.

5. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 4, characterized in that, The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computation node. include: Deep reinforcement learning networks are deep Q-networks, which use three-dimensional state tensors. Inputting the deep Q-network yields the raw output, i.e., the raw Q-value. And perform a correction operation using a mask vector to adjust the original Q value. The correction is as follows: ； in, This represents the original Q value. This represents the corrected Q-value vector; Get the current optimal computing node: ,in This represents the index of the optimal computing node.

6. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 5, characterized in that, Selecting a unique optimal computing node through a resource selection strategy; including: The resource selection strategy is one of the completion priority strategy, availability priority strategy, and high utilization priority strategy. Based on the scheduling goals of the cloud computing platform, the resource selection strategy is used to select a unique optimal computing node from multiple optimal computing nodes. If the cloud computing platform needs to complete the entire batch of tasks quickly, the completion priority strategy is used to select the computing node with the earliest completion time as the sole optimal computing node; as shown below; ； in, Indicates the completion time. This is the end time of the preceding task. The estimated execution time for the current task. Indicates the earliest execution time of the current task. The earliest available time for the compute node; select the completion time. The smallest computing node is the optimal computing node; If the cloud computing platform prioritizes response speed, then an availability-first strategy is used to select the compute node with the earliest available time as the sole optimal compute node; as shown below: ； in, Indicates available time; select available time. The smallest computing node is the optimal computing node; If the goal of a cloud computing platform is to save costs and improve resource utilization, then a high utilization priority strategy is used to select the optimal computing node; as shown below: ； in, Indicates utilization rate. This refers to the time that the machine, i.e., the computing node, has been used. For machine availability time, Indicates all that satisfy A set of tasks; selecting the utilization rate The largest computing node is the optimal computing node.

7. The deep reinforcement learning-driven multi-task cooperative scheduling method according to claim 6, characterized in that, Update the weight parameters using an optimization algorithm; including: An algorithm combining a progressive acceleration mechanism and a gradient update mechanism is used to update the weights of the deep reinforcement learning network. The update formula is as follows: ； in, Indicates the updated weights. This indicates the network parameters before the update. For the asymptotic acceleration term through the momentum coefficient Dynamically adjusted update stride, This represents the temporary target weight calculated based on the current batch of data. This indicates an update to the stride control factor; For gradient update terms, The learning rate decays over time. This represents the gradient vector of the loss function with respect to the current weights.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, The steps of implementing a deep reinforcement learning-driven multi-task cooperative scheduling method when the processor executes the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The steps of implementing a deep reinforcement learning-driven multi-task cooperative scheduling method when the computer program is executed by the processor.

10. A deep reinforcement learning-driven multi-task collaborative scheduling system, characterized in that, include: The data acquisition module is configured to: collect the running data of the computing cluster, perform normalization processing, and convert it into a three-dimensional state tensor; The priority scoring module is configured to: calculate a comprehensive priority score based on the information of the cloud computing tasks to be scheduled, and select the cloud computing task with the highest priority score as the current scheduling object; The optimal computing node acquisition module is configured to: traverse the computing nodes in the computing cluster and generate a mask vector based on the current scheduling object; Construct a deep reinforcement learning network by inputting a three-dimensional state tensor into the deep reinforcement learning network for training, and use an optimization algorithm to update the weight parameters to obtain a trained deep reinforcement learning network. The optimal computation nodes are selected using a pre-trained deep reinforcement learning network, including: The three-dimensional state tensor is input into the deep reinforcement learning network to obtain the original output. The original output is then corrected using a mask vector to obtain the optimal computing node. If there are multiple optimal computing nodes, a unique optimal computing node is selected through a resource selection strategy. The optimal computing node is used to execute the scheduling object, thus completing the task scheduling.

Images