A deep learning-based computing resource optimization method and system

Abstract

The application discloses a kind of based on deep learning's computing power resource optimization method and system, including following steps: acquisition node, task, link and instance data, generate computing power resource state dataset;State data is encoded to node state, task demand, link instance and scheduling constraint, obtain multidimensional scheduling state vector;Build computing power scheduling interactive environment;The multidimensional scheduling state vector is input into improved A3C multithreaded deep reinforcement learning resource scheduling model, generates candidate policy set through task adaptation state attention coding, asynchronous Actor grouping exploration, action mask, Critic value evaluation and resource constraint feedback correction;Optimized scheme is obtained after combination verification and conflict resolution and is executed;Based on the execution result, generate closed-loop reward feedback sample and update model online.The application belongs to the technical field of computing power resource scheduling, realizes computing power resource dynamic allocation, conflict inhibition and continuous optimization.

Description

Technical Field

[0001] This invention relates to the field of computing resource scheduling technology, specifically to a computing resource optimization method and system based on deep learning. Background Technology

[0002] With the rapid growth of artificial intelligence training, cloud computing services, edge computing applications, and large-scale data processing tasks, the task types, resource requirements, and operating status in the computing resource pool exhibit significant dynamic changes. Different tasks have varying degrees of dependence on CPU, GPU, memory, network links, and container instance resources. Some tasks have high concurrency, high GPU memory usage, or low latency execution requirements. If fixed rules, static thresholds, or manual configuration methods are still used for resource allocation, problems such as uneven node load, excessively long task waiting times, coexistence of idle and congested GPU resources, untimely instance scaling, and excessive overhead for cross-node migration can easily occur.

[0003] Existing computing resource scheduling methods typically make scheduling decisions based on task priority, remaining node resources, or preset load thresholds. While these methods can meet basic resource allocation needs, they struggle to simultaneously consider the interrelationships between task requirements, node status, link status, container instance status, and execution feedback. As the computing resource pool expands, task types increase, or node status changes frequently, traditional methods struggle to generate timely optimization strategies adapted to the current operating environment, easily leading to localized resource overload, increased task execution latency, and decreased system service stability.

[0004] Deep reinforcement learning can learn scheduling strategies through interaction with the environment, making it suitable for handling dynamic resource optimization problems. However, when ordinary A3C models are directly applied to computing power scheduling scenarios, they still suffer from problems such as insufficient state representation, excessive exploration of impossible actions, delayed feedback on resource constraints, and fixed reward weights. These issues lead to low model training efficiency, unstable scheduling strategies, and even resource conflicts during task migration and instance scaling.

[0005] Therefore, how to provide a method and system for optimizing computing resources based on deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a deep learning-based method and system for optimizing computing resources. This invention fully utilizes deep reinforcement learning, state attention encoding, action mask constraints, and closed-loop feedback update techniques. It details the processes of acquiring computing resource states, constructing multi-dimensional scheduling state vectors, establishing an interactive environment for computing resource scheduling, generating candidate strategies for the improved A3C multi-threaded deep reinforcement learning resource scheduling model, multi-task combination verification, scheduling execution, and online model updates. It can adaptively generate computing resource optimization schemes under dynamic changes in task requirements, node load, link status, and instance running status. It has the advantages of high resource utilization, stable task scheduling, minimal exploration of non-compliant actions, low cross-node scheduling conflicts, and strong continuous model optimization capabilities.

[0007] A method and system for optimizing computing resources based on deep learning according to an embodiment of the present invention includes the following steps: S1. Collect node, task, link and instance data, and obtain computing resource status dataset after synchronization, cleaning, correction and normalization; S2. Encode the node status, task requirements, link instances and scheduling constraints of the computing power resource status dataset to obtain a multi-dimensional scheduling status vector; S3. Construct a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, set up a scheduling action space and a reward feedback set, and obtain the training environment; S4. Input the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model, and obtain a set of candidate computing power resource optimization strategies through task adaptation state attention encoding, asynchronous Actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction. S5. Perform combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate set of computing power resource optimization strategies to obtain the computing power resource optimization scheme. S6. Execute node allocation, quota adjustment, task migration, instance scaling up and down, and queue rearrangement according to the computing power resource optimization plan to obtain the scheduling execution result; S7. Generate closed-loop reward feedback samples based on the scheduling execution results, and update and improve the A3C multi-threaded deep reinforcement learning resource scheduling model online to obtain the updated computing resource optimization model.

[0008] Optionally, step S2 includes the following steps: S21. Integrate the node's remaining computing power, memory resources, load fluctuations, and health status into a node state vector; S22. Integrate task computation amount, priority, waiting time, input data size and service level constraints into a task requirement vector; S23. Encode link bandwidth, transmission delay, congestion status, instance resource quota, and instance health status into a link-instance joint vector; S24. Combine the node state vector, task requirement vector, and link instance joint vector with scheduling constraints, and add resource upper limit, task latency, migration number and instance stability constraints to obtain a multi-dimensional scheduling state vector.

[0009] Optionally, step S3 includes the following steps: S31. Establish a state space based on a multi-dimensional scheduling state vector to represent computing power supply, task requirements, link transmission and instance running status; S32. Perform action coding on target node selection, resource quota adjustment, migration target selection, instance replica change, and resource limit adjustment to obtain a comprehensive action space; S33. Configure resource utilization rewards, task completion rewards, latency penalties, migration overhead penalties, load imbalance penalties, and service stability rewards to obtain the training environment.

[0010] Optionally, step S4 includes the following steps: S41. Construct an improved A3C multi-threaded deep reinforcement learning resource scheduling model. The model adopts a collaborative generation strategy of global shared parameters, asynchronous Actor threads, Critic value evaluation network, action mask pre-masking processing, resource constraint feedback correction processing, and reward weight correction processing. S42. Using the task requirement vector as the query object, calculate its matching weight with the node state vector and the link instance joint vector to obtain the task node adaptation features, task link transmission features and task instance carrying features. S43. The above features are integrated into scheduling decision features and input into multiple asynchronous Actor threads for parallel exploration to obtain multiple sets of candidate resource optimization actions; S44. Perform action masking pre-masking on multiple sets of candidate resource optimization actions, delete individual actions that violate resource, latency, link, migration count and instance health constraints, and obtain a set of feasible candidate actions; S45. Perform Critic value evaluation and resource constraint feedback correction on the set of feasible candidate actions, and adjust the action selection probability according to the action value and constraint risk to obtain a set of candidate computing power resource optimization strategies.

[0011] Optionally, step S43 includes the following steps: S431. Based on the task's pre-computational power consumption, video memory consumption, input data size, and maximum allowable latency, the task type classification results are obtained; S432. Based on the remaining node resources, load fluctuations, health status, temperature changes, and historical fault records, obtain the node status classification results; S433. Based on the task type division results and node status division results, allocate exploration sub-environments to different asynchronous Actor threads and generate corresponding candidate resource optimization actions; S434. Asynchronously aggregate the actions output by each asynchronous Actor thread, delete duplicate actions, mutual exclusion actions, and actions that conflict with the execution object, and obtain multiple sets of candidate resource optimization actions.

[0012] Optionally, S44 includes the following steps: S441. Generate a set of task constraints based on the task's minimum computing power requirements, maximum waiting time, maximum execution latency, data size, and allowed number of migrations; S442. Generate a set of available resources for the target node based on its remaining computing power, remaining memory, node health status, and number of tasks it can handle. S443. Match the candidate resource optimization actions with the task constraint set, the node available resource set, and the link status, and filter out individual actions such as insufficient resources, latency exceeding the limit, link congestion, migration number exceeding the limit, instance abnormality, and quota conflict to obtain a set of feasible candidate actions.

[0013] Optionally, S45 includes the following steps: S451. Evaluate the resource utilization, task latency, waiting time and node load changes after the execution of feasible candidate actions to obtain the basic value of the actions. S452. Calculate the resource consumption risk, migration overhead risk, link congestion risk and instance stability risk corresponding to feasible candidate actions to obtain the constraint risk value; S453. Using the constraint risk value as a value correction item and penalty item, the basic value of the action is corrected to obtain the action constraint correction value. S454. Based on the action constraint correction value, sort and filter feasible candidate actions to obtain a set of candidate computing power resource optimization strategies.

[0014] Optionally, step S5 includes the following steps: S51. Match the computing power consumption, memory consumption, and number of instance replicas after the combined execution of multiple candidate strategies with the remaining resource status of the computing power resource pool to obtain the combined resource verification result. S52. Delete candidate strategy combinations that cause continuous high load on a single node, widening load differences between nodes, cross-node latency exceeding limits, or failure to meet service level constraints, and obtain the combined correction result. S53. Identify conflict situations such as multiple tasks occupying the same resource quota, multiple migration actions pointing to the same node, and expansion / shrinkage actions and migration actions affecting each other, and obtain the combined conflict resolution results. S54. Based on the combined resource verification results, combined correction results, and combined conflict resolution results, a comprehensive screening is conducted to obtain a computing resource optimization scheme.

[0015] Optionally, step S7 includes the following steps: S71. Collect data on resource utilization, task latency, waiting time, migration overhead, node load fluctuation, instance health changes, and service stability to obtain a scheduling feedback dataset. S72. Compare the scheduling feedback dataset with the multi-dimensional scheduling state vector before scheduling to obtain changes in resource utilization, task latency, migration overhead, load balancing, and service stability. S73. Positive rewards are given to actions that improve resource utilization and reduce task latency, while penalties are given to actions that increase task latency, cause excessive migration overhead, cause load imbalance, and decrease instance stability, resulting in a composite reward outcome. S74. Perform reward weight correction processing based on composite reward results, and adjust the corresponding penalty weight according to the increase in latency, increase in migration overhead or decrease in stability during continuous scheduling cycles. S75. The composite reward result and penalty weight are fed back to the asynchronous Actor thread, the Critic value evaluation network and the global shared parameters to update the policy gradient, state value error, action mask parameters and resource constraint feedback parameters, thus obtaining the updated computing power resource optimization model.

[0016] A deep learning-based computing resource optimization system according to an embodiment of the present invention includes the following modules: The resource status acquisition module collects running data of computing nodes, task queue data, network link data and container instance data in the computing resource pool. It performs time synchronization, field cleaning, abnormal field correction and normalization on the collected data to generate a computing resource status dataset. The scheduling status encoding module performs node status encoding, task requirement encoding, link instance encoding, and scheduling constraint encoding on the computing resource status dataset to generate a multi-dimensional scheduling status vector. The scheduling environment construction module builds a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, sets up a scheduling action space and a reward feedback set, and generates a training environment. The A3C strategy generation module inputs the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model. Through task adaptation state attention encoding, asynchronous actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction, it generates a set of candidate computing power resource optimization strategies. The strategy screening and correction module performs combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate set of computing resource optimization strategies to generate a computing resource optimization scheme. The scheduling and execution module performs node allocation, quota adjustment, task migration, instance scaling, and queue rearrangement according to the computing resource optimization plan, and generates scheduling and execution results. The closed-loop update module generates closed-loop reward feedback samples based on the scheduling execution results, and updates and improves the A3C multi-threaded deep reinforcement learning resource scheduling model based on the closed-loop reward feedback samples, generating an updated computing resource optimization model.

[0017] The beneficial effects of this invention are as follows: This invention collects computing node operation data, task queue data, network link data, and container instance data, and performs synchronization, cleaning, correction, and normalization processing on the above data to form a unified computing resource status dataset. This enables the system to comprehensively reflect node resource reserves, task computing requirements, link transmission conditions, and instance operation status under the same status expression framework, thus solving the problems of scattered status sources and incomplete resource perception in existing scheduling methods.

[0018] This invention constructs a multi-dimensional scheduling state vector by encoding node status, task requirements, link status, instance operation, and scheduling constraints. It also uses a task adaptation state attention encoding layer to extract the matching relationship between tasks and nodes, links, and instances. This enables the improved A3C model to determine more suitable scheduling features based on the actual resource requirements of different tasks. Compared with the method of allocating only based on the remaining resources of nodes, this improves the matching accuracy between tasks and computing power nodes.

[0019] This invention utilizes an asynchronous Actor grouping exploration mechanism to allocate exploration sub-environments to different Actor threads based on task type and node status. This enables multiple Actor threads to learn resource allocation, task migration, and instance scaling strategies in parallel under different computing power scenarios. This reduces the adaptation pressure of a single-policy network to complex scheduling environments and improves the generation efficiency of candidate resource optimization actions and the coverage of scheduling strategies.

[0020] This invention pre-masks individual actions such as resource insufficiency, latency exceeding limits, link congestion, migration number exceeding limits, instance health abnormalities, and resource quota conflicts through an action masking layer. This reduces the invalid exploration of ineffective actions by the reinforcement learning model and avoids the scheduling policy from generating actions that obviously violate resource constraints before execution, thereby improving the feasibility and training stability of the policy generation process.

[0021] This invention calculates resource occupancy risk, migration overhead risk, link congestion risk, and instance stability risk through a resource constraint feedback layer, and uses the constraint risk value as a value correction term for the Critic value network and a penalty term for the reward feedback set. This reduces the selection probability of high-risk actions in value assessment and subsequent model updates, solving the problems of lagging resource constraint feedback and disconnect between policy updates and actual execution risks in ordinary A3C models.

[0022] This invention performs multi-task combination feasibility verification, global load balancing correction, cross-node latency constraint verification, and combination action conflict resolution on a set of candidate computing resource optimization strategies. It can identify resource competition, migration conflicts, and instance scaling conflicts that occur when multiple candidate strategies are executed simultaneously, thus avoiding the problem that individual actions may be feasible but cause global resource imbalance after combined execution, thereby improving the overall executability of computing resource optimization schemes.

[0023] This invention generates closed-loop reward feedback samples by collecting scheduling execution results, and adjusts the corresponding penalty weights according to the increase in latency, migration overhead, or stability within a continuous scheduling cycle. This enables the improved A3C multi-threaded deep reinforcement learning resource scheduling model to be continuously updated with the running status of the computing resource pool. Compared with the scheduling model with a fixed reward function, it improves the adaptive optimization capability and long-term service stability in dynamic task scenarios. Attached Figure Description

[0024] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a deep learning-based computing resource optimization method and system proposed in this invention; Figure 2 The diagram shows the structure of the improved A3C multi-threaded deep reinforcement learning resource scheduling model proposed in this invention. Figure 3 This is a flowchart of the closed-loop reward feedback and online model update proposed in this invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0026] refer to Figures 1-3 A method and system for optimizing computing resources based on deep learning, comprising the following steps: S1. Collect node, task, link and instance data, and obtain computing resource status dataset after synchronization, cleaning, correction and normalization; S2. Encode the node status, task requirements, link instances and scheduling constraints of the computing power resource status dataset to obtain a multi-dimensional scheduling status vector; S3. Construct a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, set up a scheduling action space and a reward feedback set, and obtain the training environment; S4. Input the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model, and obtain a set of candidate computing power resource optimization strategies through task adaptation state attention encoding, asynchronous Actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction. S5. Perform combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate set of computing power resource optimization strategies to obtain the computing power resource optimization scheme. S6. Execute node allocation, quota adjustment, task migration, instance scaling up and down, and queue rearrangement according to the computing power resource optimization plan to obtain the scheduling execution result; S7. Generate closed-loop reward feedback samples based on the scheduling execution results, and update and improve the A3C multi-threaded deep reinforcement learning resource scheduling model online to obtain the updated computing resource optimization model.

[0027] In this embodiment, S1 specifically includes: collecting node, task, link and instance data, and obtaining a computing resource status dataset after synchronization, cleaning, correction and normalization.

[0028] Node data reflects the real-time operating status and historical stability of each computing node in the computing resource pool. Node data includes the computing node number, node type, CPU resource reserve, GPU resource reserve, GPU memory reserve, memory reserve, storage reserve, current node load, node load fluctuation, node temperature, node health status, number of tasks carried by the node, historical fault records, and node availability status. Node types can include CPU-based nodes, GPU-based nodes, storage-enhanced nodes, and hybrid heterogeneous nodes. For nodes without GPUs, their GPU resource reserve and GPU memory reserve are marked as zero or unavailable to prevent subsequent models from misclassifying nodes without GPUs as target nodes capable of carrying GPU tasks. For nodes in maintenance, offline, fault isolation, or resource freeze states, the system marks their availability status as unschedulable, preventing them from entering the candidate scheduling object set.

[0029] Task data reflects the resource requirements and scheduling constraints of tasks pending or currently executing. Task data includes task number, task type, task priority, pre-computational power requirement, estimated CPU usage, estimated GPU memory usage, input data size, maximum allowable waiting time, maximum allowable execution latency, whether task migration is allowed, pre-waiting time, service level constraints, current task status, and the corresponding business queue. For training tasks, the system focuses on collecting estimated runtime, memory requirements, data read size, and whether interrupt recovery is allowed; for inference tasks, the system focuses on collecting request concurrency, response latency constraints, instance replica requirements, and short-term traffic changes; for batch processing tasks, the system focuses on collecting input data size, estimated processing time, and task deadline. Through this task data, the system can determine the dependence of different tasks on computing resources, link transmission, and instance capacity.

[0030] Link data reflects the data transmission conditions between computing nodes and between computing nodes and data storage nodes. Link data includes available bandwidth, transmission latency, packet loss, link congestion status, the node where the task data resides, the transmission path between the target execution node, cross-rack transmission identifiers, and cross-region transmission identifiers. Link status is used to determine whether task migration or cross-node execution will introduce excessive data transmission overhead. For tasks with large input data volumes, if the target node has sufficient remaining computing power but the link between it and the data node is severely congested, the system will not directly use that target node as the optimal execution node. Instead, it will consider the link transmission cost as an important factor in subsequent status coding and policy evaluation.

[0031] Instance data reflects the task-carrying capacity of container instances or service instances. Instance data includes the container instance ID, the node where the instance resides, the number of instance replicas, the instance resource quota, the instance startup time, the instance health status, the number of instance restarts, the number of tasks the instance carries, the instance interface response status, and the instance resource usage. Instance health status is determined based on instance heartbeats, restart counts, interface response status, and abnormal resource usage. Instances that do not return heartbeats for three consecutive sampling periods, restart more than twice within 10 minutes, or whose CPU or memory usage exceeds 90% of the quota limit for five consecutive minutes are marked as abnormal instances. Instance health status is used to determine whether the target node can stably carry the scheduled tasks, avoiding tasks being assigned to nodes with seemingly sufficient resources but unstable instance operation.

[0032] The system performs unified time synchronization processing on the above data, aligning data from different sources according to the same sampling period. In this embodiment, the sampling period is set to 10 seconds. For inference service scenarios with higher real-time requirements, the sampling period can be shortened to 5 seconds; for batch processing or offline training scenarios, the sampling period can be set to 30 seconds. The system aggregates node, task, link, and instance data in units of scheduling cycles, and performs alignment correction on data with large sampling time deviations. For missing fields, the system completes them using the most recent valid value, the average value of similar nodes, or an unavailable flag, based on the field type; for out-of-bounds fields, the system corrects them based on resource limits, operating status range, and node hardware configuration; for duplicate records, the system retains the record with the latest sampling time and the highest field completeness. After synchronization, cleaning, correction, and normalization processing, the system obtains a computing resource status dataset.

[0033] Normalization is used to ensure that data of different scales can enter the same encoding process. CPU resource reserves, GPU resource reserves, memory reserves, and storage reserves can be normalized according to the node hardware limits; task waiting time and task execution latency can be normalized according to the service level constraints of the corresponding tasks; link bandwidth and transmission latency can be normalized according to the link capacity range within the computing resource pool; instance health status can be converted into a health score, with a score range of 0 to 100. Through normalization, the system can reduce the impact of differences in data scales on model training and improve the stability of subsequent state encoding.

[0034] In this embodiment, S2 specifically includes: encoding the computing power resource status dataset with node status, task requirements, link instances and scheduling constraints to obtain a multi-dimensional scheduling status vector.

[0035] Node status coding is used to characterize whether a computing node is currently suitable for undertaking new tasks, migrating tasks, or expanding instances. The system integrates the node's remaining computing power, memory resources, GPU memory resources, load fluctuations, temperature changes, health status, and historical fault information into a node status vector. The node's remaining computing power includes the number of remaining CPU cores, the remaining GPU computing power, the GPU memory allowance, and the available memory capacity. Node load fluctuations are determined by the magnitude of resource usage changes over six consecutive sampling periods. When the CPU, GPU, or memory utilization rate is consistently higher than 85% and does not decrease for three consecutive sampling periods, the node is classified as a high-load node; when the node temperature exceeds 80℃, the health status score is lower than 70 points, or the number of faults exceeds three in the past 24 hours, the node is classified as a risk node; when the node resource utilization rate is between 40% and 85% and the health status is normal, the node is classified as a medium-load node; when the node resource utilization rate is lower than 40% and there are no abnormal health records, the node is classified as a low-load node. By using node status coding, the system can identify whether node resources are truly available, thus avoiding assigning tasks to nodes with large load fluctuations or declining health.

[0036] Task requirement encoding is used to characterize the actual requirements of the task to be processed for computing resources, execution latency, and link transmission. The system integrates task computation volume, priority, waiting time, input data size, maximum allowable latency, expected GPU memory requirements, whether migration is allowed, and service level constraints into a task requirement vector. Tasks can be classified according to expected CPU utilization, expected GPU memory utilization, input data size, and maximum allowable latency. When the expected CPU utilization of a task exceeds 70% and the expected GPU memory utilization is less than 30%, it is classified as a compute-intensive task; when the expected GPU memory utilization of a task exceeds 60% of the target node's memory capacity, it is classified as a memory-intensive task; when the task's input data size exceeds 10GB or the cross-node read ratio exceeds 50%, it is classified as a storage access-intensive task; when the task's maximum allowable waiting time is less than 30 seconds or the response latency requirement is less than 1 second, it is classified as a latency-sensitive task. If a task meets multiple type conditions simultaneously, the system retains primary and secondary type tags, enabling subsequent asynchronous Actor threads to explore based on different resource bottlenecks.

[0037] Link instance encoding characterizes the transmission conditions between the data location and the target execution node, as well as whether the instance on the target node has the capacity to handle the load. Link status reflects whether task data can be transmitted in a timely manner, while instance status reflects whether the task can be stably handled; both together determine the execution reliability of the task after cross-node scheduling. Therefore, this invention encodes link bandwidth, transmission latency, link congestion status, instance resource quota, number of instance replicas, and instance health status into a joint link instance vector. If the available link bandwidth is lower than the task transmission requirements, the transmission latency exceeds the task service level constraints, or the target instance is in an abnormal state, it will not be directly identified as the best scheduling target even if the target node has sufficient remaining computing power. The joint link instance vector enables subsequent models to simultaneously consider network transmission conditions and instance handling conditions, reducing the problem of task execution instability after cross-node migration.

[0038] Scheduling constraint encoding is used to transform resource limits, task latency, migration counts, instance stability, and service level requirements into model-recognizable constraint features. Resource limit constraints include the upper limits of node CPU, GPU, memory, video memory, and storage resources; task latency constraints include maximum wait time, maximum execution latency, and response latency requirements; migration count constraints include whether a task is allowed to migrate, the maximum number of migrations, and the maximum migration pause duration; instance stability constraints include instance health score, instance restart count, and the range of instance replicas; and service level constraints include task priority, completion deadline, and business queue requirements. The system fuses the node state vector, task demand vector, and link instance joint vector, and adds the above scheduling constraint features to obtain a multi-dimensional scheduling state vector. This multi-dimensional scheduling state vector serves as input to the subsequent training environment and the improved A3C model, enabling the model to simultaneously perceive resource supply, task demand, network status, instance status, and execution constraints.

[0039] In this embodiment, S3 specifically includes: constructing a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, setting a scheduling action space and a reward feedback set, and obtaining a training environment.

[0040] The state space consists of multi-dimensional scheduling state vectors, reflecting the overall operational status of the current computing resource pool. This state space includes not only the resource availability of individual nodes, but also the link transmission status between nodes, the health status of container instances, the congestion level of task queues, and task service level constraints. Through this state space, the model can identify complex scheduling scenarios such as nodes appearing idle but with congested links, nodes having sufficient resources but instances exhibiting abnormalities, tasks with low priority but excessively long waiting times, sufficient GPU resources but insufficient VRAM, and sufficient CPU resources but frequent container instance restarts. Compared to scheduling methods that only use remaining node resources as input, the state space constructed in this invention has a more complete resource awareness capability.

[0041] The action space includes resource allocation actions, task migration actions, and elastic scaling actions. Resource allocation actions include selecting target nodes for pending tasks, adjusting CPU quotas, GPU quotas, memory quotas, video memory quotas, and storage quotas. Task migration actions include identifying tasks to be migrated, identifying source nodes, identifying target nodes, determining migration trigger conditions, and determining the migration execution order. Elastic scaling actions include adding instance replicas, reducing instance replicas, adjusting instance resource limits, adjusting task queue entry rates, and adjusting service instance carrying relationships. The system uniformly encodes these actions into a comprehensive action space, enabling the model to select resource allocation, migration, or scaling operations within the same decision framework.

[0042] The reward feedback set includes resource utilization rewards, task completion rewards, latency penalties, migration overhead penalties, load imbalance penalties, and service stability rewards. When a scheduling action improves resource utilization and reduces task completion latency, the system generates a positive reward; when a scheduling action leads to increased task waiting time, excessive migration overhead, node load imbalance, or decreased instance stability, the system generates a penalty feedback. Resource utilization rewards encourage the model to reduce idle computing resources; task completion rewards encourage the model to prioritize tasks with longer waiting times or higher service levels; latency penalties suppress task execution delays; migration overhead penalties prevent frequent migrations that waste resources; load imbalance penalties prevent tasks from concentrating on a few nodes; and service stability rewards encourage the model to select healthy instances and stable links. Through this reward feedback set, the model can learn the impact of different scheduling actions on resource utilization, task latency, and service stability through multiple interactions.

[0043] The training environment simulates the state changes after scheduling actions are executed. Whenever the improved A3C multi-threaded deep reinforcement learning resource scheduling model outputs an action, the training environment updates the node resource availability, task queue status, link occupancy status, and instance replica status based on the current computing resource status, action type, and execution constraints, and generates corresponding reward feedback. For already deployed and running computing resource pools, the training environment can also receive actual scheduling execution results, enabling the model to be updated online based on real feedback. Therefore, the training environment supports both offline pre-training and continuous online optimization.

[0044] In this embodiment, S4 specifically includes: inputting the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model, and obtaining a set of candidate computing power resource optimization strategies through task adaptation state attention encoding processing, asynchronous Actor group exploration processing, action mask pre-masking processing, Critic value evaluation processing, and resource constraint feedback correction processing.

[0045] An improved A3C multi-threaded deep reinforcement learning resource scheduling model employs globally shared parameters, multiple asynchronous Actor threads, and a Critic value evaluation network working collaboratively. Multiple asynchronous Actor threads generate candidate resource optimization actions in different exploration sub-environments, while the Critic value evaluation network evaluates the value of these actions. The globally shared parameters are used to receive updates from each asynchronous Actor thread, enabling the model to learn scheduling strategies in parallel for different task types and node states. Ordinary A3C models tend to generate a large number of inactive actions when facing computing power scheduling scenarios and struggle to distinguish scheduling differences under different task types and node state combinations. This invention improves the A3C model by enabling it to possess task adaptation state awareness before action generation, group exploration capabilities during action generation, inactive action masking capabilities before action evaluation, and resource constraint feedback and correction capabilities during action evaluation.

[0046] The task adaptation state attention encoding process uses the task requirement vector as the query object, calculating the matching weights between the task requirement vector and the node state vector and the joint vector of link instances, respectively, to obtain task node adaptation features, task link transmission features, and task instance carrying features. Task node adaptation features reflect the degree of matching between task resource requirements and the remaining resources of the node; task link transmission features reflect the cost of transmitting task data to the target node; and task instance carrying features reflect whether the container instance on the target node can stably carry the task. The system integrates these features into scheduling decision features and inputs them into multiple asynchronous Actor threads. Through this processing, the model no longer only focuses on the remaining resources of the node, but dynamically focuses on the node, link, and instance information most relevant to the task based on the current task requirements.

[0047] Asynchronous Actor grouping and exploration processes allocate exploration sub-environments based on task type and node state classifications. Task type classifications include computationally intensive tasks, memory-intensive tasks, storage access-intensive tasks, and latency-sensitive tasks; node state classifications include low-load nodes, medium-load nodes, high-load nodes, and risky nodes. For computationally intensive tasks, asynchronous Actor threads prioritize exploring nodes with high CPU resource availability and low load fluctuations; for memory-intensive tasks, they prioritize nodes with sufficient GPU memory availability and stable temperatures; for storage access-intensive tasks, they prioritize nodes close to the task data location, with high link bandwidth and low transmission latency; for latency-sensitive tasks, they prioritize nodes with low link latency, stable instance health, and short task queues. Different asynchronous Actor threads handle the action generation process under different combinations of task types and node states, reducing the adaptability pressure of a single-policy network to complex scenarios.

[0048] Each asynchronous Actor thread generates resource allocation actions, task migration actions, and elastic scaling actions within its corresponding exploration sub-environment, and writes these actions to a local action cache. The system asynchronously aggregates the actions output by multiple asynchronous Actor threads, removing duplicate actions, mutual exclusion actions, and actions that conflict with execution objects, resulting in multiple sets of candidate resource optimization actions. Duplicate actions refer to multiple Actor threads providing identical resource allocation results for the same task; mutual exclusion actions refer to the same task being assigned to two different nodes simultaneously; and actions that conflict with execution objects refer to multiple actions simultaneously modifying the number of replicas of the same instance or the same resource quota. Through asynchronous aggregation, the system can preserve the strategy diversity brought by multi-threaded exploration while reducing the number of obviously conflicting actions entering subsequent processing stages.

[0049] Action masking pre-masking is used to remove obviously infeasible individual actions before they enter the value evaluation stage. The system generates a task constraint set based on the task's minimum computing power requirements, maximum waiting time, maximum execution latency, data size, and allowed migration count; it generates a node available resource set based on the target node's remaining computing power, remaining memory, remaining GPU memory, node health status, and the number of tasks it can handle; then, it matches candidate resource optimization actions with the task constraint set, node available resource set, and link status. When a candidate action has insufficient resources, latency exceeding limits, link congestion, migration count exceeding limits, instance anomalies, or quota conflicts, the system masks the action, resulting in a set of feasible candidate actions.

[0050] Action masking parameters include masking conditions corresponding to resource margin constraints, task latency constraints, link congestion constraints, migration count constraints, and instance health constraints. Resource margin constraints mask actions with insufficient target node resources; task latency constraints mask actions expected to cause task waiting or execution latency exceeding service level constraints; link congestion constraints mask actions with excessively high cross-node transmission costs; migration count constraints mask actions exceeding the allowed number of migrations; and instance health constraints mask actions from abnormal or unstable target instances. Through action masking pre-masking, the model can reduce the proportion of ineffective actions entering the training and execution phases, decrease invalid explorations, and improve training efficiency and policy feasibility.

[0051] The Critic value assessment process evaluates changes in resource utilization, task latency, waiting time, and node load after the execution of feasible candidate actions, yielding the basic value of the action. The resource constraint feedback correction process calculates the resource occupancy risk, migration overhead risk, link congestion risk, and instance stability risk corresponding to feasible candidate actions, obtaining constraint risk values. Resource constraint feedback parameters include weights for resource occupancy risk, migration overhead risk, link congestion risk, and instance stability risk. The system uses the constraint risk values ​​as value correction and penalty terms to adjust the basic value of the action, resulting in the action constraint-corrected value. A higher constraint risk value results in a lower selection weight for the corresponding action in the Actor's output action distribution; a lower constraint risk value and a higher Critic value result in a higher retention priority for the corresponding action.

[0052] For actions that improve resource utilization but have excessive migration overhead, the system reduces their action constraint correction value; for actions that reduce latency but decrease instance stability, the system adds stability penalty feedback; for actions that improve resource utilization, reduce task latency, have controllable migration overhead, and improve instance stability, the system increases their selection probability. Through this processing, resource constraints are no longer just external verification conditions before execution, but directly affect the action value evaluation and subsequent strategy updates in the A3C model, enabling the model to gradually reduce the output probability of high-risk actions. Thus, the system obtains a set of candidate computing resource optimization strategies.

[0053] In this embodiment, S5 specifically includes: performing combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate computing resource optimization strategy set to obtain a computing resource optimization scheme.

[0054] Action masking pre-masking primarily addresses the feasibility of individual actions, while S5 further addresses the global consistency issue when multiple candidate strategies are executed in combination. The system matches the computational power consumption, memory consumption, GPU memory consumption, storage consumption, and instance replica count after executing multiple candidate strategies in combination with the remaining resource status of the computational resource pool to obtain a combined resource verification result. When multiple candidate strategies are individually feasible but would occupy the same resource quota on the same node after simultaneous execution, the system identifies this as a combined resource conflict. If the combined resource verification result shows insufficient remaining resources on the target node, the system deletes candidate strategies with lower priority, shorter waiting times, or smaller migration benefits, prioritizing resource allocation to high-priority tasks or latency-sensitive tasks.

[0055] The system performs global load balancing correction on the differences in node load distribution and load change trends after the execution of candidate strategy combinations. If a candidate strategy combination leads to a single node consistently high load, a continued widening of load differences between nodes, risky nodes continuing to carry high-priority tasks, or low-load nodes remaining unutilized for an extended period, the system adjusts the priority of the candidate strategy combination. Global load balancing correction not only focuses on whether individual nodes have sufficient resources but also on the load distribution of the entire computing resource pool, ensuring that the scheduling results do not result in a state of local congestion or resource idleness coexisting.

[0056] The system verifies cross-node latency, determining whether task waiting time, task execution latency, data transmission latency, and migration pause duration meet task service level constraints. For latency-sensitive tasks, if a candidate strategy combination introduces a long cross-node transmission path, the system lowers the priority of that strategy combination. For storage access-intensive tasks, if the target node is far from the data node and the link is congested, the system prioritizes nodes closer to the data location. For batch processing tasks that allow delayed execution, the system can postpone their execution to periods of resource idleness, thereby freeing up resources for high-priority tasks.

[0057] The system further identifies conflicts such as multiple tasks occupying the same resource quota, multiple migration actions pointing to the same target node, instances scaling up / down actions interfering with task migration actions, queue reordering actions changing the execution order of high-priority tasks, and multiple scaling actions causing excessive instantaneous resource consumption on nodes. For conflicts involving multiple tasks occupying the same resource quota, the system determines the reserved actions based on task priority, waiting time, and service level constraints. For conflicts involving multiple migration actions pointing to the same target node, the system reallocates migration targets based on the remaining resources and link status of the target node. For conflicts involving instances scaling up / down actions interfering with task migration actions, the system prioritizes actions that ensure service stability. After combined feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution, the system obtains a computing resource optimization scheme that meets the execution conditions.

[0058] In this embodiment, S6 specifically includes: performing node allocation, quota adjustment, task migration, instance scaling and queue rearrangement according to the computing power resource optimization scheme to obtain the scheduling execution result.

[0059] The computing resource optimization scheme includes the mapping between tasks and target nodes, resource quota adjustments, a migration task list, instance scaling instructions, and queue rearrangement results. The system determines the target computing nodes for tasks to be processed based on the optimization scheme and writes the task identifier, target node identifier, and resource quota into the scheduling execution queue. For newly entered tasks, the system allocates CPU, GPU, memory, video memory, and storage resources based on task requirements and target node status. For tasks that are currently executing but have increased node load, the system determines whether to perform task migration based on migration overhead and link status. For services with continuously increasing request volume, the system performs instance scaling up based on instance health status and remaining resources; for services with decreasing request volume and low instance resource usage, the system performs instance scaling down.

[0060] Node allocation determines the actual execution location of a task. The system allocates tasks to target nodes based on a computing resource optimization scheme and writes the target node identifier, the task's required resource quota, and the task's execution priority into the resource management component. Quota adjustment adjusts the upper limits of CPU, GPU, memory, video memory, and storage resources according to task requirements. If the task is memory-intensive, the system prioritizes ensuring GPU video memory quotas; if the task is compute-intensive, the system prioritizes ensuring CPU or GPU computing resources; if the task is latency-sensitive, the system prioritizes ensuring low-load nodes and low-latency links.

[0061] Task migration is used to transfer running tasks from a source node to a target node. Before migration, the system records the task status, task context, input / output paths, and resource usage information of the source node, and determines whether the target node meets the runtime environment requirements. During migration, the system controls the pause of tasks on the source node, context replication, startup of the target node environment, and resumption of task execution. After migration, the system releases the resources occupied by the source node and updates the node identifier where the task resides. For tasks that are not allowed to migrate or whose migration cost is too high, the system does not perform the migration action, but instead alleviates the load pressure by adjusting other tasks, queue order, or the number of instance replicas.

[0062] Instance scaling is used to adjust the number of replicas and resource limits of service instances. During scaling up, the system determines the number of new replicas based on the remaining resources of the target node, instance startup time, and changes in service traffic. During scaling down, the system prioritizes releasing instances with low load, normal health status, and no high-priority tasks. After instance scaling up or down is complete, the system verifies instance heartbeats, interface response status, and resource usage to confirm that the scaling action has taken effect.

[0063] Queue reordering adjusts the task execution order based on task priority, waiting time, service level constraints, available node resources, and estimated execution time. High-priority tasks with waiting times nearing their upper limit are prioritized for execution; tasks with lower resource requirements and faster completion times are inserted into idle resource windows; tasks requiring large amounts of data to be transferred across nodes are executed after link congestion delays or moved to computing nodes closer to the data node; tasks with lower service level constraints and permissible delays can be postponed to avoid impacting high-priority tasks. Through queue reordering, the system can reduce the average task waiting time and minimize resource fragmentation.

[0064] After scheduling is completed, the system records the node allocation results, quota adjustment results, task migration results, instance scaling results, and queue rearrangement results, thus obtaining the scheduling execution result. The scheduling execution result includes whether the task was successfully allocated, the actual executing node, changes in resource quotas, migration time, changes in instance replicas, changes in task waiting time, changes in task execution latency, and changes in node load. This scheduling execution result is not only used to output the optimization effect of this round of computing resources but also serves as the data foundation for the next round of online model updates.

[0065] In this embodiment, S7 specifically includes: generating closed-loop reward feedback samples based on the scheduling execution results, and updating and improving the A3C multi-threaded deep reinforcement learning resource scheduling model online to obtain the updated computing power resource optimization model.

[0066] The system collects data on resource utilization, task completion latency, task waiting time, migration overhead, node load fluctuations, instance health changes, and service stability after scheduling execution, resulting in a scheduling feedback dataset. Resource utilization includes CPU utilization, GPU utilization, GPU memory utilization, memory utilization, and storage utilization; task completion latency includes the total time from task entry into the queue to completion; task waiting time includes the queuing time of the task; migration overhead includes task pause time, context copying time, data transfer time, and target node recovery time; node load fluctuations include changes in node resource usage before and after scheduling; and service stability includes instance health score, number of restarts, and interface response success rate. The system compares the scheduling feedback dataset with the multi-dimensional scheduling state vector before scheduling to obtain changes in resource utilization, task latency, migration overhead, load balancing, and service stability.

[0067] When a scheduling action improves resource utilization and reduces task latency, the system assigns a positive reward to the corresponding action. When a scheduling action leads to increased task latency, excessive migration overhead, node load imbalance, or decreased instance stability, the system assigns a penalty, resulting in a composite reward. For actions that improve resource utilization but significantly increase task latency, the system does not only award a high reward for improved resource utilization but also considers latency penalties for a comprehensive evaluation. For actions that reduce task latency but have excessive migration overhead, the system reduces their long-term selection value. For actions that improve short-term resource utilization but decrease instance health scores, the system increases stability penalties to reduce similar actions in subsequent scheduling.

[0068] The system performs reward weight adjustment processing based on the composite reward results. When the average task latency increases over three consecutive scheduling cycles, the system increases the latency penalty weight; when migration overhead increases over three consecutive scheduling cycles, the system increases the migration penalty weight; when resource utilization improves but instance health status score decreases, the system increases the stability penalty weight; when the load difference between nodes continuously widens, the system increases the load imbalance penalty weight. Through reward weight adjustment processing, the model can avoid long-term bias towards a single resource utilization indicator, while neglecting task latency, migration cost, and service stability.

[0069] The system feeds back the composite reward results and penalty weights to the asynchronous Actor thread, the Critic value evaluation network, and globally shared parameters to update the policy gradient, state value error, action mask parameters, and resource constraint feedback parameters, resulting in an updated computing resource optimization model. Updating the action mask parameters includes adjusting the masking conditions corresponding to resource availability constraints, latency constraints, link congestion constraints, migration count constraints, and instance health constraints. Updating the resource constraint feedback parameters includes adjusting the resource occupancy risk weight, migration overhead risk weight, link congestion risk weight, and instance stability risk weight. When the system detects that a certain type of action frequently causes conflicts or leads to a decrease in service stability during the execution phase, the corresponding risk weight is increased, reducing the probability of selecting that type of action in subsequent action value evaluations. The updated model continues to be used for candidate policy generation in the next scheduling cycle, achieving a continuous closed loop in the computing resource optimization process.

[0070] This embodiment also provides a deep learning-based computing resource optimization system that executes the above-described method. The system includes a resource status acquisition module, a scheduling status encoding module, a scheduling environment construction module, an A3C policy generation module, a policy filtering and correction module, a scheduling execution module, and a closed-loop update module.

[0071] The resource status acquisition module collects node runtime data, task queue data, network link data, and container instance data from the computing resource pool, and performs synchronization, cleaning, correction, and normalization processing to generate a computing resource status dataset. This module can connect to resource management platforms, container orchestration platforms, task queue systems, and network monitoring components to acquire multi-source runtime data according to a unified sampling period. The scheduling status encoding module encodes node status, task requirements, link instances, and scheduling constraints in the computing resource status dataset, generating a multi-dimensional scheduling status vector. The scheduling environment construction module builds a computing resource scheduling interaction environment based on the multi-dimensional scheduling status vector, sets the scheduling action space and reward feedback set, and generates a training environment.

[0072] The A3C strategy generation module inputs a multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model. Through task adaptation state attention encoding, asynchronous Actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction, it generates a set of candidate computing resource optimization strategies. The strategy selection and correction module performs feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate strategy set to generate a computing resource optimization scheme. The scheduling execution module executes node allocation, quota adjustment, task migration, instance scaling, and queue rearrangement according to the computing resource optimization scheme, generating scheduling execution results. The closed-loop update module generates closed-loop reward feedback samples based on the scheduling execution results and updates the improved A3C multi-threaded deep reinforcement learning resource scheduling model based on these samples, resulting in an updated computing resource optimization model.

[0073] The modules described above have clearly defined data transfer relationships. The resource status acquisition module outputs a dataset of computing resource status to the scheduling status encoding module; the scheduling status encoding module outputs a multi-dimensional scheduling status vector to the scheduling environment construction module and the A3C policy generation module; the scheduling environment construction module outputs training environment parameters to the A3C policy generation module; the A3C policy generation module outputs a set of candidate computing resource optimization policies to the policy screening and correction module; the policy screening and correction module outputs the computing resource optimization scheme to the scheduling execution module; the scheduling execution module outputs the scheduling execution result to the closed-loop update module; and the closed-loop update module outputs the updated model parameters and sends them back to the A3C policy generation module. Through the collaboration of these modules, the system can form a closed-loop process from status acquisition, status encoding, policy generation, policy verification, scheduling execution to model update.

[0074] Example 1: To verify the feasibility of this invention in practice, it was applied to an intelligent scheduling scenario of a computing resource pool in a certain region. This computing resource pool includes general-purpose CPU nodes, GPU accelerated nodes, memory-enhanced nodes, and containerized inference nodes, mainly undertaking tasks such as model training, batch data processing, online inference, image analysis, and periodic report generation. In this scenario, different tasks have significantly different dependencies on computing power, GPU memory, memory, link transmission, and execution latency. Traditional scheduling methods based on fixed thresholds tend to allocate high-memory tasks to nodes with insufficient GPU memory, or continue to push low-priority tasks onto some GPU nodes when they are under high load, resulting in increased task waiting time, uneven node load, and delayed instance scaling.

[0075] In this embodiment, node operation data, task queue data, network link data, and container instance data are first collected from the computing resource pool. Node operation data includes CPU utilization, GPU utilization, VRAM availability, RAM availability, node temperature, load fluctuation, and health status; task queue data includes task type, task priority, pre-computing power consumption, input data size, maximum allowable latency, and waiting time; network link data includes inter-node bandwidth, transmission latency, and link congestion status; container instance data includes the number of instance replicas, instance resource quota, instance health status, and the number of tasks carried by the instance. After time synchronization, field cleaning, abnormal field correction, and normalization, the above data forms a unified computing resource status dataset.

[0076] During the state representation phase, the system integrates node remaining computing power, memory resources, load fluctuations, and health status into a node state vector; integrates task computation volume, priority, waiting time, input data size, and service level constraints into a task requirement vector; and encodes link bandwidth, transmission latency, congestion status, instance resource quotas, and instance health status into a link-instance joint vector. Furthermore, resource limits, task latency, migration counts, and instance stability constraints are added to these vectors to obtain a multi-dimensional scheduling state vector. This multi-dimensional scheduling state vector can simultaneously reflect the fit between task requirements and node carrying capacity, as well as the link transmission risks and instance stability risks that may arise from cross-node scheduling.

[0077] During the model strategy generation phase, the system inputs a multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model. This model uses the task requirement vector as the query object, calculating the matching weights between tasks and node states, and link instance states, forming task node adaptation features, task link transmission features, and task instance carrying features. Subsequently, multiple asynchronous Actor threads conduct parallel explorations for different task types and different combinations of node states, generating candidate resource optimization actions such as resource allocation, task migration, and instance scaling. Through this processing, high-memory tasks are preferentially matched with GPU nodes that have sufficient memory reserves and stable link states, while latency-sensitive tasks are preferentially matched with low-load nodes that are in good health.

[0078] During the action selection phase, the system performs action masking pre-masking on candidate resource optimization actions, masking individual actions such as insufficient resources, latency exceeding limits, link congestion, migration count exceeding limits, instance anomalies, and quota conflicts, resulting in a set of feasible candidate actions. Subsequently, the system performs Critic value evaluation and resource constraint feedback correction on feasible candidate actions, using resource occupancy risk, migration overhead risk, link congestion risk, and instance stability risk as value correction items and reward / penalty items to reduce the selection probability of high-risk actions, thus obtaining a set of candidate computing power resource optimization strategies.

[0079] Before executing a strategy, the system performs a global check on the resource consumption, number of instance replicas, cross-node latency, and service level constraints after executing multiple candidate strategies in combination. It also identifies combination conflicts such as multiple tasks occupying the same resource quota, multiple migration actions pointing to the same node, and scaling up and down actions affecting each other. Through combination feasibility checks and conflict resolution, the system obtains the final computing resource optimization scheme and executes node allocation, quota adjustment, task migration, instance scaling up and down, and queue rearrangement accordingly.

[0080] During the closed-loop update phase, the system collects data on resource utilization, task completion latency, waiting time, migration overhead, node load fluctuations, instance health changes, and service stability after scheduling execution, and compares this data with the multi-dimensional scheduling state vector before scheduling. For actions that improve resource utilization and reduce task latency, the system assigns positive rewards; for actions that increase latency, cause excessive migration overhead, lead to load imbalance, or decrease instance stability, the system assigns penalty feedback. When increased latency, increased migration overhead, or decreased stability occur within consecutive scheduling cycles, the system adjusts the penalty weights accordingly and sends the composite reward result back to the asynchronous Actor thread, the Critic value evaluation network, and globally shared parameters, thereby obtaining the updated computing resource optimization model.

[0081] To verify the actual effectiveness of the scheme, this embodiment sets up three scheduling methods for comparison: conventional threshold scheduling, ordinary A3C scheduling, and the method of the present invention. Conventional threshold scheduling mainly allocates tasks based on the remaining resources of nodes and a preset load threshold; ordinary A3C scheduling uses reinforcement learning to generate resource allocation actions, but does not introduce task-adaptive state attention encoding processing, action mask pre-masking processing, resource constraint feedback correction processing, and reward weight correction processing; the method of the present invention adopts the complete technical process described in the claims. Test samples include training tasks, inference tasks, image analysis tasks, and batch processing tasks, covering small-batch low-latency tasks, medium-computation tasks, and high-memory tasks.

[0082] Table 1 Comparison of computing resource optimization effects under different scheduling methods

[0083] As shown in Table 1, the method of this invention significantly outperforms conventional threshold scheduling and ordinary A3C scheduling methods in terms of both average GPU utilization and average CPU utilization. This is because this invention does not simply allocate tasks based on the remaining resources of nodes, but rather encodes task requirements, node states, link instance states, and scheduling constraints into a unified multi-dimensional scheduling state vector. Furthermore, it establishes a matching relationship between tasks and nodes, links, and instances through task adaptation state attention encoding. Therefore, high-memory tasks, low-latency tasks, and batch processing tasks can be assigned to more suitable nodes, reducing the situation where some nodes are overloaded while others are idle.

[0084] Regarding task waiting time and task P95 completion latency, the method of this invention reduces these to 92s and 305s respectively, demonstrating its effective reduction in average waiting pressure and tail latency. While the conventional A3C scheduling method can generate scheduling actions through reinforcement learning, the lack of action masking pre-masking means the model still explores actions with insufficient resources, link congestion, or high migration overhead, requiring more trial and error for the scheduling strategy to converge. The method of this invention pre-masks inoperable actions and utilizes resource constraint feedback correction to reduce the probability of selecting high-risk actions, making the strategy generation process more stable.

[0085] In terms of the number of cross-node migrations and migration rollbacks, the method of this invention reduces them to 49 and 3 respectively, indicating that the method can reduce unnecessary cross-node migrations and avoid migration rollbacks caused by insufficient resources of the target node, poor link transmission conditions, or abnormal instance status. This result demonstrates the role of resource constraint feedback correction processing, which introduces resource occupancy risk, migration overhead risk, link congestion risk, and instance stability risk into the Critic value assessment process, ensuring that the action value reflects not only short-term resource utilization but also post-execution stability.

[0086] The method of this invention also outperforms in terms of high-load node duration, number of instance abnormal restarts, and service stability score. This result demonstrates that multi-task combination feasibility verification, global load balancing correction, and action conflict resolution can identify potential resource contention and scaling conflicts that may arise when multiple candidate strategies are combined before policy execution, avoiding the problem of individual actions being feasible but causing global load imbalance after combined execution. Closed-loop reward feedback samples and reward weight correction further enable the model to continuously adjust penalty weights based on actual execution deviations, thereby improving long-term operational stability.

[0087] As can be seen from the data in Table 1, the method of this invention improves state representation capability through task-adaptive state attention encoding, enhances candidate action generation efficiency through asynchronous Actor grouping exploration, reduces the proportion of inactive actions through action masking pre-masking, improves the accuracy of action value assessment through resource constraint feedback correction, and enhances the model's online adaptability through closed-loop reward feedback and reward weight correction. Compared with conventional threshold scheduling and ordinary A3C scheduling, this invention can achieve higher resource utilization, lower task latency, fewer migration rollbacks, and better service stability under dynamic task loads.

[0088] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing computing resources based on deep learning, characterized in that, Includes the following steps: S1. Collect node, task, link and instance data, and obtain computing resource status dataset after synchronization, cleaning, correction and normalization; S2. Encode the node status, task requirements, link instances and scheduling constraints of the computing power resource status dataset to obtain a multi-dimensional scheduling status vector; S3. Construct a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, set up a scheduling action space and a reward feedback set, and obtain the training environment; S4. Input the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model, and obtain a set of candidate computing power resource optimization strategies through task adaptation state attention encoding, asynchronous Actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction. S5. Perform combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate set of computing power resource optimization strategies to obtain the computing power resource optimization scheme. S6. Execute node allocation, quota adjustment, task migration, instance scaling up and down, and queue rearrangement according to the computing power resource optimization plan to obtain the scheduling execution result; S7. Generate closed-loop reward feedback samples based on the scheduling execution results, and update and improve the A3C multi-threaded deep reinforcement learning resource scheduling model online to obtain the updated computing resource optimization model.

2. The method for optimizing computing resources based on deep learning according to claim 1, characterized in that, S2 includes the following steps: S21. Integrate the node's remaining computing power, memory resources, load fluctuations, and health status into a node state vector; S22. Integrate task computation amount, priority, waiting time, input data size and service level constraints into a task requirement vector; S23. Encode the link bandwidth, transmission delay, congestion status, instance resource quota, and instance health status into a link-instance joint vector; S24. Combine the node state vector, task requirement vector, and link instance joint vector with scheduling constraints, and add resource upper limit, task latency, migration number and instance stability constraints to obtain a multi-dimensional scheduling state vector.

3. The method for optimizing computing resources based on deep learning according to claim 1, characterized in that, S3 includes the following steps: S31. Establish a state space based on a multi-dimensional scheduling state vector to represent computing power supply, task requirements, link transmission and instance running status; S32. Perform action coding on target node selection, resource quota adjustment, migration target selection, instance replica change, and resource limit adjustment to obtain the comprehensive action space; S33. Configure resource utilization rewards, task completion rewards, latency penalties, migration overhead penalties, load imbalance penalties, and service stability rewards to obtain the training environment.

4. The method for optimizing computing resources based on deep learning according to claim 1, characterized in that, S4 includes the following steps: S41. Construct an improved A3C multi-threaded deep reinforcement learning resource scheduling model. The model adopts a collaborative generation strategy of global shared parameters, asynchronous Actor threads, Critic value evaluation network, action mask pre-masking processing, resource constraint feedback correction processing, and reward weight correction processing. S42. Using the task requirement vector as the query object, calculate its matching weight with the node state vector and the link instance joint vector to obtain the task node adaptation features, task link transmission features and task instance carrying features. S43. The above features are integrated into scheduling decision features and input into multiple asynchronous Actor threads for parallel exploration to obtain multiple sets of candidate resource optimization actions; S44. Perform action masking pre-masking on multiple sets of candidate resource optimization actions, delete individual actions that violate resource, latency, link, migration count and instance health constraints, and obtain a set of feasible candidate actions; S45. Perform Critic value evaluation and resource constraint feedback correction on the set of feasible candidate actions, and adjust the action selection probability according to the action value and constraint risk to obtain a set of candidate computing power resource optimization strategies.

5. The method for optimizing computing resources based on deep learning according to claim 4, characterized in that, S43 includes the following steps: S431. Based on the task's pre-computational power consumption, video memory consumption, input data size, and maximum allowable latency, the task type classification results are obtained; S432. Based on the remaining node resources, load fluctuations, health status, temperature changes, and historical fault records, obtain the node status classification results; S433. Based on the task type division results and node status division results, allocate exploration sub-environments to different asynchronous Actor threads and generate corresponding candidate resource optimization actions; S434. Asynchronously aggregate the actions output by each asynchronous Actor thread, delete duplicate actions, mutual exclusion actions, and actions that conflict with the execution object, and obtain multiple sets of candidate resource optimization actions.

6. The method for optimizing computing resources based on deep learning according to claim 4, characterized in that, S44 includes the following steps: S441. Generate a set of task constraints based on the task's minimum computing power requirements, maximum waiting time, maximum execution latency, data size, and allowed number of migrations; S442. Generate a set of available resources for the target node based on its remaining computing power, remaining memory, node health status, and number of tasks it can handle. S443. Match the candidate resource optimization actions with the task constraint set, the node available resource set, and the link status, and filter out individual actions such as insufficient resources, latency exceeding the limit, link congestion, migration number exceeding the limit, instance abnormality, and quota conflict to obtain a set of feasible candidate actions.

7. The method for optimizing computing resources based on deep learning according to claim 4, characterized in that, S45 includes the following steps: S451. Evaluate the resource utilization, task latency, waiting time and node load changes after the execution of feasible candidate actions to obtain the basic value of the actions. S452. Calculate the resource consumption risk, migration overhead risk, link congestion risk and instance stability risk corresponding to feasible candidate actions to obtain the constraint risk value; S453. Using the constraint risk value as a value correction item and penalty item, the basic value of the action is corrected to obtain the action constraint correction value. S454. Based on the action constraint correction value, sort and filter feasible candidate actions to obtain a set of candidate computing power resource optimization strategies.

8. The method for optimizing computing resources based on deep learning according to claim 1, characterized in that, S5 includes the following steps: S51. Match the computing power consumption, memory consumption, and number of instance replicas after the combined execution of multiple candidate strategies with the remaining resource status of the computing power resource pool to obtain the combined resource verification result. S52. Delete candidate strategy combinations that cause continuous high load on a single node, widening load differences between nodes, cross-node latency exceeding limits, or failure to meet service level constraints, and obtain the combined correction result. S53. Identify conflict situations such as multiple tasks occupying the same resource quota, multiple migration actions pointing to the same node, and expansion / shrinkage actions and migration actions affecting each other, and obtain the combined conflict resolution results; S54. Based on the combined resource verification results, combined correction results, and combined conflict resolution results, a comprehensive screening is conducted to obtain a computing resource optimization scheme.

9. The method for optimizing computing resources based on deep learning according to claim 1, characterized in that, S7 includes the following steps: S71. Collect data on resource utilization, task latency, waiting time, migration overhead, node load fluctuation, instance health changes, and service stability to obtain a scheduling feedback dataset. S72. Compare the scheduling feedback dataset with the multi-dimensional scheduling state vector before scheduling to obtain changes in resource utilization, task latency, migration overhead, load balancing, and service stability. S73. Positive rewards are given to actions that improve resource utilization and reduce task latency, while penalties are given to actions that increase task latency, cause excessive migration overhead, cause load imbalance, and decrease instance stability, resulting in a composite reward outcome. S74. Perform reward weight correction processing based on composite reward results, and adjust the corresponding penalty weight according to the increase in latency, increase in migration overhead or decrease in stability during continuous scheduling cycles. S75. The composite reward result and penalty weight are fed back to the asynchronous Actor thread, the Critic value evaluation network and the global shared parameters to update the policy gradient, state value error, action mask parameters and resource constraint feedback parameters, thus obtaining the updated computing power resource optimization model.

10. A deep learning-based computing resource optimization system, characterized in that, The method for optimizing computing resources based on deep learning as described in any one of claims 1 to 9 is characterized by comprising the following modules: The resource status acquisition module collects running data of computing nodes, task queue data, network link data and container instance data in the computing resource pool. It performs time synchronization, field cleaning, abnormal field correction and normalization on the collected data to generate a computing resource status dataset. The scheduling status encoding module performs node status encoding, task requirement encoding, link instance encoding, and scheduling constraint encoding on the computing resource status dataset to generate a multi-dimensional scheduling status vector. The scheduling environment construction module builds a computing resource scheduling interaction environment based on a multi-dimensional scheduling state vector, sets up a scheduling action space and a reward feedback set, and generates a training environment. The A3C strategy generation module inputs the multi-dimensional scheduling state vector into the improved A3C multi-threaded deep reinforcement learning resource scheduling model. Through task adaptation state attention encoding, asynchronous actor grouping exploration, action mask pre-masking, Critic value evaluation, and resource constraint feedback correction, it generates a set of candidate computing power resource optimization strategies. The strategy screening and correction module performs combination feasibility verification, global load balancing correction, cross-node latency verification, and action conflict resolution on the candidate set of computing resource optimization strategies to generate a computing resource optimization scheme. The scheduling and execution module performs node allocation, quota adjustment, task migration, instance scaling, and queue rearrangement according to the computing resource optimization plan, and generates scheduling and execution results. The closed-loop update module generates closed-loop reward feedback samples based on the scheduling execution results, and updates and improves the A3C multi-threaded deep reinforcement learning resource scheduling model based on the closed-loop reward feedback samples, generating an updated computing resource optimization model.

Images