Distributed computing resource dynamic scheduling and optimization method based on deep reinforcement learning

Abstract

The application provides a distributed computing resource dynamic scheduling and optimization method based on deep reinforcement learning, relates to the field of computing resource scheduling, and comprises the following steps: determining an adaptive resource state recognition strategy and an adaptive reinforcement learning decision scheme based on a processing time threshold value analysis; performing computing resource state prediction according to the adaptive resource state recognition strategy; performing computing resource scheduling optimization by using the adaptive reinforcement learning decision scheme, according to predicted task data and a predicted resource state distribution, and taking the processing time threshold value as a constraint; and performing computing resource scheduling according to an optimal resource scheduling scheme. The method aims to solve the technical problem that a traditional computing resource scheduling method based on fixed rules cannot perceive and adapt to dynamically changing actual workloads and resource states, and thus cannot balance scheduling precision, efficiency and resource utilization; the method can maximize scheduling quality under the premise of ensuring response speed, and significantly improves the efficiency, practicality and precision of computing resource scheduling.

Description

Technical Field

[0001] This invention relates to the field of computing resource scheduling, and in particular to a method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning. Background Technology

[0002] With the rapid development of cloud computing, big data and artificial intelligence technologies, distributed computing platforms have become a key infrastructure supporting the digital transformation of society. In such platforms, how to efficiently and rationally schedule massive computing tasks to distributed computing nodes is the core technology that determines the performance, efficiency and cost of the entire system.

[0003] Currently, the dominant scheduling methods in production environments are mostly based on fixed preset rules, such as prioritizing task allocation to nodes with the most abundant resources or following a first-come, first-served queue principle. While these methods are simple to implement and fast in decision-making, their inherent drawbacks are becoming increasingly apparent. First, their scheduling logic is static and rigid, unable to perceive and adapt to dynamic fluctuations in task load and resource status. When faced with sudden traffic surges or complex heterogeneous workloads, this strategy either sacrifices scheduling quality and resource optimization space in pursuit of scheduling speed, resulting in low resource utilization; or it is too simplistic to perform fine-grained scheduling, leading to wasted computing power. Second, traditional methods lack forward-looking decision-making capabilities, scheduling only based on the current instantaneous state of the system, which easily leads to local optima rather than global optima. For example, densely scheduling tasks on currently idle but soon-to-be-fully-loaded nodes may create performance bottlenecks in the future. Summary of the Invention

[0004] The purpose of this invention is to provide a method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning. This method addresses the technical problem that traditional fixed-rule-based computing resource scheduling methods cannot perceive and adapt to dynamically changing actual workloads and resource states, leading to a difficulty in balancing scheduling accuracy, efficiency, and resource utilization. The method includes: performing computing resource scheduling urgency analysis based on predicted task data within a preset time zone to obtain a processing time threshold; determining an appropriate resource state identification strategy and an appropriate reinforcement learning decision scheme based on the processing time threshold analysis; predicting computing resource states according to the appropriate resource state identification strategy to obtain the predicted resource state distribution within the preset time zone; using the processing time threshold as a constraint, and employing the appropriate reinforcement learning decision scheme, optimizing computing resource scheduling based on the predicted task data and predicted resource state distribution to output an optimal resource scheduling scheme; and executing distributed computing resource scheduling within the preset time zone according to the optimal resource scheduling scheme.

[0005] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: obtaining historical task data sequences within historical time zones based on computing resource scheduling logs; constructing a task prediction model using a long short-term memory network to predict and obtain predicted task data within the preset time zone based on the historical task data sequences, wherein the time interval between the historical time zone and the preset time zone is the same; performing task load and task priority analysis based on the predicted task data, and outputting the total task load and task priority; and evaluating the urgency of computing resource scheduling based on the total task load and task priority, and outputting a processing time threshold, wherein the processing time threshold is positively correlated with the total task load and negatively correlated with the task priority.

[0006] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: using the ratio of the processing time threshold to the average historical processing time threshold as a scheme compensation coefficient; setting the product of the scheme compensation coefficient and the preset number of monitoring indicators as the number of adaptive resource monitoring indicators; calling the monitoring indicators according to the number of adaptive resource monitoring indicators to obtain an adaptive monitoring indicator set as an adaptive resource status identification strategy, wherein the preset number of monitoring indicators is 3; and setting the product of the scheme compensation coefficient and the preset number of learning iterations as the number of adaptation optimization iterations as an adaptive reinforcement learning decision scheme.

[0007] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: configuring a sequence of resource monitoring indicators, wherein the monitoring indicators arranged sequentially in the sequence include the number of allocable CPU cores, allocable memory capacity, allocable GPU resources, allocable local temporary storage space, CPU utilization, memory utilization, GPU computing unit utilization, network bandwidth utilization, disk space utilization, and network packet transmission latency; and selecting monitoring indicators from the sequence of resource monitoring indicators in order of the number of suitable resource monitoring indicators to obtain a set of suitable monitoring indicators.

[0008] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: screening historical distributed computing resource monitoring data according to the adapted monitoring index set to obtain the distribution of adapted historical resource monitoring data sequences within the historical time zone; collecting a sample resource monitoring data sequence distribution set based on the distributed computing resource operation log, and setting the sample historical resource monitoring data sequence distribution of different sample resource monitoring data sequences within the historical time zone as the sample predicted resource monitoring data sequence distribution to obtain a sample predicted resource monitoring data sequence distribution set; using the sample resource monitoring data sequence distribution set as input and the sample predicted resource monitoring data sequence distribution set as supervision, training a long short-term memory network until convergence to obtain a resource state distribution predictor; and using the resource state distribution predictor to predict the predicted resource state distribution within the preset time zone based on the historical resource monitoring data sequence distribution.

[0009] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: determining several computing resource scheduling spaces for several distributed computing nodes based on the predicted resource state distribution; randomly allocating tasks in the several computing resource scheduling spaces with the total task load as a constraint, outputting a first resource scheduling scheme, and iteratively selecting multiple resource scheduling schemes; performing task processing analysis according to the predicted task data, the predicted resource state distribution, and the multiple resource scheduling schemes to obtain multiple predicted processing times; setting resource scheduling schemes with predicted processing times less than the processing time threshold as feasible resource scheduling schemes to obtain several feasible resource scheduling schemes; and using the adaptive reinforcement learning decision scheme, optimizing computing resource scheduling based on the predicted task data, the predicted resource state distribution, and the several feasible resource scheduling schemes to output the optimal resource scheduling scheme.

[0010] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: performing computing resource scheduling simulations based on the predicted task data, predicted resource state distribution, and several feasible resource scheduling schemes, and outputting several scheduling simulation results, wherein the scheduling simulation results include simulated resource load distribution; performing load balancing evaluation based on the several simulated resource load distributions, and outputting several load balancing coefficients; obtaining several predicted processing times for the several feasible resource scheduling schemes, and calculating several scheme fitnesss based on the several load balancing coefficients and several predicted processing times, wherein the scheme fitness is negatively correlated with the load balancing coefficients and predicted processing times; and using the adaptive reinforcement learning decision scheme, performing computing resource scheduling optimization based on the several feasible resource scheduling schemes and several scheme fitnesss, and outputting the optimal resource scheduling scheme.

[0011] Preferably, the method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning further includes: taking the feasible resource scheduling scheme as an initial solution, arranging several initial solutions in descending order of fitness to generate an initial solution sequence; setting the first solution in the initial solution sequence as the optimal solution and the remaining solutions as inferior solutions to obtain multiple inferior solutions; using the optimal solution as the adjustment direction, adjusting the multiple inferior solutions once according to a preset optimization step size to obtain multiple updated inferior solutions, wherein if the updated inferior solution does not satisfy the several computing resource scheduling spaces, no adjustment is made. The adjustment involves: evaluating the fitness of each of the multiple suboptimal solutions and reordering them by merging the optimal solutions to obtain a sequence of updated solutions; eliminating the last Q solutions in the sequence and randomly selecting Q feasible resource scheduling schemes from the available computing resources to supplement them, where Q is an integer and the number of Q decreases as the number of optimization iterations increases; continuing iterative optimization until the number of optimization iterations in the adaptive reinforcement learning decision scheme is reached, at which point optimization stops, and the optimal solution in the current updated solution sequence is set as the optimal resource scheduling scheme.

[0012] The embodiments of the present invention have the following advantages: By analyzing the urgency of computing power scheduling based on predicted task data within a preset time zone, a processing time threshold is obtained. Then, based on the analysis of the processing time threshold, an adaptive resource status identification strategy and an adaptive reinforcement learning decision scheme are determined. Further, computing power resource status is predicted according to the adaptive resource status identification strategy to obtain the predicted resource status distribution within the preset time zone. Then, constrained by the processing time threshold, the adaptive reinforcement learning decision scheme is used to optimize computing power resource scheduling based on the predicted task data and predicted resource status distribution, outputting the optimal resource scheduling scheme. Finally, distributed computing power resource scheduling within the preset time zone is executed according to the optimal resource scheduling scheme. In other words, by constructing an adaptive and predictable intelligent scheduling framework, different complexity computing power resource scheduling strategies can be intelligently switched according to real-time scenario requirements, thereby maximizing scheduling quality while ensuring response speed, significantly improving the efficiency, practicality, and accuracy of computing power resource scheduling. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating the steps of a method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning, as described in this invention. Detailed Implementation

[0014] This invention provides a dynamic scheduling and optimization method for distributed computing resources based on deep reinforcement learning. It addresses the technical problem that traditional fixed-rule-based scheduling methods cannot perceive and adapt to dynamically changing workloads and resource states, leading to a difficulty in balancing scheduling accuracy, efficiency, and resource utilization. By constructing an adaptive and predictable intelligent scheduling framework, it can intelligently switch between different complexity scheduling strategies based on real-time scenario requirements. This maximizes scheduling quality while ensuring response speed, significantly improving the efficiency, practicality, and accuracy of computing resource scheduling.

[0015] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. It should also be noted that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, not all of them.

[0016] For examples, please refer to the appendix. Figure 1 This invention provides a method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning, specifically including the following steps: S100: Performs urgency analysis of computing power scheduling based on the predicted task data within the preset time zone, and obtains the processing time threshold value.

[0017] Furthermore, step S100 of the present invention further includes: S110: Obtain historical task data sequences within historical time zones based on computing resource scheduling logs; S120: Construct a task prediction model using a Long Short-Term Memory (LSTM) network, and predict task data within the preset time zone based on the historical task data sequences, wherein the time interval between the historical time zone and the preset time zone is the same; S130: Perform task load and task priority analysis based on the predicted task data, and output the total task load and task priority; S140: Perform computing power scheduling urgency assessment based on the total task load and task priority, and output a processing time threshold, wherein the processing time threshold is positively correlated with the total task load and negatively correlated with the task priority.

[0018] Specifically, firstly, historical task data is collected based on computing resource scheduling logs to obtain historical task data sequences within historical time zones (such as the most recent 20 minutes). The task data includes at least task submission information (task ID, submission timestamp, user / queue information, etc.), task resource requirements (number of CPU cores requested, memory size, number of GPUs, storage space, etc.), and task lifecycle information (task start execution time, task completion / termination time, etc.).

[0019] Next, a task prediction model is constructed using a Long Short-Term Memory (LSTM) network. LSTM is a special type of recurrent neural network capable of effectively capturing long-term dependencies in time-series data, which is crucial for identifying daily and weekly patterns in task load. During model training, pre-constructed historical samples are input into the network for training. The model learns from the data at each time point within the input window and uses the state of its memory cells to transmit and update historical information, gradually adjusting network weights. The goal is for the model to accurately predict subsequent task data. Through repeated iterations and error backpropagation, the task prediction model gradually learns the complex dynamic patterns inherent in the historical task data sequence, such as periodic patterns like high computing demand during the day on weekdays, batch task initiation at night, and reduced load on weekends, resulting in a fully trained task prediction model. Then, the LSTM network is used to construct the task prediction model, which predicts based on the historical task data sequence, outputting predicted task data within a preset time zone, including key indicators such as the expected number of task submissions and total resource requirements. The time interval between the historical time zone and the preset time zone is the same.

[0020] Then, based on the predicted task data, task load and priority analysis are performed. On one hand, key resource requirement fields are extracted from the predicted task data, such as the number of CPU cores requested, memory size (GB), and number of GPUs for each predicted task. Subsequently, these resources are aggregated using a weighted aggregation model (which can assign different weights based on the resource scarcity and cost of the cluster (e.g., in a cluster with scarce GPUs, the weight of GPUs is much higher than that of CPUs)) to finally calculate a comprehensive quantitative indicator of "total task load". This indicator, compared to the simple "number of tasks", can more accurately predict the actual pressure on the computing power pool in the future. On the other hand, based on the contextual information contained in the predicted task data (such as the user group, project queue, business type, or explicitly marked priority tags of the task), a set of rule engines or classification models is used to classify the priority of each predicted task (e.g., into three levels: "high", "medium", and "low"). Then, the proportion of tasks at each priority level in the future preset time zone is statistically analyzed, and scores are assigned to different levels and then weighted averaged to obtain a comprehensive "task priority" score.

[0021] Finally, based on the predicted task data within the preset time zone, an urgency analysis of computing power scheduling is performed. For example, a multi-factor decision is made using a predefined urgency assessment matrix. This matrix takes the continuous total task load quantification and task priority quantification as input parameters and uses a weighted decision algorithm to calculate the corresponding urgency index. The weight coefficient of the total task load reflects the system's emphasis on throughput capacity, while the weight coefficient of task priority reflects the requirements for service quality assurance. Further, based on the calculated urgency index, a processing time threshold is dynamically output through a preset mapping relationship (such as a piecewise linear function or a lookup table mechanism). This processing time threshold is positively correlated with the total task load and negatively correlated with the task priority. For example, when high load and high priority tasks occur simultaneously, the algorithm intelligently balances these conflicting demands and outputs a suitable threshold. The final dynamically generated processing time threshold is injected as a key constraint into subsequent scheduling stages, ensuring stable operation during peak business periods and achieving optimal allocation when resources are sufficient, thus truly realizing a precise match between scheduling strategies and business needs.

[0022] S200: Based on the processing time threshold analysis, determine the appropriate resource status identification strategy and the appropriate reinforcement learning decision scheme.

[0023] Furthermore, step S200 of the present invention further includes: S210: The ratio of the processing time threshold to the average historical processing time threshold is used as the scheme compensation coefficient.

[0024] Specifically, the current dynamically acquired processing time threshold is compared with the average historical processing time threshold calculated by the system based on historical data to obtain the scheme compensation coefficient. This coefficient reflects the deviation of the current scheduling urgency from the historical average level: when the ratio is greater than 1, it indicates that the current allowable decision time is wider than the historical average level, and the system may tend to adopt a more refined scheduling strategy; when the ratio is less than 1, it means that the decision time is more urgent than usual, and a more efficient simplification strategy needs to be initiated. This compensation coefficient, as a dynamic weight parameter, provides a quantitative benchmark for subsequent adaptive matching of state recognition strategies and reinforcement learning schemes with different complexities, enabling the system to intelligently adjust the balance between accuracy and efficiency based on historical experience.

[0025] S220: The product of the scheme compensation coefficient and the preset number of monitoring indicators is set as the number of adaptive resource monitoring indicators. The monitoring indicators are called according to the number of adaptive resource monitoring indicators to obtain the set of adaptive monitoring indicators, which is used as the adaptive resource status identification strategy. The preset number of monitoring indicators is 3.

[0026] Furthermore, step S220 of the present invention further includes: S221: Configure a sequence of resource monitoring indicators, wherein the monitoring indicators arranged sequentially in the sequence include the number of allocable CPU cores, allocable memory capacity, allocable GPU resources, allocable local temporary storage space, CPU utilization, memory utilization, GPU computing unit utilization, network bandwidth utilization, disk space utilization, and network packet transmission latency; S222: Select and call monitoring indicators from the sequence of resource monitoring indicators in order of the number of adaptive resource monitoring indicators to obtain an adaptive monitoring indicator set.

[0027] Specifically, firstly, a sequence of resource monitoring indicators is configured. This sequence is strictly ordered according to the correlation between the indicators and task scheduling decisions. The monitoring indicators arranged in the sequence include the number of allocable CPU cores, allocable memory capacity, allocable GPU resources, allocable local temporary storage space, CPU utilization, memory utilization, GPU computing unit utilization, network bandwidth utilization, disk space utilization, and network packet transmission latency.

[0028] Next, the calculated scheme compensation coefficient is multiplied by the preset number of baseline monitoring indicators (set to 3) and rounded to determine the actual number of adaptive resource monitoring indicators used in the current scenario. This calculation process realizes the core mechanism of adaptively adjusting the monitoring granularity according to the urgency of scheduling: when the compensation coefficient is greater than 1 (i.e., the decision time is relatively relaxed), the system will increase the number of monitoring indicators proportionally (e.g., the number of monitoring indicators corresponding to the coefficient 1.6 = rounded (1.6 × 3) = 4) to obtain a more refined system status perception; when the compensation coefficient is less than 1 (the decision time is tight), the number of monitoring indicators will be reduced accordingly (e.g., the number of monitoring indicators corresponding to the coefficient 0.7 = 2) to ensure scheduling efficiency by reducing the complexity of status recognition. This dynamic adjustment mechanism ensures that the system can achieve an intelligent trade-off between accuracy and efficiency.

[0029] Then, according to the inherent order of the resource monitoring indicator sequence (from the number of allocable CPU cores to the network packet transmission delay), the top N indicators (N = the number of resources to be monitored) are strictly selected to form a monitoring set. This mechanism ensures that in any scheduling scenario, the system prioritizes monitoring the core indicators most closely related to task scheduling decisions (such as the amount of allocable resources), thereby achieving the optimal balance between accuracy and efficiency in the state awareness stage: when resources are scarce and quick decisions are needed, only the first 2 to 3 key indicators are monitored; when resources are abundant, the monitoring range can be expanded to the 6th to 8th performance indicators to achieve adaptive adjustment.

[0030] S230: The product of the scheme compensation coefficient and the preset number of learning iterations is set as the number of adaptation optimization iterations, which is used as the adaptation reinforcement learning decision scheme.

[0031] Specifically, the compensation coefficient of the scheme is multiplied by a preset baseline learning iteration number (e.g., 200 times) to dynamically generate the number of adaptation optimization iterations for the current scheduling scenario. This number serves as the core parameter for adapting the reinforcement learning decision scheme. This mechanism achieves the goal of intelligently adjusting the convergence depth of the algorithm according to the urgency of the scheduling: when the compensation coefficient is greater than 1 (i.e., the decision time is relatively relaxed), the system increases the number of iterations proportionally (e.g., a coefficient of 1.5 corresponds to 300 iterations), allowing the reinforcement learning model to explore and optimize more fully, thus improving the quality of the scheduling scheme; when the compensation coefficient is less than 1 (the decision time is tight), the number of iterations is reduced accordingly (e.g., a coefficient of 0.6 corresponds to 120 iterations), prioritizing the real-time scheduling response by limiting computational overhead, thereby achieving a flexible balance between optimization accuracy and execution efficiency at the algorithm level.

[0032] S300: Perform computing resource status prediction according to the adapted resource status identification strategy, and obtain the predicted resource status distribution within the preset time zone.

[0033] Furthermore, step S300 of the present invention also includes: S310: Screen historical distributed computing resource monitoring data according to the adapted monitoring index set to obtain the adapted historical resource monitoring data sequence distribution within the historical time zone; S320: Collect sample resource monitoring data sequence distribution set based on the distributed computing resource operation log, and collect the sample historical resource monitoring data sequence distribution of different sample resource monitoring data sequence distributions within the historical time zone as the sample predicted resource monitoring data sequence distribution to obtain the sample predicted resource monitoring data sequence distribution set; S330: Use the sample resource monitoring data sequence distribution set as input and the sample predicted resource monitoring data sequence distribution set as supervision to train a long short-term memory network until convergence to obtain a resource state distribution predictor; S340: Use the resource state distribution predictor to predict the predicted resource state distribution within the preset time zone based on the historical resource monitoring data sequence distribution.

[0034] Specifically, firstly, according to the adapted monitoring indicator set, specific resource indicators (such as only "allocateable CPU cores", "allocateable memory capacity" and "CPU utilization") are selected from the cluster containing several distributed computing nodes. Then, the monitoring data of all nodes in the historical time zone are aggregated according to a unified time granularity (such as 2-minute intervals) to form a time series dataset with simplified dimensions but focusing on key resource status. This dataset accurately reflects the historical dynamic distribution pattern of cluster resources under the given monitoring dimensions, providing an efficient and targeted data foundation for subsequent resource status prediction.

[0035] The core objective of step S320 is to construct a sample dataset for training the resource prediction model. First, based on the distributed computing resource operation logs, resource monitoring data from multiple consecutive time intervals is collected to form an initial sample resource monitoring data sequence distribution set. This involves extracting historical data from the logs, including key indicators such as CPU utilization, memory allocation, and network I / O, and dividing the data into fixed time windows (e.g., hourly). The resource status distribution of all nodes within each time window constitutes a sample sequence. Next, for each sample resource monitoring data sequence, the actual resource monitoring data sequence for the subsequent time period (e.g., the next 20 minutes) within the historical time zone is collected, serving as the corresponding sample predicted resource monitoring data sequence distribution. In this way, each input sequence (historical state) is paired with its corresponding output sequence (future state), ultimately forming a sample predicted resource monitoring data sequence distribution set containing multiple "history-future" sequence pairs, providing a complete data foundation for subsequent supervised learning training.

[0036] Furthermore, the sample resource monitoring data sequence distribution set is used as the training input, and the corresponding sample predicted resource monitoring data sequence distribution set is used as the supervision signal to train the Long Short-Term Memory (LSTM) network. During the training process, the LSM network learns the complex mapping relationship from historical resource state sequences to future resource state distributions through its unique gating mechanism. The model continuously adjusts the network parameters through the time backpropagation algorithm to minimize the difference between the predicted output and the actual future state (e.g., by using the mean squared error loss function) until the model's performance on the validation set tends to stabilize and reaches the preset accuracy requirements. Finally, a converged resource state distribution predictor is obtained, which can accurately predict the distribution of cluster resources in a specific future period based on current and historical resource monitoring data.

[0037] Finally, the historical resource monitoring data sequence distribution obtained in step S310 (i.e., the actual resource status sequence of the most recent complete historical time zone) is input into the trained and converged resource status distribution predictor to calculate the dynamic changes of cluster resources in the future preset time zone, and finally output a predicted resource status distribution that includes the complete status of each node at each future time point on the adapted monitoring index set.

[0038] S400: Using the processing time threshold as a constraint, and employing the adaptive reinforcement learning decision scheme, optimize the scheduling of computing resources based on the predicted task data and the predicted resource status distribution, and output the optimal resource scheduling scheme.

[0039] Furthermore, step S400 of the present invention further includes: S410: Determine several computing resource scheduling spaces for several distributed computing power nodes based on the predicted resource status distribution; S420: With the total task load as a constraint, randomly allocate tasks in the several computing resource scheduling spaces, output a first resource scheduling scheme, and iteratively select multiple resource scheduling schemes; S430: Perform task processing analysis based on the predicted task data, the predicted resource status distribution, and the multiple resource scheduling schemes to obtain multiple predicted processing times; S440: Set the resource scheduling scheme with a predicted processing time less than the processing time threshold as a feasible resource scheduling scheme to obtain several feasible resource scheduling schemes.

[0040] Specifically, firstly, for each node, based on the predicted resource status distribution, key allocable resource indicators (such as the number of allocable CPU cores, allocable memory capacity, etc.) are extracted from its predicted resource status. Taking into account the node's existing load, resource reservation strategy, and predicted resource usage trends, a matrix of available resources containing the dynamic changes of all nodes within a preset time zone is finally generated by subtracting the security threshold and committed resource amount from the resource capacity. This matrix clearly quantifies the remaining task processing capacity space of each node in various future time segments.

[0041] Next, using the Monte Carlo random sampling method, under the premise of meeting the total task load resource requirements, multiple rounds of random task allocation are performed within the several computing power resource scheduling spaces: each round of allocation randomly maps the resource requirements in the predicted task data to the available resource space of different nodes, generating feasible solutions that meet the load constraints. After multiple iterations, a set of multiple resource scheduling schemes, including the first resource scheduling scheme, is output. These schemes exhibit different task-node matching patterns while maintaining the balance of total resources, providing diverse initial decision samples for subsequent reinforcement learning optimization.

[0042] Furthermore, based on the predicted task data and predicted resource status distribution, an end-to-end performance simulation model is used to perform an analysis of each resource scheduling scheme. That is, based on the task characteristics (such as computational load and dependencies) in the predicted task data and the node performance expectations (such as CPU / memory / network load) provided by the predicted resource status distribution, the entire process of queuing, execution and completion of tasks on the allocation nodes is simulated. Finally, the predicted processing time of all tasks under each scheme is quantitatively output, providing an accurate performance evaluation basis for subsequent scheme selection.

[0043] Finally, the predicted processing time for each resource scheduling scheme is examined one by one. Only those schemes with a maximum or average predicted processing time less than or equal to the threshold are retained and marked as feasible resource scheduling schemes. In this way, while ensuring scheduling timeliness, a set of feasible solutions that meet the real-time requirements is selected from the initial scheme set, providing a candidate range that meets efficiency constraints for subsequent optimal scheme selection.

[0044] S450: Using the adaptive reinforcement learning decision scheme, optimize the computing power resource scheduling based on the predicted task data, the predicted resource state distribution, and several feasible resource scheduling schemes, and output the optimal resource scheduling scheme.

[0045] Furthermore, step S450 of the present invention further includes: S451: Perform computing power resource scheduling simulations based on the predicted task data, predicted resource status distribution, and several feasible resource scheduling schemes, and output several scheduling simulation results, wherein the scheduling simulation results include simulated resource load distribution; S452: Perform load balancing evaluation based on several simulated resource load distributions, and output several load balancing coefficients; S453: Obtain several predicted processing times for the several feasible resource scheduling schemes, and calculate several scheme fitnesss based on the several load balancing coefficients and several predicted processing times, wherein the scheme fitness is negatively correlated with the load balancing coefficients and predicted processing times.

[0046] Specifically, computing power resource scheduling simulations are performed based on the predicted task data, predicted resource status distribution, and several feasible resource scheduling schemes. For example, a high-fidelity scheduling simulator can be constructed to dynamically deduce each feasible resource scheduling scheme: the task sequence in the predicted task data is injected into the simulation environment constructed by the predicted resource status distribution according to the allocation strategy specified in the scheme, and the resource occupation and release during the task execution process are tracked in real time. Finally, the scheduling simulation results corresponding to each scheme are output, wherein the simulated resource load distribution accurately records the dynamic utilization curves of key resources such as CPU and memory of each node in the future preset time zone.

[0047] Next, by statistically analyzing several simulated resource load distributions, the cluster balance level of each scheduling scheme is quantitatively evaluated. For example, the variance of resource utilization rate of each scheme across all nodes is calculated. Combined with indicators such as the range of resource utilization between nodes and the Gini coefficient, several load balance coefficients are output. The smaller the coefficient value, the more uniform the resource distribution. When it is zero, it indicates that the ideal balance state has been achieved, thus providing a key quantitative basis for scheme selection based on load balance.

[0048] Finally, using a multi-objective fusion algorithm, the load balancing coefficient and predicted processing time of each feasible resource scheduling scheme are normalized and then weighted and fused according to their negative correlation. By assigning appropriate weights to the two indicators, the reciprocal or negative standardization method is used to process them so that the scheme with the smaller load balancing coefficient and the shorter predicted processing time receives a higher scheme fitness score. Finally, a fitness sequence that can be quantified and compared for the comprehensive performance of the schemes is generated, providing a unified evaluation standard for the selection of the optimal scheme and obtaining several scheme fitnesss of the several feasible resource scheduling schemes.

[0049] S454: Using the adaptive reinforcement learning decision scheme, optimize the computing resource scheduling based on the several feasible resource scheduling schemes and the fitness of several schemes, and output the optimal resource scheduling scheme.

[0050] Furthermore, step S454 of the present invention further includes: S4541: Using the feasible resource scheduling scheme as the initial solution, arrange several initial solutions in descending order of fitness to generate an initial solution sequence; S4542: Set the first solution in the initial solution sequence as the optimal solution and the remaining solutions as inferior solutions to obtain multiple inferior solutions; S4543: Using the optimal solution as the adjustment direction, adjust the multiple inferior solutions once according to a preset optimization step size to obtain multiple updated inferior solutions, wherein if the updated inferior solution does not satisfy the several computing power resource scheduling spaces, no adjustment is performed; S4544: Adjust the multiple updated inferior solutions... The suboptimal solutions are evaluated for fitness and then merged with the optimal solutions to reorder them, resulting in an updated solution sequence. S4545: The last Q solutions in the updated solution sequence are eliminated, and Q feasible resource scheduling schemes are randomly selected from the several computing resource scheduling spaces to supplement them, where Q is an integer and the number of Q decreases as the number of optimization iterations increases. S4546: Iterative optimization continues until the number of adaptation optimization iterations in the adaptive reinforcement learning decision scheme is reached, then optimization stops, and the optimal solution in the current updated solution sequence is set as the optimal resource scheduling scheme.

[0051] Specifically, firstly, all feasible resource scheduling schemes selected are used as the initial solution set for the optimization process. These schemes are then sorted in descending order of their fitness scores calculated in step S453, generating an initial solution sequence ranked from best to worst in terms of overall performance, with the scheme with the highest fitness at the top of the sequence. Next, the highest-fitting solution ranked first in the sorted initial solution sequence is selected as the current optimal solution, serving as the benchmark for subsequent iterations. Simultaneously, all other solutions with lower fitness in the sequence are uniformly designated as the inferior solution set.

[0052] Further, guided by the resource allocation pattern of the current optimal solution, each inferior solution is locally perturbed according to a preset optimization step size (e.g., adjusting the allocation of 3% of task nodes). By randomly selecting the node allocation of some tasks in the inferior solution, it is made to move closer to the corresponding allocation method of the optimal solution, thereby generating multiple updated inferior solutions. During the adjustment process, it is verified in real time whether the new solution still meets the capacity constraints of the computing power resource scheduling space. Any adjustment scheme that may cause resource overrun is discarded to ensure that all updated solutions remain feasible. Then, through the evaluation process in step S453, the load balancing coefficient and predicted processing time of each updated solution under the new resource allocation state are calculated, and its scheme fitness is re-derived. Subsequently, these newly evaluated updated solutions are merged with the original optimal solutions to form a new candidate solution set, and are uniformly sorted in descending order according to the scheme fitness. Finally, a sequence of updated solutions that integrates historical best solutions and emerging improved solutions is generated, providing a set of high-quality solutions that have been screened and sorted for the next round of iterative optimization.

[0053] Next, the elimination quantity Q is dynamically calculated based on the current iteration round. A larger Q value is set in the early stages of optimization (e.g., 30% of the population size). By eliminating a large number of low-fitness solutions and supplementing with random new solutions, the search range is significantly expanded, avoiding premature convergence. As the number of iterations increases, the Q value decreases linearly or stepwise, dropping to a smaller value (e.g., 5% of the population size) in the later stages of optimization. At this point, the focus is on reducing random interference and concentrating resources on fine-tuning existing high-quality solutions. Furthermore, the last Q solutions in the updated solution sequence are removed. Simultaneously, within the constraints of the computing power resource scheduling space, Q new feasible resource scheduling schemes are generated to replenish the population. This dynamic Q-value adjustment strategy ensures both the global exploration capability in the early stages of the algorithm and the local optimization accuracy in the later convergence stage, enabling the optimization process to both escape local optima and quickly converge to a high-quality solution domain.

[0054] The iterative optimization continues. In each iteration, steps S4543 to S4545 are repeated: inferior solutions are adjusted based on the current best solution, fitness is re-evaluated and ranked, and the solution set is eliminated and replenished according to the dynamic Q-value. This iterative mechanism allows the solution set quality to continuously evolve in the cycle of "selection-adjustment-evaluation-update," maintaining population diversity while ensuring that the search direction converges towards the high-performance region. When the cumulative number of iterations reaches the critical threshold of the fitness optimization iteration count, the optimization process is immediately terminated, and the highest fitness solution ranked first in the final updated solution sequence is selected as the optimal resource scheduling scheme. This termination condition based on the preset number of iterations ensures that the scheduling decision is strictly controlled within the specified processing time threshold, achieving a precise balance between optimization quality and computational efficiency.

[0055] S500: Execute the distributed computing power resource scheduling within the preset time zone according to the optimal resource scheduling scheme.

[0056] Specifically, according to the optimal resource scheduling scheme, the tasks to be processed are distributed to the corresponding distributed computing power nodes in batches, and the resource quotas required for task execution (such as the number of CPU cores, memory capacity, GPU resources, etc.) are configured synchronously. During the scheduling and execution process, the task deployment status and resource usage are continuously monitored to ensure that the actual allocation is consistent with the scheme expectation. At the same time, an anomaly handling mechanism is established to activate contingency plans for emergencies such as node failure or resource contention, and finally achieve accurate and efficient allocation of computing power resources within the preset time zone.

[0057] In summary, the distributed computing resource dynamic scheduling and optimization method based on deep reinforcement learning provided by this invention has the following technical effects: By analyzing the urgency of computing power scheduling based on predicted task data within a preset time zone, a processing time threshold is obtained. Then, based on the analysis of the processing time threshold, an adaptive resource status identification strategy and an adaptive reinforcement learning decision scheme are determined. Further, computing power resource status is predicted according to the adaptive resource status identification strategy to obtain the predicted resource status distribution within the preset time zone. Then, constrained by the processing time threshold, the adaptive reinforcement learning decision scheme is used to optimize computing power resource scheduling based on the predicted task data and predicted resource status distribution, outputting the optimal resource scheduling scheme. Finally, distributed computing power resource scheduling within the preset time zone is executed according to the optimal resource scheduling scheme. In other words, by constructing an adaptive and predictable intelligent scheduling framework, different complexity computing power resource scheduling strategies can be intelligently switched according to real-time scenario requirements, thereby maximizing scheduling quality while ensuring response speed, significantly improving the efficiency, practicality, and accuracy of computing power resource scheduling.

[0058] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0059] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning, characterized in that, The methods include: Based on the predicted task data within the preset time zone, an urgency analysis of computing power scheduling is performed to obtain processing time thresholds, including: Historical task data sequences within historical time zones are obtained based on computing resource scheduling logs; A task prediction model is constructed using a long short-term memory network. Based on the historical task data sequence, predicted task data within the preset time zone is obtained, wherein the time interval between the historical time zone and the preset time zone is the same. Based on the predicted task data, analyze the task load and task priority, and output the total task load and task priority; Based on the total task load and task priority, the urgency of computing power scheduling is evaluated, and a processing time threshold is output. The processing time threshold is positively correlated with the total task load and negatively correlated with the task priority. Based on the processing time threshold analysis, the appropriate resource state identification strategy and appropriate reinforcement learning decision scheme are determined, including: The ratio of the processing time threshold to the historical average processing time threshold is used as the scheme compensation coefficient. The product of the compensation coefficient of the scheme and the number of preset monitoring indicators is set as the number of adaptive resource monitoring indicators. The monitoring indicators are called according to the number of adaptive resource monitoring indicators to obtain the set of adaptive monitoring indicators, which is used as the adaptive resource status identification strategy. The number of preset monitoring indicators is 3. The product of the compensation coefficient of the above scheme and the preset number of learning iterations is set as the number of adaptation optimization iterations, which is used as the adaptation reinforcement learning decision scheme. The computing resource status is predicted according to the adapted resource status identification strategy, and the predicted resource status distribution within the preset time zone is obtained. Using the processing time threshold as a constraint, and employing the adaptive reinforcement learning decision-making scheme, the optimal resource scheduling scheme is output based on the prediction task data and the prediction resource state distribution, including: Based on the predicted resource status distribution, several computing resource scheduling spaces for several distributed computing power nodes are determined; With the total task load as a constraint, tasks are randomly allocated in the plurality of computing power resource scheduling spaces, a first resource scheduling scheme is output, and multiple resource scheduling schemes are obtained through iterative selection. Based on the predicted task data, predicted resource status distribution, and multiple resource scheduling schemes, task processing analysis is performed to obtain multiple predicted processing times. The resource scheduling scheme with a predicted processing time less than the processing time threshold is set as a feasible resource scheduling scheme, resulting in several feasible resource scheduling schemes. Using the aforementioned adaptive reinforcement learning decision-making scheme, based on the predicted task data, predicted resource state distribution, and several feasible resource scheduling schemes, computational resource scheduling optimization is performed, and the optimal resource scheduling scheme is output, including: Based on the predicted task data, the predicted resource status distribution, and several feasible resource scheduling schemes, computing power resource scheduling simulations are performed respectively, and several scheduling simulation results are output. The scheduling simulation results include the simulated resource load distribution. Based on several simulated resource load distributions, a load balancing assessment is performed, and several load balancing coefficients are output. Several predicted processing times for several feasible resource scheduling schemes are obtained, and several scheme fitnesss are calculated based on several load balancing coefficients and several predicted processing times, wherein the scheme fitness is negatively correlated with the load balancing coefficients and the predicted processing times. Using the adaptive reinforcement learning decision scheme, the computing power resource scheduling is optimized based on the several feasible resource scheduling schemes and the fitness of several schemes, and the optimal resource scheduling scheme is output. Distributed computing resources within the preset time zone are scheduled according to the optimal resource scheduling scheme.

2. The method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning according to claim 1, characterized in that, The set of adaptive monitoring indicators is obtained by calling the monitoring indicators according to the number of adapted resources monitored, including: Configure a sequence of resource monitoring indicators, wherein the monitoring indicators arranged in sequence include the number of allocable CPU cores, allocable memory capacity, allocable GPU resources, allocable local temporary storage space, CPU utilization, memory utilization, GPU computing unit utilization, network bandwidth utilization, disk space utilization, and network packet transmission latency. According to the number of resources to be monitored, the monitoring indicators are selected from the beginning to the end of the resource monitoring indicator sequence to obtain the set of adaptive monitoring indicators.

3. The method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning according to claim 1, characterized in that, According to the adapted resource status identification strategy, the computing resource status is predicted, and the predicted resource status distribution within the preset time zone is obtained, including: Based on the aforementioned set of adaptive monitoring indicators, historical distributed computing resource monitoring data is screened to obtain the distribution of adaptive historical resource monitoring data sequences within historical time zones. Based on the distributed computing power resource operation log, the sample resource monitoring data sequence distribution set is collected, and the historical resource monitoring data sequence distribution of different sample resource monitoring data sequences in the historical time zone is collected as the sample predicted resource monitoring data sequence distribution set. Using the sample resource monitoring data sequence distribution set as input and the sample predicted resource monitoring data sequence distribution set as supervision, a long short-term memory network is trained until convergence to obtain a resource state distribution predictor; The resource status distribution predictor is used to obtain the predicted resource status distribution within the preset time zone based on the historical resource monitoring data sequence distribution prediction.

4. The method for dynamic scheduling and optimization of distributed computing resources based on deep reinforcement learning according to claim 1, characterized in that, Using the aforementioned adaptive reinforcement learning decision-making scheme, the computational resource scheduling is optimized based on several feasible resource scheduling schemes and several scheme fitnesss, and the optimal resource scheduling scheme is output, including: The feasible resource scheduling scheme is used as the initial solution, and several initial solutions are arranged in descending order of fitness to generate an initial solution sequence; The first solution in the initial solution sequence is set as the optimal solution, and the remaining solutions are set as inferior solutions to obtain multiple inferior solutions; Using the optimal solution as the adjustment direction, the multiple inferior solutions are adjusted once according to a preset optimization step size to obtain multiple updated inferior solutions. If the updated inferior solution does not satisfy the multiple computing resource scheduling spaces, no adjustment is performed. The fitness of each of the multiple suboptimal solutions updated once is evaluated, and the optimal solutions are merged and reordered to obtain a sequence of solutions updated once. The last Q solutions of the updated solution sequence are eliminated, and Q feasible resource scheduling schemes are randomly selected from the plurality of computing power resource scheduling spaces to supplement them, where Q is an integer and the number of Q decreases as the number of optimization iterations increases; Continue iterative optimization until the number of adaptation optimization iterations in the adaptive reinforcement learning decision scheme is reached, then stop optimization and output the optimal solution of the current updated solution sequence as the optimal resource scheduling scheme.

Images