Heterogeneous computing network resource collaborative scheduling optimization method based on adaptive multi-agent

Abstract

The invention discloses a heterogeneous computing network resource collaborative scheduling optimization method based on self-adaptive multi-agent, and aims to solve the scheduling problem caused by resource heterogeneity, load dynamics and task high concurrency in a heterogeneous computing network system. According to the method, cross-domain resource collaboration is realized by constructing three sub-domain adaptive agents of a computing resource domain, a network resource domain and a storage resource domain and a global collaboration layer. According to the method, a deep reinforcement learning algorithm and an 'LSTM + GNN' fusion model are integrated, and multi-target adaptive optimization of resource utilization rate, task time delay, service quality and energy consumption is achieved through closed-loop optimization of state perception, strategy generation, value evaluation and strategy updating. The heterogeneous computing network resource fine-grained sensing, cross-domain cooperative scheduling and multi-target dynamic optimization are realized, the resource utilization rate and the task completion rate are high, the service quality and the energy consumption performance are good, and the dynamic response capability and the overall performance of the heterogeneous computing network system are improved.

Description

Technical Field

[0001] This invention relates to a heterogeneous computing network resource collaborative scheduling method, and more particularly to a heterogeneous computing network resource collaborative scheduling optimization method based on adaptive multi-agent systems. Background Technology

[0002] With the rapid development of new infrastructure and the increasing deployment of edge devices, heterogeneous computing network systems have become a key support. These systems need to handle the differentiated characteristics of computing, network, and storage resources, while also coping with high concurrency tasks and dynamically fluctuating loads.

[0003] Traditional resource scheduling methods mainly fall into two categories: 1. Static rule-based scheduling: such as shortest task priority or round-robin scheduling, which rely on preset strategies. While simple in design, they lack adaptability and cannot handle resource heterogeneity and load fluctuations, leading to resource misallocation. 2. Single-agent scheduling: combining deep reinforcement learning, which optimizes resource allocation through autonomous decision-making by the agent. However, it has limited scope, focusing only on a local resource domain and ignoring the strong correlation between computing, network, and storage resources, making it difficult to achieve cross-domain collaboration and multi-objective balancing.

[0004] The limitations of existing technologies include the following: 1. Fixed rules cannot adapt to heterogeneous resources: Unified allocation logic leads to fine-grained parameter mismatch, such as matching resource types but unbalanced performance parameters. 2. Single agent local optimization trap: Agents are biased towards a single goal, causing global imbalance. 3. Lagging dynamic capture: Fixed rule parameters are based on experience and cannot respond to millimeter-level load fluctuations; single agents rely on globally aggregated data, resulting in insufficient real-time state awareness. 4. Lack of multi-objective balancing and disturbance resistance mechanisms: Fixed reward function weights cannot dynamically adapt to diverse task requirements. 5. Lack of forward-looking reward function: Without load prediction integration, the long-term stability of the strategy is poor. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a heterogeneous computing network resource collaborative scheduling optimization method that can achieve cross-domain collaborative scheduling and multi-objective dynamic optimization.

[0006] Technical solution: The heterogeneous computing network resource collaborative scheduling optimization method based on adaptive multi-agent as described in this invention includes the following steps:

[0007] (1) Constructing domain-specific adaptive agents: Design adaptive agents for computing resource domain, network resource domain and storage resource domain respectively. The adaptive agents include a state perception module, a policy network module, a value evaluation module and a policy update module, forming an independent closed-loop optimization structure. When processing single resource domain tasks, each agent independently executes the resource allocation policy generated based on local state perception.

[0008] (2) Design a global collaboration layer: connect the domain-specific adaptive agents, receive and synchronize key information of each single-domain agent in real time through the information interaction and synchronization module, the global strategy fusion module integrates the local strategies of each single-domain agent to generate a global scheduling strategy, the cross-domain conflict detection and coordination module detects global scheduling strategy conflicts and coordinates their resolution, and the global feedback optimization module optimizes the global scheduling strategy based on the system execution results to achieve cross-domain strategy collaboration;

[0009] (3) Perceiving the state of heterogeneous computing networks: Real-time resource data, task load description, temporal feature extraction and spatial feature extraction are collected through the state perception module, and the current resource load quantitative representation, resource demand prediction in the next 5-15 minutes, task feature-resource demand matching degree, system bottleneck and potential conflict warning information are output.

[0010] (4) Generate and execute local policies: Through the policy network module, with the output of the state awareness module as input, generate local allocation policies for each resource domain based on the deep reinforcement learning algorithm; for tasks involving only a single resource domain, directly execute the local policy to complete the scheduling; the local policy includes task binding and resource quota for the computing domain, path selection and bandwidth quota for the network domain, and data binding and IO priority setting for the storage domain.

[0011] (5) Quantitative evaluation: The value evaluation module performs multi-dimensional quantitative evaluation of the local strategy execution effect, including resource utilization, task latency, energy consumption and priority satisfaction rate, and outputs feedback signals.

[0012] (6) Iterative optimization: Based on the feedback signal, the policy network parameters are dynamically adjusted through the policy update module to achieve single-domain closed-loop iterative optimization;

[0013] (7) Generate a global scheme: When the task is multi-resource domain collaboration, the local policies output by each adaptive single agent are integrated through the global collaboration layer to perform cross-domain policy fusion and conflict resolution, and a global collaborative scheduling scheme is generated. The global collaboration layer generates a global collaborative scheduling strategy for computing, network and storage resources based on the performance indicators, resource demand matching degree and global state vector of each local policy, and ensures global feasibility through conflict detection.

[0014] Furthermore, step 3, sensing the state of the heterogeneous computing network, includes the following steps:

[0015] (31) To comprehensively characterize the system resource status, a dynamic state vector of computing resources, network resources, and storage resources is constructed by using multi-source heterogeneous data fusion technology;

[0016] (32) Characterize task characteristics and system requirements, and construct task feature vectors based on features such as task type, priority, execution time, and resource requirement patterns;

[0017] (33) Proactively predict the short-term load of heterogeneous computing networks. Based on the “LSTM+GNN” fusion model, LSTM captures the periodic and trend characteristics of historical load, and GNN models the spatial correlation characteristics of resource topology and task dependence, accurately predicting the fine-grained load distribution and bottleneck risks in the next 5-15 minutes.

[0018] (34) Output heterogeneous resource state vectors and task load descriptions. Adopt a multi-level data fusion strategy to process the original monitoring data, task features and spatiotemporal features in a unified manner. Perform data preprocessing, feature extraction, state vector construction and dynamic update in sequence to form a system state representation for decision-making.

[0019] Furthermore, according to the collaborative scheduling optimization method according to claim 2, the "LSTM+GNN" fusion model in step 33 includes LSTM temporal modeling and GNN spatial resource modeling; the LSTM performs temporal modeling through a gating mechanism, including a forget gate, input gate, candidate memory, output gate, and memory, and outputs a single-node load prediction value:

[0020]

[0021] in: For the load on a single node within the future forecast window, This is the parameter matrix for the corresponding gating unit. The hidden state at the current moment and Simultaneously input into the model The bias term of the gated unit; the GNN network takes the node feature matrix formed by combining the LSTM prediction vectors of all nodes as input, abstracts the network topology into an undirected weighted graph, and fuses spatial features through graph convolution operation to output a global load prediction matrix.

[0022] Preferably, in step 4, each domain agent selects at least one of the three deep reinforcement learning algorithms—DDPG, PPO, and SAC—to generate a local allocation strategy based on the characteristics of the resource domain.

[0023] Furthermore, the deep neural network architecture of the deep reinforcement learning algorithm described in step 4 includes an input layer, a hidden layer, and an output layer; the input layer concatenates the input feature vectors and unifies the data dimensions through "feature alignment technology"; the hidden layer includes a fully connected layer that abstracts the fused features and extracts associated features, and an attention mechanism layer for achieving multi-objective adaptive optimization; the output layer outputs the corresponding action according to the algorithm type.

[0024] Furthermore, the quantitative assessment described in step 5 includes the following steps:

[0025] (51) By combining deviation calculation and grade judgment, scores are given to the four indicators of resource utilization, task delay, energy consumption and priority satisfaction rate respectively;

[0026] (52) Calculate the comprehensive score by combining dynamic weights;

[0027] (53) Output a comprehensive evaluation report, strategy feedback signals and a list of optimization suggestions.

[0028] Furthermore, the dynamic weight calculation in step 52 is based on adaptive adjustment of task priority and system state, including a dynamic weighted reward function that integrates resource utilization, task delay, energy consumption and priority satisfaction rate. The dynamic weighted reward function incorporates the prediction results of the "LSTM+GNN" fusion model.

[0029] Furthermore, in step 7, the global collaboration layer dynamically selects at least one strategy fusion algorithm from weighted average fusion, voting selection fusion, or maximum value screening fusion based on the characteristics of the computing network scenario. Based on the performance indicators of each local strategy, the resource demand matching degree, and the global state vector, it generates a global collaborative scheduling strategy for computing, network, and storage resources.

[0030] Furthermore, the weighted average fusion algorithm dynamically calculates the fusion weight of each agent based on the deviation of the policy execution results from the global performance target within the current scheduling cycle, the adaptive base weights, and the resource domain load status. The weight formula is as follows:

[0031]

[0032] in, Let i be the fusion weight of agent i at time t. The base weights are given, and λ is the deviation influence coefficient. This represents the performance deviation rate.

[0033] Furthermore, the generation of the global scheme in step 7 includes the following steps:

[0034] (71) Information interaction and synchronization: receiving local information from each single-domain agent and integrating it into a global state vector with consistent specifications through the information interaction and synchronization module; the global state vector is used as the input of subsequent modules.

[0035] (72) Global policy fusion: The global policy fusion module extracts the local policy vectors of each agent from the global state vector and performs dimensional alignment. Based on the characteristics of the computing network scenario, the policy fusion algorithm is selected to generate the global policy.

[0036] (73) The cross-domain conflict detection and coordination module checks whether the constraints are violated based on the current resource status and global policy. When a conflict is detected, the policy is adjusted according to the nature of the conflict, a coordinated global policy is generated and the possibility is checked again. The cross-domain conflict detection and coordination module outputs the adjusted global scheduling policy and global feedback information.

[0037] (74) The global feedback optimization module calculates the comprehensive reward value and optimizes the global scheduling strategy based on the system execution results, and feeds back the reward signal and the optimized global scheduling strategy to each domain agent.

[0038] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: (1) Through domain-specific intelligent agent collaboration and dynamic resource allocation, the utilization rate of computing, network and storage resources is significantly optimized, reducing resource idleness and waste; (2) Based on forward-looking load prediction and real-time scheduling, the task execution delay is effectively shortened, while balancing energy consumption and ensuring service quality; (3) By utilizing the closed-loop optimization mechanism, the system can dynamically respond to load fluctuations and sudden risks, improving overall robustness and long-term operational stability; (4) Through the global collaboration layer, cross-domain collaboration and multi-objective adaptive optimization of heterogeneous computing network resources are realized, significantly improving resource utilization, task completion efficiency, service quality and energy consumption performance. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the global architecture of the present invention;

[0040] Figure 2 This is a schematic diagram of the domain-specific intelligent agent module of the present invention;

[0041] Figure 3 This is for the timing modeling of the LSTM in this invention. Detailed Implementation

[0042] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0043] As shown in the accompanying drawings, the present invention includes the following steps:

[0044] I. Constructing a Domain-Specific Adaptive Intelligent Agent

[0045] For heterogeneous media such as computing, network and storage resources in heterogeneous computing network systems, adaptive agents capable of independently completing complex resource scheduling tasks and possessing full-function closed-loop capabilities are designed to distinguish between computing resource domain, network resource domain and storage resource domain. The adaptive agent includes a state perception module, a policy network module, a value evaluation module and a policy update module. Each module deeply integrates cutting-edge technologies such as deep reinforcement learning (DRL), spatiotemporal prediction and multi-objective optimization, enabling a single agent to have the following core capabilities: multi-dimensional state perception, adaptive policy generation, forward-looking resource scheduling, and closed-loop feedback optimization.

[0046] 1. Implementation of the State Awareness Module

[0047] This module is responsible for collecting and processing multi-dimensional data from the heterogeneous computing network environment, realizing multi-dimensional and fine-grained heterogeneous computing network status awareness, and outputting in real time the representation of the current resource load of the system, the prediction of resource demand in the next 5-15 minutes, the matching degree between task characteristics and resource demand, and the early warning information of system bottlenecks and potential conflicts, which are used as input for intelligent decision-making by subsequent modules.

[0048] (1) Multi-source heterogeneous data are fused and processed to construct dynamic state vectors of computing resources, network resources and storage resources respectively, which are used to comprehensively characterize the resource load of the system at the current moment;

[0049] (2) Based on parameters such as task type, task priority, expected execution time and resource demand pattern, a task feature vector is constructed to achieve accurate characterization of task load;

[0050] (3) Extracting temporal and spatial features of resources and tasks based on the “LSTM+GNN” fusion model. Specifically, Long Short-Term Memory (LSTM) network is used to perform temporal modeling on historical task execution data to extract the periodic patterns and trend changes of task execution, and to predict resource demand in future periods; at the same time, Graph Neural Network (GNN) is used to model the topology of computing network resources and task dependencies to extract the spatial correlation features of the system, providing a structured basis for subsequent cross-resource domain collaborative scheduling.

[0051] “LSTM+GNN” fusion model construction and training:

[0052] like Figure 3 As shown, LSTM timing modeling uses a gating mechanism, where:

[0053] Forgotten Gate: It determines how much information to retain from historical cell states.

[0054] Input Gate: Used to control the proportion of new information incorporated.

[0055] Candidate memory units: Similar to the RNN's next hidden state calculation formula, it is used to capture new features of the current input.

[0056] Output gate: Used to adjust the hidden state generated at the current moment.

[0057] Memory element: Adjust the ratio of historical information to current new information in the current memory cell based on the input gate and forget gate.

[0058] Single-node load prediction output:

[0059] The hidden state of the LSTM at the last time step Input a fully connected layer to get the future Single-node load forecast for the step:

[0060]

[0061] in: The state information to be input into the model at time T; : Represents the hidden state at the previous time step and Simultaneously input into the model. , , etc.: Represents the parameter matrix of the corresponding gating unit. , , "etc." indicates the bias term of the corresponding gating unit. The activation function maps features to Mitigating gradient explosion Single-node load time series. The load on a single node within the future forecast window. For the predicted duration.

[0062] GNN spatial resource modeling uses a graph representation of its computational network topology: the computational network is abstracted as an undirected weighted graph. in: This represents the set of nodes in a computational network; This represents the set of edges in the computational network; It is an adjacency matrix. For the edge The weight value.

[0063] First, combine the LSTM prediction vectors of all nodes according to the node dimension to form the node feature matrix:

[0064]

[0065] As input to a Graph Neural Network (GNN), each row corresponds to a node's time-series prediction feature for a future window. The GNN then weights and fuses the node's own time-series prediction features with those of its neighbors, thereby capturing the spatial relationships between different nodes based on factors such as resource competition, link bandwidth, and topology. The graph convolutional layer is calculated as follows: the input to the GNN is the node feature matrix. The single-node prediction value output by the LSTM is used as the core feature and can be expressed by the formula as follows: Then, the GNN uses graph convolution operations to fuse the features of its own nodes with the features of its neighboring nodes, resulting in a spatially correlated node feature matrix. in and The feature matrix output by the GNN after representing the spatial relationships between nodes through the convolutional layer parameters. The input is fed into the fully connected layer to obtain the global load prediction matrix. , .

[0066] After completing the above LSTM temporal modeling and GNN spatial modeling structures, in order to enable the "LSTM+GNN" fusion model to accurately characterize the dynamic changes of heterogeneous computing networks, this invention performs end-to-end training on the model, and the specific steps are as follows.

[0067] Step 1: Training Sample Construction

[0068] Based on historical system operation data, a continuous length of [length missing] is constructed for each node in the computational network. Historical state sequence The state vector contains multi-dimensional resource metrics such as CPU / GPU utilization, link bandwidth usage, latency, and storage I / O. This will be followed by... The actual load of the step As a supervisory label, it enables overall prediction training, where the input sequences of all nodes constitute the overall training sample, eliminating the need to train each node individually.

[0069] Step 2: End-to-end forward reasoning process

[0070] First, the historical state sequence of each node is input into the LSTM network to obtain its initial prediction vector for the future window. Then, the LSTM prediction outputs of all nodes are combined according to the node dimension to form a node feature matrix. and the adjacency matrix of the constructed computing network. The inputs are combined into a Graph Neural Network (GNN). The GNN uses graph convolution operations to fuse the features of a node itself with the features of its neighboring nodes using edge weights, outputting a node feature matrix that includes spatial dependencies. Finally, through mapping using the fully connected layer, the future state of all nodes in the entire computing network is obtained. Prediction results of steps .

[0071] Step 3: Unify loss function and parameter update

[0072] To uniformly optimize the overall future load prediction of the entire computing network, this invention constructs a global multi-node prediction loss function:

[0073]

[0074] in For the number of nodes, The prediction step size is determined by this loss function. This loss is used to simultaneously update all parameters in both the LSTM and GNN through backpropagation, achieving joint optimization of temporal feature extraction and spatial feature modeling, thus forming a consistent spatiotemporal prediction capability.

[0075] Step 4: Training Mode: Offline Pre-training + Online Incremental Update

[0076] The model is first pre-trained offline on historical data to bring the prediction error to a stable range. During system operation, the state awareness module continuously generates the latest monitoring data and performs online incremental updates in small batches. The model parameters are fine-tuned according to the latest load pattern to adapt to the dynamic, sudden, and high-concurrency computing network environment.

[0077] Step 5: Connecting the prediction results with subsequent modules

[0078] The “LSTM+GNN” fusion model outputs the node-level load prediction results, potential bottleneck trends and conflict area prompts for the next 5 to 15 minutes in each scheduling cycle, providing forward-looking state input for the policy network module, value evaluation module and global collaboration layer. It is an important foundation for the adaptive agent proactive scheduling optimization of this invention.

[0079] 2. Implementation of the Policy Network Module

[0080] The adaptive agent policy network module uses DDPG / PPO / SAC deep reinforcement learning algorithms as its core, combined with the forward-looking capability of LSTM+GNN load prediction, to generate multi-objective adaptive resource allocation strategies, providing decision support for global collaborative scheduling.

[0081] (1) Deep Neural Network Architecture Design

[0082] Input layer: The feature vectors of the four inputs are concatenated, and the data dimensions are unified through "feature alignment technology" to avoid feature bias.

[0083] Hidden layer: A combination of "fully connected layer + attention mechanism layer" is adopted. The fully connected layer abstracts the fused features and extracts the associated features of "load-prediction-matching-warning". The attention mechanism layer (such as multi-head self-attention) gives higher weight to high-priority features (to achieve multi-objective adaptive optimization).

[0084] Output layer: Outputs corresponding actions according to the algorithm type—DDPG outputs continuous action values, PPO / SAC outputs action probability distributions, and the action space fully covers task allocation to CPU / GPU, network path selection, and storage I / O scheduling.

[0085] (2) Algorithm Differentiation Selection and Implementation

[0086] Computational resource agents: Their action space (such as CPU core allocation and GPU memory quota) is continuous, so the DDPG algorithm is adopted.

[0087] Network resource agents: Their action space (such as path selection) is discrete, so the PPO algorithm is used.

[0088] Storage resource agent: Its actions need to balance multiple objectives such as IO priority, so the SAC algorithm is adopted. This algorithm encourages exploration by maximizing the expected reward while maximizing the entropy of the policy.

[0089] (3) State-space design of a single agent

[0090] The state space fully receives the four outputs of the state awareness module and transforms them into vector forms that the algorithm can process. Specifically: the current resource load quantification dimension is transformed into a normalized numerical vector (CPU / GPU load rate, memory usage, etc.); the short-term demand prediction dimension is transformed into a time-series prediction vector (peak resource demand and task density distribution in the next 5-15 minutes); the task-resource matching degree dimension is transformed into a matching degree matrix vector (adaptation score of each task and resource); and the bottleneck and conflict warning dimension is transformed into a binary / hierarchical vector (system bottleneck node identifier, warning level, etc.).

[0091] (4) Action space design for a single agent

[0092] The action space must cover all core operations of single-domain resource scheduling and must be completely consistent with the "resource allocation action instructions" output by the policy network, specifically categorized by single domain:

[0093] Computation domain: The specific operations in the action space include task binding with computing nodes and computing resource quotas. The action form is a hybrid form, where binding is discrete and quotas are continuous.

[0094] Network domain: The specific operations in the action space include task transmission path selection and link bandwidth quota. The action form is a hybrid form, in which the path is discrete and the bandwidth is continuous.

[0095] Storage domain: The specific operations in the action space include binding task data to storage nodes / partitions and setting storage IO priorities. The action form is a hybrid form, in which the binding is discrete and the priority is continuous.

[0096] 3. Implementation of the value assessment module

[0097] The value assessment module, based on the output of the policy network module and actual execution data, quantitatively evaluates the effectiveness of single-domain resource scheduling strategies, providing accurate feedback for the subsequent policy update module. The model uses a bias calculation plus a ranking method to score four core dimensions: resource utilization, latency, energy consumption, and priority fulfillment rate, and then combines dynamic weights to calculate a comprehensive score.

[0098] (1) The value assessment module receives three types of input ( , , ), from feedback Four core performance indicators were extracted: resource utilization, task latency, energy consumption, and priority fulfillment rate. Performance deviation rates were calculated for each indicator. ;in For feedback The actual performance values ​​extracted from them The target performance set for the system (determined by task priority or policy objectives);

[0099] (2) Then, based on the grade determination, the deviation rate is mapped to the individual score. (For example Achieve multi-objective adaptive optimization with a single item score of 0.6 in the range of 0.15–0.30;

[0100] (3) Introduce a dynamic weighting mechanism, incorporating task priority, real-time performance deviation, and prediction risk into the scoring process. The dynamic weights of the four types of indicators are defined as follows: ;in This is the deviation penalty coefficient. The basic weights are used for the indicators. The comprehensive score is calculated based on the dynamic weights. Calculate the reward value .

[0101] Final model output:

[0102] (1) Comprehensive evaluation report: includes individual scores, overall scores, and deviation rates of each of the four dimensions, clearly identifying the advantages and disadvantages of the strategy.

[0103] (2) Policy feedback signal: The comprehensive score is converted into a reward value and directly input into the policy update module as the basis for adjusting the policy parameters of the DDPG / PPO / SAC algorithm (e.g., the Critic network of DDPG updates the value assessment based on the reward value, thereby optimizing the action generation of the Actor network).

[0104] (3) Optimization suggestion list: Based on the deviation analysis, targeted suggestions are proposed to provide directional reference for strategy updates and avoid blind exploration of the algorithm.

[0105] 4. Implementation of the strategy update module

[0106] The core function of the strategy update module is to generate feedback signals (reward signals) based on the output of the value assessment module. Based on the comprehensive evaluation results and optimization suggestion list, and combined with the parameter update logic of deep reinforcement learning algorithms (DDPG / PPO / SAC), the model parameters of the policy network module are dynamically adjusted. Since the policy network module can support different algorithms (DDPG, PPO, SAC) with different optimization logics, the parameter update paths differ, requiring separate implementation schemes. Taking the parameter update of the DDPG (Deep Deterministic Policy Gradient) algorithm as an example...

[0107] DDPG adopts an Actor-Critic dual-network architecture, and parameter updates require simultaneous optimization of the action generation network (Actor) and the value evaluation network (Critic).

[0108] In the Critic network update, our goal is to optimize the accuracy of the Critic network's evaluation of "action value," enabling it to more accurately determine the "long-term benefit of the current action in the current state." This is implemented by using a feedback reward value based on the value evaluation module. Current status Next state Construct the temporal difference (TD) loss function:

[0109] in, These are the current Critic network parameters. The target Critic network parameters (via a "soft update" mechanism, i.e.) , (Use a value between 0.001 and 0.01 to avoid parameter oscillation) It is a discount factor (0.9~0.99), balancing immediate rewards and long-term benefits. The loss function is minimized using a gradient descent algorithm (such as Adam), and the value is updated. .

[0110] Our goal in updating the Actor network is to optimize its ability to generate "high-value actions," ensuring that its output resource allocation actions maximize the value evaluated by the Critic network. This is achieved by constructing a policy loss function that is "value-oriented" to the output of the Critic network:

[0111]

[0112] in, These are the current Actor network parameters. These are actions generated by the Actor. Updated using the gradient ascent algorithm (maximizing the Q-value). Meanwhile, referring to the "optimization suggestions" in the value assessment module, corresponding penalty terms are added to the gradient calculation to guide the action towards optimization.

[0113] The algorithm dynamically adjusts the learning rate based on the feedback reward value. If the policy performance decreases after the update, it triggers "parameter backtracking," restoring the optimal historical parameters before the update to ensure system stability. Finally, the module outputs the updated policy network parameters; for DDPG, it outputs the updated Actor network. Critic Network and target network parameters .

[0114] PPO limits the policy update magnitude by pruning the objective function, maintaining good stability in the discrete action space. It is suitable for discrete or weakly mixed action decision-making scenarios in the network resource domain where path selection is the primary focus. This invention directly adopts its standard proximal policy optimization structure in the network domain without additional modifications. SAC, based on the maximum entropy reinforcement learning framework, enhances exploration capabilities by adding an entropy term to the reward, exhibiting stronger optimization capabilities for continuous or continuously dominant action spaces. It is suitable for scheduling scenarios in the computation and storage domains where continuous actions exist, such as resource quotas, bandwidth ratios, and I / O priorities. This invention reuses its standard Actor-Critic framework in these two types of resource domains and adopts a multi-output head form to support the joint expression of discrete and continuous actions based on the action structure.

[0115] II. Specific Implementation of the Global Collaboration Layer (Meta-Agent)

[0116] The global collaboration layer connects computing, network, and storage domain-specific adaptive agents. Through lightweight strategy information sharing, global goal-oriented strategy fusion, and dynamic resolution of cross-domain conflicts, it integrates single-domain local optimization strategies into a globally optimal scheduling scheme, while avoiding interference with the autonomy of single-domain agents.

[0117] 1. Components of the global collaboration layer

[0118] The global collaboration layer includes an information interaction and synchronization module, a global strategy fusion module, a cross-domain conflict detection and coordination module, and a global feedback optimization module. Among them:

[0119] (1) Information Interaction and Synchronization Module: This module serves as the data entry point for the collaboration layer, responsible for receiving key information from single-domain agents and achieving real-time synchronization to ensure that fusion decisions are based on the latest state. First, the information interaction and synchronization module is responsible for constructing a globally unified state space. Each single-domain agent (computation domain...) Network domain With storage domain Based on its local observations and predictions, state vectors are generated respectively. Local policies obtained from mapping These strategies will be collected uniformly in this module, along with real-time resource load data. (Real-time resource load for each resource domain, such as CPU / GPU utilization, memory usage, bandwidth, storage usage, etc.), future load prediction. (Future resource load prediction data generated by the LSTM+GNN hybrid model) and bottleneck early warning information (Bottleneck warning information generated by each domain agent, indicating potential resource conflicts or bottlenecks), organized into a globally consistent state vector. This information is then output to the next module. This ensures that subsequent fusion decisions are based on the latest state within the same time slice, avoiding strategy deviations caused by information asynchrony.

[0120] (2) Global Strategy Fusion Module: This module is the core decision-making unit of the collaboration layer, responsible for receiving data from the computing domain. Network domain and storage domain Local strategies based on the global objective function The various local strategies are weighted and fused to generate a global scheduling strategy. First, the information interaction and synchronization module integrates the key information of each single-domain agent into a global state space. This is then passed to this module. Upon receiving the global state space, the module extracts the local policy vectors of each individual domain, and then performs weighted fusion based on these local policies to generate a global scheduling policy. This module first extracts the local policy vectors from the global state space. Extract the local policy vector generated by each single-domain agent Then, based on the global state space... It includes the real-time status and prediction information of each resource domain, and calculates the dynamic weight of each resource domain. To reflect the importance of each resource domain in global scheduling, a fusion algorithm is then used to weight and combine the policy vectors to generate a global scheduling policy. In this way, this module ensures collaboration and optimization among multiple resource domains, makes reasonable decisions using the global state space, and achieves optimal scheduling of global performance in a dynamic environment.

[0121] (3) Cross-domain conflict detection and coordination module: This module is responsible for detecting conflicts in the global scheduling strategy and coordinating possible cross-domain resource conflicts to ensure the feasibility and stability of the global scheduling strategy. First, the module receives the global scheduling strategy from the global strategy fusion module. Based on the current resource status (such as load, bandwidth, storage, etc.) and the global scheduling policy, conflict detection is performed. If resource constraint conflicts are found or constraints that do not meet the global objectives are not met, the module will resolve the conflicts by adjusting weights or modifying certain resource allocations to ensure stable system operation. This module first checks the global scheduling policy. Do they meet the hard constraints and global constraints of each resource domain? This ensures the generated global scheduling policy is feasible. If the global policy violates resource constraints (such as exceeding computing resources, bandwidth, or storage limits), adjustments are necessary. This module will then base its decisions on the global scheduling policy and real-time load data for each resource domain. and bottleneck early warning information The module detects cross-domain resource conflicts. Once a conflict is detected, it will take appropriate adjustment strategies based on the nature of the conflict: resource weight adjustment, task allocation adjustment, or resource reallocation. After adjusting the strategies, the module will perform a feasibility check on the global strategies again to ensure that the adjusted global strategies comply with system constraints and that coordination between resource domains does not compromise the global objectives. Finally, this module will output the adjusted global scheduling strategy. and global feedback information .

[0122] (4) Global Feedback Optimization Module: This module is responsible for optimizing the global scheduling strategy based on the system execution results. It monitors the system's execution performance, calculates reward values, and feeds them back to the agents in each resource domain for strategy adjustments. The goal of this module is to ensure the system maintains efficient and stable operation under dynamic load, task demands, and resource status changes through continuous optimization of the global scheduling strategy. The global feedback optimization module utilizes execution feedback and reward signals to guide the continuous improvement of the global scheduling strategy to achieve multi-objective optimization, such as task latency, resource utilization, and energy consumption. First, this module optimizes the global scheduling strategy based on the system's execution results. The execution effect of the global strategy Calculate the reward value of the current global scheduling policy. .in, It is an aggregation function that calculates a comprehensive reward by combining multiple factors such as task latency, energy efficiency, and resource utilization. Then, based on the reward value... This module will optimize the global scheduling strategy. The global scheduling strategy is adjusted based on feedback from reward signals to ensure optimal goal performance. Specifically, the module updates the global objective function based on the new reward value and feedback signals. The parameters are adjusted to better meet the requirements for long-term stable system operation. Finally, the module outputs the optimized global scheduling strategy. .

[0123] 2. Implementation of the strategy fusion algorithm

[0124] In heterogeneous computing networks, under different load scenarios, resource domain states, and policy quality distributions, a single fusion method cannot guarantee the stability and optimality of global scheduling. Therefore, differentiated fusion methods must be selected based on the characteristics of the computing network scenario. The global policy fusion module selects differentiated fusion methods based on the characteristics of the computing network scenario, including weighted average fusion, voting selection fusion, and maximum value selection fusion. This ensures that the generated global policy not only aligns with the capabilities of each resource domain but also meets the global objectives.

[0125] (1) Weighted average fusion

[0126] Weighted average fusion combines the local policies of each agent using dynamic weights in a linear combination. The weights are dynamically adjusted based on the contribution of each agent's policy to global performance, ensuring that the fused global policy considers multi-domain optimization objectives and avoids local optima. It is suitable for scenarios with stable network load and no significant bottlenecks.

[0127] Dynamic weight calculation

[0128] Weight Based on basic weights Together with the performance deviation correction term, the correction term reflects the degree of deviation between the agent's current policy and the global objective. , The deviation influence coefficient is usually taken as Avoid excessive fluctuations in weighting.

[0129] Dynamic weight formula:

[0130]

[0131] The constraints are ,in This is to prevent intelligent agents from being ignored due to low weight, and to ensure cross-domain collaboration.

[0132] in The formula for calculating the performance deviation rate is as follows:

[0133]

[0134] Among them, performance deviation rate Measuring intelligent agents The deviation between the performance of local strategies and the global objective. The larger, The further the strategy deviates from the goal. for The local performance target value. The meanings are shown in Table 1.

[0135] symbol meaning a set of domain agents, = a computing agent, = a network agent, = a storage agent The global state vector of the computing network contains m performance metrics. It is an intelligent agent The local policy vector, yes strategy space Time-t intelligent agent Fusion weights intelligent agent The local policy performance function, quantization Contribution to the global state The global performance constraint function defines the hard constraints that the computational network must satisfy.

[0136] Global strategy integration

[0137] Let the local policy vectors of each agent at time t be... At this point, it is necessary to first align the policy dimensions (this can be done using zero-expansion to fill in the largest dimension among the three), resulting in the following aligned policy matrix:

[0138]

[0139] Global Strategy For weighted linear combinations:

[0140]

[0141] Constraints: ,in This provides a globally feasible strategy space to ensure the feasibility of strategies. The strategy must meet certain hard constraints.

[0142] (2) Voting selection fusion and maximum value filtering fusion

[0143] The voting-based fusion approach is based on the majority rule principle. Each single-domain agent independently votes on a set of discrete candidate policies, ultimately selecting the policy with the highest number of votes. This method does not rely on continuous scoring and is suitable for scenarios where the quality differences between policies of different agents are small, or where it is necessary to avoid the influence of individual anomalous policies on the global decision, thus significantly improving the robustness of the decision.

[0144] The maximum value selection fusion method compares the quantitative scores (such as Q-scores, dominance values, or performance metrics) given by each agent for its local policy, and directly selects the policy with the highest score. This method is suitable for scenarios where a policy in a certain resource domain is significantly better than other policies, and where it is necessary to quickly select the globally optimal solution.

[0145] Both of the above-mentioned fusion methods belong to the policy selection rules commonly used in multi-agent collaboration and integrated decision-making. Their decision-making logic is clear and their implementation is intuitive. They can be directly completed based on existing deep reinforcement learning structures without the need for additional complex mechanisms.

[0146] III. Overall System Workflow

[0147] Initialization: Start all agents and the global collaboration layer, load the pre-trained model or initialize randomly. Configure system parameters.

[0148] Operation process:

[0149] 1. The state awareness module periodically collects data and runs the "LSTM+GNN" fusion model to generate future load predictions.

[0150] 2. The policy network of each agent generates local resource allocation actions based on the current state vector.

[0151] 3. Local actions are reported to the global collaboration layer for fusion and conflict resolution, generating the final global action instructions.

[0152] 4. The action command is issued to the resource pool for execution.

[0153] 5. The value assessment module collects system metrics after execution and calculates rewards.

[0154] 6. The policy update module updates the policy network parameters of each agent using reward signals. Simultaneously, the global collaboration layer adjusts its fusion weight rules based on the results of this round.

[0155] Repeat steps 1 to 6 to achieve continuous optimization.

Claims

1. A method for collaborative scheduling optimization of heterogeneous computing network resources based on adaptive multi-agent systems, characterized in that, Includes the following steps: (1) Constructing domain-specific adaptive agents: Design adaptive agents for computing resource domain, network resource domain and storage resource domain respectively. The adaptive agents include a state perception module, a policy network module, a value evaluation module and a policy update module, forming an independent closed-loop optimization structure. When processing single resource domain tasks, each agent independently executes the resource allocation policy generated based on local state perception. (2) Design a global collaboration layer: connect the domain-specific adaptive agents, receive and synchronize key information of each single-domain agent in real time through the information interaction and synchronization module, the global strategy fusion module integrates the local strategies of each single-domain agent to generate a global scheduling strategy, the cross-domain conflict detection and coordination module detects global scheduling strategy conflicts and coordinates their resolution, and the global feedback optimization module optimizes the global scheduling strategy based on the system execution results to achieve cross-domain strategy collaboration; (3) Perceiving the state of heterogeneous computing networks: Real-time resource data, task load description, temporal feature extraction and spatial feature extraction are collected through the state perception module, and the current resource load quantitative representation, resource demand prediction in the next 5-15 minutes, task feature-resource demand matching degree, system bottleneck and potential conflict warning information are output. (4) Generate and execute local policies: Through the policy network module, with the output of the state awareness module as input, generate local allocation policies for each resource domain based on the deep reinforcement learning algorithm; for tasks involving only a single resource domain, directly execute the local policy to complete the scheduling; the local policy includes task binding and resource quota for the computing domain, path selection and bandwidth quota for the network domain, and data binding and IO priority setting for the storage domain. (5) Quantitative evaluation: The value evaluation module performs multi-dimensional quantitative evaluation of the local strategy execution effect, including resource utilization, task latency, energy consumption and priority satisfaction rate, and outputs feedback signals. (6) Iterative optimization: Based on the feedback signal, the policy network parameters are dynamically adjusted through the policy update module to achieve single-domain closed-loop iterative optimization; (7) Generate a global scheme: When the task is multi-resource domain collaboration, the local policies output by each adaptive single agent are integrated through the global collaboration layer to perform cross-domain policy fusion and conflict resolution, and a global collaborative scheduling scheme is generated. The global collaboration layer generates a global collaborative scheduling strategy for computing, network and storage resources based on the performance indicators, resource demand matching degree and global state vector of each local policy, and ensures global feasibility through conflict detection.

2. The collaborative scheduling optimization method according to claim 1, characterized in that, Step 3, sensing the state of the heterogeneous computing network, includes the following steps: (31) To comprehensively characterize the system resource status, a dynamic state vector of computing resources, network resources, and storage resources is constructed by using multi-source heterogeneous data fusion technology; (32) Characterize task characteristics and system requirements, and construct task feature vectors based on features such as task type, priority, execution time, and resource requirement patterns; (33) Proactively predict the short-term load of heterogeneous computing networks. Based on the "LSTM+GNN" fusion model, LSTM captures the periodic and trend characteristics of historical load, and GNN models the spatial correlation characteristics of resource topology and task dependence, accurately predicting the fine-grained load distribution and bottleneck risks in the next 5-15 minutes. (34) Output heterogeneous resource state vectors and task load descriptions. Adopt a multi-level data fusion strategy to process the original monitoring data, task features and spatiotemporal features in a unified manner. Perform data preprocessing, feature extraction, state vector construction and dynamic update in sequence to form a system state representation for decision-making.

3. The collaborative scheduling optimization method according to claim 2, characterized in that, The "LSTM+GNN" fusion model described in step 33 includes LSTM temporal modeling and GNN spatial resource modeling; the LSTM performs temporal modeling through a gating mechanism, including a forget gate, input gate, candidate memory, output gate, and memory, and outputs a single-node load prediction value: in: For the load on a single node within the future forecast window, This is the parameter matrix for the corresponding gating unit. The hidden state at the current moment and Simultaneously input into the model For the bias term of the gating unit; The GNN network takes the node feature matrix formed by combining the LSTM prediction vectors of all nodes as input, abstracts the network topology into an undirected weighted graph, and fuses spatial features through graph convolution operations to output a global load prediction matrix.

4. The collaborative scheduling optimization method according to claim 1, characterized in that, In step 4, each domain agent selects at least one of the three deep reinforcement learning algorithms—DDPG, PPO, and SAC—to generate a local allocation strategy based on the characteristics of the resource domain.

5. The collaborative scheduling optimization method according to claim 1, characterized in that, The deep neural network architecture of the deep reinforcement learning algorithm described in step 4 includes an input layer, a hidden layer, and an output layer. The input layer concatenates the input feature vectors and unifies the data dimensions through "feature alignment technology". The hidden layer includes a fully connected layer that abstracts the fused features and extracts related features, and an attention mechanism layer for multi-objective adaptive optimization. The output layer outputs the corresponding action according to the algorithm type.

6. The collaborative scheduling optimization method according to claim 1 or 2, characterized in that, The quantitative assessment described in step 5 includes the following steps: (51) By combining deviation calculation and grade judgment, scores are given to the four indicators of resource utilization, task delay, energy consumption and priority satisfaction rate respectively; (52) Calculate the comprehensive score by combining dynamic weights; (53) Output a comprehensive evaluation report, strategy feedback signals and a list of optimization suggestions.

7. The collaborative scheduling optimization method according to claim 6, characterized in that, The dynamic weight calculation in step 52 is based on adaptive adjustment of task priority and system state, including a dynamic weighted reward function that integrates resource utilization, task delay, energy consumption and priority satisfaction rate. The dynamic weighted reward function incorporates the prediction results of the "LSTM+GNN" fusion model.

8. The collaborative scheduling optimization method according to claim 1, characterized in that, In step 7, the global collaboration layer dynamically selects at least one strategy fusion algorithm from weighted average fusion, voting selection fusion, or maximum value screening fusion based on the characteristics of the computing network scenario. Based on the performance indicators of each local strategy, the resource demand matching degree, and the global state vector, it generates a global collaborative scheduling strategy for computing, network, and storage resources.

9. The collaborative scheduling optimization method according to claim 8, characterized in that, The weighted average fusion algorithm dynamically calculates the fusion weight of each agent based on the deviation of the policy execution results from the global performance target within the current scheduling cycle, the adaptive base weights, and the resource domain load status. The weight formula is as follows: in, Let i be the fusion weight of agent i at time t. The base weights are given, and λ is the deviation influence coefficient. This represents the performance deviation rate.

10. The collaborative scheduling optimization method according to claim 1, characterized in that, Step 7, generating the global solution, includes the following steps: (71) Information interaction and synchronization: receiving local information from each single-domain agent and integrating it into a global state vector with consistent specifications through the information interaction and synchronization module; the global state vector is used as the input of subsequent modules. (72) Global policy fusion: The global policy fusion module extracts the local policy vectors of each agent from the global state vector and performs dimensional alignment. Based on the characteristics of the computing network scenario, the policy fusion algorithm is selected to generate the global policy. (73) The cross-domain conflict detection and coordination module checks whether the constraints are violated based on the current resource status and global policy. When a conflict is detected, the policy is adjusted according to the nature of the conflict, a coordinated global policy is generated and the possibility is checked again. The cross-domain conflict detection and coordination module outputs the adjusted global scheduling policy and global feedback information. (74) The global feedback optimization module calculates the comprehensive reward value and optimizes the global scheduling strategy based on the system execution results, and feeds back the reward signal and the optimized global scheduling strategy to each domain agent.

Images