A method and system for multi-agent cooperative scheduling of terminal clusters
By constructing a dynamic spatiotemporal state graph and a graph attention network, the agent performs task offloading and computing power allocation in the terminal cluster, solving the scheduling imbalance problem under dynamic topology and heterogeneous computing power, and realizing efficient collaborative scheduling and resource matching.
Patent Information
- Application Number
- CN202611268602.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-20
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies struggle to achieve effective collaborative scheduling in dynamic topology and heterogeneous computing power terminal cluster environments, leading to task allocation imbalances and collaborative failures, and failing to respond in real time to network topology changes and differences in computing power among heterogeneous nodes.
A dynamic spatiotemporal state graph is constructed, and the state of nodes and links is weighted and aggregated through a graph attention network. The agent outputs action instructions for task unloading and computing power allocation based on local feature vectors. A multi-head attention mechanism and a time decay factor are adopted to enhance the adaptability and accuracy of scheduling decisions.
It achieves decentralized collaborative scheduling in dynamic topology and heterogeneous computing scenarios, reduces communication latency and single point of failure risk, improves the fine-grained matching capability between tasks and resources, and enhances the adaptability and accuracy of scheduling decisions.
Smart Images

Figure CN123123439A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, specifically to a method and system for multi-agent collaborative scheduling of terminal clusters. Background Technology
[0002] With the evolution of distributed computing technology, the scale of terminal clusters continues to expand, making collaborative scheduling technology a core element in allocating computing tasks and network resources. In traditional terminal cluster scheduling architectures, a centralized master node allocation model is typically used for task scheduling. The master node acts as the control center, periodically collecting status information such as computing power utilization, memory usage, and task queue length from all worker nodes within the cluster. Simultaneously, it aggregates network link connectivity and bandwidth capacity data between nodes. Based on the collected global status information, the master node executes centralized scheduling decision logic, uniformly allocating tasks to be processed to each worker node for execution and specifying the corresponding network transmission path and computing power quota. This centralized scheduling model, in environments with limited node size and relatively stable network topology, can achieve comprehensive resource planning and load balancing based on a global perspective, meeting the scheduling needs of routine computing tasks.
[0003] Building upon this foundation, to address the scaling needs of large-scale nodes, some existing technologies introduce global scheduling algorithms or centralized reinforcement learning models based on the homogeneous node assumption. These solutions presuppose that each node in the terminal cluster possesses similar or identical computing power and communication bandwidth, treating the entire cluster as a homogeneous resource pool. During scheduling decisions, the algorithm focuses on the macroscopic matching of task arrival rate and overall resource utilization, distributing tasks according to a fixed scheduling policy or a pre-trained global policy network. When scheduling is required, the scheduler directly selects idle nodes from the homogeneous resource pool for offloading based on the computational load required by the task, eliminating the need for fine-grained joint evaluation of the heterogeneous computing power characteristics of individual nodes and the dynamic changes in links between nodes. This simplifies the complexity and state space dimension of scheduling decisions.
[0004] However, in real-world terminal cluster operating environments, node movement, failures, or hibernation cause the network topology to be constantly changing dynamically, and the differences in hardware configuration and load among nodes result in highly heterogeneous computing power. Centralized master nodes struggle to obtain accurate global states in real time under dynamic topologies, and communication delays and single-point-of-failure risks lead to delayed or interrupted scheduling instructions. Global scheduling algorithms based on the isomorphic assumption lack awareness of topological dynamics and heterogeneous computing power, resulting in a severe mismatch between scheduling decisions and the actual processing capabilities of nodes. Tasks are frequently assigned to nodes with insufficient computing power or broken links, causing task queuing, retransmission, or even dropping. Existing technologies fail to jointly represent dynamic network topology and heterogeneous computing power states within the same structure and achieve decentralized decision-making based solely on local observations, leading to core technical problems of collaborative failure and scheduling imbalance in scenarios where dynamic topologies and heterogeneous computing power coexist. Summary of the Invention
[0005] The purpose of this invention is to provide a method and system for multi-agent collaborative scheduling of terminal clusters, which can effectively solve the problems in the background art.
[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A multi-agent collaborative scheduling method for a terminal cluster includes: collecting the real-time computing power utilization and network link connectivity status of each node in the terminal cluster, and constructing a dynamic spatiotemporal state graph by using the nodes as graph nodes and the network links as edges. Each agent acts as a graph node, and performs weighted aggregation of the state features of the adjacent nodes through a graph attention network. The attention weights of the weighted aggregation are jointly determined by the computing power margin of the adjacent nodes and the link latency. Based on the weighted aggregated local feature vectors, the agent outputs action instructions for task unloading and computing power allocation through a policy network.
[0007] Preferably, constructing a dynamic spatiotemporal state graph by using the nodes as graph nodes and the network links as edges includes: extracting the remaining computing resources, memory usage, and task queue length of the nodes as node attribute features of the graph nodes; The bandwidth capacity, transmission latency, and packet loss rate of the network link are extracted as edge attribute features of the edge; The node attribute features and edge attribute features are concatenated and dimensionality-reduced to generate a multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph.
[0008] Preferably, the attention weight of the weighted aggregation is jointly determined by the computing power margin of the adjacent nodes and the link latency, including: calculating the absolute value of the computing power difference between the graph node and the adjacent node, and performing a nonlinear mapping on the absolute value of the computing power difference to obtain a computing power association score; Calculate the reciprocal of the link delay between the graph node and its neighboring node, and perform a nonlinear mapping on the reciprocal of the link delay to obtain a communication association score; The computing power association score and the communication association score are multiplied and normalized to obtain the attention weight of the neighboring node to the graph node.
[0009] Preferably, the action instructions for task unloading and computing power allocation by the agent based on the weighted aggregated local feature vector through the policy network include: inputting the local feature vector into the fully connected layer of the policy network to extract the scheduling preference features of the agent; Based on the scheduling preference features, calculate the target node probability distribution for task unloading and the resource allocation probability distribution for computing power allocation in the preset action space; Based on the probability distribution of the target node and the probability distribution of the resource configuration, discrete action instructions for task unloading and computing power allocation are generated.
[0010] Preferably, before the agent outputs the action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector, the method further includes: obtaining the agent's own observation vector at the current time step, wherein the own observation vector includes the agent's local task arrival rate and local computing power utilization rate. The self-observation vector and the weighted aggregated neighbor vector are subjected to feature concatenation and activation processing to generate the local feature vector that integrates local topological information and self-state information.
[0011] Preferably, after generating the multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph, the method further includes: introducing a time decay factor to attenuate the historical features in the multidimensional state embedding vector within consecutive time steps. The real-time features collected at the current time step are residually connected with the historical features attenuated by the time decay factor to update the state representation of the graph nodes and edges in the dynamic spatiotemporal state graph, so as to capture the spatiotemporal evolution pattern of the terminal cluster.
[0012] Preferably, multiplying and normalizing the computing power association score and the communication association score to obtain the attention weight of the neighboring node to the graph node includes: using a multi-head attention mechanism, inputting the computing power association score and the communication association score into multiple independent attention heads respectively; Each attention head calculates the sub-attention weights between the graph node and its neighboring nodes in different subspaces; All the sub-attention weights are concatenated and linearly transformed to obtain the attention weights that incorporate multi-dimensional scheduling constraints.
[0013] Preferably, before the agent outputs the action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector, it further includes: obtaining a set of tasks to be scheduled, extracting the latency sensitivity and computational complexity of each task to be scheduled in the set of tasks to be scheduled, and constructing a task feature matrix; The task feature matrix and the local feature vector are cross-attention calculated to extract the matching degree feature between the task to be scheduled and the current state of the terminal cluster. The matching degree feature is used as an additional input condition for the policy network.
[0014] Preferably, the weighted aggregation of the state features of the adjacent nodes of the graph node through the graph attention network further includes: when each agent transmits the state features and the action instructions, performing random sparsification processing on the state features and the action instructions, retaining only the feature components whose absolute value is greater than a preset pruning threshold. The receiving end performs zero-filling restoration on the information that has undergone the random sparsification process, and triggers asynchronous parameter updates of the policy network based on the local clock, thereby reducing the communication load between the agents.
[0015] A multi-agent collaborative scheduling system for a terminal cluster includes: a state acquisition processor, used to acquire the real-time computing power utilization and network link connectivity status of each node in the terminal cluster, and to construct a dynamic spatiotemporal state graph by using the nodes as graph nodes and the network links as edges. A graph attention aggregation processor is used to enable each agent to act as a graph node, and to perform weighted aggregation of the state features of the adjacent nodes of the graph node through a graph attention network. The attention weight of the weighted aggregation is jointly determined by the computing power margin of the adjacent nodes and the link latency. The scheduling decision processor enables the agent to output action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector.
[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention constructs a dynamic spatiotemporal state graph by collecting real-time computing power utilization and network link connectivity status of each node in a terminal cluster. It jointly represents heterogeneous computing power and topological dynamics within the graph structure, breaking the assumption of a homogeneous resource pool and enabling scheduling decisions to have a perceptual basis for topological and computing power differences. Each agent, acting as a graph node, performs weighted aggregation of the state features of adjacent nodes through a graph attention network. The attention weights for this aggregation are jointly determined by the computing power margin and link latency of adjacent nodes. This allows agents to extract state features based solely on observations of their local topological neighborhood, eliminating strong dependence on global state information and overcoming the communication latency and single-point-of-failure defects of centralized scheduling in dynamic topologies. Based on local feature vectors that integrate local topological information and their own state information, agents output action commands for task unloading and computing power allocation through a policy network, achieving decentralized collaborative decision-making. This avoids mismatch between scheduling commands and the actual processing capabilities of nodes, solving the core technical problems of collaborative failure and scheduling imbalance in scenarios where dynamic topologies and heterogeneous computing power coexist.
[0017] 2. This invention introduces a time decay factor to attenuate historical features in the multidimensional state embedding vector, and performs residual connection between the real-time features of the current time step and the attenuated historical features to update the state representation of the dynamic spatiotemporal state graph. This captures the spatiotemporal evolution of the terminal cluster and enhances the adaptability of scheduling decisions under continuous temporal changes. A multi-head attention mechanism is employed to calculate the sub-attention weights between graph nodes and adjacent nodes in different subspaces, integrating multidimensional scheduling constraints to improve the dimensionality and accuracy of feature extraction under complex constraints. The latency sensitivity and computational complexity of the task to be scheduled are extracted to construct a task feature matrix. Cross-attention calculation is performed between the task feature matrix and local feature vectors to extract the matching degree features between the task to be scheduled and the current state of the terminal cluster as additional input conditions for the policy network, achieving fine-grained matching of task requirements and node resources. Random sparsity processing is performed when transmitting state features and action instructions between agents, retaining only feature components with absolute values greater than a pruning threshold. After restoration at the receiving end, asynchronous parameter updates of the policy network are triggered based on the local clock, reducing the amount of communication data and synchronization waiting overhead between agents. Attached Figure Description
[0018] Figure 1 This is a flowchart illustrating the step-by-step construction process of the dynamic spatiotemporal state diagram of the terminal cluster according to the present invention. Figure 2 This is a flowchart of the node multidimensional embedding vector temporal update process based on the time decay factor of the present invention. Figure 3 This is a flowchart illustrating the step-by-step calculation process of single-head attention weight under the joint constraints of computing power and link latency of the present invention. Figure 4This is a flowchart of the multi-head attention mechanism for weighted aggregation of neighborhood node features in this invention. Figure 5 This is a flowchart of the task cross-attention matching and policy network input feature preprocessing process of the present invention; Figure 6 A flowchart is provided for the generation of the full process of intelligent agent feature sparse transmission, asynchronous parameter update and scheduling instructions in this invention. Detailed Implementation
[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0020] Please refer to Figure 1This embodiment provides a multi-agent collaborative scheduling method and system for terminal clusters, collecting real-time computing power utilization and network link connectivity status of each node in the terminal cluster. Specifically, each node periodically collects its own CPU utilization, GPU utilization, memory usage, and task queue length through its local operating system interface, with a collection period set to 100 milliseconds. In this embodiment, CPU utilization is collected by reading the system status interface provided by the operating system. For Linux kernel-based operating systems, the CPU line data in the / proc / stat file is read, and the proportion of time slices in user mode, kernel mode, and idle mode is calculated to obtain the overall CPU utilization. For Windows kernel-based operating systems, CPU utilization is obtained by calling the Processor%ProcessorTime counter in the Windows performance counters. GPU utilization is collected by calling the programming interface provided by the GPU manufacturer. For NVIDIA GPUs, the query interface of the nvidia-smi tool is called to obtain information such as GPU utilization and video memory usage; for AMD GPUs, the query interface of the rocm-smi tool is called to obtain the corresponding information. Memory usage is collected by reading the operating system's memory status information. For Linux systems, the MemTotal and MemUsed fields in the / proc / meminfo file are read to calculate the ratio of used memory to total memory. For Windows systems, the memory usage is obtained by calling the Memory%CommittedBytesInUse counter in the Windows performance counters. Task queue length is collected by counting the number of tasks waiting to be executed in the node's local task scheduler. The task scheduler maintains a first-in, first-out (FIFO) task queue, where each task entering the node is stored before execution begins.
[0021] Simultaneously, each node periodically sends probe packets to its neighboring nodes. Each probe packet is 64 bytes in size, and the sending period is 50 milliseconds. Link latency is calculated by measuring the round-trip time of the probe packets, and the packet loss rate is calculated by counting the number of lost probe packets. Link connectivity is determined by detecting whether probe packets from neighboring nodes are received. The network link connectivity, transmission latency, and packet loss rate are collected through the periodic sending of probe packets. In this embodiment, the probe packets are encapsulated using the User Datagram Protocol (UDP) and include the sender's identifier, sending timestamp, and sequence number. Each node maintains a list of neighboring nodes, containing nodes currently connected to that node. Each node sends probe packets to all nodes in its neighboring node list at 50-millisecond intervals and simultaneously receives probe packets from other nodes. When a node receives a probe packet from a node, it records the receiving timestamp, calculates the round-trip time of the link based on the sending and receiving timestamps, and uses half of the round-trip time as the one-way transmission latency. The packet loss rate is calculated by counting the number of lost probe packets out of 10 consecutively sent packets. If a node does not receive a probe data packet from a certain node for three consecutive cycles, it determines that the link between that node and itself is broken and removes that node from the list of neighboring nodes; if a node receives a probe data packet from a node that is not in the list of neighboring nodes, it determines that the link between that node and itself is connected and adds that node to the list of neighboring nodes.
[0022] A dynamic spatiotemporal state graph is constructed by treating nodes as graph nodes and network links as edges. In this embodiment, the dynamic spatiotemporal state graph is represented as follows: ,in It is the set of graph nodes at the t-th time step, and each graph node corresponds to a computing node in the terminal cluster; Let be the set of edges at time step t. Each edge corresponds to a network link between two nodes in the graph. An edge exists in the edge set if and only if the network link between the two nodes is connected. The topology of the dynamic spatiotemporal state graph changes in real time with the connectivity of the network links in the terminal cluster. When the link between two nodes changes from disconnected to connected, the corresponding edge is added to the dynamic spatiotemporal state graph; when the link between two nodes changes from connected to disconnected, the corresponding edge is deleted from the dynamic spatiotemporal state graph. The topology and attribute characteristics of nodes and edges of the dynamic spatiotemporal state graph are updated once at each time step. The update period is consistent with the state acquisition period, which is 100 milliseconds. The construction process of the dynamic spatiotemporal state graph is completed independently on each node. Each node only maintains a local dynamic spatiotemporal state graph centered on itself, which includes the state information of itself and all neighboring nodes. It does not need to maintain global state information, thereby reducing the overhead of state storage and transmission.
[0023] Furthermore, the remaining computing resources, memory utilization, and task queue length of nodes are extracted as node attribute features of the graph nodes. The remaining computing resources are calculated by subtracting the used computing resources from the node's total computing power. The total computing power is determined by the node's hardware configuration, and the used computing power is obtained by multiplying the real-time CPU utilization and GPU utilization by the total computing power. Memory utilization is the ratio of the node's used memory capacity to its total memory capacity. The task queue length is the number of tasks currently waiting to be executed on the node. The bandwidth capacity, transmission latency, and packet loss rate of network links are extracted as edge attribute features. Bandwidth capacity is the maximum data transmission rate that the link can support; transmission latency is the time required for data to be transmitted from one node to another; and packet loss rate is the ratio of the number of data packets lost during transmission to the total number of data packets sent.
[0024] The node attribute features and edge attribute features are concatenated and dimensionality-reduced to generate a multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph. Specifically, for each graph node i, the edge attribute features of all its adjacent edges are collected. The attribute feature vector of node i is concatenated with the edge attribute feature vectors of all its adjacent edges to obtain a high-dimensional feature vector. The feature concatenation operation concatenates the node attribute feature vector with the edge attribute feature vectors of all its adjacent edges in sequence to form a high-dimensional feature vector. For example, when node i has 3 adjacent nodes, the dimension of the attribute feature vector of node i is 3, the dimension of the edge attribute feature vector of each adjacent edge is 3, and the dimension of the concatenated high-dimensional feature vector is 3 + 3 × 3 = 12. This high-dimensional feature vector is input into an embedding mapping multilayer perceptron, which contains two fully connected layers. The first layer maps the concatenated high-dimensional feature vector to a 2d dimension, and the second layer maps it to a d dimension. The activation function is ReLU, resulting in the initial multidimensional state embedding vector of node i. The first layer of the embedding mapping multilayer perceptron has a weight matrix dimension of (3 + 3 × K) × 2d, where K is the average number of neighboring nodes of a node, and the bias vector dimension is 2d. The second layer has a weight matrix dimension of 2d × d, and the bias vector dimension is d. In this embodiment, d is set to 64, therefore the output dimension of the first layer is 128, and the output dimension of the second layer is 64. The parameters of the embedding mapping multilayer perceptron are updated during training using a backpropagation algorithm, so that the generated state embedding vector can better reflect the resource state and communication link state of the nodes. The mathematical expression for this process is:
[0025] in, It is the initial multidimensional state embedding vector of node i, with dimension d; It is an embedded mapping multilayer perceptron; It is the attribute feature vector of node i, with a dimension of 3; It is the set of neighboring nodes of node i; It is the edge attribute feature vector between node i and node j, with a dimension of 3; This indicates the feature concatenation operation. Table 1 describes the node attribute features and edge attribute features.
[0026] Table 1. Description of Node and Edge Attribute Features Feature type Feature Name Dimension Physical meaning Range of values Node attribute features Computational resource reserves 1 The proportion of the total computing power currently available to the node [0,1] Node attribute features Memory usage 1 The percentage of node's used memory capacity out of total memory capacity [0,1] Node attribute features Task queue length 1 Number of tasks currently waiting to be executed on the node [0,+∞) Edge attribute features Bandwidth capacity 1 The maximum data transmission rate that the link can support [0,+∞) Edge attribute features Transmission delay 1 The time required for data to be transmitted on the link [0,+∞) Edge attribute features Packet loss rate 1 The proportion of data packets lost during link transmission [0,1] Table 1 lists detailed information on node and edge attribute features in the dynamic spatiotemporal state graph, including feature type, feature name, dimension, physical meaning, and value range. Node attribute features reflect the resource status of a single computing node, while edge attribute features reflect the communication link status between two computing nodes. All features have been normalized to ensure consistent value ranges across different features, facilitating subsequent feature concatenation and mapping.
[0027] In a preferred embodiment, reference Figure 2 After generating a multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph, a time decay factor is introduced to attenuate the historical features in the multidimensional state embedding vector within consecutive time steps. The real-time features collected at the current time step are residually connected with the historical features attenuated by the time decay factor to update the state representations of graph nodes and edges in the dynamic spatiotemporal state graph, thereby capturing the spatiotemporal evolution of the terminal cluster. The time decay factor ranges from [0,1]. When the time decay factor is 0, the state representation is entirely determined by the current real-time features, without retaining any historical feature information; when the time decay factor is 1, the state representation is entirely determined by historical features, without updating any real-time feature information. In this embodiment, the initial value of the time decay factor is 0.7, meaning the weight of the historical features is 0.7 and the weight of the current real-time features is 0.3. The mathematical expression for this process is:
[0028] in, It is the updated state embedding vector of node i at time step t; It is the time decay factor, with a value range of [0,1]. It is the state embedding vector of node i at time step t-1; It is the initial state embedding vector of node i at time step t, which is calculated from the real-time features collected at the current time step.
[0029] The time decay factor is adjusted based on the rate of change of the terminal cluster's state. When the cluster state changes rapidly, the time decay factor is decreased to increase the weight of the current real-time features; when the cluster state changes slowly, the time decay factor is increased to retain more historical feature information. During the adaptive adjustment of the time decay factor, the cosine similarity between the state embedding vectors of two consecutive time steps is calculated. The formula for calculating the cosine similarity is... ,in Represents the dot product of vectors. The L2 norm of the vector is used. When the cosine similarity is less than 0.3, it indicates that the cluster state changes rapidly, so the time decay factor is reduced by 0.1; when the cosine similarity is greater than 0.7, it indicates that the cluster state changes slowly, so the time decay factor is increased by 0.1. The value of the time decay factor is restricted to [0.2, 0.9] to avoid instability in the state representation due to excessively small or large values. The residual connection structure makes the update process of the state representation smoother, avoiding drastic fluctuations in the state representation caused by abrupt changes in real-time features. This embodiment improves the ability of the dynamic spatiotemporal state diagram to capture the spatiotemporal evolution of the terminal cluster by adaptively adjusting the time decay factor.
[0030] Each agent acts as a graph node, and a weighted aggregation of the state features of its neighboring nodes is performed through a graph attention network. The attention weights for this aggregation are jointly determined by the computational margin of the neighboring nodes and the link latency. The graph attention network employs a two-layer structure: the first layer aggregates the state features of first-order neighbor nodes, and the second layer aggregates the state features of second-order neighbor nodes. The input to the first layer is the initial state embedding vector of the node, and the output is the node state vector aggregating the features of first-order neighbors. The input to the second layer is the output of the first layer, and the output is the node state vector aggregating the features of second-order neighbors. Through this two-layer graph attention network, the agent can observe the state information of second-order neighbors within the local topology, expanding the agent's observation range and improving the accuracy of scheduling decisions. Each layer of the graph attention network uses a multi-head attention mechanism with four attention heads. Each attention head has an output dimension of 16, resulting in a total dimension of 64 after concatenation, consistent with the dimension of the state embedding vector.
[0031] Specifically, refer to Figure 3 The algorithm calculates the absolute value of the computing power difference between a graph node and its neighboring nodes, and performs a nonlinear mapping on this absolute value to obtain a computing power association score. It also calculates the reciprocal of the link delay between a graph node and its neighboring nodes, and performs a nonlinear mapping on this reciprocal to obtain a communication association score. Finally, it multiplies the computing power association score and the communication association score and normalizes the result to obtain the attention weights of neighboring nodes to the graph node.
[0032] The mathematical expression for calculating the computing power-related score is:
[0033] in, It is the computing power correlation score between node i and node j; It is the computing power reserve of node i. It is the remaining computing power of node j; It is the absolute value of the difference in computing power; and These are learnable weights and bias parameters; The Sigmoid activation function maps the output to the [0,1] interval. The computing power correlation score reflects the degree of computing power complementarity between two nodes. The larger the absolute value of the computing power difference, the stronger the computing power complementarity between the two nodes, and the higher the computing power correlation score. The computing power margin ranges from [0,1], so the absolute value of the computing power difference also ranges from [0,1]. The Sigmoid activation function maps the absolute value of the input computing power difference to the [0,1] interval. When the absolute value of the computing power difference is 1, the computing power correlation score is approximately 0.731; when the absolute value of the computing power difference is 0, the computing power correlation score is approximately 0.5.
[0034] The mathematical expression for calculating the communication association score is as follows:
[0035] in, It is the communication association score between node i and node j; It is the link delay between node i and node j; It is the reciprocal of the link delay; and These are learnable weights and bias parameters; It's the Sigmoid activation function. The communication association score reflects the quality of communication between two nodes; the lower the link latency, the higher the communication association score. The link latency is measured in milliseconds, and its range is [1, 1000] milliseconds. Therefore, the reciprocal of the link latency ranges from [0.001, 1]. The Sigmoid activation function maps the input reciprocal of the link latency to the [0, 1] interval. When the link latency is 1 millisecond, the communication association score is approximately 0.731; when the link latency is 1000 milliseconds, the communication association score is approximately 0.50025.
[0036] The mathematical expression for calculating attention weights is:
[0037] in, The attention weight is the value of node j to node i. The numerator is the product of the computational power correlation score and the communication correlation score, which comprehensively considers both computational power complementarity and communication quality. The denominator is the sum of the products of the computational power correlation scores and communication correlation scores of all neighboring nodes of node i, used to normalize the attention weight so that the sum of the attention weights of all neighboring nodes is 1. By multiplying the computational power correlation score and the communication correlation score, a correlation score that comprehensively considers computational power complementarity and communication quality is obtained. Then, the attention weight is obtained through normalization, so that the attention weight can simultaneously reflect the influence of both factors.
[0038] In a preferred embodiment, reference Figure 4 A multi-head attention mechanism is employed, inputting computational power correlation scores and communication correlation scores into multiple independent attention heads. Each attention head calculates sub-attention weights between graph nodes and their neighbors within a different subspace. All sub-attention weights are concatenated and linearly transformed to obtain attention weights that incorporate multi-dimensional scheduling constraints. Each attention head uses an independent... , , , The parameters are updated independently during training. The mathematical expression for this process is:
[0039] in, It is the neighbor aggregation vector of node i after multi-head attention aggregation; K is the number of attention heads; It is the sub-attention weight of node j to node i calculated by the k-th attention head. Each attention head is calculated using an independent set of parameters. It is the state embedding vector of node j at time step t; This indicates a splicing operation, which combines the aggregated results of K attention heads together; It outputs a linear transformation matrix that maps the concatenated high-dimensional vector to the d-dimensional dimension. The multi-head attention head subspace constraints are explained in Table 2.
[0040] Table 2 Explanation of Multi-Head Attention Head Subspace Constraints Attention Head Number Subspace focus constraint type Parameter initialization method 1 Computational complementarity constraints Initialize with a Gaussian distribution, with a mean of 1 and a variance of 0.1. 2 Communication quality constraints Initialize with a Gaussian distribution, with a mean of 1 and a variance of 0.1. 3 Memory resource constraints The distribution is initialized with a Gaussian distribution, with a mean of 0.5 and a variance of 0.1. 4 Task load constraints The distribution is initialized with a Gaussian distribution, with a mean of 0.5 and a variance of 0.1. Table 2 lists the subspace constraint types and corresponding parameter initialization methods for each attention head in the multi-head attention mechanism. Each attention head is trained using an independent parameter set, and different parameter initialization methods guide each attention head to focus on different scheduling constraint dimensions. The first attention head... The initial value is set to 2. The initial value is set to 0, making the computing power-related score dominant in this attention head; the second attention head's... The initial value is set to 2. The initial value is set to 0, making the communication association score dominant in this attention head; the third attention head's and All initial values are set to 1. and The initial values are all set to 0, and a weighted term for memory usage is introduced, with the initial weighting coefficient set to 0.5; the fourth attention point... and All initial values are set to 1. and All initial values are set to 0, and a weighting term for the task queue length is introduced, with the initial weighting coefficient set to 0.5. Computational complementarity constraints and communication quality constraints serve as the primary scheduling constraints, with corresponding parameters having relatively high initial mean values, making these two constraints dominant in attention weight calculation. Memory resource constraints and task load constraints serve as auxiliary scheduling constraints, with corresponding parameters having relatively low initial mean values, used to supplement consideration of the impact of other resource dimensions. This embodiment enhances the feature aggregation process's ability to fuse multi-dimensional states of the communication link by introducing an edge attribute feature weighting term into the multi-head attention mechanism.
[0041] In a preferred embodiment, before outputting action instructions for task unloading and computing power allocation through the policy network, the agent obtains its own observation vector at the current time step based on the weighted aggregated local feature vector. The self-observation vector includes the agent's local task arrival rate and local computing power utilization rate. The self-observation vector and the weighted aggregated neighbor vector are subjected to feature concatenation and activation processing to generate a local feature vector that integrates local topology information and its own state information. The self-observation vector includes two features: local task arrival rate and local computing power utilization rate. The local task arrival rate refers to the number of tasks arriving at the node per unit time, calculated by counting the number of tasks arriving at the node in the most recent second. The local computing power utilization rate refers to the proportion of the node's currently used computing power to its total computing power, complementing the computing resource reserve in the node attribute features, i.e., computing resource reserve = 1 - local computing power utilization rate. The mathematical expression for this process is:
[0042] in, It is the local feature vector of node i; It is the self-observation vector of node i at time step t, with a dimension of 2; and These are learnable fusion weights and bias parameters; It is an activation function.
[0043] The agent's own observation vector is concatenated with the aggregated neighbor vectors to obtain a feature vector of dimension 2 + 64 = 66. This feature vector is then input into a fusion multilayer perceptron, which contains a fully connected layer with a weight matrix of dimension 66 × 64, a bias vector of dimension 64, and uses ReLU as the activation function. The output is a local feature vector of dimension 64. The parameters of the fusion multilayer perceptron are updated during training using backpropagation, ensuring that the generated local feature vector better integrates the agent's own state information with the aggregated neighbor information. The feature concatenation operation fuses the agent's own state information with the aggregated information of neighboring nodes, so that the local feature vector simultaneously contains the agent's own load state and the resource state within the local topology.
[0044] In a preferred embodiment, reference Figure 5 The agent, based on weighted aggregated local feature vectors, obtains a set of tasks to be scheduled before outputting action instructions for task unloading and computing power allocation through the policy network. The latency sensitivity and computational complexity of each task in the set are extracted to construct a task feature matrix. Cross-attention calculation is performed between the task feature matrix and the local feature vectors to extract the matching degree feature between the task to be scheduled and the current state of the terminal cluster. This matching degree feature is used as an additional input condition for the policy network. The set of tasks to be scheduled refers to the set of tasks that a node currently needs to schedule; each task is assigned a unique task identifier upon arrival at the node. The latency sensitivity and computational complexity of each task to be scheduled are extracted. The latency sensitivity ranges from [0,1], with a larger value indicating a more stringent latency requirement; the computational complexity ranges from [0,1], with a larger value indicating a greater demand for computing resources. Task feature matrix. The dimension is N×2, where N is the number of tasks to be scheduled, and each row corresponds to a feature vector of a task to be scheduled. The mathematical expression for this process is:
[0045] in, It is the matching degree feature vector of node i; It is a query vector, composed of local feature vectors. Obtained through linear transformation; It is a key vector, derived from the task feature matrix. Obtained through linear transformation; It is a value vector, derived from the task feature matrix. Obtained through linear transformation; It is the dimension of the key vector; It is a scaling factor used to prevent the gradient of the Softmax function from vanishing due to an excessively large dot product result. The function is used to calculate the attention weights between the task and the node state.
[0046] Query vector The dimension is key vector The dimension is value vector The dimension is ,in and The value of is 32. Linear transformation matrix. Its dimensions are 64×32. Its dimension is 2×32. The dimension is 2×32. The result of cross-attention calculation is a matching degree feature vector with a dimension of 1×32, which reflects the overall matching degree between the set of tasks to be scheduled and the current state of node i. The matching degree feature vector is concatenated with the local feature vector to obtain a feature vector with a dimension of 64+32=96, which is used as the input to the policy network. The matching degree feature reflects the matching degree between each task to be scheduled and the current resource state of node i. Tasks with high latency sensitivity will have a higher matching degree with nodes with low link latency, and tasks with high computational complexity will have a higher matching degree with nodes with large computing power margin. The features of the tasks to be scheduled are described in Table 3.
[0047] Table 3. Description of characteristics of tasks to be scheduled Task type Delay sensitivity computational complexity Example Application Real-time reasoning task high middle Image object detection, speech recognition Offline training task Low high Deep learning model training and big data analysis Data preprocessing tasks middle Low Data cleaning and feature extraction Control command task Extremely high Low Industrial equipment control, automatic driving commands Table 3 lists common types of tasks to be scheduled, along with their corresponding latency sensitivity, computational complexity, and example applications. Latency sensitivity reflects a task's tolerance for transmission and processing latency; higher latency sensitivity indicates stricter latency requirements. Computational complexity reflects the amount of computational resources required by a task; higher computational complexity requires a larger computing power quota. Based on latency sensitivity and computational complexity, tasks can be categorized into different types, facilitating subsequent task-node matching. In a preferred embodiment, the task feature matrix also includes data transmission volume characteristics. Data transmission volume refers to the amount of data that needs to be transmitted between nodes during task execution. When constructing the task feature matrix, data transmission volume is used as a third feature dimension, forming the task feature vector along with latency sensitivity and computational complexity. During cross-attention calculation, the data transmission volume characteristic is matched with the link bandwidth capacity characteristic; tasks with large data transmission volumes are preferentially matched with links with large bandwidth capacity, thereby reducing data transmission time. This embodiment improves the fine-grained matching of tasks and node resources by introducing data transmission volume characteristics into the task feature matrix.
[0048] The agent, based on a weighted aggregated local feature vector, outputs action instructions for task unloading and computing power allocation through a policy network. Specifically, the local feature vector is concatenated with matching degree features and input into the fully connected layer of the policy network to extract the agent's scheduling preference features. The policy network contains three fully connected layers, with the first layer having an input dimension of... , where d is the dimension of the local feature vector. The dimension of the matching feature vector is 2d for the first layer output and d for the second layer output. The third layer has two independent output branches: the first branch outputs the probability distribution of the target node for task unloading, and the second branch outputs the probability distribution of resource allocation for computing power allocation. In the three fully connected layers of the policy network, the first layer has a weight matrix dimension of 96×128 and a bias vector dimension of 128, using ReLU as the activation function; the second layer has a weight matrix dimension of 128×64 and a bias vector dimension of 64, also using ReLU. In the two output branches of the third layer, the task unloading branch has a weight matrix dimension of 64×(K+1) and a bias vector dimension of K+1, where K is the number of neighboring nodes of node i. The output is processed by the Softmax function to obtain the probability distribution of the target node. The computing power allocation branch has a weight matrix dimension of 64×M and a bias vector dimension of M, where M is the number of computing power allocation tiers. The output is processed by the Softmax function to obtain the probability distribution of resource allocation. In this embodiment, M is 10, and each tier corresponds to 10% of the computing power quota. That is, the first tier corresponds to 10% of the computing power quota, the second tier corresponds to 20% of the computing power quota, and so on, with the tenth tier corresponding to 100% of the computing power quota.
[0049] Based on scheduling preference characteristics, the probability distribution of target nodes for task unloading and the probability distribution of resource allocation for computing power allocation are calculated within a preset action space. The action space for task unloading includes all neighboring nodes of node i and local execution options, and the dimension of the target node probability distribution is... Each element corresponds to the probability of selecting the target node for task unloading. The computing power allocation action space contains M preset computing power allocation levels, each level corresponding to a different computing power quota. The resource configuration probability distribution has a dimension of M, and each element corresponds to the probability of selecting that computing power level.
[0050] Based on the probability distributions of target nodes and resource allocation, discrete action instructions for task unloading and computing power allocation are generated through sampling. The sampling process uses a categorical distribution sampling method. The target node for task unloading is randomly sampled according to the target node probability distribution, and the computing power quota allocated to the task is randomly sampled according to the resource allocation probability distribution. During sampling, a target node is randomly selected based on the target node probability distribution. If the selected target node is for local execution, the task is executed locally on node i; if the selected target node is an adjacent node, the task is unloaded to that adjacent node for execution. A computing power tier is randomly selected based on the resource allocation probability distribution, and the corresponding proportion of computing power quota is allocated to the task. The generated action instructions include the identifier of the task to be scheduled, the identifier of the target node, and the allocated computing power quota.
[0051] In a preferred embodiment, reference Figure 6 The weighted aggregation of state features of adjacent nodes in a graph through a graph attention network also includes random sparsification of state features and action commands when agents transmit state features and action commands, retaining only feature components with absolute values greater than a preset pruning threshold. The receiving end performs zero-padding to restore the randomly sparsified information and updates the asynchronous parameters of the policy network based on a local clock, reducing the communication load between agents. Specifically, for the feature vector to be transmitted, a random mask vector of the same dimension as the feature vector is first generated. Each element of the mask vector follows a Bernoulli distribution with a probability of p of being 1. The feature vector and the mask vector are multiplied element-wise to obtain a pre-sparsed feature vector. Then, the feature vector with absolute values less than a preset pruning threshold is selected from the pre-sparsed feature vector. The elements are set to 0 to obtain the final sparsified feature vector. During the random sparsification process, the initial sparsity p is 0.3, meaning that an average of 30% of the feature components are retained. A preset pruning threshold is used. The value is 0.01, meaning that feature components with an absolute value less than 0.01 are set to 0. For example, for a feature vector with a dimension of 64, after random masking, an average of 19.2 feature components are retained. Then, after pruning thresholding, feature components with an absolute value less than 0.01 are further removed, and the final number of retained feature components is about 15, which greatly reduces the amount of data transmitted.
[0052] After receiving the sparse feature vector, the receiving end fills the non-zero elements into their corresponding positions according to the index information of the feature vector, and fills the remaining positions with 0, thus restoring the original 64-dimensional feature vector. Each agent triggers the parameter update process of the policy network based on its own local clock. The parameter update uses the policy gradient method, calculating the gradient based on the local reward signal and updating the parameters of the policy network without waiting for other agents to complete their parameter updates. Asynchronous parameter updates use the PPO algorithm. Each agent collects trajectory data locally, which includes information such as state, action, and reward. After collecting enough trajectory data, it calculates the policy gradient and updates the parameters of the policy network. The objective function for parameter updates is a pruned alternative objective function, limiting the magnitude of the policy update to within a certain range. between, The value is set to 0.2 to avoid overly drastic policy updates that could lead to training instability. Each agent's parameter updates are performed independently, without needing to synchronize with other agents, thus reducing synchronization waiting overhead.
[0053] In a preferred embodiment, the sparsity rate of randomized sparsity is adaptively adjusted, dynamically adjusting the sparsity rate based on the communication load between agents. Specifically, the average communication data volume of agents within the most recent time window is calculated. When the average communication data volume is greater than a first threshold, the sparsity rate is increased, reducing the amount of data transmitted; when the average communication data volume is less than a second threshold, the sparsity rate is decreased, increasing the amount of transmitted feature information; when the average communication data volume is between the first and second thresholds, the sparsity rate remains unchanged. The first threshold is set to 1 MB / s, and the second threshold is set to 0.1 MB / s. Adaptively adjusting the sparsity rate minimizes the communication load between agents while ensuring the integrity of feature information. This embodiment achieves a balance between communication load and feature information integrity by adaptively adjusting the sparsity rate of randomized sparsity.
[0054] This embodiment describes the overall process of multi-agent collaborative scheduling of terminal clusters. By constructing a dynamic spatiotemporal state graph to jointly represent the computing power heterogeneity and topology dynamics of the terminal cluster, a graph attention network is used to aggregate state features that rely only on local observations, and a policy network is used to output decentralized scheduling decision instructions, thereby realizing the collaborative scheduling of terminal clusters in scenarios where dynamic topology and computing power coexist.
[0055] The terminal cluster multi-agent collaborative scheduling system includes a state acquisition processor, a graph attention aggregation processor, and a scheduling decision processor. The state acquisition processor collects real-time computing power utilization and network link connectivity status of each node in the terminal cluster, constructing a dynamic spatiotemporal state graph with nodes as graph nodes and network links as edges. The graph attention aggregation processor uses a graph attention network to weightedly aggregate the state features of adjacent nodes, with the attention weights determined jointly by the computing power margin and link latency of adjacent nodes. The scheduling decision processor enables agents to output action commands for task unloading and computing power allocation based on the weighted aggregated local feature vectors through a policy network. Each processor can be deployed on each node of the terminal cluster to achieve distributed collaborative scheduling.
Claims
1. A multi-agent cooperative scheduling method for terminal clusters, characterized in that, include: The system collects the real-time computing power utilization and network link connectivity status of each node in the terminal cluster, and constructs a dynamic spatiotemporal state graph by using the nodes as graph nodes and the network links as edges. Each agent acts as a graph node, and performs weighted aggregation of the state features of the adjacent nodes through a graph attention network. The attention weights of the weighted aggregation are jointly determined by the computing power margin of the adjacent nodes and the link latency. Based on the weighted aggregated local feature vectors, the agent outputs action instructions for task unloading and computing power allocation through a policy network.
2. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, Constructing a dynamic spatiotemporal state graph using the nodes as graph nodes and the network links as edges includes: extracting the remaining computing resources, memory usage, and task queue length of the nodes as node attribute features of the graph nodes; The bandwidth capacity, transmission latency, and packet loss rate of the network link are extracted as edge attribute features of the edge; The node attribute features and edge attribute features are concatenated and dimensionality-reduced to generate a multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph.
3. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, The attention weight of the weighted aggregation is jointly determined by the computing power margin of the adjacent nodes and the link latency, including: calculating the absolute value of the computing power difference between the graph node and the adjacent node, and performing a nonlinear mapping on the absolute value of the computing power difference to obtain a computing power association score; Calculate the reciprocal of the link delay between the graph node and its neighboring node, and perform a nonlinear mapping on the reciprocal of the link delay to obtain a communication association score; The computing power association score and the communication association score are multiplied and normalized to obtain the attention weight of the neighboring node to the graph node.
4. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, The agent outputs action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector, including: inputting the local feature vector into the fully connected layer of the policy network to extract the agent's scheduling preference features; Based on the scheduling preference features, calculate the target node probability distribution for task unloading and the resource allocation probability distribution for computing power allocation in the preset action space; Based on the probability distribution of the target node and the probability distribution of the resource configuration, discrete action instructions for task unloading and computing power allocation are generated.
5. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, Before the agent outputs the action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector, the method further includes: obtaining the agent's own observation vector at the current time step, wherein the own observation vector includes the agent's local task arrival rate and local computing power utilization rate. The self-observation vector and the weighted aggregated neighbor vector are subjected to feature concatenation and activation processing to generate the local feature vector that integrates local topological information and self-state information.
6. The terminal cluster multi-agent cooperative scheduling method according to claim 2, characterized in that, After generating the multidimensional state embedding vector for each graph node in the dynamic spatiotemporal state graph, the method further includes: introducing a time decay factor to decay the historical features in the multidimensional state embedding vector within a continuous time step; The real-time features collected at the current time step are residually connected with the historical features attenuated by the time decay factor to update the state representation of the graph nodes and edges in the dynamic spatiotemporal state graph, so as to capture the spatiotemporal evolution pattern of the terminal cluster.
7. The terminal cluster multi-agent cooperative scheduling method according to claim 3, characterized in that, The process of multiplying and normalizing the computing power association score and the communication association score to obtain the attention weight of the neighboring node to the graph node includes: using a multi-head attention mechanism, inputting the computing power association score and the communication association score into multiple independent attention heads respectively; Each attention head calculates the sub-attention weights between the graph node and its neighboring nodes in different subspaces; All the sub-attention weights are concatenated and linearly transformed to obtain the attention weights that incorporate multi-dimensional scheduling constraints.
8. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, Before the agent outputs the action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector, it also includes: obtaining a set of tasks to be scheduled, extracting the latency sensitivity and computational complexity of each task to be scheduled in the set of tasks to be scheduled, and constructing a task feature matrix. The task feature matrix and the local feature vector are cross-attention calculated to extract the matching degree feature between the task to be scheduled and the current state of the terminal cluster. The matching degree feature is used as an additional input condition for the policy network.
9. The terminal cluster multi-agent cooperative scheduling method according to claim 1, characterized in that, The weighted aggregation of the state features of the adjacent nodes of the graph node through the graph attention network further includes: when each agent transmits the state features and the action instructions, the state features and the action instructions are randomly sparsified, and only feature components with an absolute value greater than a preset pruning threshold are retained. The receiving end performs zero-filling restoration on the information that has undergone the random sparsification process, and triggers asynchronous parameter updates of the policy network based on the local clock, thereby reducing the communication load between the agents.
10. A multi-agent collaborative scheduling system for terminal clusters, characterized in that, include: The state acquisition processor is used to collect the real-time computing power utilization and network link connectivity status of each node in the terminal cluster, and construct a dynamic spatiotemporal state graph by using the nodes as graph nodes and the network links as edges. A graph attention aggregation processor is used to enable each agent to act as a graph node, and to perform weighted aggregation of the state features of the adjacent nodes of the graph node through a graph attention network. The attention weight of the weighted aggregation is jointly determined by the computing power margin of the adjacent nodes and the link latency. The scheduling decision processor enables the agent to output action instructions for task unloading and computing power allocation through the policy network based on the weighted aggregated local feature vector.
