A micro-service scheduling method and system based on double-layer queue feedback and topology awareness

Abstract

The application relates to a micro-service scheduling method and system based on double-layer queue feedback and topology sensing, and the application introduces a shared graph attention network (GAT), deeply fuses and encodes residual computing power of a heterogeneous computing node and time-varying link quality, and gives the scheduling system a structure sensing capability of a global perspective; by establishing a dynamic feasibility action mask and a hierarchical degradation mechanism, the generation of illegal scheduling instructions is cut off from the root, and the physical executability of each decision is ensured. A virtual queue tracking mechanism is introduced to realize nanosecond-level real-time cumulative evaluation of scheduling actions in the same time slot on node load, and fine load sharing is realized in a concurrent storm; a Lyapunov drift and punishment principle is introduced to construct a composite reward function, guiding the DSAC agent to complete joint optimization of delay, energy consumption and queue stability in adaptive exploration.

Description

Technical Field

[0001] This invention relates to a microservice scheduling method and system based on two-layer queue feedback and topology awareness, belonging to the field of microservice scheduling and deep reinforcement learning technology. Background Technology

[0002] With the development of 6G communication technology, Computing Network (CPN), as a new network architecture, realizes the ubiquitous computing paradigm of "computing power as a service" by deeply integrating distributed computing resources with communication networks. In the CPN architecture, microservices serve as the basic scheduling unit, carrying diverse computing tasks such as video analytics and intelligent inference. Each microservice request carries specific computing resource requirements (such as CPU / GPU type, computing power, and memory) and Quality of Service (QoS) constraints (such as maximum tolerable latency).

[0003] However, microservice scheduling in a computing power network environment faces three core challenges:

[0004] 1. Heterogeneous resource constraints: Nodes at different levels (core, region, edge) in the computing network have significant differences in hardware architecture, computing power and memory size. Microservices must meet specific hardware architecture compatibility requirements.

[0005] 2. Dynamic network environment: Link bandwidth, transmission delay and packet loss rate fluctuate in real time with the network congestion status, and static scheduling strategies cannot adapt to this time-varying characteristic.

[0006] 3. Conflicts in Multi-Objective Optimization: Scheduling decisions must simultaneously minimize total scheduling cost, meet SLA (Service Level Agreement) latency constraints, and maintain global load balancing. These objectives are inherently contradictory; for example, solely pursuing low latency may lead to core node overload and a surge in energy costs.

[0007] Existing methods are mainly divided into heuristic methods and deep reinforcement learning (DRL) methods. Heuristic methods rely on empirical rules, which makes it difficult to achieve stable performance in dynamic and complex environments. Although existing DRL methods have adaptive capabilities, they are prone to generating illegal scheduling instructions during the action exploration phase (such as assigning them to nodes with insufficient resources), and most of them adopt a multilayer perceptron (MLP) architecture that directly flattens features, which seriously loses the topological connection patterns and spatial structure information between nodes. In addition, traditional Markov decision processes (MDPs) update the state according to time slots, which is difficult to cope with the instantaneous queue backlog problem caused by concurrent microservices. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention provides a microservice scheduling method and system based on two-layer queue feedback and topology awareness. This invention introduces a shared graph attention network (GAT) to deeply fuse and encode the remaining computing power of heterogeneous computing nodes with time-varying link quality, endowing the scheduling system with a global perspective of structural awareness. By establishing a dynamic feasible action mask and a hierarchical degradation mechanism, the generation of illegal scheduling instructions is cut off at the source, ensuring the physical executability of each decision. A virtual queue tracking mechanism is introduced to achieve nanosecond-level real-time cumulative evaluation of node load by scheduling actions within the same time slot, enabling fine-grained load sharing during concurrency storms. A composite reward function is constructed using the Lyapunov drift plus penalty principle to guide the DSAC agent to jointly optimize latency, energy consumption, and queue stability through adaptive exploration.

[0009] The technical solution of the present invention is as follows:

[0010] The first aspect of this invention provides a microservice scheduling method based on two-level queue feedback and topology awareness, comprising:

[0011] Step 1: Construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a comprehensive feature vector of the nodes;

[0012] Step 2: Obtain microservice requests based on the microservice set, and calculate the minimum end-to-end transmission delay baseline for each microservice request;

[0013] Step 3: Construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer;

[0014] The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector;

[0015] The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality;

[0016] By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector;

[0017] Step 4: Concatenate the microservice request with the embedding vector of each compute node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each compute node, and obtain the target scheduling node based on the original action logic value;

[0018] Step 5: Update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward;

[0019] Step 6: Input the multidimensional composite reward into the Critic network to obtain the target expected value. Through continuous iterative updates, the Actor network converges and outputs the optimal target scheduling node.

[0020] According to a preferred embodiment of the present invention, the specific implementation process of step 1 includes:

[0021] The computing network topology includes a core layer, a regional layer, and an edge layer;

[0022] Based on the computing power network topology, a discrete time-varying graph is constructed. ;in, E(t) represents the set of nodes, including computing nodes in the core layer, region layer, and edge layer; E(t) represents the set of edges in time slot t, including communication links between computing nodes. Represents the node feature matrix, ,in, This represents the node comprehensive feature vector of the Nth node, where N represents the total number of computing nodes in the computing power network; Represents the edge feature matrix, ,in, This represents a directed link, or communication link, between any nodes i and j.

[0023] For any directed link Construct a 4D edge feature vector This includes physical connectivity status indicators and time-varying available bandwidth. Link transmission delay and inter-node link packet loss rate ;

[0024] In the node feature matrix In the diagram, the node-comprehensive feature vector of any node i includes 14-dimensional features, as shown below:

[0025] ;

[0026] in, This represents the comprehensive feature vector of the i-th node. Static identifier characteristics include: maximum computing power resources, maximum memory capacity, hardware architecture and accelerator type, and node deployment level identifier;

[0027] Represents dynamic state characteristics, including: remaining computing power Remaining memory Internal load rate External reach Cumulative energy consumption ;in, This indicates that the current time slot has received load. Indicates maximum load capacity. Represents an instantaneous virtual queue; Indicates service rate, service rate , For service rate coefficient;

[0028] The policy attributes include: service capability level, security domain level, unit computing power price, and system priority weight.

[0029] This indicates the overall scheduling pressure on the nodes.

[0030] Further optimization of the overall scheduling pressure of nodes ;

[0031] in, , , These represent queue pressure weight, resource utilization weight, and network pressure weight, respectively. Represents an instantaneous virtual queue; Indicates the utilization rate of node basic resources. This indicates the bottleneck pressure at the node level;

[0032] Basic resource utilization rate As shown below:

[0033] ;

[0034] in, Represents a node That is, nodes In the time slot of Utilization rate Represents a node In the time slot Real-time memory utilization;

[0035] Assuming node-based The first-order neighbor set is Then the node-level network bottleneck pressure As shown below:

[0036] ;

[0037] in, , , , These represent bandwidth weight, latency weight, packet loss weight, and jitter weight, respectively. Represents the first-order neighbor set The node, This represents the link utilization rate, where Indicates the current bandwidth. Indicates the maximum bandwidth of the link. Indicates the link normalized transmission delay. This represents normalized link jitter, as shown below:

[0038] ;

[0039] in, Indicates the standard deviation of time delay. This represents the round-trip time of the k-th probe packet sent by the network probe, where n represents the number of probe packets sent. Indicates the average delay. ;

[0040] Normalized link jitter is:

[0041] ;

[0042] in, This indicates the preset maximum jitter threshold;

[0043] Further preferred, at each node There are nodes on it Physical workload queue and nodes Instant virtual queue ;

[0044] The physical workload queue represents the actual backlog of tasks, and the physical workload queue is updated between each time slot according to the following dynamic equation:

[0045] ;

[0046] in, This indicates the physical workload queue for the next time slot. For time slots Internal allocation to nodes Total computing requirements Indicates the service rate; at the beginning of each time slot, an immediate virtual queue initialization is performed: .

[0047] According to a preferred embodiment of the present invention, step 2 includes the following specific implementation process:

[0048] In each time slot The collection of microservices that arrive is ; where each microservice request Represented as:

[0049] ;

[0050] in, To meet computing resource requirements, For memory requirements, For the maximum tolerable latency, For architecture compatibility requirements, i.e., the required hardware architecture;

[0051] Deploy each microservice request to a node that meets the constraints, i.e. ;in, It is a Boolean scheduling decision variable, that is, when a microservice request... Assigned to the i-th computing node hour, ,otherwise ;

[0052] Taking the gateway node that initiates the microservice request as the source, calculate the minimum end-to-end transmission delay benchmark from the source node to all candidate nodes i in the entire network using Dijkstra's algorithm. ;in, An identifier index for microservice requests.

[0053] According to a preferred embodiment of the present invention, step 3 specifically includes the following steps:

[0054] Through the edge feature mapping layer, from discrete time-varying maps Extracting nodes spliced ​​vector The data is then input into a multilayer perceptron network for nonlinear mapping to obtain a topology quality encoding vector, as shown below:

[0055] ;

[0056] ;

[0057] in, Indicates the time-varying available bandwidth. , Indicates link transmission delay, i.e. , Indicates link The concatenated vector, Represents the topology quality encoding vector. Represents a multilayer perceptron;

[0058] The feature concatenation and attention computation layer includes M attention computation heads, which concatenate the nodes in the discrete time-varying graph. Node comprehensive feature vector ,node That is, nodes Node comprehensive feature vector and topology quality encoding vector The common input feature concatenation and attention calculation layer obtains M attention weights through M attention calculation heads;

[0059] The M attention weights are weighted and summed through a multi-head aggregation activation layer, and the dense embedding is output through the ELU activation function, as shown below;

[0060] ;

[0061] in, This represents the attention weight of the m-th attention calculation head. Represents the node embedding vector. Represents the weight matrix. Indicates the activation function; Represents a node The node-comprehensive feature vector.

[0062] Further preferably, the feature concatenation and attention calculation layer obtains M attention weights through M attention calculation heads; including:

[0063] The m-th attention calculation head will input the feature, i.e. , and Perform vector concatenation as follows:

[0064] ;

[0065] in, Represents the joint eigenvector. This represents a vector concatenation operation. This represents the linear transformation matrix corresponding to the m-th attention computation head;

[0066] After concatenation, the joint feature vector is mapped to unnormalized attention coefficients, and then passed through the LeakyReLU activation function and the Softmax normalization function to obtain attention weights that comprehensively consider node state and link quality, as shown below:

[0067] ;

[0068] in, This represents the attention weight of the m-th attention calculation head. This represents the activation function. Indicates the first The learnable weight vector of each attention computation head.

[0069] According to a preferred embodiment of the present invention, step 4 includes the following specific implementation process:

[0070] microservice requests Perform dimensional broadcast copying and compare it with the embedding vectors of each node. After concatenation, input the Actor network to obtain the original action logic values. ;

[0071] The target compute node and microservice requests are compared in real time to generate a dynamic action mask, also known as a feasibility mask, as shown below:

[0072] ;

[0073] in, This represents the dynamic action mask of node i for the current task; This indicates a conditional indicator function, which has a value of 1 when the condition inside the indicator function is true, and a value of 0 otherwise. Indicates the actual hardware architecture type of node i;

[0074] If all nodes in the computing network do not satisfy the constraint, that is, all nodes... If both are 0, then calculate the memory satisfaction rate. ,when At that time, a dynamic action mask is generated. Record memory relaxation penalty cost ,in Indicates the system threshold. This represents the memory page swapping penalty constant; As shown below:

[0075] ;

[0076] in, Indicates a dynamic action mask;

[0077] If all nodes If the result is still 0, then a static lookup table mapping function is introduced. The compatibility score is obtained through a static lookup table mapping function, allowing nodes with a compatibility score greater than 0 to participate in the candidate selection, and generating a dynamic action mask. :

[0078] ;

[0079] The cost of architecture conversion penalty is calculated based on compatibility loss. :

[0080] ;

[0081] in, To convert the penalty coefficient, Indicates multiplication operation;

[0082] The final dynamic action mask is the feasibility mask. for or or , all nodes Applied to action logic values: ,in, Indicates the action logic value. For penalty constants; for action logic values Apply the Softmax function to obtain the probability distribution, and perform random sampling to obtain the target scheduling node's output and execution action in the current state.

[0083] A further preferred static lookup table mapping function Compatibility is obtained through a pre-established compatibility mapping table; including:

[0084] Enumerate the hardware architecture types supported by the computing power network, select each microservice request, and run them on nodes with different hardware architecture types. Record the execution success status, average execution latency, throughput, unit task completion time, and execution efficiency.

[0085] Microservice requests on the required hardware architecture Using the execution efficiency on other nodes as a benchmark, the execution efficiency of microservice requests on other nodes' actual hardware architecture types is compared with the benchmark to obtain the compatibility score. If a microservice request cannot run on other nodes' actual hardware architecture types, the compatibility score is 0. If it can run directly without cross-compilation, runtime conversion, or a compatibility layer, it is considered to be natively compatible, and the compatibility score is 1. If it needs to run through cross-compilation, runtime conversion, or a compatibility layer, the compatibility score is between 0 and 1. Finally, the compatibility score is written to a compatibility mapping table, and the static lookup mapping function obtains the corresponding compatibility score by looking up the compatibility mapping table.

[0086] According to a preferred embodiment of the present invention, step 5 includes the following specific implementation process:

[0087] Within the same time slot, after each microservice scheduling decision is completed, the target scheduling node is obtained. After deploying the current microservice request to the target scheduling node, calculate and update the real-time virtual queue of the target scheduling node:

[0088] ;

[0089] in, Represents an updated, real-time virtual queue;

[0090] Synchronously perform closed-loop recalculation of the comprehensive scheduling pressure representation to obtain the updated node comprehensive scheduling pressure. As shown below:

[0091] ;

[0092] Get It is directly used as the state input for the next microservice decision cycle within the same time slot;

[0093] Calculate the topology congestion index for the current task scheduling path. As shown below:

[0094] ;

[0095] in, This indicates the distance a microservice request travels from the initiating gateway to the target scheduling node. The available bandwidth of the bottleneck link on the end-to-end routing path, that is, the minimum available bandwidth among all connected links on the path. Indicates the transmission delay reference;

[0096] Calculate topology congestion penalty Relaxing penalties with masking ;in, This represents the congestion penalty coefficient. This indicates the penalty for easing memory usage restrictions. Indicates the cost of architecture conversion penalties;

[0097] Introduce a scheduling pressure penalty term that reflects the immediate state of an action. :

[0098] ;

[0099] in, This represents the stress penalty coefficient; the final environmental reward is calculated as follows:

[0100] ;

[0101] in, This refers to multidimensional composite rewards, also known as environmental rewards. Indicates the stable drift term in the queue. This represents the cumulative real-time virtual queue length of node i before the execution of this scheduling action. This indicates a comprehensive cost penalty item. This represents the Lyapunov control parameters. This indicates the weight of the delay penalty in the cost penalty item. This represents the total latency of microservice request processing and queuing on node i. , This represents the energy consumption coefficient per unit computing power of node i. This indicates the energy consumption penalty weight in the cost penalty item. This indicates a service quality violation penalty when the predicted delay exceeds the maximum tolerable delay. Apply a fixed negative penalty at the time. This represents the load balancing incentive. This represents the load balancing incentive constant factor; Indicates topology congestion penalty;

[0102] Using the Welford algorithm to evaluate environmental rewards Perform online normalization and scaling truncation, and store the results in the experience replay pool.

[0103] According to a preferred embodiment of the present invention, step 6 includes the following specific implementation process:

[0104] Environmental rewards are input into the Critic network, which consists of two parallel subnetworks. and The target Q value is calculated by taking the minimum value of the sub-network output, as shown below:

[0105] ;

[0106] in, This represents the expected temporal difference objective value required to update the Critic network, i.e., the objective Q value. Indicates the discount factor for future rewards; Indicates the temperature coefficient; This represents the next state that the environment transitions to after an action is performed, i.e., a discrete time-varying graph and a microservice request; This indicates the next action under state s', which is to schedule the current microservice request to be executed on the compute node; , This represents the action value assessment output by the Critic network for two independent objectives;

[0107] Huber Loss is used to update the Critic network parameters, including the temperature coefficient. As a learnable optimization variable, when the current information entropy of the agent's output policy is lower than the preset target information entropy benchmark, it is adaptively reduced by the optimizer. Conversely, it increases. As shown below:

[0108] ;

[0109] in, This represents the optimization loss function used for adaptively updating the temperature coefficient. ; This represents the mathematical expectation of the action sampling distribution under the current policy; This represents the policy probability of the output action in the current state; The preset target information entropy benchmark is the negative logarithm of the size of the discrete action space, i.e. ;in, This represents the size of the discrete action space, i.e., the number of current candidate scheduling nodes.

[0110] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of a microservice scheduling method based on two-level queue feedback and topology awareness.

[0111] A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the steps of a microservice scheduling method based on two-level queue feedback and topology awareness.

[0112] A second aspect of the present invention provides a microservice scheduling system based on two-layer queue feedback and topology awareness, comprising:

[0113] The discrete time-varying graph construction module is configured to: construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a node comprehensive feature vector;

[0114] The microservice request extraction module is configured to: obtain microservice requests based on the microservice set and calculate the minimum end-to-end transmission delay baseline for the microservice requests;

[0115] The node embedding vector acquisition module is configured to: construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer;

[0116] The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector;

[0117] The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality;

[0118] By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector;

[0119] The target scheduling node acquisition module is configured to: concatenate the microservice request with the embedding vector of each computing node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each computing node, and obtain the target scheduling node based on the original action logic value;

[0120] The reward construction module is configured to: update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward;

[0121] The expected value acquisition module is configured to input the multidimensional composite reward into the Critic network, obtain the target expected value, and through continuous iterative updates, the Actor network reaches convergence and outputs the optimal target scheduling node.

[0122] The beneficial effects of this invention are as follows:

[0123] 1. Break through the bottleneck of feature flattening and achieve deep full-domain topology and link quality awareness: By designing a shared GAT encoder, "edge features" are embedded in attention weight calculation, so that the system output implicitly contains communication path smoothness information, accurately identify inferior targets with "strong computing power but congested links" and reduce end-to-end latency.

[0124] 2. Eliminating invalid explorations from the algorithm's core, ensuring absolute scheduling feasibility under extreme constraints: A unique indicator function action masking mechanism transforms hardware constraints into rigorous numerical filtering, achieving zero illegal instruction issuance. Combined with a hierarchical relaxation strategy, it prevents scheduling deadlocks when resources are exhausted.

[0125] 3. Overcome the problem of instantaneous congestion (herding effect) of concurrent tasks and achieve fine-grained queue stability: The dual-track mechanism of virtual queues breaks the lag of traditional MDP state updates, enabling the system to keenly detect the queuing pressure accumulated at the nanosecond level and achieve dynamic diversion, preventing local node overload and crash.

[0126] 4. Resolve conflicts between multiple objectives and ensure SLA while reducing costs and increasing efficiency: The composite reward framework tends to compress the computation time of core nodes under low load, and is controlled by drift penalty to activate edge nodes under high load, thereby minimizing global costs while adhering to the bottom line of microservice SLA.

[0127] 5. High training efficiency and dynamic adaptability brought by shared architecture and maximum entropy framework: The DSAC framework improves the breadth of exploration against network jitter, and the shared GAT underlying encoder accelerates feature convergence while significantly reducing inference computing power consumption. Attached Figure Description

[0128] Figure 1 This is a flowchart of the microservice scheduling method based on two-layer queue feedback and topology awareness of the present invention;

[0129] Figure 2 This is a deployment architecture diagram of the microservice scheduling system of the present invention;

[0130] Figure 3 This is a schematic diagram of the two-layer queue timely feedback mechanism of the present invention. Detailed Implementation

[0131] The present invention will be further described below with reference to the embodiments and accompanying drawings, but is not limited thereto.

[0132] Example 1

[0133] Terminology Explanation:

[0134] 1. Dijkstra's Algorithm: A classic algorithm for calculating the single-source shortest path in a graph structure. In this invention, it is mainly used to pre-calculate the prior benchmark of end-to-end communication delay between nodes based on the real-time link status of the computing network.

[0135] 2. Lyapunov Drift and Penalty Principle: Originating from stochastic network optimization theory. By minimizing the weighted sum of the "drift term" (representing the backlog instability of the node queue system) and the "penalty term" (representing energy consumption or delay cost), network performance indicators are jointly optimized while strictly ensuring that the system queue does not collapse.

[0136] 3. Welford Algorithm: A numerically stable online statistical algorithm that updates the mean and variance of streaming data in real time with a single pass. This invention uses it for online normalization of dynamic reward signals in reinforcement learning to prevent network divergence caused by fluctuations in reward scale.

[0137] 4. Huber Loss: A piecewise loss function that combines the advantages of mean squared error (MSE) and mean absolute error (MAE). When the error is small, it exhibits squared loss to ensure convergence accuracy, while when the error is too large, it exhibits absolute value linear loss, thus having strong robustness to extreme abnormal gradients caused by high concurrency bursts.

[0138] 5. Actor Network: The policy network of the intelligent agent, used to receive dynamic graph embedding vectors that integrate the global topology state of the computing power network and the characteristics of microservices. Through multilayer perceptron (MLP) and activation function, it directly outputs the initial probability distribution logic value of each computing node as the scheduling target.

[0139] 6. Critic Network: A value evaluation network for agents, used to evaluate the long-term expected returns (Q-value) of scheduling actions made by the Actor network under the current computing power network state. It calculates the timing difference error through the output Q-value, and guides and optimizes the Actor network to iterate parameters in the direction of low latency, low energy consumption, and congestion prevention.

[0140] A microservice scheduling method based on two-level queue feedback and topology awareness, such as... Figure 1 As shown, it includes:

[0141] Step 1: Construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a comprehensive feature vector of the nodes;

[0142] Step 2: Obtain microservice requests based on the microservice set, and calculate the minimum end-to-end transmission delay baseline for each microservice request;

[0143] Step 3: Construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer;

[0144] The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector;

[0145] The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality;

[0146] By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector;

[0147] Step 4: Concatenate the microservice request with the embedding vector of each compute node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each compute node, and obtain the target scheduling node based on the original action logic value;

[0148] Step 5: Update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward;

[0149] Step 6: Input the multidimensional composite reward into the Critic network to obtain the target expected value. Through continuous iterative updates, the Actor network converges and outputs the optimal target scheduling node.

[0150] Example 2

[0151] The microservice scheduling method based on two-layer queue feedback and topology awareness described in Example 1 differs in that:

[0152] The specific implementation process of step 1 includes:

[0153] like Figure 2As shown, the computing power network topology includes a core layer, a regional layer, and an edge layer. (The core layer is the "global computing power hub" of the computing power network, usually composed of large-scale data centers or supercomputing centers at the national or provincial level; the regional layer is the "mid-level aggregation node" connecting the core and the edge, usually composed of cloud computing centers or metropolitan area network backbone nodes at the prefecture-level city level; the edge layer is the "nerve ending" of the computing power network, widely and dispersedly deployed in physical locations closest to end users or data sources; it is specifically used to handle lightweight real-time microservices that are extremely sensitive to latency and have high concurrency (such as real-time control of industrial internet, autonomous driving inference, and real-time analysis of video security); for example, an edge industrial gateway deployed in a smart manufacturing workshop, or an edge computing board directly embedded at a 5G base station at an intersection).

[0154] Based on the computing power network topology, a discrete time-varying graph is constructed. ;in, E(t) represents the set of nodes, including computing nodes in the core layer, regional layer, and edge layer (divided according to the physical deployment location of computing nodes, the maximum computing power resource scale, and the hierarchical routing affiliation of network topology); E(t) represents the set of edges in time slot t, including the communication links between computing nodes. Represents the node feature matrix, ,in, This represents the node comprehensive feature vector of the Nth node, where N represents the total number of computing nodes in the computing power network; Represents the edge feature matrix, ,in, This represents a directed link, or communication link, between any nodes i and j.

[0155] For any directed link Constructing 4D edge feature vectors This includes physical connectivity indicators (which extract the physical connectivity of the network topology directly by periodically sending probe packets through monitoring components deployed on the network management plane, such as probe services like HpcMonitor), and time-varying available bandwidth. Link transmission delay and inter-node link packet loss rate (Obtained directly through network probe services, Blackbox Exporter, HpcMonitor, or ICMP Ping probe services deployed on the network management plane);

[0156] The discrete time-varying graph G(t) can simultaneously describe the state of computing nodes, network topology connections, and link quality changes within time slot t, and serves as the input graph structure tensor for subsequent shared graph attention networks; in the node feature matrix In the diagram, the node-comprehensive feature vector of any node i includes 14-dimensional features, as shown below:

[0157] ;

[0158] in, This represents the comprehensive feature vector of the i-th node. Static identifier characteristics include: maximum computing power resources, maximum memory capacity, hardware architecture and accelerator type, and node deployment level identifier (distinguishing between core / regional / edge layers).

[0159] Represents dynamic state characteristics, including: remaining computing power Remaining memory Internal load rate External reach Cumulative energy consumption ;in, This indicates that the current time slot has received load. Indicates maximum load capacity. Represents an instantaneous virtual queue; Indicates service rate, service rate , This is the service rate coefficient; (its value typically ranges from...) Between, for example, take It is designed to reserve space for node operating system (such as Ubuntu kernel overhead) and underlying container scheduling. (Safety buffer computing power)

[0160] The policy attributes include: service capability level (automatically determined and generated during the resource registration phase based on node hardware configuration, supported software operating environment, and types of computing services available; for example, ordinary CPU nodes, high-performance GPU nodes, and supercomputing nodes correspond to different service capability levels), security domain level (configured and generated by the network management platform according to the computing power network security domain division policy, reflecting the security zone and access permissions of the node), unit computing power price (generated based on the billing standards published by the computing power resource operation platform or the resource price configured by the resource provider, representing the cost of using a unit of computing resources), and system priority weight (pre-configured by the system administrator or scheduling policy module according to the computing power network scheduling policy, reflecting the priority scheduling level of the node when undertaking critical tasks, important businesses, or real-time businesses).

[0161] This indicates the overall scheduling pressure on the nodes;

[0162] Overall scheduling pressure of nodes ;

[0163] in, , , These represent queue pressure weight, resource utilization weight, and network pressure weight, respectively. Represents an instantaneous virtual queue; Indicates the utilization rate of node basic resources. This indicates the bottleneck pressure at the node level;

[0164] Basic resource utilization rate As shown below:

[0165] ;

[0166] in, Represents a node That is, nodes In the time slot of Utilization rate Represents a node In the time slot Real-time memory utilization;

[0167] To decouple micro-scheduling pressure from topological bottlenecks and form a closed-loop perception, maximum pressure assessment is adopted, assuming it is based on node... The first-order neighbor set is (with nodes) The set of all directly connected compute nodes indicates the node-level network bottleneck pressure. As shown below:

[0168] ;

[0169] in, , , , These represent bandwidth weight, latency weight, packet loss weight, and jitter weight, respectively (e.g., ...). , , , For high-throughput tasks (such as distributed training), For low-latency tasks (such as real-time inference), , , , ), Represents the first-order neighbor set The node, This represents the link utilization rate, where Indicates the current bandwidth. Indicates the maximum bandwidth of the link. This represents the link-normalized transmission delay (which can be mapped from the real-time round-trip time (RTT) of the probe packet to...). (interval) This represents the normalized link jitter, calculated using the standard deviation of continuous probe packet delay and mapped to the [0,1] interval, as shown below:

[0170] ;

[0171] in, Indicates the standard deviation of time delay. This represents the round-trip time of the k-th probe packet sent by the network probe, where n represents the number of probe packets sent. Indicates the average delay. ;

[0172] Normalized link jitter is:

[0173] ;

[0174] in, This indicates the preset maximum jitter threshold (e.g., 50ms, typically 20-100ms).

[0175] Unlike existing traditional reinforcement learning scheduling methods based on unified state updates in time slots, this invention updates the virtual queue state immediately after each microservice scheduling decision is completed within the same time slot and reconstructs the comprehensive scheduling pressure representation. This mechanism enables subsequent microservices arriving concurrently in the same batch to perceive the resource consumption and pressure fluctuations caused by the preceding scheduling behavior in real time, realizing the micro-state flow within the time slot and fundamentally eliminating local scheduling congestion in high-concurrency scenarios.

[0176] Define a two-level queue state model: at each node There are nodes on it Physical workload queue and nodes Instant virtual queue ;

[0177] The physical workload queue represents the actual backlog of tasks, and the physical workload queue is updated between each time slot according to the following dynamic equation:

[0178] ;

[0179] in, This indicates the physical workload queue for the next time slot. For time slots Internal allocation to nodes Total computing requirements This represents the service rate (FLOPS / slot); to record the load in real time during the micro-concurrency scheduling process within a single slot, an immediate virtual queue initialization is performed at the beginning of each slot: .

[0180] The specific implementation process of step 2 includes:

[0181] Microservice requests arrive according to a Poisson process (the generation of microservice requests exhibits randomness and independence in time, meaning the number of requests arriving per unit time follows a Poisson distribution, thus accurately simulating the bursty task arrival patterns in real computing networks), and the microservice request arrival rate is... (per slot), in each slot The collection of microservices that arrive is ; where each microservice request Represented as:

[0182] ;

[0183] in, For computing resource requirements (FLOPS), For memory requirements (GB) For the maximum tolerable latency, For architecture compatibility requirements, i.e., the required hardware architecture;

[0184] Deploy each microservice request to a node that meets the constraints, i.e. ;in, It is a Boolean scheduling decision variable, that is, when a microservice request... Assigned to the i-th computing node hour, ,otherwise This constraint ensures the atomicity and indivisibility of the task.

[0185] Taking the gateway node that initiates the microservice request as the source, the minimum end-to-end transmission delay benchmark to the source node among all candidate nodes i in the entire network is calculated using Dijkstra's algorithm. ;in, An identifier index for microservice requests; (pre-computed during environment model building to provide a latency baseline for the optimal transmission path in the topology when subsequently generating agent state features and calculating Service Level Agreement (SLA) rewards and penalties).

[0186] The specific implementation process of step 3 includes:

[0187] Through the edge feature mapping layer, from discrete time-varying maps Extracting nodes spliced ​​vector The data is then input into a multilayer perceptron network for nonlinear mapping to obtain a topology quality encoding vector, as shown below:

[0188] ;

[0189] ;

[0190] in, Indicates the time-varying available bandwidth. , Indicates link transmission delay, i.e. , Indicates link The concatenated vector, Represents the topology quality encoding vector. Represents a multilayer perceptron;

[0191] The feature concatenation and attention computation layer includes M attention computation heads, which concatenate the nodes in the discrete time-varying graph. Node comprehensive feature vector ,node That is, nodes Node comprehensive feature vector and topology quality encoding vector The common input feature concatenation and attention calculation layer obtains M attention weights through M attention calculation heads;

[0192] The M attention weights are weighted and summed through a multi-head aggregation activation layer, and the dense embedding is output through the ELU activation function, as shown below;

[0193] ;

[0194] in, This represents the attention weight of the m-th attention calculation head. Represents the node embedding vector. Represents the weight matrix. Indicates the activation function; Represents a node The node-comprehensive feature vector.

[0195] The feature concatenation and attention computation layer obtains M attention weights through M attention computation heads; including:

[0196] The m-th attention calculation head will input the feature, i.e. , and Perform vector concatenation as follows:

[0197] ;

[0198] in, Represents the joint eigenvector. This represents a vector concatenation operation. This represents the linear transformation matrix corresponding to the m-th attention computation head;

[0199] After concatenation, the joint feature vector is mapped to unnormalized attention coefficients, and then passed through the LeakyReLU activation function and the Softmax normalization function to obtain attention weights that comprehensively consider node state and link quality, as shown below:

[0200] ;

[0201] in, This represents the attention weight of the m-th attention calculation head. This represents the activation function. Indicates the first The learnable weight vector of each attention computation head.

[0202] The specific implementation process of step 4 includes:

[0203] microservice requests Perform dimensional broadcast replication to match the total number of computation nodes, and then copy the data to the embedding vectors of each node. After concatenation, input the Actor network to obtain the original action logic values. ;

[0204] The system compares the target computing node (any computing node in the network that is a candidate) with the microservice request in real time, and introduces a three-level hierarchical feasible domain relaxation mechanism to generate a dynamic action mask, i.e., a feasibility mask, as shown below:

[0205] ;

[0206] in, This indicates that node i is for the current task (the microservice request being assigned). The dynamic action mask; This indicates a conditional indicator function, which has a value of 1 when the condition inside the indicator function is true, and a value of 0 otherwise. Indicates the actual hardware architecture type of node i (such as general CPU or GPU accelerator card architecture).

[0207] If all nodes in the computing network do not satisfy the constraint, that is, all nodes... If both are 0, then calculate the memory satisfaction rate. ,when At that time, virtual memory swapping is triggered, generating a dynamic action mask. Record memory relaxation penalty cost ,in This represents the system threshold (ranging from 0.7 to 0.9). Represents the memory page swapping penalty constant (e.g.) =1.2, the possible range is [0.5, 2.0]). As shown below:

[0208] ;

[0209] in, Indicates a dynamic action mask;

[0210] If all nodes If the result is still 0, then a static lookup table mapping function is introduced. The model is based on the cross-compilation degradation rate of microservice containers across different hardware. It reflects the efficiency loss during cross-platform compilation or runtime conversion; for example, CUDA to CUDA is 1.0, CUDA to ROCm is 0.85, and CUDA to regular CPU is 0. Compatibility is obtained through a static lookup table mapping function, allowing nodes with a compatibility greater than 0 to participate in the candidate pool, and generating dynamic action masks. :

[0211] ;

[0212] The cost of architecture conversion penalty is calculated based on compatibility loss. :

[0213] ;

[0214] in, For conversion penalty coefficient ( ), Indicates multiplication operation;

[0215] The final dynamic action mask is the feasibility mask. for or or , all nodes Applied to action logic values: ,in, Indicates the action logic value. To penalize constants (usually taken as very large numbers, e.g.) ,when When (not feasible), subtract the original score. This will become an extremely small negative number, causing the probability of an infeasible node being selected to approach zero; for action logic values... Applying the Softmax function to obtain the probability distribution, and then performing random sampling, yields the target scheduling node. That is, the action that is output and executed in the current state.

[0216] Static lookup table mapping function Compatibility is obtained through a pre-established compatibility mapping table; including:

[0217] Enumerate the hardware architecture types supported by the computing power network (such as CPU, CUDA GPU, ROCm GPU, ARM CPU, domestic accelerator, etc.), select each microservice request (microservice container or benchmark task), and run them on nodes with different hardware architecture types. Record the execution success status, average execution latency, throughput, unit task completion time, and execution efficiency (ratio of completed tasks to execution time).

[0218] Microservice requests on the required hardware architecture Using the execution efficiency on other nodes as a benchmark, the execution efficiency of microservice requests on other nodes' actual hardware architecture types is compared with the benchmark to obtain the compatibility score. If a microservice request cannot run on other nodes' actual hardware architecture types, the compatibility score is 0. If it can run directly without cross-compilation, runtime conversion, or a compatibility layer, it is considered to be natively compatible, and the compatibility score is 1. If it needs to run through cross-compilation, runtime conversion, or a compatibility layer, the compatibility score is between 0 and 1. Finally, the compatibility score is written to a compatibility mapping table, and the static lookup mapping function obtains the corresponding compatibility score by looking up the compatibility mapping table.

[0219] The specific implementation process of step 5 includes:

[0220] like Figure 3 As shown, within the same time slot, after each microservice scheduling decision is completed, the target scheduling node is obtained. After deploying the current microservice request to the target scheduling node, calculate and update the real-time virtual queue of the target scheduling node:

[0221] ;

[0222] in, Represents an updated, real-time virtual queue;

[0223] Synchronously perform closed-loop recalculation of the comprehensive scheduling pressure representation to obtain the updated node comprehensive scheduling pressure. As shown below:

[0224] ;

[0225] Get It directly serves as the state input for the next microservice decision cycle within the same time slot, thereby fully establishing a micro-adaptive closed loop of "state - decision - queue accumulation - pressure recalculation - new state";

[0226] Subsequently, an environmental reward feedback model was constructed to calculate the topology congestion index for the current task scheduling path. As shown below:

[0227] ;

[0228] in, This indicates the distance a microservice request travels from the initiating gateway to the target scheduling node. The available bandwidth of the bottleneck link on the end-to-end routing path, that is, the minimum available bandwidth among all connected links on the path. Indicates the transmission delay reference;

[0229] Calculate topology congestion penalty Relaxing penalties with masking ;in, This represents the congestion penalty coefficient. This indicates the penalty for easing memory usage restrictions. Indicates the cost of architecture conversion penalties;

[0230] To reinforce the stress loop at the reinforcement learning level, a scheduling stress penalty term reflecting the immediate state of the action is introduced. :

[0231] ;

[0232] in, Indicates the pressure penalty coefficient ( The final environmental reward is calculated as follows:

[0233] ;

[0234] in, This refers to multidimensional composite rewards, also known as environmental rewards. Indicates the stable drift term in the queue. This represents the cumulative real-time virtual queue length of node j before the execution of this scheduling action. This represents the comprehensive cost penalty item, among which Let Lyapunov control parameters ([10, 100]) represent the weights of the system's trade-offs between maintaining virtual queue stability (congestion prevention) and pursuing low latency / low energy consumption (cost reduction and efficiency improvement). The larger the value, the more the system tends to optimize latency and energy consumption, but the risk of queue backlog also increases accordingly. This indicates the weight of the delay penalty in the cost penalty item. ), This represents the total latency of microservice request processing and queuing on node i. , This represents the energy consumption coefficient per unit computing power of node i (e.g., for CPU nodes, it is typically 1). Joules / GFLOPS, GPU node is Joules / GFLOPS, statically determined by the physical characteristics of the hardware architecture type (such as CPU or GPU accelerator card) of a node, characterize the energy efficiency of the node in performing computing tasks. This indicates the energy consumption penalty weight in the cost penalty item. ), This indicates a service quality violation penalty when the predicted delay exceeds the maximum tolerable delay. Apply a fixed negative penalty (-50 or -100) at that time. This represents the load balancing incentive. This represents the load balancing incentive constant factor ( ); Indicates topology congestion penalty;

[0235] Using the Welford algorithm to evaluate environmental rewards Perform online normalization and scaling truncation, and store the results in the experience replay pool.

[0236] The specific implementation process of step 6 includes:

[0237] Environmental rewards are input into the Critic network, which consists of two parallel subnetworks. and The target Q-value is calculated by taking the minimum value of the sub-network output to mitigate overestimation, as shown below:

[0238] ;

[0239] in, The expected value of the temporal difference (TD) required to update the Critic network is represented by the target Q value. This represents the discount factor for future rewards (usually a constant close to 1). Represents the temperature coefficient (temperature coefficient) Instead of adding or subtracting a fixed value each time, it will... (Consider the network parameters as being updatable via gradient descent). This represents the next state that the environment transitions to after an action is performed, i.e., a discrete time-varying graph and a microservice request; This indicates the next action under state s', which is to schedule the current microservice request to be executed on the compute node; , This represents the action value assessment output by the Critic network for two independent objectives. After an action is executed, the environment transitions to the next state according to the following rules: updating the virtual queue of the target node and updating the remaining computing power. Remaining memory update After all microservice requests within a time slot have been scheduled, the actual workload queue is updated according to the dynamic equation, and the next microservice request to be scheduled is... The randomness of state transitions, which are taken out of the arrival queue, comes from the Poisson process of microservice arrival and the random fluctuations in link bandwidth.

[0240] Huber Loss is used to update the Critic network parameters. During the smoothing phase, mean squared error (MSE) is used to ensure accuracy, while during the abrupt change phase, mean absolute error (MAE) is used to suppress extreme abnormal gradients caused by sudden changes in the virtual queue. (When the temporal difference between the true Q-value and the target Q-value is small, mean squared error (MSE) is used to ensure the model converges precisely to a local optimum. When encountering abrupt changes in the queue due to high-concurrency microservices, resulting in abnormal samples with large reward fluctuations, the model automatically degenerates to mean absolute error (MAE) to limit the gradient backpropagation amplitude and prevent network parameter collapse.) Temperature coefficients are also considered. As a learnable optimization variable, when the current information entropy of the agent's output policy is lower than the preset target information entropy benchmark, it is adaptively reduced by the optimizer. Strengthening the system's willingness to explore; conversely, increasing To encourage the use of known high-optimal actions (exploration); maximum entropy frame temperature coefficient Instead of adding or subtracting a fixed value each time, it will... The network parameters can be viewed as being updatable via gradient descent, as shown below:

[0241] ;

[0242] in, This represents the optimization loss function used for adaptively updating the temperature coefficient. ; This represents the mathematical expectation of the action sampling distribution under the current policy; This represents the policy probability of the output action in the current state; The preset target information entropy benchmark, used to control the strategy to maintain appropriate stochastic exploration capability, is the negative logarithm of the discrete action space size. ;in, This represents the size of the discrete action space, i.e., the number of current candidate scheduling nodes; when the action information entropy At that time, optimizing the loss function guides Gradient reduction (typically based on the Adam optimizer, learning rate) Each update may involve minor adjustments. (with a range of 5), and vice versa;

[0243] Through continuous iteration and updates, the network parameters of the Actor policy of the intelligent agent gradually converge. In the actual deployment phase, the central controller inputs the network topology and microservice requests collected in real time into the converged Actor network, and uses the network forward propagation and feasibility masking mechanism to quickly output the optimal node allocation scheme, thereby completing the microservice scheduling based on topology awareness.

[0244] The entire end-to-end training process includes four core steps: (1) Experience collection: The agent interacts with the environment to collect state transition data. (2) Network evaluation: Sample small batches of data from the replay pool, and update the parameters of the dual Critic network based on minimizing the temporal difference error using Huber Loss; (3) Policy optimization: Freeze the Critic parameters, calculate the gradient and update the parameters of the Actor network and the underlying shared GAT extractor by maximizing the expected Q value and policy entropy; (4) Adaptive adjustment: Adjust the parameters according to the actual information entropy of the current output action and the target information entropy. The difference is used by the Adam optimizer along Gradient direction update temperature coefficient Iterate in a loop until the loss converges and the scheduling success rate meets the target.

[0245] Taking a computing power network to handle high-concurrency microservice request scenarios as an example, the internal operation mechanism of the system's scheduling based on the method of this invention is as follows:

[0246] State awareness and topology preprocessing stage:

[0247] In the time slot At the start, the central controller scans the entire network (covering the core, regional, and edge topologies), extracting the 14-dimensional state of each node and the 4-dimensional features of each link. The system then runs Dijkstra's algorithm to pre-calculate the end-to-end transmission delay matrix. Initialize the virtual queues on each node. .

[0248] Concurrent request parsing and feature input stage:

[0249] The first request in the arriving set of microservice requests is resolved. (Requires 800 FLOPS, 32 GB of memory, and GPU architecture). The system broadcasts and concatenates its features to construct a state tensor and inputs it into a shared GAT encoder.

[0250] Graph encoding and action space masking stage:

[0251] The GAT encoder assigns extremely low attention weights to nodes on congested paths by incorporating features such as link bandwidth. Subsequently, a masking mechanism utilizes... Indicator function calculation Quickly subtract the output logic value of nodes with only 16GB of remaining memory or no GPU. This ensures that the probability of these nodes in the Softmax distribution is strictly 0.

[0252] Random sampling and state transition execution:

[0253] Based on the classification distribution of the output, the scheduling engine will... The node is assigned to a feasible and well-connected "core node 3". Upon assignment, a virtual queue update is triggered: the virtual queue for "core node 3" is updated. Increase the load pressure by 800 FLOPS and deduct the physical resource margin.

[0254] Composite reward-driven and dynamic distribution:

[0255] Calculate the single-step reward subject to Lyapunov penalty based on delay and energy consumption. The second task in the extraction set. When making decisions, since the virtual load of "core node 3" has increased, the drift penalty mechanism in the reward function will automatically force the model to change its strategy and distribute traffic to edge nodes with idle computing power. This micro-level nanosecond-level adjustment naturally forms a macro-level global load balance, avoiding the paralysis phenomenon caused by traditional greedy algorithms on core nodes.

[0256] Example 3

[0257] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the microservice scheduling method based on two-level queue feedback and topology awareness as described in Embodiment 1 or 2.

[0258] Example 4

[0259] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the microservice scheduling method based on two-level queue feedback and topology awareness as described in Embodiment 1 or 2.

[0260] Example 5

[0261] A microservice scheduling system based on two-level queue feedback and topology awareness includes:

[0262] The discrete time-varying graph construction module is configured to: construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a node comprehensive feature vector;

[0263] The microservice request extraction module is configured to: obtain microservice requests based on the microservice set and calculate the minimum end-to-end transmission delay baseline for the microservice requests;

[0264] The node embedding vector acquisition module is configured to: construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer;

[0265] The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector;

[0266] The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality;

[0267] By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector;

[0268] The target scheduling node acquisition module is configured to: concatenate the microservice request with the embedding vector of each computing node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each computing node, and obtain the target scheduling node based on the original action logic value;

[0269] The reward construction module is configured to: update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward;

[0270] The expected value acquisition module is configured to input the multidimensional composite reward into the Critic network, obtain the target expected value, and through continuous iterative updates, the Actor network reaches convergence and outputs the optimal target scheduling node.

Claims

1. A microservice scheduling method based on two-level queue feedback and topology awareness, characterized in that, include: Step 1: Construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a comprehensive feature vector of the nodes; Step 2: Obtain microservice requests based on the microservice set, and calculate the minimum end-to-end transmission delay baseline for each microservice request; Step 3: Construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer; The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector; The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality; By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector; Step 4: Concatenate the microservice request with the embedding vector of each compute node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each compute node, and obtain the target scheduling node based on the original action logic value; Step 5: Update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward; Step 6: Input the multidimensional composite reward into the Critic network to obtain the target expected value. Through continuous iterative updates, the Actor network converges and outputs the optimal target scheduling node.

2. The microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 1, characterized in that, The specific implementation process of step 1 includes: The computing network topology includes a core layer, a regional layer, and an edge layer; Based on the computing power network topology, a discrete time-varying graph is constructed. ;in, E(t) represents the set of nodes, including computing nodes in the core layer, region layer, and edge layer; E(t) represents the set of edges in time slot t, including communication links between computing nodes. Represents the node feature matrix, ,in, This represents the node comprehensive feature vector of the Nth node, where N represents the total number of computing nodes in the computing power network; Represents the edge feature matrix, ,in, This represents a directed link, or communication link, between any nodes i and j. For any directed link Construct a 4D edge feature vector This includes physical connectivity status indicators and time-varying available bandwidth. Link transmission delay and inter-node link packet loss rate ; In the node feature matrix In the diagram, the node-comprehensive feature vector of any node i includes 14-dimensional features, as shown below: ； in, This represents the comprehensive feature vector of the i-th node. Static identifier characteristics include: maximum computing power resources, maximum memory capacity, hardware architecture and accelerator type, and node deployment level identifier; Represents dynamic state characteristics, including: remaining computing power Remaining memory Internal load rate External reach Cumulative energy consumption ;in, This indicates that the current time slot has received load. Indicates maximum load capacity. Represents an instantaneous virtual queue; Indicates service rate, service rate , For service rate coefficient; The policy attributes include: service capability level, security domain level, unit computing power price, and system priority weight. This indicates the overall scheduling pressure on the nodes; Overall scheduling pressure of nodes ; in, , , These represent queue pressure weight, resource utilization weight, and network pressure weight, respectively. Represents an instantaneous virtual queue; Indicates the utilization rate of node basic resources. This indicates the bottleneck pressure at the node level; Basic resource utilization rate As shown below: ； in, Represents a node That is, nodes In the time slot of Utilization rate Represents a node In the time slot Real-time memory utilization; Assuming node-based The first-order neighbor set is Then the node-level network bottleneck pressure As shown below: ； in, , , , These represent bandwidth weight, latency weight, packet loss weight, and jitter weight, respectively. Represents the first-order neighbor set The node, This represents the link utilization rate, where Indicates the current bandwidth. Indicates the maximum bandwidth of the link. Indicates the link normalized transmission delay. This represents normalized link jitter, as shown below: ； in, Indicates the standard deviation of time delay. This represents the round-trip time of the k-th probe packet sent by the network probe, where n represents the number of probe packets sent. Indicates the average delay. ; Normalized link jitter is: ； in, This indicates the preset maximum jitter threshold; At each node There are nodes on it Physical workload queue and nodes Instant virtual queue ; The physical workload queue represents the actual backlog of tasks, and the physical workload queue is updated between each time slot according to the following dynamic equation: ； in, This indicates the physical workload queue for the next time slot. For time slots Internal allocation to nodes Total computing requirements Indicates the service rate; at the beginning of each time slot, an immediate virtual queue initialization is performed: .

3. The microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 2, characterized in that, The specific implementation process of step 2 includes: In each time slot The collection of microservices that arrive is ; where each microservice request Represented as: ； in, To meet computing resource requirements, For memory requirements, For the maximum tolerable latency, For architecture compatibility requirements, i.e., the required hardware architecture; Deploy each microservice request to a node that meets the constraints, i.e. ;in, It is a Boolean scheduling decision variable, that is, when a microservice request... Assigned to the i-th computing node hour, ,otherwise ; Taking the gateway node that initiates the microservice request as the source, calculate the minimum end-to-end transmission delay benchmark from the source node to all candidate nodes i in the entire network using Dijkstra's algorithm. ;in, An identifier index for microservice requests.

4. The microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 3, characterized in that, The specific implementation process of step 3 includes: Through the edge feature mapping layer, from discrete time-varying maps Extracting nodes spliced ​​vector The data is then input into a multilayer perceptron network for nonlinear mapping to obtain a topology quality encoding vector, as shown below: ； ； in, Indicates the time-varying available bandwidth. , Indicates link transmission delay, i.e. , Indicates link The concatenated vector, Represents the topology quality encoding vector. Represents a multilayer perceptron; The feature concatenation and attention computation layer includes M attention computation heads, which concatenate the nodes in the discrete time-varying graph. Node comprehensive feature vector ,node That is, nodes Node comprehensive feature vector and topology quality encoding vector The common input feature concatenation and attention calculation layer obtains M attention weights through M attention calculation heads; The M attention weights are weighted and summed through a multi-head aggregation activation layer, and the dense embedding is output through the ELU activation function, as shown below; ； in, This represents the attention weight of the m-th attention calculation head. Represents the node embedding vector. Represents the weight matrix. Indicates the activation function; Represents a node The node-comprehensive feature vector; The feature concatenation and attention computation layer obtains M attention weights through M attention computation heads; including: The m-th attention calculation head will input the feature, i.e. , and Perform vector concatenation as follows: ； in, Represents the joint eigenvector. This represents a vector concatenation operation. This represents the linear transformation matrix corresponding to the m-th attention computation head; After concatenation, the joint feature vector is mapped to unnormalized attention coefficients, and then passed through the LeakyReLU activation function and the Softmax normalization function to obtain attention weights that comprehensively consider node state and link quality, as shown below: ； in, This represents the attention weight of the m-th attention calculation head. This represents the activation function. Indicates the first The learnable weight vector of each attention computation head.

5. A microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 4, characterized in that, The specific implementation process of step 4 includes: microservice requests Perform dimensional broadcast copying and compare it with the embedding vectors of each node. After concatenation, input the Actor network to obtain the original action logic values. ; The target compute node and microservice requests are compared in real time to generate a dynamic action mask, also known as a feasibility mask, as shown below: ； in, This represents the dynamic action mask of node i for the current task; This indicates a conditional indicator function, which has a value of 1 when the condition inside the indicator function is true, and a value of 0 otherwise. Indicates the actual hardware architecture type of node i; If all nodes in the computing network do not satisfy the constraint, that is, all nodes... If both are 0, then calculate the memory satisfaction rate. ,when At that time, a dynamic action mask is generated. Record memory relaxation penalty cost ,in Indicates the system threshold. This represents the memory page swapping penalty constant; As shown below: ； in, Indicates a dynamic action mask; If all nodes If the result is still 0, then a static lookup table mapping function is introduced. The compatibility score is obtained through a static lookup table mapping function, allowing nodes with a compatibility score greater than 0 to participate in the candidate selection, and generating a dynamic action mask. : ； The cost of architecture conversion penalty is calculated based on compatibility loss. : ； in, To convert the penalty coefficient, Indicates multiplication operation; The final dynamic action mask is the feasibility mask. for or or , all nodes Applied to action logic values: ,in, Indicates the action logic value. For penalty constants; for action logic values Apply the Softmax function to obtain the probability distribution, and perform random sampling to obtain the target scheduling node's output and execution action in the current state; Static lookup table mapping function Compatibility is obtained through a pre-established compatibility mapping table; including: Enumerate the hardware architecture types supported by the computing power network, select each microservice request, and run them on nodes with different hardware architecture types. Record the execution success status, average execution latency, throughput, unit task completion time, and execution efficiency. Microservice requests on the required hardware architecture Using the execution efficiency on other nodes as a benchmark, the execution efficiency of microservice requests on other nodes' actual hardware architecture types is compared with the benchmark to obtain the compatibility score. If a microservice request cannot run on other nodes' actual hardware architecture types, the compatibility score is 0. If it can run directly without cross-compilation, runtime conversion, or a compatibility layer, it is considered to be natively compatible, and the compatibility score is 1. If it needs to run through cross-compilation, runtime conversion, or a compatibility layer, the compatibility score is between 0 and 1. Finally, the compatibility score is written to a compatibility mapping table, and the static lookup mapping function obtains the corresponding compatibility score by looking up the compatibility mapping table.

6. A microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 5, characterized in that, The specific implementation process of step 5 includes: Within the same time slot, after each microservice scheduling decision is completed, i.e. the target scheduling node is obtained and the current microservice request is deployed to the target scheduling node, the real-time virtual queue of the target scheduling node is calculated and updated: ； in, Represents an updated, real-time virtual queue; Synchronously perform closed-loop recalculation of the comprehensive scheduling pressure representation to obtain the updated node comprehensive scheduling pressure. As shown below: ； Get It is directly used as the state input for the next microservice decision cycle within the same time slot; Calculate the topology congestion index for the current task scheduling path. As shown below: ； in, This indicates the distance a microservice request travels from the initiating gateway to the target scheduling node. The available bandwidth of the bottleneck link on the end-to-end routing path, that is, the minimum available bandwidth among all connected links on the path. Indicates the transmission delay reference; Calculate topology congestion penalty Relaxing penalties with masking ;in, This represents the congestion penalty coefficient. This indicates the penalty for easing memory usage restrictions. Indicates the cost of architecture conversion penalties; Introduce a scheduling pressure penalty term that reflects the immediate state of an action. : ； in, This represents the stress penalty coefficient; the final environmental reward is calculated as follows: ； in, This refers to multidimensional composite rewards, also known as environmental rewards. Indicates the stable drift term in the queue. This represents the cumulative real-time virtual queue length of node i before the execution of this scheduling action. This indicates a comprehensive cost penalty item. This represents the Lyapunov control parameters. This indicates the weight of the delay penalty in the cost penalty item. This represents the total latency of microservice request processing and queuing on node i. , This represents the energy consumption coefficient per unit computing power of node i. This indicates the energy consumption penalty weight in the cost penalty item. This indicates a service quality violation penalty when the predicted delay exceeds the maximum tolerable delay. Apply a fixed negative penalty at the time. This represents the load balancing incentive. This represents the load balancing incentive constant factor; Indicates topology congestion penalty; Using the Welford algorithm to evaluate environmental rewards Perform online normalization and scaling truncation, and store the results in the experience replay pool.

7. A microservice scheduling method based on two-layer queue feedback and topology awareness according to claim 6, characterized in that, The specific implementation process of step 6 includes: Environmental rewards are input into the Critic network, which consists of two parallel subnetworks. and The target Q value is calculated by taking the minimum value of the sub-network output, as shown below: ； in, This represents the expected temporal difference objective value required to update the Critic network, i.e., the objective Q value. Indicates the discount factor for future rewards; Indicates the temperature coefficient; This represents the next state that the environment transitions to after an action is performed, i.e., a discrete time-varying graph and a microservice request; This indicates the next action under state s', which is to schedule the current microservice request to be executed on the compute node; , This represents the action value assessment output by the Critic network for two independent objectives; Huber Loss is used to update the Critic network parameters, including the temperature coefficient. As a learnable optimization variable, when the current information entropy of the agent's output policy is lower than the preset target information entropy benchmark, it is adaptively reduced by the optimizer. Conversely, it increases. As shown below: ； in, This represents the optimization loss function used for adaptively updating the temperature coefficient. ; This represents the mathematical expectation of the action sampling distribution under the current policy; This represents the policy probability of the output action in the current state; The preset target information entropy benchmark is the negative logarithm of the size of the discrete action space, i.e. ;in, This represents the size of the discrete action space, i.e., the number of current candidate scheduling nodes.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the microservice scheduling method based on two-level queue feedback and topology awareness as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the microservice scheduling method based on two-level queue feedback and topology awareness as described in any one of claims 1-7.

10. A microservice scheduling system based on two-layer queue feedback and topology awareness, characterized in that, include: The discrete time-varying graph construction module is configured to: construct a discrete time-varying graph based on the computing power network topology, collect multi-dimensional node features, and construct a node comprehensive feature vector; The microservice request extraction module is configured to: obtain microservice requests based on the microservice set and calculate the minimum end-to-end transmission delay baseline for the microservice requests; The node embedding vector acquisition module is configured to: construct a shared graph attention network, including an edge feature mapping layer, a feature concatenation and attention calculation layer, and a multi-head aggregation activation layer; The discrete time-varying graph is input into the shared graph attention network, and the topological connectivity quality is captured through the edge feature mapping layer to obtain the topological quality encoding vector; The discrete time-varying graph and the topology quality encoding vector are input together into the feature concatenation and attention calculation layer to obtain multiple attention weights that comprehensively consider node state and link quality; By using a multi-head aggregation activation layer, multiple attention weights are weighted and summed to obtain the node embedding vector; The target scheduling node acquisition module is configured to: concatenate the microservice request with the embedding vector of each computing node, calculate the original action logic value through the Actor network; obtain the feasibility mask of each computing node, and obtain the target scheduling node based on the original action logic value; The reward construction module is configured to: update the real-time virtual queue and overall scheduling pressure of the target scheduling node, and calculate the multi-dimensional composite reward; The expected value acquisition module is configured to input the multidimensional composite reward into the Critic network, obtain the target expected value, and through continuous iterative updates, the Actor network reaches convergence and outputs the optimal target scheduling node.

Images