A method and system for dynamically scheduling computing power resources based on spatiotemporal joint perception

Abstract

This invention relates to a dynamic scheduling method for computing resources based on spatiotemporal joint perception, comprising: receiving computing requests containing geographic location information in real time; constructing the current computing network topology state and the historical state sequence of the last W time steps based on real-time network data; inputting the state into a spatiotemporal feature extraction model to obtain a fused state feature vector, which is processed through nested fusion, i.e., first using a graph neural network (GNN) to encode the spatial features of the network topology state at each time step, then inputting the obtained spatial feature vector sequence into a time series model to output a fused state feature vector; finally, inputting the vector into a reinforcement learning agent to output a scheduling action. Through the above design, this invention effectively improves the globality and foresight of scheduling decisions, achieving the technical effects of increasing task success rate, reducing service latency, and balancing system load.

Description

Technical Field

[0001] This invention relates to the fields of mobile communication and edge computing technology, and in particular to a method for dynamic scheduling of computing resources based on spatiotemporal joint sensing. Background Technology

[0002] With the accelerated deployment of integrated "vehicle-road-cloud" systems, 5G-V2X (vehicle-to-everything) technology has become a key technology for achieving high-level autonomous driving and intelligent traffic management. This network carries safety-critical services such as cooperative platooning and intersection collision avoidance, which typically combine computationally intensive characteristics with extremely low latency requirements (e.g., end-to-end latency in emergency braking scenarios needs to be controlled within 20 milliseconds). To address this challenge, the multi-access edge computing (MEC) paradigm has emerged, which, by offloading computing resources to the network edge, can theoretically significantly reduce task processing latency.

[0003] However, the high-speed mobility of vehicles causes computational requests to fluctuate dramatically and become random in both time and space, resulting in highly non-stationary resource demands in edge networks. In such highly dynamic environments, existing resource scheduling methods, especially those based on Deep Reinforcement Learning (DRL), exhibit two main limitations:

[0004] First, there is a disconnect in the perception of spatiotemporal features. Existing technologies struggle to achieve deep integration of spatiotemporal dimensions. Some solutions employ Graph Neural Networks (GNNs) to characterize the spatial relationships within the network topology, but often overlook the long-term temporal evolution of resource demands, exhibiting a "spatial awareness, temporal shortsightedness" deficiency. Other solutions utilize sequence models such as Long Short-Term Memory (LSTM) networks to predict future resource states, but treat nodes in the network as isolated individuals, failing to effectively capture the topological relationships between node states and their neighborhood dynamics. This fragmented modeling approach fails to reflect the physical essence that "node states are driven by both their own history and the dynamics of their neighbors," resulting in a sharp degradation in scheduling performance when facing complex traffic patterns such as urban "tidal flows."

[0005] Second, the optimization objectives lack a global foresight. Most existing solutions focus excessively on optimizing immediate Quality of Service (QoS) metrics at the single-task level (such as latency and success rate of individual tasks), neglecting effective consideration and constraints on the long-term health of system-wide resources. This short-sighted optimization strategy leads to uneven resource distribution among edge nodes when agents pursue immediate rewards—some nodes experience task failures or QoS drops due to momentary overload, while other nodes remain idle. This not only fails to maintain a steady state of resource utilization at the global level but also makes it difficult to prevent local congestion, severely restricting the stability and reliability of the system in long-term operation.

[0006] Therefore, there is an urgent need in this field for a new method for dynamic scheduling of computing resources that can deeply integrate spatiotemporal characteristics and has a global and long-term optimization perspective, in order to cope with the severe challenges brought about by highly dynamic network environments such as 5G-V2X.

[0007] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0008] The technical problem this application aims to solve is: "How to achieve efficient, balanced and forward-looking scheduling of computing resources in a highly dynamic network environment, so as to collaboratively ensure the quality of real-time task services and the long-term operational stability of the system."

[0009] The technical solution adopted in this application to solve the above-mentioned technical problems is as follows.

[0010] This application provides a method for dynamic scheduling of computing resources based on spatiotemporal joint sensing, including: Real-time reception of computing power requests, which must include at least the geographical location information of the requesting node; Based on computing power requests and real-time network data, construct the current computing power network topology state and a historical state sequence containing the most recent W time steps, where W is a positive integer greater than 1; The current computing network topology state and historical state sequence are input into the spatiotemporal feature extraction model to obtain the fused state feature vector; Among them, the spatiotemporal feature extraction model is processed through nested fusion, which includes: using a graph neural network (GNN) to encode the spatial features of the computing network topology state at the current moment and the computing network topology state at each time step in the historical state sequence, respectively, to obtain the spatial feature vector of the corresponding time step; inputting the obtained series of spatial feature vectors into the time series model, and outputting the fused state feature vector representing the spatiotemporal dynamics of the system by the time series model. The fused state feature vector is input into the reinforcement learning agent, which then outputs scheduling actions in response to the computing power request.

[0011] In some embodiments, the computing power request may also include the required computing resource size and the maximum tolerable latency.

[0012] In some embodiments, the topology of the computing power network is represented by a graph structure, in which: the nodes of the graph correspond to computing nodes in the computing power network; the feature vector of each node of the graph is defined based on the attributes of its corresponding computing node, wherein the attributes of the computing node include: the geographical coordinates of the computing node, the current proportion of remaining computing power resources, and the node type code; the edges of the graph correspond to the computing power network connections between computing nodes, each edge represents a network link, and the feature vector of each edge includes the transmission delay of the network link.

[0013] In some embodiments, the graph neural network (GNN) includes the graph attention network (GAT).

[0014] In some embodiments, the time series model includes a Long Short-Term Memory (LSTM) network or a Transformer model.

[0015] In some embodiments, the reinforcement learning agent includes a deep reinforcement learning (DRL) agent; the deep reinforcement learning (DRL) agent includes a dual deep Q network (DDQN).

[0016] In some embodiments, the reinforcement learning agent is trained by maximizing cumulative rewards, and its reward function includes at least two of the following: a base reward, a compound delay penalty, an adaptive long-term resource utilization reward, and an asymptotic penalty for transient overload.

[0017] In some embodiments, the scheduling action indicates that the current computing power request is allocated to the target computing node; the dynamic scheduling method for computing power resources further includes: performing a scheduling action, completing resource allocation and updating the system status.

[0018] In some embodiments, a dynamic scheduling system for computing resources is also provided, comprising: The request receiving module is used to receive computing power requests in real time. The computing power request must contain at least the geographical location information of the requesting node. The state construction module is used to construct the current computing power network topology state and a historical state sequence containing the most recent W time steps based on computing power requests and real-time network data, where W is a positive integer greater than 1. The feature extraction module is used to input the current computing network topology state and the historical state sequence into the spatiotemporal feature extraction model to obtain a fused state feature vector. The spatiotemporal feature extraction model processes the data through nested fusion, which includes: using a graph neural network (GNN) to encode the spatial features of the current computing network topology state and the computing network topology state at each time step in the historical state sequence, obtaining spatial feature vectors for the corresponding time steps; inputting the obtained series of spatial feature vectors into the time series model, which then outputs a fused state feature vector representing the spatiotemporal dynamics of the system. The intelligent decision-making module is used to input the fused state feature vector into the reinforcement learning agent, which then outputs scheduling actions in response to the computing power request.

[0019] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the dynamic scheduling method for computing resources of this application.

[0020] The present invention has the following beneficial effects: This invention, through the technical feature of "receiving computing power requests in real time, with each request containing at least the geographical location information of the requesting node," continuously acquires the raw input data required for scheduling decisions, ensuring the system's real-time response capability to dynamic external environments and providing a data foundation for subsequent accurate perception and decision-making. Through the technical feature of "constructing the current computing power network topology state and a historical state sequence containing the most recent W time steps based on computing power requests and real-time network data," it comprehensively and multidimensionally depicts the instantaneous spatial layout and temporal evolution of the computing power network, providing a system state representation with rich contextual information for the spatiotemporal feature extraction model, overcoming the limitations of relying solely on instantaneous network snapshots. Furthermore, by "inputting the current computing power network topology state and the historical state sequence into the spatiotemporal feature extraction model to obtain a fused state feature vector," and this spatiotemporal feature extraction model employs "nested fusion processing" (i.e., "using a graph neural network (GNN) to spatially encode the current computing power network topology state and the computing power network topology state at each time step in the historical state sequence to obtain the corresponding time feature vector"), this invention achieves this. The technical features of the step size spatial feature vector; inputting a series of spatial feature vectors into a time series model, and outputting a fused state feature vector representing the spatiotemporal dynamics of the system by the time series model) achieve deep coupling and joint learning of spatial topological relationships and temporal dynamic laws. This nested structure ensures that the time series model analyzes not the original, isolated node states, but a high-level abstract feature sequence that already contains its spatial neighborhood relationships. This enables more accurate capture of the spatiotemporal evolution pattern of the system state, significantly improving the accuracy of predicting future load and resource competition risks, and laying a solid foundation for forward-looking scheduling. Through the technical feature of "inputting the fused state feature vector into a reinforcement learning agent, and outputting a scheduling action for the computing power request by the reinforcement learning agent", the advantages of reinforcement learning agents in handling high-dimensional, continuous state spaces and complex decision-making problems can be utilized to automatically learn and output a scheduling strategy that approximates the optimal based on the deeply fused spatiotemporal state features. This data-driven decision-making method is superior to heuristic methods that rely on fixed rules and can adapt to complex and non-stationary network environments.

[0021] In summary, this invention constructs a complete, data-driven intelligent scheduling process through the synergistic effect of the aforementioned technical features. This process achieves accurate and joint perception of the spatiotemporal dynamics of the computing power network, and makes adaptive and forward-looking optimization decisions based on this perception. This effectively addresses spatiotemporal fluctuations in computing power demand, enhances the global and long-term nature of scheduling decisions, and ultimately elevates the computing power resource scheduling mode from "passive response" to "active prediction and optimization." This achieves a global and long-term optimal balance between service quality and resource efficiency in a highly dynamic and complex network environment.

[0022] Other beneficial effects of the present invention will be further described below. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 Flowchart for real-time scheduling of STAR-GQN computing resources; Figure 2 A flowchart illustrating the specific design process of the GNN model; Figure 3 The spatiotemporal joint modeling framework diagram is shown, with time window (time step) W=3 as an example; Figure 4 Execute the scheduling decision process for the DDQN model; Figure 5 Training process for DDQN models; Figure 6 It adopts a three-tier computing power architecture of cloud-data center-base station; Figure 7 To analyze decision delays for different models in simulation and performance evaluation; Figure 8 This represents the average reward performance of different models in different scenarios during simulation and performance evaluation. Figure 9 To demonstrate the task completion rate of different models in different scenarios during simulation and performance evaluation; Figure 10 The success rate of the STAR-GQN model for tasks with different latency requirements is shown. Detailed Implementation

[0024] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0026] This invention proposes a dynamic scheduling method for computing resources based on spatiotemporal joint perception. Its goal is to promote the shift of the scheduling paradigm from "passive response" to "active prediction" through a deeply coupled spatiotemporal feature fusion architecture and a dual-dimensional reward mechanism that takes into account both immediate and long-term goals. In a highly dynamic 5G-V2X network environment, this method aims to achieve a balance between task success rate, latency optimization, and long-term steady-state system resources.

[0027] In some embodiments, the innovative aspects of this invention include, but are not limited to: A unified scheduling framework, STAR-GQN (Spatio-Temporal Adaptive Real-time Graph Q-Network), is constructed. This framework creatively integrates a Graph Attention Network (GAT) for spatial topology encoding with a time series model (LSTM or Transformer) in a nested manner, and incorporates it into a Dual Deep Q-Network (DDQN). Its technological advancement lies in the fact that the input to the sequence model is already a joint feature representation incorporating spatial embeddings and task attributes, thereby achieving true spatiotemporal joint learning at the model's underlying level and improving the accuracy of future load predictions.

[0028] This invention designs a dual-dimensional joint optimization reward function: going beyond the traditional modeling approach that only optimizes immediate QoS, it introduces the dimension of "long-term system health." This function employs an adaptive segmentation mechanism: on the one hand, it ensures task success rate and latency through immediate rewards; on the other hand, it strictly constrains global oversaturation trends and instantaneous node overload through resource utilization penalties. This mechanism proactively guides the agent to consider the long-term balance of global resources when making decisions, enhancing the system's resilience.

[0029] In some embodiments, the method of the present invention captures the topological characteristics of the computing power network through a graph neural network, learns the temporal dynamics of the request load through a time series prediction model, and makes forward-looking decisions using a deep reinforcement learning agent, thereby achieving efficient and adaptive scheduling of cross-domain computing power resources.

[0030] In some embodiments, Figure 1 A flowchart illustrating the real-time scheduling method for computing resources provided in this application embodiment. This method is primarily executed by a scheduling center (which can be deployed in a server room within the core network), and specifically includes the following steps:

[0031] 1. Step S1: Real-time data input.

[0032] Execution entity: Data interface module of the scheduling center.

[0033] Scenario: This method runs continuously and receives external input in real time.

[0034] Specific details: This module continuously receives user request streams from various base stations. Each computing power request includes at least the geographical location of its occurrence, the required computing resources, and the maximum tolerable latency, among other constraints.

[0035] The data interface module parses, verifies, and standardizes the received raw request information, packages it into a unified data structure, and outputs it to the downstream environment simulation layer.

[0036] 2. Step S2: The environment simulation layer constructs the current state.

[0037] Execution entity: the environment simulator of the scheduling center.

[0038] Scenario: Whenever a new scheduling decision requirement is triggered.

[0039] Specific details: The environment simulator constructs two core state representations based on real-time network data: Topology State: Obtain real-time information about the global computing network, including the set of all server rooms, the current remaining computing resources of each room, and the network link status between rooms (such as bandwidth utilization and transmission latency). Construct this information into a graph structure, where nodes represent server rooms, edges represent network connections, and the attributes of nodes and edges correspond to their dynamic states.

[0040] Historical State Sequence: Extract the system state sequence over the most recent W time steps from the state buffer. This sequence records the dynamic changes in network topology and resource states. W is tested and analyzed using time window lengths commonly used in LSTM and transformer models, comparing the performance of different time window models. In practical applications, the time window size can be selected according to specific circumstances.

[0041] The environment simulator aggregates data from the network monitoring system, generates a graph data structure, and maintains a fixed-length state queue to store historical state sequences. Finally, the topology state and historical state sequences are output together to the spatiotemporal feature extraction module.

[0042] 3. Step S3: Spatiotemporal feature extraction.

[0043] Execution entity: Spatiotemporal feature extraction module of the scheduling center (integrating GNN and LSTM / Transformer).

[0044] Scenario: Upon receiving the topology state and historical state sequence output by the environment simulation layer, execute immediately.

[0045] Specific details: This step aims to deeply integrate network features from spatial and temporal dimensions.

[0046] S31: Space processing.

[0047] Execution entity: Graph neural network model.

[0048] Specific content: Perform convolution operation on the current topology state graph to learn the embedded representation of each node (data center) in the global network topology.

[0049] Detailed design: refer to Figure 2 : Flowchart of the specific design process of the GNN model; Input feature definition: The input to GNN is designed to comprehensively characterize the static properties and dynamic states of the computing network.

[0050] Node characteristics: The feature vector of each node (representing a computing node, such as a base station, server room, or cloud center) is defined as [x coordinate, y coordinate, computing power resource size, node type].

[0051] x-coordinates and y-coordinates represent the geographic location information of the nodes, implicitly used to calculate the physical distance between nodes. They are standardized during initialization, with coordinate values ​​scaled to the range of 0-100.

[0052] Computing resource size: This represents the amount of computing resources currently available to the node, and is a dynamically changing attribute. It is normalized using the ratio of the current remaining computing power to the total computing power of the node.

[0053] Node type: Use codes to identify the level or type of a node (e.g., 1 represents a base station, 2 represents a server room, and 0 represents the cloud) to help the model understand the functional differences between different nodes.

[0054] Edge characteristics: Each edge (representing a network connection between nodes) is characterized by its transmission delay. This characteristic dynamically reflects the communication quality of the current network link.

[0055] Network topology: Defines the connection relationships between nodes in the form of an adjacency matrix, describing the physical or logical connectivity of the computing network.

[0056] Graph Attention Layer: This design employs a two-layer cascaded Graph Attention Layer (GAT) to progressively aggregate neighborhood information. The initial feature dimension of each node is 4 (corresponding to coordinates, computational power, and type encoding), mapped to a 64-dimensional hidden layer via a linear layer before being input into the first GAT layer. The core formula of the attention mechanism is: For the central node u and its neighbor v, calculate the normalized attention coefficient. :

[0057]

[0058] This mechanism allows the model to assign appropriate weights to different neighbors, rather than treating them equally. Each layer of GAT uses four independent attention heads. Node u is in the... The output features of a layer are formed by concatenating the outputs of each attention point (||).

[0059]

[0060] Layer 1: Local Feature Extraction: This layer primarily focuses on the relationships between directly adjacent nodes. For example, for a base station node, this layer aggregates information about its connected base stations and the server room it directly belongs to, thereby optimizing the feature representation within a local scope.

[0061] Second layer: Global feature extraction: By stacking a second graph attention layer, the receptive field of each node is expanded, allowing aggregation of information from multi-hop neighbors. This enables the features of a base station node to include state information from indirectly connected server rooms or even cloud centers, thereby capturing long-range dependencies in the network and forming a global perspective.

[0062] Global Pooling: After processing through two graph attention layers, the updated high-level feature vector for each node is obtained. To generate a fixed-length summary representing the entire network state, this design employs a global attention pooling layer. This layer calculates the importance weight for each node (obtained through an MLP and softmax function) and performs a weighted summation of the features from all nodes, ultimately outputting a 64-dimensional graph-level feature vector. .

[0063]

[0064] This design allows the model to adaptively learn the importance of each node in representing the overall network state. For example, a critical data center node with limited resources might be assigned a higher weight.

[0065] Output: A set of node embedding vectors containing rich spatial structure information.

[0066] S32: Timing processing.

[0067] Execution subject: LSTM model or Transformer model.

[0068] Specifically, this step involves analyzing the chronologically ordered sequence of historical spatial features output by the spatial processing (S31) module to capture the temporal evolution of the system state. The input to this step is not the original historical state, but rather a sequence of graph-level feature vectors preprocessed by a GNN, thus achieving nested deep fusion of spatiotemporal features.

[0069] Detailed design: refer to Figure 3 This application proposes a "spatiotemporal joint modeling" framework implemented in the feature extraction module of the scheduling center. This framework works collaboratively by nesting a parameter-shared graph neural network (GNN) before each time step input of a temporal model (LSTM / Transformer), performing deep feature extraction and fusion of the computing network state from both spatial and temporal dimensions. Its workflow is as follows: Figure 3 Each step in the process:

[0070] Nested processing flow: For decision time t, the system maintains a historical state sequence of length W. The same set of GNN model parameters was used to process each state in the sequence. Output the corresponding spatial feature vectors in sequence. Meanwhile, from each state Extract the feature vector of the task request (computing power request) at the corresponding time. (This represents information such as the computational requirements and latency constraints of the tasks to be scheduled at the current moment). Subsequently, the spatial feature vector at each moment... Its corresponding request feature vector The data is then concatenated to form a fused feature vector that includes spatiotemporal and task information. This process embeds spatial encoding and immediate request information together within the feature representation at each time step.

[0071] Spatiotemporal feature sequence formation: The feature vector sequence obtained from the above steps, which integrates spatial and request information. It serves as the direct input to the timing module (LSTM or Transformer). Instead of processing the raw topology data, the timing module analyzes the evolution of this feature sequence, which has undergone both spatial abstraction and task-aware encoding, thereby capturing the dynamic patterns of the system state under the combined effects of spatiotemporal dimensions and load demands.

[0072] Continuous dynamic extraction based on LSTM: Input dimension is The sum of the dimension (64) and the dimension of the requested feature (5) is (69). A unidirectional single-layer LSTM is used, with the hidden state dimension set to 128. The hidden state of the last time step of the LSTM is used as the temporal feature. After the spatial feature sequence is input into the LSTM module, the model recursively processes the time flow using its internal gating mechanism. Through the step-by-step propagation of hidden states, LSTM can effectively capture the continuous evolution trajectory of network load and resource consumption. At the current time step, its output feature vector completes a deep aggregation of local transient fluctuations and short-term trends within the observation window, forming a temporally enhanced contextual representation.

[0073] Global dependency capture based on Transformer: First, the 69-dimensional features are projected to 128 dimensions through a linear layer, and learnable positional encodings are added. Then, a 2-layer Transformer encoder is input, with each layer using a 4-head self-attention mechanism, and the feedforward network has a dimension of 256. The features corresponding to the last time step in the encoder's output sequence are taken as... Compared to recurrent architectures, the Transformer encoder offers a different paradigm for sequence modeling through positional encoding and multi-head self-attention. This mechanism allows the model to compute correlation weights between any time steps within a sequence in parallel, thus overcoming the local receptive field limitations of recursive structures and directly establishing long-range global dependencies. When characterizing macroscopic traffic patterns with strong periodicity and phase regularity, such as "urban tidal flows," the Transformer can output a comprehensive representation focusing on global evolutionary features.

[0074] Complementarity of Two Paradigms and Scenario Adaptation: In this framework, LSTM and Transformer serve as optional temporal foundations, jointly forming a deep perception system for the spatiotemporal dynamics of the system. Their core value lies in the strong complementarity of their information processing paradigms: LSTM excels at capturing local, continuous short-term dependencies, demonstrating computational efficiency and robustness when dealing with micro-fluctuations caused by the high mobility of individual vehicles; while Transformer, with its global perspective, possesses a natural advantage in handling highly non-stationary macro-periodic fluctuations across time windows (such as extreme tidal traffic). By integrating these two differentiated modelers, the scheduling framework can flexibly adapt to the actual complexity of the road network dynamics, ensuring that the strategy maintains excellent foresight and robustness under ever-changing spatiotemporal fluctuations.

[0075] The innovation of this framework lies in the fact that it does not analyze the network state at a single moment in isolation, but concatenates the spatial features extracted by GNN in the time dimension, and then uses LSTM / Transformer to mine its implicit temporal patterns. This allows the scheduling decision to consider both the current resource layout and the development trend of the system state, significantly improving the foresight and adaptability of the scheduling strategy.

[0076] Output: A fusion state feature vector representing the recent dynamics and trends of the system is obtained.

[0077] 4. Step S4: Intelligent decision-making.

[0078] Execution entity: DDQN policy network of the scheduling center.

[0079] Scenario: After receiving the fused state feature vector.

[0080] Specific details: Based on the current fusion state, calculate the Q-value of all possible scheduling actions and select the optimal action.

[0081] Detailed design: refer to Figure 4 The DDQN model executes the scheduling decision process; The DDQN module is the core decision-making component of this application. It receives a 128-dimensional feature vector containing spatiotemporal dynamic information, output by the temporal prediction model. This vector is then input into a two-layer fully connected network (i.e., the decision head) to predict Q-values: The first fully connected layer maps the 128-dimensional input to a 128-dimensional hidden layer, followed by a ReLU activation function. The second fully connected layer (output layer) maps the 128-dimensional hidden layer to the action space. This layer does not use an activation function and directly outputs the raw Q-value estimate for each action. Its design details are as follows:

[0082] Action Space: The action space defines all possible decisions an agent can make. When a new computing power request arrives, the agent must allocate a target computing node to it. The action space is a discrete space whose size is the total number of schedulable nodes in the network (N) plus 1 (representing the cloud). Specifically, it is defined as:

[0083] Actions 0 to N-1: Each action corresponds to a uniquely identified server room. Selecting this action means that the current computing power request will be scheduled to this server room for processing.

[0084] Action N: This action indicates that the current computing power request will be offloaded to the cloud for processing.

[0085] This design ensures that the scheduling strategy can cover all possible resource allocation objectives.

[0086] Environmental interaction and reward function: After the agent performs an action, the environment will provide a reward signal, which is the key to guiding the agent to learn the optimal strategy.

[0087] Reward function: After the agent performs an action, the environment will provide multi-objective reward signals. As the core criterion for guiding strategy optimization, the reward function designed in this study aims to balance the immediate quality of service (QoS) of the task with the long-term evolutionary stability of the edge resource pool, and its definition is as follows:

[0088]

[0089] Basic Rewards ( ): This reward is only issued when the task is completed within the maximum tolerable latency. To reduce the load on the core network and incentivize edge processing, the system assigns a higher reward weight to local edge offloading, reflecting the scheduling orientation of "edge priority, cloud compensation".

[0090] Composite delay penalty ( ):

[0091] It consists of a global latency mitigation item and a Service Level Agreement (SLA) penalty item for breach of contract.

[0092] The former through Linear penalties reduce the overall system response time; the latter imposes heavy penalties on behaviors that exceed latency constraints, rigidly guaranteeing the achievement of the Service Level Agreement (SLA). In the formula... Indicates the latency required to execute the scheduling policy. This indicates the maximum tolerable delay for the request. and The value is determined based on actual business needs, where The value should be greater than .

[0093] Adaptive long-term resource utilization reward ( ):

[0094] To improve the robustness of the strategy under fluctuating flow (such as tidal flow), we introduce a method based on request arrival rate. Dynamically adjusted load-aware adaptive threshold To balance resource availability and system throughput.

[0095] in This represents the baseline optimal utilization rate under low load conditions. It is the traffic scaling sensitivity factor. It is the maximum allowed increment. This indicates the "warning threshold". Specific parameter settings are determined based on business requirements and simulation experiments, and can be adjusted according to actual needs.

[0096] The reward system employs a piecewise linear function design: it incentivizes increased throughput when utilization falls below the optimal threshold; the gain decreases as a warning upon entering the warning zone; and it switches to a penalty once the saturation threshold is exceeded. This mechanism smooths the incentive signal, prevents the agent from falling into momentarily efficient local optima, and optimizes resource distribution from a long-term perspective. and The value is determined based on actual business needs.

[0097] Instantaneous overload progressive penalty ( ):

[0098] Fine-grained constraints are implemented on resource utilization for specific nodes. By setting a stepped penalty gradient, the observation cost is significantly increased when nodes approach saturation. This design aims to suppress local "utilization explosion," prevent tasks from clustering around near-saturated nodes, and ensure high system availability under high concurrency demands. and The value is determined based on actual business needs, where The value should be greater than .

[0099] in( The value of ) is determined by the simulation pre-experiment, and the other parameters can be adjusted according to actual business needs.

[0100] Q-score prediction and action selection: Q-value prediction: such as Figure 3 As shown, the fused state feature vector output by the LSTM / Transformer module is fed into a fully connected neural network, which maps the high-dimensional features to the action space. Its output is a vector of dimension (N+1), where each value is the Q-value of the corresponding action. The Q-value represents the expected cumulative discounted reward that can be obtained by choosing the action given the current spatiotemporal network state.

[0101] Action selection: During the decision-making phase, the scheduling system selects the action corresponding to the maximum value in the Q-value vector as the final scheduling decision.

[0102] Network training mechanism: refer to Figure 5 DDQN model training process; To ensure stable convergence during the deep reinforcement learning process, this paper adopts the Double DQN (DDQN) training architecture and introduces a soft update mechanism. The specific training process is as follows:

[0103] Dual-network topology: Two deep neural networks with identical parameter structures are constructed, namely a policy network and an object network. The policy network is responsible for generating action policies in real time and updating parameters through the Adam optimizer, with an initial learning rate set to 1e. -4 This is coupled with the ReduceLROnPlateau learning rate scheduling strategy: if the loss function does not decrease within 50 consecutive training steps, the learning rate is reduced to half its original value, with a minimum learning rate not lower than 1e. -6The target network provides a stable estimate of the target value during computation, mitigating overestimation during training. This set of parameters represents the experimentally validated base configuration. In practical deployments, more refined scheduling strategies such as learning rate warmup or cosine annealing can be employed, and the initial learning rate can be fine-tuned within the range [1e-6, 1e-4] using grid search or Bayesian optimization to further improve the final performance.

[0104] Experience data storage and decoupled sampling: Experience data sequences generated by the interaction between the agent and the simulation environment are stored. These sequences include the current environment state, the actions performed, the immediate reward received, the next state, and a marker indicating whether the task has terminated. This data is stored in a fixed-capacity experience replay pool. During training, micro-batch data is obtained from the pool through random sampling to eliminate temporal correlations between training samples and ensure the independent and identically distributed characteristics of the data. During training, the experience replay pool has a fixed capacity of 1000 complete experience sequences, each containing state-action-reward transition data for W consecutive time steps. Eight sequences are randomly sampled as a batch during training, and batch processing and parallel computation are used to improve training efficiency. To stabilize the training process, the gradient of the policy network is pruned, limiting its L2 norm to no more than 1.0.

[0105] Training update (per step): Within each training step, the following processing logic is executed sequentially for each sampled micro-batch of data: Target value calculation: First, the policy network selects the optimal action with the highest estimated value based on the state at the next time step; then, the target network evaluates the value of this specific action and calculates the target estimate of the dual-depth Q-network by combining the immediate reward and a preset discount factor. The discount factor γ is set to 0.95 to balance the weights of immediate reward and long-term return.

[0106] Policy network update: Calculate the actual output value of the policy network for the current state-action pair and measure the mean squared error loss between it and the aforementioned target estimate. The optimizer then adjusts the parameters of the policy network in real time based on this deviation gradient.

[0107] Exploration and Exploitation Balancing Mechanism: An ε-greedy strategy is employed for action selection, with the exploration rate ε decaying linearly during training. The initial exploration rate is set to 1.0 (completely random exploration), and it decays linearly to 0.01 (basic exploitation) within 300,000 training steps. The decay function is as follows:

[0108] This design ensures that the state space is fully explored in the early stages of training, and that the learned strategies are stably utilized in the later stages.

[0109] Target network smooth soft update: Two target network update mechanisms are provided, which can be switched via parameters: Soft update (default): Continuously fine-tunes the target network. At each training step, the current weight parameters of the policy network are weighted and fused with the existing weight parameters of the target network by a smoothing coefficient τ=0.01. This smooth transition of parameters ensures the continuity and stability of the target value during the training process.

[0110] Hard update (optional): Every 500 training steps, the policy network parameters are completely copied to the target network. .

[0111] Both mechanisms can ensure the stability of the target value estimation and mitigate training oscillations.

[0112] Training stability assurance and convergence monitoring: In the current training framework, several key technologies have been implemented to ensure the stable convergence of the STAR-GQN deep reinforcement learning model: First, gradient pruning (limiting the L2 norm of the gradient to no more than 1.0) prevents gradient explosion during backpropagation; second, a complete training state snapshot and rollback mechanism has been established, regularly saving all information including network parameters, optimizer state, experience replay pool, and random number state, ensuring that the model can quickly recover to a stable checkpoint when training fluctuations occur, thus guaranteeing the reproducibility of the experiment and the robustness of long-term training; finally, the training process is monitored in real time by tracking the round reward curve, Q-network loss curve, and the decay of the exploration rate ε to determine the model's convergence status. To further improve sample efficiency, exploration capabilities, and final policy performance, the framework has good scalability. It can integrate priority experience replay to make more efficient use of historical data, introduce noisy networks to achieve more intelligent state-related exploration, and adopt multi-step rewards to accelerate credit allocation. It can also upgrade the basic DDQN algorithm to more advanced algorithm frameworks such as Proximal Policy Optimization (PPO) or Soft Actor-Critic (SAC) to deal with more complex decision-making scenarios.

[0113] Output: A specific scheduling instruction, which is an action selected from the action space, indicating which target data center or cloud to schedule the current computing power request to.

[0114] 5. Step S5: Schedule execution.

[0115] Execution entity: the executor of the scheduling center.

[0116] Scenario: After receiving the scheduling action output by the DDQN policy network.

[0117] Specific tasks include: executing scheduling decisions and calculating relevant metrics. This includes:

[0118] Resource allocation: Send instructions to the target data center (or cloud platform) to allocate computing resources to process the task.

[0119] Latency calculation: Estimate the total latency from the time the task is sent to the time it is completed, including transmission latency and execution latency.

[0120] Resource Update: Update the resource status of the target computer room in the environment simulation layer according to the allocation results.

[0121] The actuator communicates with the target computing node through control signaling to complete resource reservation and task assignment, and estimates performance based on the network model and computing model.

[0122] To verify the effectiveness of the proposed algorithm, the simulation environment design used for training and testing is detailed below. The network infrastructure considers a three-layer computing network architecture comprising the cloud, edge data centers, and base stations. Figure 6 For example, this architecture is designed to provide hierarchical computing services for geographically dispersed vehicles.

[0123] Model training and testing were conducted within a normalized 10km x 10km urban area, utilizing one remote cloud center, 22 local edge computing rooms, and 100 base stations. The overall scheduling logic is as follows: when a new computing power request occurs in the vehicle-road network... When the request occurs, the base station closest to the location where the request occurred will be the first to respond. Receive request information The base station then forwards the request information to its server room. This data center is connected to all other server rooms via a backhaul network, allowing it to select the most suitable data center from among them. Allocate computing resources as requested. If no suitable data center can provide the required resources, allocate them to the cloud. Request computing resources.

[0124] The following section describes key details regarding request information and delay calculations in the simulation.

[0125] The computation tasks are dynamically generated based on the standard 5G-V2X service profile and are divided into 6 categories. Their typical parameters are shown in Table 1 below: Table 1: Types and Parameters of Vehicle-to-Everything (V2X) Computing Power Requests

[0126] The calculation of latency is simulated based on actual business requirements, and some calculation methods are simplified. The specific calculation process is as follows:

[0127] Connection relationships and edge delays: The nodes are connected via a wired fiber optic network, and the latency is mainly determined by the physical distance.

[0128] Vehicle to base station: wireless air interface transmission, latency It is relatively high and variable, dynamically calculated in the model based on the distance between the requested location and the nearest base station.

[0129]

[0130] From the base station to its local equipment room: connected via a fronthaul network (fiber optic cable). Base station To its respective computer room Delay The calculation method is as follows:

[0131] .

[0132] Between computer rooms: connected via a backhaul network (fiber optic cable). Any two computer rooms... and Fiber optic transmission delay The calculation method is as follows:

[0133] .

[0134] There is a relatively high latency between the data center and the cloud; it should be set to a fixed value. .

[0135] End-to-end overall delay calculation: For a request Scheduled to compute node (This could be an edge data center) or in the cloud The process, its total delay It consists of transmission delay and processing delay.

[0136] Transmission delay : Transmission delay is the total time it takes for data to travel through a network, calculated as follows:

[0137] in: Request location to nearest base station Air interface delay.

[0138] Base station To its home computer room The pre-game delay.

[0139] From the local equipment room of the nearest base station To the target data center The return network latency, if Local data center If , then this item is 0. For the cloud Then this item is .

[0140] Processing delay : Processing latency is the time required for a computing node to execute a task, and it is related to the task requirements and the node's computing power.

[0141] in For nodes Server processing power (unit: CPU units / millisecond). To simplify the model, it is usually assumed that all edge nodes have the same processing power, and the cloud's processing power is much higher than that of the edge.

[0142] Total delay: The total end-to-end delay is:

[0143] One of the necessary conditions for successful scheduling is: .

[0144] While the simulation model described above strives for realism, it still contains simplifications. For example, it uses a deterministic propagation delay model and does not simulate physical layer channel fluctuations and network protocol stack overhead. This may lead to a gap between simulation and reality. To mitigate this issue, online learning techniques can be employed during the algorithm deployment phase. This involves using data streams generated by the real system to fine-tune and adapt the agent, continuously aligning it with actual network dynamics.

[0145] Performance metrics: To objectively verify the scheduling performance of the STAR-GQN framework proposed in this invention in a dynamic 5G-V2X network environment, this embodiment uses the following multi-dimensional core indicators for performance evaluation: Average reward: Based on the defined reward function, it comprehensively measures the global benefits of task success rate, latency overhead, long-term load balancing, and instantaneous overload suppression.

[0146] Task completion rate: refers to the percentage of tasks successfully processed under the dual constraints of computing resource sufficiency and end-to-end latency.

[0147] Average end-to-end latency: The total time taken from task generation and transmission to execution feedback, which directly reflects the system's ability to guarantee latency-sensitive services.

[0148] Performance Analysis: Through systematic comparative experiments with various baseline strategies in diverse scenarios covering different traffic patterns and load intensities, this invention demonstrates the following significant technological advancements: Decision Delay Analysis: refer to Figure 7 In simulation and performance evaluation, decision latency is considered separately as a key component of end-to-end latency. The forward inference process of the STAR-GQN framework in this paper, although integrating GAT, sequence models, and DDQN, primarily occurs in the "scheduling center" of the resource cluster. By optimizing the trained model (e.g., model pruning, quantization) and deploying it on a server equipped with a GPU accelerator, the inference time for a single scheduling decision can be compressed to the millisecond level. In the simulation experiments in Chapter 5, "decision latency" is continuously monitored as an independent metric. The results show that even under high load and concurrent requests, its average value remains stable between 4-8 milliseconds, lower than the maximum tolerable latency for various V2X services (especially emergency braking services requiring less than 20ms). Therefore, the decision overhead of this algorithm will not become a bottleneck for system latency and can meet the stringent real-time scheduling requirements of the 5G-V2X environment.

[0149] Robustness of predictive scheduling paradigms to complex environments: refer to Figure 8 , 9 Experimental results show that, under different load intensities, STAR-GQN consistently outperforms heuristic strategies such as random scheduling, all-local processing, all-cloud processing, and lowest utilization priority in both compound benefit and task completion rate. Especially in... In high-load, tidal scenarios, this invention can maintain a task completion rate of over 75%, significantly higher than the baseline. This verifies that the invention's shift from a "reactive" to a "predictive" scheduling paradigm can effectively address the high dynamism of the 5G-V2X environment.

[0150] Further analysis shows that STAR-GQN (LSTM) achieves the best performance in various scenarios with a time window of W=10 and STAR-GQN (Transformer) with W=20, with higher task completion rates than other models. This observation indicates that LSTM is more suitable for modeling short- to medium-term temporal dependencies, while Transformer can capture longer-term patterns in complex load dynamics by leveraging a larger temporal receptive field. While traditional GNN-DQN outperforms simple heuristics, its scheduling decisions are based solely on instantaneous network states, giving it spatial adaptability but exhibiting short-sightedness in the temporal dimension. In dynamic environments, this can lead to short-sighted behavior, such as aggressively offloading tasks to currently underutilized nodes without considering potential short-term load increases. By introducing LSTM or Transformer modules, the proposed framework integrates temporal context and historical dependency modeling. This makes scheduling decisions more forward-looking and highlights the importance of joint spatiotemporal awareness for improving long-term cumulative rewards in dynamic networks.

[0151] Spatiotemporal joint perception's role in curbing short-sighted decision-making: Thanks to the deep integration of GNN and temporal models (LSTM / Transformer), this invention significantly outperforms the GNN-DQN model, which only has spatial awareness, in terms of long-term cumulative rewards. This mechanism endows the scheduler with the ability to remember and predict the evolution of network topology, enabling the system to proactively avoid potential resource competition risks and achieve a smoother and more balanced resource distribution than the pure spatial model, effectively suppressing the "load oscillation" phenomenon.

[0152] Scene adaptation capability of heterogeneous temporal modelers: The framework of this invention offers extremely high flexibility in component design: Local spatiotemporal correlation: Under the standard mode where traffic is relatively stable, STAR-GQN (LSTM) uses its gating mechanism to accurately capture the continuous local time delay characteristics caused by vehicle movement, exhibiting extremely high decision stability.

[0153] Long-range periodic correlation: Under the tidal pattern with macro-fluctuation regularity, STAR-GQN (Transformer) models long-range dependence through self-attention mechanism, effectively linking the traffic characteristics of the current state with those of similar historical phases (such as morning and evening rush hours), and achieving more accurate resource pre-allocation.

[0154] Rigid Quality of Service (QoS) Guarantee for Critical Business Operations: refer to Figure 10Under extreme conditions where the system approaches saturation, this invention forces the strategy to execute "predictive resource reservation" through a momentary overload penalty term in the reward function. Experiments show that even under global heavy load, the system can still maintain an extremely high success rate for ultra-low latency services such as emergency braking. This result demonstrates the unique technical value of this invention in ensuring safety-critical applications in the Internet of Vehicles, achieving a deep integration of resource efficiency and system reliability.

[0155] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This invention is described with reference to flowchart illustrations and / or block diagrams of control methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the control methods, apparatus, and computer program products according to embodiments of the invention. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0157] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0158] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0159] The background section of this invention may include background information about the problems or environment in which the invention is being developed, and is not necessarily a description of prior art. Therefore, the content included in the background section does not constitute an admission of prior art by the applicant.

[0160] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for dynamic scheduling of computing resources based on spatiotemporal joint sensing, characterized in that, include: Real-time reception of computing power requests, wherein the computing power requests include at least the geographical location information of the requesting node; Based on the computing power request and real-time network data, construct the current computing power network topology state and a historical state sequence containing the most recent W time steps, where W is a positive integer greater than 1; The current computing network topology state and the historical state sequence are input into the spatiotemporal feature extraction model to obtain a fused state feature vector; The spatiotemporal feature extraction model is processed through nested fusion, which includes: using a graph neural network (GNN) to encode the spatial features of the computing network topology at the current moment and the computing network topology at each time step in the historical state sequence, respectively, to obtain spatial feature vectors for the corresponding time steps; inputting the obtained series of spatial feature vectors into the time series model, and having the time series model output the fused state feature vector representing the spatiotemporal dynamics of the system; The fused state feature vector is input into the reinforcement learning agent, which then outputs a scheduling action in response to the computing power request.

2. The dynamic scheduling method for computing resources according to claim 1, characterized in that, The computing power request also includes the required computing resource size and the maximum tolerable latency.

3. The method according to claim 1, characterized in that, The computing power network topology is represented by a graph structure, in which: the nodes of the graph correspond to computing nodes in the computing power network; the feature vector of each node in the graph is defined based on the attributes of its corresponding computing node, wherein the attributes of the computing node include: the geographical coordinates of the computing node, the current proportion of remaining computing power resources, and the node type code; the edges of the graph correspond to the computing power network connections between computing nodes, each edge represents a network link, and the feature vector of each edge includes the transmission delay of the network link.

4. The dynamic scheduling method for computing resources according to claim 1, characterized in that, The graph neural network (GNN) includes the graph attention network (GAT).

5. The dynamic scheduling method for computing resources according to claim 1, characterized in that, The time series model includes Long Short-Term Memory (LSTM) networks or Transformer models.

6. The dynamic scheduling method for computing resources according to claim 1, characterized in that, The reinforcement learning agent includes a deep reinforcement learning (DRL) agent; the deep reinforcement learning (DRL) agent includes a dual deep Q network (DDQN).

7. The method for dynamic scheduling of computing resources according to claim 1, characterized in that, The reinforcement learning agent is trained by maximizing cumulative rewards, and its reward function includes at least two of the following: basic reward, compound delay penalty, adaptive long-term resource utilization reward, and transient overload progressive penalty.

8. The method for dynamic scheduling of computing resources according to claim 1, characterized in that, The scheduling action indicates that the current computing power request will be allocated to the target computing node; the dynamic scheduling method for computing power resources further includes: executing the scheduling action, completing resource allocation and updating the system status.

9. A dynamic scheduling system for computing resources, characterized in that, include: The request receiving module is used to receive computing power requests in real time, wherein the computing power request includes at least the geographical location information of the requesting node; The state construction module is used to construct the current computing power network topology state and a historical state sequence containing the most recent W time steps based on the computing power request and real-time network data, where W is a positive integer greater than 1. The feature extraction module is used to input the current computing network topology state and the historical state sequence into the spatiotemporal feature extraction model to obtain a fused state feature vector. The spatiotemporal feature extraction model processes the data through nested fusion, which includes: using a graph neural network (GNN) to encode the spatial features of the current computing network topology state and the computing network topology state at each time step in the historical state sequence, respectively, to obtain spatial feature vectors for the corresponding time steps; inputting the obtained series of spatial feature vectors into a time series model, which then outputs the fused state feature vector representing the spatiotemporal dynamics of the system. The intelligent decision-making module is used to input the fused state feature vector into the reinforcement learning agent, and the reinforcement learning agent outputs a scheduling action in response to the computing power request.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the dynamic scheduling method for computing resources as described in any one of claims 1-8.

Images