Internet of vehicles resource slicing method fusing ST-GNN and multiple agents

Abstract

The invention provides an Internet of Vehicles resource slicing method fusing ST-GNN and multiple agents, and the method comprises the steps: a hierarchical resource slicing frame with a shared resource pool as the center is used for integrating the resource distribution problems of a base station and an MEC controller; all areas of the Internet of Vehicles take ST-GNN as a base, and a task traffic prediction result is optimized in combination with a Transform encoder. The MADRL algorithm is used for coordinating resource decision behaviors of base stations and other different role nodes in a region, the Actor network is deployed on the base stations, and the Critic network is deployed on an MEC controller so as to realize centralized training and decentralized decision. And finally, the ST-GNN and MADRL algorithms are optimized in an intra-region and cross-region combined fine tuning mode, and efficient management and optimization of resource slices are realized. Simulation results show that the method is superior to an existing method in the aspects of slice isolation performance, decision-making efficiency and resource utilization rate.

Description

Technical Field

[0001] The present invention relates to the application of artificial intelligence technology in the Internet of Things, and specifically to a vehicle networking resource slicing method that integrates ST-GNN and multi-agent Background Art

[0002] With the continuous improvement of traffic safety and intelligent requirements, scholars have proposed to connect vehicles as network communication nodes to the network, and vehicles, pedestrians, and base stations communicate with each other to form a vehicle networking system. The communication resources in vehicle networking are scarce, and the rapid growth of the number of vehicles and diverse application requirements (such as high-definition map updates, real-time video transmission, autonomous driving decisions) have led to increased resource competition. Different applications have significantly different requirements for network performance and greatly different resource demands. In order to ensure and meet the QoS of various services in vehicle networking, the network needs to have higher bandwidth, lower latency, and greater capacity. At this time, the 5G network with flexible orchestration, mutual isolation, and extremely high performance becomes the first choice in the vehicle networking scenario.

[0003] Network slicing [1] can divide the physical network into multiple virtual networks (i.e., slices). Each slice can customize the resource allocation strategy according to the needs of specific applications to meet the differentiated QoS requirements. Independently allocate communication, computing, and storage resources according to the needs of different application scenarios. The MEC controller provides flexible and refined QoS guarantees for various applications. There are essential differences between vehicle networking resource slicing and the mobile resource slicing of traditional user terminals (such as smart phones). Traditional mobile resource slicing is oriented towards personal user terminals, and the resource demand is centered on bandwidth; while vehicle networking resource slicing is oriented towards vehicle and base station clusters, and the resource demand is centered on multi-objective constraints such as latency, reliability, bandwidth, and computing collaboration.

[0004] The characteristics of vehicle network resource slicing make the traditional slicing technologies based on static policies or single-domain optimization inapplicable directly. Designing new resource slicing architectures and algorithms has become the core requirement at present to achieve dynamic topology adaptation, cross-domain collaborative scheduling, and service difference guarantee. The high-speed mobility of vehicles in the vehicle network causes frequent changes in the network topology, and the traditional static slicing strategy fails because it cannot perceive the link state and vehicle position in real time. Ways such as prediction-driven resource pre-allocation, distributed slice migration mechanism, and lightweight slice protocol can achieve dynamic adaptation. For example, based on vehicle motion trajectory prediction (such as Kalman filter [2] or Long Short-Term Memory (LSTM) network [3]), slice resources (such as resource blocks, computing instances) are pre-allocated within the coverage area of the base station that the vehicle is about to enter; slice context information (such as session status, QoS parameters) is transmitted through vehicle-to-vehicle (V2V) direct communication, reducing the dependence on the core network and thus reducing the migration delay; a vehicle-network digital twin is constructed to simulate the slice migration process in real time and verify the feasibility, ensuring the synchronization of physical layer handover and logical layer resource reconstruction; a dedicated slice signaling protocol (such as an extension based on QUIC [4]) is designed to reduce the signaling overhead during the migration process and support millisecond-level slice reconstruction.

[0005] In large-scale vehicle network scenarios, resource allocation decisions rely on accurate traffic prediction. Considering the huge differences in traffic flow in different regions in terms of time and space, how to improve the accuracy of cross-regional traffic prediction has become a thorny problem. In the wide-area scope, the traffic prediction model based on graph neural network seems to be a preferred solution to solve this problem. Multiple base stations in each region serve as nodes in the graph structure, which can not only provide traffic monitoring but also act as local controllers for resource slicing decisions. Considering that due to factors such as road structure and vehicle density, the distribution of vehicle task requests is uneven in the blocks under the coverage of different base stations, the region needs to introduce an MEC controller for overall scheduling to reasonably allocate the resources of multiple base stations in the region. In the wide-area scope, the regional resource slicing model based on multi-level agents seems to be able to effectively solve this problem. Multiple base stations in the region act as agents, and the MEC controller acts as the overall controller, which can not only share the base station decision information but also schedule the base station resources. When the resources of a certain base station are insufficient to cope with the sudden traffic, the MEC controller can introduce this part of the traffic into the shared resource pool

[0006] Although the introduction of methods such as graph neural network models and multi-level agents provides a feasible way to solve resource slicing in large-scale vehicle network scenarios, considering the particularity of multi-base station resource scheduling, how to achieve the efficient collaboration of prediction models and decision models still faces the following challenges:

[0007] 1) Modeling of spatio-temporal characteristics of large-scale vehicle network traffic. Vehicle network traffic exhibits significant spatio-temporal heterogeneity (such as morning and evening rush hours, sudden accidents), and traditional methods are difficult to capture non-Euclidean spatial relationships and long-term dependencies. Existing models based on RNN or CNN cannot effectively model the graph structure characteristics of vehicle task traffic. Zhu et al. [5] proposed a graph convolutional neural network for traffic prediction, using graph convolution to model urban road networks and capturing the topological structure of the graph with Graph Convolutional Networks (GCN) to handle spatio-temporal traffic prediction tasks. Feng et al. [6] proposed a novel deep learning-based multi-component spatio-temporal graph convolutional network to solve the traffic flow prediction problem. Although the above methods have good traffic prediction effects in a single region, they cannot well capture the spatio-temporal correlations of data when facing the complex environment of multi-region vehicle networks.

[0008] 2) Multi-level resource collaboration mechanism. Traditional methods rely on fixed rules to allocate resources [7], making it difficult to cope with the local resource supply-demand imbalance caused by vehicle mobility. If the efficient integration of centralized resources and local resources cannot be achieved, it will lead to resource fragmentation and response delays. Wang et al. [8] proposed a network slicing resource management algorithm based on deep reinforcement learning, which uses two long short-term memory networks and the advantage actor-critic algorithm for broadband allocation decisions. Lin et al. [9] proposed a federated slicing resource management scheme based on generative adversarial network-assisted multi-agent reinforcement learning, using the multi-agent competitive DDQN method and the federated learning framework to achieve collaborative decision-making for slicing resource management. However, none of the above methods proposed a hierarchical resource collaboration mechanism, which may lead to local resource supply-demand imbalances.

[0009] 3) Efficient collaboration between the regional decision-making model and the global prediction model. In a large-scale vehicle networking scenario, independent training of models in each region may lead to policy fragmentation and resource isolation, and there are significant differences in vehicle flow resources between regions. The effectiveness of the decision-making model within a region depends on the accuracy of the global prediction model. Zhang et al.

[10] proposed a prediction-based 5G network slicing algorithm that pre-isolates virtual node resources and virtual link resources in the 5G network according to the prediction results. When a network slicing request arrives, containers are directly launched to complete the dynamic creation of network slices. Peng et al.

[11] proposed an intelligent collaborative management technology for virtual resources based on teleoperation in vehicle networking, introduced mobile edge computing technology at the terminal, conducted virtual resource collaborative scheduling and intelligent fault prediction technology analysis on the network side, and analyzed intelligent vehicle integrated control services and technologies related to the SDN cloud control center on the platform side to achieve intelligent applications for teleoperated vehicles. However, none of the above methods reflect the collaborative optimization of the prediction and decision-making parts, and the overall effect is still lacking. Summary of the Invention

[0010] To address the above challenges, the present invention proposes a vehicle networking resource slicing framework that integrates a Spatial-Temporal Graph Neural Network (ST-GNN) and Multi-Agent Reinforcement Learning (MARL). This framework adopts a collaborative paradigm of pre-training and alternating joint fine-tuning to optimize the slicing isolation effect.

[0011] A vehicle networking resource slicing method integrating the ST-GNN and multi-agents of the present invention, the vehicle networking resource slicing method is applied to a hierarchical resource slicing framework. In this framework, a centralized controller is deployed in the cloud, and a local controller is deployed on the edge side; a Mobile Edge Computing (MEC) controller serves as the local controller; for an MEC controller in the same region of the vehicle networking and multiple base stations under its jurisdiction:

[0012] The physical resource blocks of the MEC controller are virtualized into a resource pool indirectly shared by the base stations within the region under its jurisdiction; the resources on each base station are virtualized into multiple slicing regions, and each slice supports a customized service type; the task traffic (task request volume) data received by the base station is divided into different types of task traffic and scheduled to the corresponding slices for processing according to the type; when the resources of a certain slice on the base station are insufficient, the arriving tasks are scheduled to the resource blocks allocated to the base station from the shared resource pool;

[0013] The vehicle networking resource slicing method is to obtain the spatio-temporal characteristics of task traffic through a global task traffic prediction model, and use this as the input of the local slicing window prediction model (which can adopt a slicing window optimizer), so as to obtain the corresponding slicing window length result;

[0014] The output results of the global task traffic prediction model and the local slicing window prediction model are input into the resource slicing decision model to obtain the optimal resource slicing decision;

[0015] The global task traffic prediction model is a task traffic prediction model based on the spatio-temporal graph neural network ST-GNN, and it is a global model; in ST-GNN, the nodes of the graph represent the base station location characteristics, vehicle flow characteristics, and task type characteristics, and the edges of the graph represent the base station physical distance characteristics and communication link quality characteristics;

[0016] The resource slicing decision model is a resource slicing decision model based on multi-agent deep reinforcement learning MADRL, and it is a regional model;

[0017] Each base station deploys a slicing window prediction model respectively;

[0018] In the resource slicing decision model, the MEC controller integrates the Critic network; each base station is regarded as an agent, has an independent Actor network and shares the Critic network integrated by the MEC controller;

[0019] The base station agent observes the local state and outputs actions accordingly; the spatio-temporal characteristics of the task traffic input by the agent interact with the environment based on the actions generated by the Actor network, responds to task requests in real time, and outputs the action decision for resource slicing allocation;

[0020] The MEC controller is used for global coordination and local decision-making; the MEC controller collects the real-time states of all base station agents, runs the Critic network, evaluates the Q value of the joint actions of the base station agents, generates cross-base station cooperation signals, and reallocates resources for each base station at the beginning of each slicing window or when encountering burst task traffic.

[0021] The main contributions of the present invention are reflected in the following three aspects:

[0022] 1) Pre-training of the road network traffic model based on ST-GNN and Transformer.

[0023] This framework models the road network topology through ST-GNN, converts spatio-temporal data such as base station locations and vehicle flows into graph signals, and introduces the Transformer encoder into the construction of the traffic prediction model, enhancing the spatio-temporal correlation of the prediction model for road network traffic in multiple regions and improving the prediction accuracy.

[0024] 2) Pre-training of the regional resource slicing model based on multi-agent

[0025] This slicing model adopts a hierarchical attention design. The base station, as a distributed agent, uses ST-GNN to aggregate vehicle node information, and the global Transformer captures the resource competition relationship between base stations. Combined with the global perspective of the centralized MEC controller, it coordinates resource scheduling.

[0026] 3) Joint fine-tuning of the regional decision-making model and the global prediction model

[0027] In this framework, the regional controller coordinates the base station multi-agents, integrates the global resource pool and local resources, and realizes the efficient collaboration of "centralized optimization - distributed execution"; the cloud aggregates the Critic network parameters from each regional MEC controller, and ensures the global policy consistency through the federated learning mechanism. This framework adopts an alternating collaborative strategy to alleviate gradient conflicts and accelerate the convergence speed. Brief Description of the Drawings

[0028] Figure 1 Represents the hierarchical resource slicing framework (taking 1 base station as an example);

[0029] Figure 2 Represents the joint fine-tuning framework;

[0030] Figure 3(a) and Figure 3(b) respectively represent the reinforcement learning reward convergence analysis curve and the probability density analysis curve;

[0031] Figures 4(a) to 4(c) Respectively represent the influence curves of the resource pressure of 3 base stations on performance isolation;

[0032] Figures 5(a) to 5(c) Respectively represent the influence of the pressure on the shared resource pool of 3 base stations on performance isolation. Detailed Implementation Manner

[0033] The present invention will be further described below in conjunction with the drawings and the detailed implementation manner.

[0034] 1. Overview

[0035] The high-speed mobility of vehicles makes the network topology and resource requirements continuously change dynamically, and the traffic flows between adjacent road network regions show obvious spatio-temporal heterogeneity, which exacerbates the difficulty of dynamic scheduling of slice resources. In response to the above challenges, the present invention proposes a vehicle network resource slicing method that integrates the spatio-temporal graph neural network ST-GNN and multi-agent.

[0036] First, a hierarchical resource slicing framework centered on the shared resource pool is constructed to integrate the resource allocation problems of base stations and MEC controllers.

[0037] Secondly, a joint fine-tuning mechanism of multi-agent deep reinforcement learning (MADRL) based on spatio-temporal graph neural network (ST-GNN) prediction is constructed. This framework is divided into two parts: intra-region optimization and cloud cross-region optimization. All regions are based on ST-GNN and combined with a Transformer encoder to optimize the prediction results. The MADRL algorithm is used to coordinate the resource decision-making behaviors of different role nodes such as base stations within the region. Among them, the Actor network is deployed on the base station, and the Critic network is deployed on the MEC controller to achieve centralized training and decentralized decision-making.

[0038] Finally, the ST-GNN and MADRL algorithms are optimized through intra-region and cross-region joint fine-tuning to achieve efficient management and optimization of resource slicing.

[0039] The simulation results show that the present invention is superior to the existing methods in terms of slice isolation performance, decision-making efficiency, and resource utilization rate.

[0040] 2. System Model and Problem Modeling

[0041] The present invention adopts a global prediction model based on ST-GNN and a cross-region resource slicing model based on the MADRL algorithm, and improves the resource slicing decision-making performance through alternating joint fine-tuning of the global and regional models.

[0042] The meanings of the parameters used in the present invention are shown in Table 1.

[0043] Table 1 Definitions of Symbols and Variables

[0044]

[0045] 2.1 Hierarchical Resource Slicing Framework

[0046] The present invention designs a hierarchical resource slicing framework to provide support for large-scale vehicle networking services through the joint management of MEC server shared resources and base station resources. The resources of the MEC controller are virtualized into a resource pool that can be indirectly shared by base stations within the jurisdiction area. The resources on each base station are virtualized into multiple slice regions, and each slice supports a class of customized services. The vehicle traffic data received by the base station is divided into different task traffic, and according to the task type, the arriving tasks are scheduled to the corresponding slices for processing. When the resources of a certain slice on the base station are insufficient, the arriving tasks do not occupy the unused resources of other slices on the base station, but are scheduled to the resource blocks allocated to the base station from the shared resource pool.

[0047] Take Figure 1 as an example to illustrate the slice window and resource allocation strategy of the present invention.

[0048] Assume that time is divided into multiple consecutive and fixed slice windows, and each slice window contains multiple discrete time slots. The set and cardinality of base station n are denoted as and N respectively, and the set and cardinality of the corresponding serving slices are denoted as and M n . The set and cardinality of the slice windows are denoted as and H. On base station n, the set and cardinality of the time slots contained in slice window are denoted as and At the end of slice window h - 1, the local controller of base station n re - determines M n .

[0049] 2.2 Slice Performance Isolation Metrics

[0050] Performance isolation is a prerequisite for slice resource co - existence, that is, to ensure that the overload of one slice does not affect other slices. The present invention introduces a soft performance isolation metric for a hierarchical resource slice framework. Let the total number of RBs held by base station n be B n . The total number of RBs held by the shared resource pool is denoted as B'. In slice window h, the number of RBs allocated to slice m of base station n is denoted as satisfying

[0051]

[0052] Assume that the number of RBs reserved for base station n by the MEC controller in the shared resource pool is satisfying

[0053]

[0054] At the end of slice window h - 1, the MEC controller re - partitions the number of RBs of base station n on the shared resource pool If the actual resource demand of slice m of base station n in time slot is The number of RBs occupied by all slices on base station n in the shared resource pool is calculated as

[0055]

[0056] Within window h, the number of RBs held by each slice and the shared resource pool remains unchanged.

[0057] The performance isolation quality of base station n in time slot is normalized to

[0058]

[0059] At When the slice performance isolation violates the rules (i.e., the shared resources actually occupied by base station n exceed the allocated resources), ρ t = 0 at this time. In When the shared resources actually occupied by base station n are less than the allocated resources (i.e., the shared resource pool is temporarily occupied), ρ n,t ∈ [0, 1), which means that the performance isolation at this time is "discounted"; when , base station n can meet the current demand and does not need to occupy the shared resource pool, and ρ n,t = 1 at this time.

[0060] 2.3 Problem Modeling

[0061] The optimization problem of the present invention not only involves the division of local slice resources but also includes the resource allocation of the shared resource pool. For base station n, under the slice window h (given ), the problem of maximizing the slice performance isolation quality is modeled as

[0062]

[0063] By extending to multiple consecutive slice windows, the long-term optimization problem of the performance isolation quality regarding base station n is modeled as

[0064]

[0065] The essence of problem is how to adjust the occupation ratio of slices at the base station and allocate the resources occupied by the base station in the shared resource pool to maximize the long-term performance isolation quality.

[0066] Section 3 explores how to perform joint fine-tuning through ST-GNN traffic prediction and MADRL to optimize the decision-making results.

[0067] 3. Solution Design

[0068] 3.1 Pre-training of the Global Task Traffic Prediction Model

[0069] To capture the spatio-temporal correlation of adjacent road network regions, this subsection combines the Spatiotemporal Heterogeneous Graph (SHG) and the graph neural network to characterize features such as the base station location information and traffic flow density. The two are combined to construct the ST-GNN prediction model.

[0070] The task requests generated by vehicles usually have randomness and independence and conform to the Poisson process. Assume that within a unit time, the number of task requests generated by vehicles in the coverage area of base station n is y n , and the task request intensity is λ n。y n Follows a Poisson distribution, denoted as y n ~Poisson(λ n )。Assume that within a unit time, the traffic volume in the coverage area of base station n is v n , and the average task request rate of each vehicle is ρ. The task request intensity λ n satisfies

[0071] λ n = ρ·v n (3-1)

[0072] Let represent the graph representation of the base station, where the nodes V represent characteristics such as the base station location, traffic volume, and task type, and the edges E represent characteristics such as the physical distance between base stations and the communication link quality. The spatio-temporal embedding extracted by base station n through ST-GNN is denoted as containing the traffic state and resource load in this area. The physical distance and communication link quality between base stations n and n′ are respectively denoted as

[0073] and To enhance the spatial position perception ability, the learnable position encoding of base station n is denoted as

[0074]

[0075] Given g n and PE n , the spatio-temporal features of base station n are concatenated into

[0076]

[0077] The result of Equation (3-3) is input into the multi-layer Transformer Encoder. After being processed by the encoder, the spatio-temporal features are enhanced to

[0078]

[0079] The result of Equation (3-4) and g n are input into the residual connection function to obtain the output of base station n, denoted as

[0080]

[0081] Let the predicted task request volume of base station n be which is used as the input of MADRL. By constructing a fully connected network, the task request volume in the future time period is predicted as

[0082]

[0083] Concl and yn , the loss function of ST-GNN is constructed as

[0084]

[0085] The chain rule is used to calculate the loss versus the model parameters θ G The gradient of

[0086]

[0087] Its ST-GNN trainable parameters θ G Updated to

[0088]

[0089] The adjustment of the slice window length can be completed by using the slice window optimizer. The slice window optimizer of each base station can adjust the window length according to the spatiotemporal characteristics of the corresponding task traffic. The slice window length is expressed as a function of traffic, and the relationship between the two is fitted as

[0090]

[0091] The premise for constructing the best fit is to find c1 and c2 that minimize the residual sum of squares (c1 determines how fast the window length changes with task traffic fluctuations; c2 represents the base value of the window length when the task traffic fluctuation is zero or very small). The slice window length gradually decreases as the task traffic fluctuation increases. The more drastic the task traffic fluctuation, the smaller the slice window length and the faster the resource update frequency. These characteristics are consistent with expectations. At the beginning of window h, the fitting function is used to predict the length of the next window h+1 (i.e.: ). Due to the differences in base station task traffic, it is difficult to use a unified fitting function to determine the window length of each base station. In order to achieve personalized window adjustment, the present invention sets a window optimizer for each base station to cope with the fluctuations in task traffic of different base stations. After receiving the task prediction result x transmitted by the prediction model, each optimizer substitutes it into the above formula to obtain the corresponding slice window length result

[0092] 3.2 Regional Self-Consistent Multi-Agent Training Scheduling Model Pre-training

[0093] Centralized training and decentralized decision-making

[12] is a way of MARL (Multi-Agent Reinforcement Learning). The training of all agents is centrally scheduled by the central controller MEC. In the training phase, agents can share the global traffic state information or the actions of other agents. This sharing can accelerate the training process and enable agents to learn more effective strategies. In the execution phase, each agent is decentralized and does not rely on the central controller. This decentralization can improve the robustness and scalability of the system. In the considered scenario, this approach has better effects compared to the other two architectures. MADDPG (Multi-Agent Deep Deterministic Policy Gradient), as a typical network of this approach, is applied to the MADRL framework of the present invention. This MADRL framework adopts a centralized Critic and a decentralized Actor.

[0094] For an MEC controller, each base station under its jurisdiction is regarded as an agent, which has an independent Actor network and shares the Critic network integrated on this MEC controller. This MEC controller and the relevant base stations act as different roles and undertake the responsibilities of global coordination and local decision-making. The MEC controller collects the real-time states of all base stations, runs the global Critic network, evaluates the Q value of the joint action, generates cross-base station cooperation signals, and reallocates resources for each base station at the beginning of each slice window or when encountering burst traffic. The base station agent, as a local decision-maker, observes the local state and outputs an action accordingly. The agent inputs the spatio-temporal features extracted by ST-GNN, interacts with the environment based on the action generated by the Actor network, responds to task requests in real time, and outputs the action decision of resource slice allocation.

[0095] Suppose the set and cardinality of the MEC controller i are denoted as and I respectively, and the set and cardinality of the base station n in the jurisdiction area i are denoted as and N respectively i . The state received by and the action output by the Actor network deployed on the base station n are denoted as s n and a n respectively. The local state is input into the Actor network. The Critic network inputs the joint state s = (s1, s2,..., s n ) and the joint action a = (a1, a2,..., a n ) of all agents, and outputs the full

[0096] The local Q-value Q(s,a). Combining with the Transformer multi-head attention mechanism, within the coverage of area i, the local features of N i base stations are concatenated into a matrix To enhance spatial location perception, the learnable position encoding of base station n within area i is mapped to a learnable variable:

[0097]

[0098] The local features and position encoding of base station n are input into the multi-layer Transformer Encoder, and the enhanced features are represented as

[0099]

[0100] Based on the multi-head attention mechanism, the global features are represented as

[0101]

[0102] where is a learnable parameter. Taking the mean of along the base station dimension, the global state representation is represented as

[0103]

[0104] The local features g of each base station n n are respectively concatenated with the global feature g g , and the result is input into the Critic network to obtain the following global Q-value

[0105]

[0106] The Q-value of the Critic is used to guide the policy update to achieve policy improvement, and the Actor network is updated to

[0107]

[0108] To avoid overestimating the Q-value due to too rapid policy changes, the target Actor network and the target Critic network use a soft update strategy to update the parameters

[0109] θ'←τθ+(1-τ)θ'(τ<<1) (3-16)

[0110] The reward for base station n is weighted by local and collaborative rewards. The former is denoted as to encourage efficient completion of local tasks, while the latter is denoted as r g , which is calculated by the MEC controller. To punish resource competition conflicts and encourage base station cooperation, the total reward of base station n is represented as

[0111]

[0112] Among them, α n,1 and α n,2 are the local reward weight and the collaborative reward weight respectively. If the resource competition is severe, the MEC controller increases α n,2 to encourage the cooperation of each base station; if it is necessary to prioritize the local tasks of base station n, then increase α n,1 .

[0113] 3.3 Alternating fine-tuning of the global model and the regional model

[0114] Figure 2 A region-centered joint fine-tuning framework is given. In the vehicle network resource slicing scenario, there is a two-way dependence between spatio-temporal feature extraction and resource allocation decision-making. The decision made by MADRL depends on the spatio-temporal features extracted by ST-GNN. In traditional staged training, since the prediction and decision models are decoupled, the feature space of ST-GNN and the decision of MADRL cannot be guaranteed to be aligned. The proposed joint fine-tuning framework is based on ST-GNN and uses Transformer Encoder to enhance the output features. Multiple Actor networks in MADRL receive the features and integrate the state inputs by the shared Critic network of the MEC controller.

[0115] In the process of model collaboration between the global and regional levels, there is a deeply coupled two-way dependence between spatio-temporal feature extraction and resource decision allocation. Traditional staged training models are difficult to meet the dynamic collaboration requirements. Staged training will lead to the separation of the feature space and the policy space, while joint fine-tuning can achieve the co-evolution of the two through end-to-end gradient backpropagation; there must be different degrees of distribution drift in the pre-trained ST-GNN and MARL models, while joint fine-tuning continuously adapts to environmental changes through an online learning mechanism.

[0116] To address the above problems, the present invention develops an alternating joint fine-tuning strategy to achieve parameter alignment between the prediction model and the decision model. This strategy adopts the design principle of "centralized training and distributed execution", covering the optimization of both the internal area and the cross-regional cloud level:

[0117] 1) Joint fine-tuning of ST-GNN and the Actor network on base station n. The goal of this part is to make the spatio-temporal features extracted by ST-GNN adapt to the decision-making needs of the Actor network through end-to-end optimization, achieve feature-policy collaboration; and ensure that the base station Actor can quickly respond to dynamic task requests, while avoiding resource allocation conflicts and solving local robustness.

[0118] 2) Joint fine-tuning of multiple Critic networks. The goal of this part is to coordinate the Critic networks in each region through the cloud, ensure that the resource scheduling strategies in different regions are optimal from a global perspective, and achieve global consistency; and integrate the experiences of multiple regions through the cloud to improve the model generalization ability.

[0119] Next, take region i as an example to explain the parameter update process within the region. In this process, the MEC controller fixes the Critic network and freezes the Critic parameters Update the parameters θ G and θ A of the ST-GNN and Actor networks. To constrain the semantic consistency between the features g n extracted by the ST-GNN and the features enhanced by the Transformer Encoder , a predictive consistency loss is introduced, denoted as

[0120]

[0121] The corresponding policy gradient loss is denoted as

[0122]

[0123] where is the action probability output by the Actor network, and Q fixed is the global Q value calculated by the fixed Critic network. The loss calculation corresponding to base station n is

[0124]

[0125] The gradients of the ST-GNN and Actor networks are calculated as

[0126]

[0127] and

[0128]

[0129] To align the gradients, the gradient direction cosine similarity of the ST-GNN and Actor is calculated by the MEC controller as

[0130]

[0131] If then there is a conflict in the gradient direction, and the ST-GNN parameters are temporarily frozen for 1 training step to align the gradients of the two.

[0132] Based on equations (3-20) and (3-21), the parameters of the ST-GNN are updated as

[0133]

[0134] Combining equations (3-20) and (3-22), the Actor network parameters of base station n are updated to

[0135]

[0136]

[0137] Next, the cloud integrates the updated parameters of each region. These two parts of the process are summarized in Algorithm 1. In Algorithm 1, base station n calculates the ST-GNN and Actor network parameters θ G and θ A (see lines 3-8). The Critic parameters of each region are uploaded to the cloud, and the MEC controller calculates the loss function of each region (see lines 9-10). Federated averaging is used to calculate the global Critic parameters and the global loss function. The cloud combines the regularization loss function to obtain the total loss function (see lines 12-15). The cloud softly updates the parameters to the Critic networks of each region (see lines 16-18).

[0138] The intra-region fine-tuning and inter-region fine-tuning in Algorithm 1 are repeated alternately until the validation set loss converges or the early stopping condition is triggered. The alternating joint fine-tuning can improve the prediction performance of the ST-GNN model, thereby enhancing the decision-making ability of the MADRL model and achieving the efficient cooperation of the two models.

[0139] 4. Experimental Design and Result Analysis

[0140] The present invention conducts simulation experiments on a high-performance server. The server is configured with an Intel Core i9-14900K processor, 64GB of DDR5 5200MHz memory, a 4TB PCIe 4.0 solid-state drive, an ASUS PRIME Z790-P WIFI D5 motherboard, and two Gigabyte RTX 4090 24GB WindForce graphics cards. To simulate the cooperation between nodes, the data and models of each ground base station, MEC controller, and cloud controller are encapsulated in independent Docker containers. The ground base station container runs the ST-GNN, slice window optimizer, and MADRL models. The MEC container deploys the Critic network to achieve regional resource scheduling. The cloud container is used to integrate the data and parameters of each MEC controller. These containers run on the same physical server. The containers interact through a virtual network to simulate message passing and cooperation in a real network environment. Docker Compose is used to dynamically increase the number of base station nodes to easily test the effect of multi-base station cooperation. Such a setup can flexibly adjust the number of nodes to simulate the road network of multi-base stations.

[0141] To observe the contributions of different strategies in the proposed scheme to the overall performance, the proposed scheme is divided into several different types and the Proposed method shown in Table 2 is used. All experiments use the overall dynamic method. Proposed adopts the Transformer multi-head attention and joint fine-tuning method. The baseline method Baseline-1 in Table 2 adopts the non-attention and joint fine-tuning method based on the ST-GNN prediction. Baseline-2 adopts the Transformer multi-head attention mechanism and independent training method based on the ST-GNN prediction. Baseline-3 adopts the non-attention and independent training method based on the ST-GNN prediction. Baseline-4 is an ablation experiment without using the ST-GNN prediction. Based on the PEMS dataset, the present invention selects multiple nodes with large differences in traffic flow as experimental objects to simulate the vehicle task traffic arrival situation in the real environment. The experimental parameter configuration is shown in Table 3.

[0142] Table 2 Settings of the baseline methods

[0143]

[0144] Table 3 Experimental parameter configuration table

[0145]

[0146]

[0147] 4.1 Convergence analysis

[0148] The joint fine-tuning evaluates the convergence of the proposed method by calculating and plotting the reward curves of each agent's Actor network. Sampling a large batch of tuples from the experience replay pool can ensure the stability of learning.

[0149] The reward convergence curves of the three base stations are shown in Fig. 3(a). For Base Station 1, it rises rapidly in the first 200 steps, rises slowly and fluctuates in the 200 - 400 steps, and tends to be stable after 400 steps. For Base Station 2, it rises uniformly in the first 400 steps, then rises slowly and tends to be stable. For Base Station 3, it rises slowly in the first 400 steps, rises violently and fluctuates in the 400 - 600 steps, and the rate slows down and tends to be stable after 600 steps. Fig. 3(b) is the probability density curve corresponding to three learning rates. It can be observed that the reward values of the three base stations are roughly distributed between 300 - 450. Compared with the other two base stations, the distribution of the reward value of Base Station 3 is slightly more dispersed.

[0150] 4.2 Influence of the isolation of base station resource pressure performance

[0151] This experiment examines the impact of the number of RBs held by the ground base stations on the expected quality of slice performance isolation. Taking three base stations in area i as the experimental objects, the number of RBs of all ground base stations is determined by the prediction results of the ST-GNN model. The size of the shared resource pool of the MEC controller in area i is set to 120. According to the traffic flow during a peak period of a certain day and the historical slice window length, RBs are allocated to the local base stations and the overall window length is divided. Three base stations are selected as the experimental objects. Due to the complex road network structure, the task traffic received by different base stations varies significantly. Therefore, a step-by-step increase method is adopted when unifying the number of RBs. As shown in Figure 4, the number of RBs ranges from 145 to 342, and the impact of five different total RB numbers on the expected quality of slice performance isolation of the base stations is tested.

[0152] It can be seen that when the number of RBs of the base station is much smaller or much larger than the task traffic received by the base station, the expected quality of slice performance isolation will be constant at 0 or 1. When the total number of RBs is 145 or less, the arrival task volume of base stations 1 and 2 far exceeds the total number of RBs they hold. Therefore, the expected quality of performance isolation of base stations 1 and 2 is degraded to 0. It can be predicted that when the number of RBs allocated to the ground base stations is relatively generous, none of the three base stations need to rely too much on the shared resource pool, and at this time, their expected quality of performance isolation is 1. When the total number of RBs is at a specific value, the performance of different base stations at different total RB numbers. When RB = 225, the difference in the expected quality of slice performance isolation of base stations 1, 2, and 3 is obvious. The expected quality of performance isolation of the method used by base station 1 is between 0.6 and 1.0, while that of base station 2 is between 0.4 and 0.5, and that of base station 3 fluctuates around 0.7 - 1.0. This phenomenon is mainly due to the huge differences in traffic flow in adjacent areas of the road network in terms of time and space.

[0153] 4.3 Impact of Shared Resource Pool Pressure on Performance Isolation

[0154] This group of experiments examines the impact of changes in the total number of RBs in the shared resource pool on the expected quality of slice performance isolation. Three base stations in area i are tested. The number of RBs of all ground base stations is determined by the prediction results of the ST-GNN model. The number of RBs of the base stations in area i is uniformly set to 225. According to the traffic flow during a peak period of a certain day and the historical slice window length, the MEC controller uniformly divides the dynamic window length for the local base stations. Three base stations are selected as the experimental objects. Due to the complex road network structure, the task traffic received by different base stations varies significantly. Therefore, a step-by-step increase method is adopted when unifying the number of RBs in the shared resource pool. As shown in Figure 5, the number of RBs ranges from 100 to 200, and the impact of five different total RB numbers on the expected quality of slice performance isolation of the base stations is tested.

[0155] When the number of RBs in the shared resource pool is large enough, the expected quality of slice performance isolation for each base station will reach 1. When the number of RBs is below 140, due to resource tension, the expected quality of performance isolation for base stations 1, 2, and 3 is less than 1. It can be foreseen that when the number of RBs in the shared resource pool is less than a certain threshold, the resources of the three base stations themselves and the shared resource pool are insufficient to handle the arriving traffic, and at this time, the base station performance isolation quality approaches 0. When the total number of RBs in the shared resource pool is at a specific value, the performance of different base stations under different total RB numbers can be observed. When RB = 140, the difference in the expected quality of performance isolation of base stations 1, 2, and 3 is obvious. The expected quality of performance isolation of the method used by base station 1 is between 0.7 and 1.0, while that of base station 2 is between 0.4 and 0.8, and base station 3 fluctuates around 0.8 - 1.0. The huge difference in the traffic received by the three base stations leads to this phenomenon.

[0156] 5. Summary

[0157] This invention proposes a vehicle - to - everything (V2X) resource slicing method based on ST - GNN enhanced MADRL. The prediction part inputs the ST - GNN prediction results into the Transformer encoder, enhancing the spatio - temporal features of the prediction. The optimized prediction results are combined with the slice window optimizer, indirectly improving the decision - making effect within the region. The alternating joint fine - tuning within the region and across regions further improves the prediction and decision - making performance of the model at large time scales. The simulation results show that compared with the benchmark method, the proposed scheme in this chapter improves the performance isolation effect of slicing. The scenarios described in this chapter are replicable and universal, not limited by the number of base stations and traffic differences. In addition, the proposed slicing scheme is scalable and can be further used to support task offloading, content distribution, and data collection.

[0158] References

[0159] [1] Filali A, Nour B, Cherkaoui S, et al. Communication and computation O - RAN resource slicing for URLLC services using deep reinforcement learning[J]. IEEE Communications Standards Magazine, 2023, 7(1): 66 - 73.

[0160] [2] Zhao F, Gao W, Lu J, et al. Real-time concentration detection of Aldust using GRU-based Kalman filtering approach[J]. Process Safety and Environmental Protection, 2024, 189: 154-163.

[0161] [3] Li Z, Yu H, Xu J, et al. Stock market analysis and prediction using LSTM: A case study on technology stocks[J]. Innovations in Applied Engineering and Technology, 2023: 1-6.

[0162] [4] De Coninck Q. Core QUIC: Enabling Dynamic, Implementation-Agnostic Protocol Exten-sions[C] / / 2024 IFIP Networking Conference(IFIP Networking). IEEE, 2024: 640-646.

[0163] [5] Zhu Kaili, Zhu Hailong, Liu Jingyu, et al. Traffic flow prediction based on graph convolutional neural network[J]. Intelligent Computer and Applications, 2019, 9(6): 168-170, 177.

[0164] [6] Feng Ning, Guo Shengnan, Song Chao, et al. Multi-component spatio-temporal graph convolutional network for traffic flow prediction[J]. Journal of Software, 2019, 30(3): 759-769.

[0165] [7] Hurwicz L. The design of mechanisms for resource allocation[J]. The American Economic Review, 1973, 63(2): 1-30.

[0166] [8] Wang Feifei, Wang Lan, Zheng Sihui, et al. Network Slicing Resource Management Algorithm Based on Deep Reinforcement Learning [J]. Journal of Terahertz Science and Electronic Information Technology, 2024, 22(7): 792-799.

[0167] [9] Lin Yan, Xia Kaiyuan, Zhang Yijin. Edge Computing Network Federation Slicing Resource Management Based on Generative Adversarial Network-Assisted Multi-Agent Reinforcement Learning [J]. Journal of Electronics & Information Technology, 2025, 47(3): 1-12.

[0168]

[10] Zhang Jiageng, Zhu Min, Du Feng, et al. Prediction-Based 5G Network Slicing Algorithm [J]. Telecommunications Science, 2021, 37(9): 74-85.

[11] Peng Xinyu, Zhou Yang, Dong Zhenjiang. Intelligent Cooperative Management Technology of Virtual Resources Based on Telecommuting Driving in Internet of Vehicles [J]. Telecommunications Science, 2020, 36(4): 61-68.

[0169]

[12] Saifullah M, Papakonstantinou K G, Andriotis C P, et al. Multi-agent Deep Reinforcement Learning with Centralized Training and Decentralized Execution for Transportation Infrastructure Management [J]. arXiv preprint arXiv:2401.12455, 2024.

Claims

1. A vehicle networking resource slicing method integrating ST-GNN and multi-agent, characterized in that The vehicle networking resource slicing method is applied to a hierarchical resource slicing framework. Under this framework, a centralized controller is deployed in the cloud, and a local controller is deployed at the edge side; A Mobile Edge Computing (MEC) controller serves as the local controller; for an MEC controller in the same area of the vehicle networking and multiple base stations under its jurisdiction: The physical resource blocks of the MEC controller are virtualized into a resource pool indirectly shared by the base stations within its jurisdiction; the resources on each base station are virtualized into multiple slice areas, and each slice supports a type of customized service type; The task traffic data received by the base station is divided into different types of task traffic and is scheduled to the corresponding slice for processing according to the type; when the resources of a certain slice on the base station are insufficient, the arriving tasks are scheduled to the resource blocks allocated for this base station in the shared resource pool; The vehicle networking resource slicing method is to obtain the spatio-temporal characteristics of the local task traffic through a global task traffic prediction model and use this as the input of the local slice window prediction model, so as to obtain the corresponding slice window length result; The output results of the global task traffic prediction model and the local slice window prediction model are input into the resource slicing decision model to obtain the optimal resource slicing decision; The global task traffic prediction model is a task traffic prediction model based on the Spatio-Temporal Graph Neural Network (ST-GNN), and it is a global model; in ST-GNN, the nodes of the graph represent the base station location characteristics, vehicle flow characteristics, and task type characteristics, and the edges of the graph represent the base station physical distance characteristics and communication link quality characteristics; The resource slicing decision model is a resource slicing decision model based on Multi-Agent Deep Reinforcement Learning (MADRL), and it is a regional model; Each base station deploys a slice window prediction model respectively; In the resource slicing decision model, the MEC controller integrates the Critic network; each base station is regarded as an agent, with an independent Actor network and sharing the Critic network integrated by the MEC controller; The base station agent observes the local state and outputs an action accordingly; based on the spatio-temporal characteristics of the task traffic input by the agent, the action generated by the Actor network interacts with the environment, responds to the task request in real time, and outputs the action decision for resource slice allocation; The MEC controller is used for global coordination and local decision-making; the MEC controller collects the real-time states of all base station agents, runs the Critic network, evaluates the Q value of the joint actions of the base station agents, generates cross-base station cooperation signals, and reallocates resources for each base station at the beginning of each slice window or when encountering burst task traffic.

2. The method for slicing vehicle network resources by fusing ST-GNN and multi-agent according to claim 1, wherein For the global task traffic prediction model: If the task requests generated by vehicles conform to a Poisson process, then: Let the amount of task requests generated by vehicles within the coverage area of base station n per unit time be y n , and the task request intensity be λ n ; y n follows a Poisson distribution, denoted as y n ~ Poisson(λ n ); Let the traffic flow in the coverage area of base station \(n\) per unit time be \(v\). n The average task request rate of each vehicle is \(\rho\). Task request intensity λ n Meet λ n = ρ·v n Let be a graphical representation representing a base station, where the nodes V represent the location characteristics, traffic flow characteristics, and task type characteristics of the base station, and the edges E represent the physical distance characteristics and communication link quality characteristics of the base station; The spatio-temporal embedding extracted by base station n through ST-GNN is denoted as It contains the traffic state and resource load of this area; The physical distance and communication link quality between base station n and base station n′ are respectively represented as and The learnable location encoding of base station n is represented as Given g n With PE n , the spatio-temporal features of base station n are spliced into The result of the above formula is input into a multi-layer Transformer Encoder. After being processed by the encoder, the spatio-temporal characteristics are enhanced to The result of the above formula is compared with g n and input into the residual connection function to obtain the output of base station n, denoted as Let the task request volume prediction of base station n be It is used as the input of MADRL; By constructing a fully connected network, the task request volume in the future period is predicted as Node with y n , the loss function of ST-GNN is constructed as The chain rule is used to compute the gradient of the loss with respect to the parameters θ of the ST-GNN model G which is denoted as Its ST-GNN trainable parameters θ G are updated to η GNN represents the learning rate, represents the gradient of the ST-GNN network.

3. The vehicle network resource slicing method integrating ST-GNN and multi-agent according to claim 1, characterized in that for Resource slicing decision model: In the EMC controller and the base stations under its jurisdiction, the training of agents is centrally scheduled by the MEC controller; during the training phase, agents share global traffic state information or the actions of other agents; during the execution phase, each agent is decentralized; Let the set and cardinality of the areas under the jurisdiction of the MEC controller be denoted as and I, respectively. The set and cardinality of the base stations n in area i are denoted as and respectively. The state received and the action output by the Actor network deployed at base station n are denoted as s n and a n ; Local state is input into the Actor network; the Critic network inputs the joint state s = (s1, s2, …, s n ) and the joint action a = (a1, a2, …, a n ), and outputs the global Q-value Q(s, a); Combined with the Transformer multi-head attention mechanism, within region i, N i local features of base stations are concatenated into a matrix The learnable position encoding of base station n within region i is mapped to a learnable variable: The local features and location encoding of base station n are input into the multi-layer Transformer Encoder, and the enhanced features are represented as Based on the multi-head attention mechanism, the global features are represented as wherein is a learnable parameter; Along the base station dimension, taking the mean, the global state representation is expressed as The local feature g of each base station n n is respectively concatenated with the global feature g g and the result is input into the Critic network to obtain the global Q value as The Q value is used to guide policy update; the Actor network is updated to Indicates the Actor network gradient; The target Actor network and the target Critic network use a soft update strategy to update parameters θ'←τθ+(1-τ)θ' (τ<<1) The rewards for base station n are divided into local and cooperative reward weights; local rewards are to encourage the efficient completion of local tasks; cooperative reward r g is calculated by the MEC controller; to penalize resource competition conflicts and encourage base station cooperation, the total reward of base station n is expressed as where α n,1 and α n,2 are the local reward weight and the collaborative reward weight, respectively; If resource competition is severe, the MEC controller increases α n,2 to encourage cooperation among base stations; if it is necessary to prioritize the local tasks of base station n, increase α n,1 .

4. The method for slicing vehicle network resources by integrating ST-GNN and multi-agent according to claim 1, wherein An alternating joint fine-tuning strategy is adopted to align the parameters of the global task traffic prediction model and the resource slicing decision model; The joint fine-tuning strategy adopts the "centralized training and distributed execution" design principle, covering optimizations at two levels: within the coverage area and across regions in the cloud: 1) During the joint fine-tuning process of the ST-GNN within the region and the Actor network on local base station n, through end-to-end optimization, the spatio-temporal features extracted by the ST-GNN are adapted to the decision-making requirements of the Actor network, realizing feature and policy coordination; 2) During the joint fine-tuning process of multiple Critic networks across regions, by coordinating the Critic networks in each region through the cloud, it is ensured that the resource scheduling strategies in different regions are optimal from a global perspective to achieve global consistency; and the experiences of each region are integrated through the cloud; During the parameter update process within region i: The MEC controller fixes the Critic network and freezes the Critic parameters Update the parameters θ of the ST-GNN and the Actor network G and θ A ; To constrain the ST-GNN to extract the feature g n and the features enhanced by the Transformer Encoder for semantic consistency, the predictive consistency loss is introduced and denoted as The corresponding policy gradient loss is represented as where is the action probability output by the Actor network, and Q fixed is the global Q value calculated by the fixed Critic network; The loss corresponding to base station n is calculated as The gradients of the ST-GNN and the Actor network are calculated as and For gradient alignment, the cosine similarity of the gradient directions of the ST-GNN and the Actor is calculated by the MEC controller as If there is a conflict in the gradient direction, temporarily freeze the ST-GNN parameters for 1 training step to align the gradients of the two; The parameters of the ST-GNN are updated to The parameters of the Actor network of base station n are updated to Then the cloud integrates the updated parameters of each region: During the parameter update process across regions: Critic parameters for each region Upload to the cloud, and the MEC controller calculates the loss function for each region; Federated averaging is used to calculate the global Critic parameters and the global loss function, and the cloud combines the regularization loss function to obtain the total loss function The cloud then softly updates the parameters to the Critic networks in each region; The intra-region fine-tuning and the inter-region fine-tuning are repeated alternately until the validation set loss converges or the early stopping condition is triggered.

Images