Internet of vehicles resource slicing method based on Transform double federated learning

Abstract

The invention discloses an Internet of Vehicles resource slicing method based on Transform dual federated learning, and the method comprises the steps: abstracting physical resources of a base station into a virtual resource pool in an Internet of Vehicles system, and carrying out the management of a controller; and the controller adjusts slice resource allocation according to the terminal request information fed back by each base station. The controller is of a double-layer controller structure; the centralized controller of the cloud is integrated with a global model, and the global model comprises a global flow prediction model and a global resource slicing decision model; the centralized controller performs base station resource slicing through a global model; a local controller connected with an adjacent base station on the edge side is integrated with a local model, and the local model comprises a local flow prediction model, a slice window optimizer and a local resource slice decision model. The local controller makes a resource slicing decision through the local model; the slicing step comprises the following steps of: segmenting physical resources of each base station by adopting a window self-adaptive segmentation method; and optimizing a decision result through local model training and a double-federated model method.

Description

Technical Field

[0001] The present invention belongs to the technical field of Internet of Vehicles, and specifically is a method for slicing Internet of Vehicles resources based on Transformer dual federated learning. Background Art

[0002] In the post-5G and 6G era, the Internet of Vehicles is facing complex and diverse application requirements, especially in scenarios such as autonomous driving, fleet collaboration, and remote driving. In order to meet these key applications, the network needs to have higher bandwidth, lower latency, and greater capacity. When autonomous vehicles are driving at high speeds, they have extremely high requirements for real-time control and environmental perception. The network must provide ultra-low latency and high reliability communication support. At the same time, the demand for transmission of multimedia content such as high-definition maps, on-board sensor data, and videos is also increasing rapidly, posing severe challenges to network bandwidth and transmission efficiency. With the emergence of emerging scenarios such as drone-assisted vehicle-road collaboration[1] and intelligent traffic control, the network must flexibly manage resources to cope with frequently changing needs.

[0003] Internet of Vehicles (IoV) slicing technology[2] has emerged as a key means to solve these problems. IoV slicing divides the physical network into multiple virtual network instances (slices), independently allocates communication, computing and storage resources according to the needs of different application scenarios, and provides flexible and refined Quality of Service (QoS) guarantees for various applications. This slicing technology can effectively manage resources in the IoV, ensuring that scenarios such as autonomous driving, fleet collaboration and remote driving can still obtain the best service support under the condition of diversified resource requirements. In 6G networks, slicing technology is combined with ultra-high reliability and ultra-low latency communication (URLLC)[3], high-bandwidth enhanced mobile broadband (eMBB)[4] and other technologies to further improve the resource utilization efficiency and service level of the IoV.

[0004] In order to further improve the intelligence of Internet of Vehicles (IoV) slices, the deep reinforcement learning (DRL) [5] method has been introduced to deal with complex resource allocation and management problems. Through continuous interaction with the dynamic environment, DRL can continuously optimize resource allocation decisions and adapt to changes in demand in different scenarios, thereby improving the flexibility and adaptability of the network. At the same time, combined with the federated learning [6] framework, the DRL model can use the data of each node for collaborative training in a distributed environment, avoiding the privacy and communication burden issues caused by large-scale centralized data collection. Through federated learning, IoV slices can support the efficient operation of large-scale IoV while ensuring data privacy, further enhancing the service capabilities and intelligence level of IoV.

[0005] Although DRL and the federated learning framework have been introduced, the intelligentization of IoV resource slicing still faces the following challenges:

[0006] 1. Personalized slice window adjustment. The network situation in different base station coverage areas is different. The different prosperity of different regions leads to different traffic volumes. Even in the same block, traffic volume changes due to peak and trough periods. These problems force the adjustment of slice windows to be personalized [7]. If the slice window is too short, the cloud controller needs to frequently allocate resources, resulting in a significant increase in control and computing costs. If the slice window is too long, the fluctuation of task traffic may significantly damage the performance isolation effect of the slice. Zhang et al. proposed a software-defined network framework with a hierarchical structure, which divides multiple slice windows of equal length and calculates the optimal resource allocation strategy in each slice window [8]. Li et al. proposed a hierarchical soft RAN slicing framework for differentiated service provisioning, which performs network-level and BS-level resource slicing on large and small time scales [9]. However, in the above methods, resources are allocated in fixed slice windows.

[0007] 2. Dual optimization of resource slicing agility and timeliness. The allocation decision of resource slicing

[10] occurs after the slice window is divided, which means that each decision of resource slicing has a certain time interval, and the decision effect usually needs to be verified after a period of time. Some people have proposed using reinforcement learning to optimize resource slicing decisions

[11] . However, under the traditional reinforcement learning framework, resource slicing decisions rely on real-time status information. When encountering emergencies, it is still impossible to adjust the slice length in time.

[0008] 3. Optimization of model scalability and generalization in road network environment. In road network environment

[12] , even the traffic flow in adjacent road network areas has significant spatiotemporal differences

[13] . For example, during peak hours, adjacent sections of two-way lanes may be extremely crowded on one side and sparse on the other. Under the huge differences in road network environment, traditional federated aggregation will cause the model to favor a certain traffic pattern and ignore other patterns. Therefore, some people use traditional attention methods

[14] to deal with the problem of differences in data from different base stations. However, in the case of high dynamics and non-uniformity of the Internet of Vehicles and huge data differences, the effect of traditional attention methods is also limited. Tao et al. introduced a multi-head attention mechanism

[15] to assign different weights to each node in the aggregation process, thereby enhancing the generalization of the global model. Some people use deep reinforcement learning models as local models to participate in federated learning. This method can not only learn the best strategies in different scenarios in the local model, but also aggregate the advantages of these strategies into the global model through federated aggregation

[16] , thereby improving the performance of the entire system. However, none of the above solves the problem of model generalization at the same time. Summary of the invention

[0009] In view of the challenges and limitations in the prior art, the present invention proposes a vehicle networking resource slicing method based on Transformer dual federated learning. In the vehicle networking system, the physical resources of the base station are abstracted into a virtual resource pool and managed by the controller. According to the terminal request information fed back by each base station, the controller adjusts the slice resource allocation.

[0010] The physical resource blocks (RBs) on each base station are virtualized into multiple slices and a resource sharing area. When the resources of a slice are insufficient, the resources in the sharing area are temporarily occupied.

[0011] The controller is a two-layer controller structure, in which a centralized controller and a local controller are deployed on the cloud and edge side respectively;

[0012] The centralized controller integrates a global model, which includes a global traffic prediction model and a global resource slicing decision model; the centralized controller performs base station resource slicing through the global model;

[0013] A local controller connected to a neighboring base station is integrated with a local model, which includes a local traffic prediction model, a slice window optimizer, and a local resource slice decision model; the local controller makes a resource slice decision through the local model;

[0014] The slicing steps include:

[0015] (1) Using a window-adaptive segmentation method to segment the physical resources of each base station;

[0016] (ii) Optimize decision-making results through local model training and dual federated model methods.

[0017] The dual federated learning framework based on Transformer multi-head attention of the present invention supports resource slicing in large-scale, diversified, and highly dynamic Internet of Vehicles scenarios. The present invention mainly includes three contributions:

[0018] First, to cope with the fluctuation and uneven spatial and temporal distribution of task traffic, a window-personalized flexible resource slicing framework is designed. The framework uses a soft indicator to measure the quality of slice performance isolation. The slice isolation quality maximization problem is modeled as a joint optimization problem of slice window adjustment, resource slicing, and resource sharing area setting.

[0019] Second, a traffic-aware personalized slice window adjustment strategy is designed to cooperate with the local resource slicing model driven by DRL (Deep Reinforcement Learning). Different from the "static" or "overall dynamic" slicing methods of the regional center, the slice window optimizer of each base station can flexibly adjust the window length according to the spatiotemporal characteristics of the local traffic flow. During peak traffic or congestion, the slice window length will be temporarily shortened to speed up resource updates; when the traffic is sparse or congestion is evacuated, the window will be extended to reduce control overhead. Supported by the slice window length and task traffic prediction results, the local DRL model effectively alleviates the problem of the gradual failure of slicing decisions over a long time scale caused by environmental changes.

[0020] Third, the resource slice models of each base station are uniformly incorporated into a dual federated learning framework that includes prediction and decision-making. At the prediction level, multiple base stations improve the accuracy of task traffic prediction and optimize the adjustment efficiency of slice windows through collaborative training. The optimization of these two dimensions further promotes the resource slice decision effect of the local DRL model. At the decision-making level, the federated DRL paradigm amplifies the advantages of local optimization and improves the performance of overall slice management. The Transformer multi-head attention mechanism and its encoder-decoder are used to aggregate the prediction and decision models of neighboring base stations to enhance the generalization of the model. Simulation results show that the proposed scheme is superior to existing benchmark learning methods in terms of slice performance isolation, resource allocation efficiency, and decision effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A scene diagram showing a vehicle networking system in a specific implementation manner;

[0022] Figure 2 Indicates resource allocation strategy;

[0023] Figure 3 Represents the edge-side model architecture that fuses prediction and decision making on the local controller;

[0024] Figure 4(a) to Figure 4(c) They represent the task traffic fluctuations for three base stations in the same period;

[0025] Figure 5(a) to Figure 5(c) They represent the fitting curves of the slice window and traffic fluctuation for three base stations respectively;

[0026] Figure 6 A dual federated learning framework that represents cloud-edge collaboration;

[0027] Figure 7(a) and Figure 7(b) respectively represent the reward convergence analysis of reinforcement learning, where: Figure 7(a) represents the reward convergence curves corresponding to the three learning rates, and Figure 7(b) represents the probability density curves corresponding to the three learning rates;

[0028] Figure 8(a) to Figure 8(c) Respectively represent the impact of static strategies on performance isolation effects for three base stations;

[0029] Figure 9(a) to Figure 9(c) Respectively represent the impact of the overall dynamic strategy on the performance isolation effect for three base stations;

[0030] Figure 10(a) to Figure 10(c) The influence of the personalized slice window length allocation strategy on the performance isolation effect for three base stations is shown respectively;

[0031] Figure 11 shows the effect of the number of resource blocks on performance isolation for three base stations;

[0032] Fig.12 The impact of different slicing window strategies on decision efficiency. DETAILED DESCRIPTION

[0033] 1 Overview

[0034] With the diversification and dynamic development of IoV application scenarios in the post-5G and 6G era, traditional resource slicing technology is difficult to cope with the complex requirements of highly dynamic and large-scale IoV. To this end, this paper proposes a dual federated learning framework based on Transformer multi-head attention to support efficient management and optimization of resource slicing.

[0035] Firstly, a window-personalized flexible resource slicing framework is designed to quantify the slice isolation performance through soft indicators, and the slice isolation quality maximization is modeled as a joint optimization problem of slice window adjustment, resource slicing and resource sharing area setting.

[0036] Secondly, a traffic-aware personalized slicing window adjustment strategy is proposed, which works together with the local deep reinforcement learning (DRL) model of the base station to dynamically adjust the window length to adapt to local traffic fluctuations and improve the timeliness and stability of slicing decisions.

[0037] Finally, in view of the limitation that traditional federated averaging or attention aggregation methods cannot effectively deal with the differences in tasks between adjacent base stations, the Transformer multi-head attention mechanism is introduced to improve the generalization performance of the global model. At the two federated levels of prediction and decision-making, deep coordination of traffic prediction and slice optimization is achieved.

[0038] Simulation results show that the proposed scheme outperforms existing methods in terms of slice isolation performance, decision efficiency and resource utilization.

[0039] 2 System Model

[0040] Through Network Function Virtualization (NFV)

[17] , the physical resources of the base station are abstracted into a virtual resource pool, which is managed by a controller based on Software-Defined Networking (SDN)

[18] . Based on the terminal request information fed back by each base station, the controller dynamically adjusts the slice resource allocation. Figure 1 As shown, the present invention considers a two-layer controller structure. The centralized controller is deployed in the cloud and uses global information to slice base station resources; the local controller is connected to the neighboring base stations and makes quick decisions based on local information. This multi-level resource management system can improve network agility while taking into account global performance optimization.

[0041] The parameter settings used in this paper are shown in Table 1.

[0042] Table 1 Symbols and variable definitions

[0043]

[0044]

[0045] 2.1 Window-Adaptive Flexible Slicing Framework

[0046] The physical resource blocks (RBs) on each base station are virtualized into multiple slices and a resource sharing area. Each slice supports a type of customized service, and the sharing area is used to alleviate instantaneous resource pressure. The traffic data received by the base station is divided into different task flows, and the arriving tasks are scheduled to the corresponding slices for processing according to the task type. When the resources of a slice are insufficient, the resources in the sharing area can be temporarily occupied.

[0047] A window-adaptive partitioning method is used to partition the physical resources of each base station. Figure 2 As an example, we explain the slice window adjustment and resource allocation strategy. Assume that time is divided into multiple continuous slice windows, each of which contains multiple discrete time slots. The set and cardinality of base stations are denoted as and N. The set and cardinality of service slices of base station n are respectively denoted as and M n The set and cardinality of slice windows are represented as and H n On base station n, the slice window The included time slot set and cardinality are represented as and At the end of slice window h-1, the local controller at base station n re-determines and M n .

[0048] The total number of RBs held by base station n is denoted as B n In slice window h, the number of RBs allocated to slice m and the shared area is expressed as and satisfy

[0049]

[0050] At the beginning of the slice window h, the resources of each slice and shared area resources In window h, the number of RBs held by each slice and shared area remains unchanged.

[0051] 2.2 Soft Isolation Metrics

[0052] Performance isolation is the premise for slice resource coexistence, that is, to ensure that the overload of one slice does not affect other slices. The actual resource requirement is The number of RBs occupied by the slice in the resource sharing area on base station n is calculated as

[0053]

[0054] The present invention introduces a soft performance isolation metric. Assume that at the beginning of window h, the number of RBs reserved by base station n for the shared area is Base station n in time slot The performance isolation quality of is normalized to

[0055]

[0056] exist (i.e., occupying all shared areas cannot meet resource requirements), the slice performance isolation is violated, and ρ n,t = 0. (ie: the shared area is sufficient to cope with resource pressure), if The resource blocks in the shared area will be temporarily occupied, ρ n,t ∈[0,1), which means that the performance isolation is “discounted”; when When slice m can meet the current demand without occupying the resources in the shared area, then ρ n,t =1.

[0057] 2.3 Problem Modeling

[0058] For base station n, under the slice window h (given ), the slice performance isolation quality maximization problem is modeled as

[0059]

[0060] Constraint (1) ensures that the resource blocks allocated to the slice and shared area do not exceed the total number of resource blocks held by the base station. Extending to multiple consecutive slice windows, the long-term optimization problem of the performance isolation quality of base station n is modeled as

[0061]

[0062] question The essence of this is how to adjust the slice window length and allocate resources between slices and shared areas to maximize the long-term performance isolation quality. Sections 3 and 4 discuss how to optimize decision results through local model training and dual federated model algorithms.

[0063] 3. Edge-side model training

[0064] On the edge side, each local controller integrates a traffic prediction model, a slice window optimizer, and a resource slice model. Traffic prediction and resource slice use stacked LSTM and DRL algorithms respectively. The traffic prediction model is responsible for predicting the task traffic of the base station. Figure 3 As an example, the traffic prediction result (expressed as X n) is input to the slice window optimizer to determine X-based n and The DRL model determines the resource allocation of slices and shared areas, and the corresponding decision vector is represented as

[0065] 3.1 Local Task Traffic Prediction

[0066] The historical traffic flow data at base station n is used to train a local model based on stacked LSTM

[19] to predict the traffic flow in the future. By increasing the depth of the neural network, the stacked structure enhances the ability to fit complex linear and nonlinear functions. The bottom network captures short-term patterns, while the high-level network captures longer-term trends and high-level features by aggregating the bottom-level information.

[0067] The training round set and cardinality of the local prediction model of base station n are expressed as and O n , the local parameters of the prediction model are expressed as At the beginning of training, base station n initializes the local parameters of the stacked LSTM Assume that the number of stacked LSTM layers is G. The input of the first layer is the original data sequence X = [x1, x2, ..., x O ], the output is in Indicates that o∈O n training rounds, vector The hidden state of the first layer. The output of the Gth layer is The prediction model at base station n is denoted as X(·). The number of RBs required for slice m in window h at base station n is predicted as

[0068]

[0069] According to the predicted value output by the model during training and the target value X, the loss function is described as

[0070]

[0071] The Back Propagation Through Time (BPTT) algorithm

[20] is used to calculate the gradient of the back propagation process. Based on (6), with respect to the local parameters The gradient of is calculated as Using SGD

[21] , Updated to

[0072]

[0073] where η n is the learning rate.

[0074] 3.2 Personalized adjustment of the slice window

[0075] As mentioned above, the requests issued by vehicles are unevenly distributed in time and space, and the proportion of various task traffic continues to fluctuate, even in adjacent road network areas. In the fixed slice window mode, the update of slice resources cannot be synchronized with the network situation, which not only leads to a decrease in isolation quality, but also generates unnecessary control overhead. In response to this, a base station-centric slice window personalization strategy is proposed.

[0076] Before designing the strategy, the present invention collects and visualizes the task traffic of three adjacent base stations in the road network during the same period. Figure 4(a) to Figure 4(c) As shown in the figure, although adjacent, the fluctuations of task traffic on these base stations are significantly different. The present invention presets different initial window lengths for these base stations and tentatively reduces the window length to explore the optimal match between the slice window length and the traffic. The numerical pairs of task fluctuations and optimal slice window lengths are observed and recorded. In order to avoid randomness, multiple initial time points are selected. The window length is expressed as a function of traffic, and the relationship between the two is fitted as

[0077]

[0078] The premise for constructing the best fit is to find c1 and c2 that minimize the residual sum of squares

[22] (c1 determines the speed at which the window length changes with traffic fluctuations; c2 represents the base value of the window length when the task traffic fluctuation is zero or very small). As shown in Figure 5, the slice window length gradually decreases as the task traffic fluctuation increases. The more drastic the task traffic fluctuation, the smaller the slice window length and the faster the resource update frequency. These characteristics are consistent with expectations. At the beginning of window h, the fitting function is used to predict the length of the next window h+1 (i.e.: ). Due to the differences in task traffic of base stations, it is difficult to use a unified fitting function to determine the window length of each base station.

[0079] In order to achieve personalized window adjustment, the present invention sets a window optimizer for each base station to cope with the fluctuation of task traffic of different base stations. Each optimizer receives the prediction result X transmitted by the prediction model. n Then, substitute into formula (7) to get the corresponding slice window length result

[0080] 3.3 Local Resource Slicing Model

[0081] Each base station independently determines the length of the slice window. In a given window, the resource allocation of slices and shared areas is also independent. Therefore, the long-term optimization problem It can be simplified to a short-term optimization under a discrete window, which belongs to a Markov decision process (MDP)

[23] problem in a finite time domain. In this problem, resource slicing refers to a local controller dividing the resources held by the base station into M n The process of isolating the quality of the slices and a shared area to maximize slice performance.

[0082] In the DRL framework, resource slicing decisions depend on the current state of the environment. Unlike typical DRL applications such as path planning

[24] , resource slicing is window-centric. In other words, the decisions output by DRL will continue to produce benefits over a certain period of time. As time goes by, the decisions will inevitably lose their effectiveness due to changes in demand. In this regard, a local resource slicing method based on fusion prediction and decision-making is proposed to "extend" the sustainability of resource slicing decisions so that the effectiveness of the decisions in a given window is maximized. In this method, the output results of the traffic prediction model in Section 3.1 and the window optimizer in Section 3.2 are input into the DRL model to help it generate the best resource slicing decision.

[0083] The decision model of base station n is abstracted as a DRL agent. Under the slice window h, the set and cardinality of the model iteration rounds are represented as ε n and E n . Iteration round e∈ε n The receiving state of the environment is denoted by s n,e , the resource allocation action for slices and shared areas is recorded as a n,e , the reward given by the environment is recorded as r n,e The model is in state s n,e Under the condition of n,e+1 |s n,e ,{a n,1 ,a n,2 ,...,a n,e}), the environment state is updated by the controller to s n,e+1 . Expressions related to training include:

[0084] State space: The resource slices on base station n depend on the prediction result X of the stacked LSTM n and the duration T output by the window optimizer n (h) In the iteration round e, the corresponding state vector is expressed as

[0085]

[0086] Action space: In iteration round e, the allocation action made by the agent when slicing resources is recorded as

[0087]

[0088] In the formula is the decision set of iteration round e, that is, the controller allocates slice resource blocks to the base station according to the action. The decision variable of each action is 0, 1 or 2, which is determined by the current state.

[0089] Reward: The goal of the system is transformed from maximizing the performance isolation quality to maximizing the reward, which reflects the quality of action execution. The total reward value in iteration round e is recorded as

[0090]

[0091] Among them, ρ n,t Derived from (3), it is used to quantify the reward obtained by completing the task in time slot t.

[0092] In the state space, the agent estimates the reward for each action and stores it in the Q table. The action value function is recorded as The maximum reward for each state in the Q table represents the maximum possible reward in the future. By querying the Q table, the agent determines the action with the maximum reward in each state as

[0093]

[0094] Substituting the Bellman equation

[25] into equation (11), the vector in the Q table is expressed as

[0095]

[0096] Where γ n is the learning rate and ν is the greedy probability.

[0097] After the local model is trained, the window optimizer and decision model finally come into play. n and Assist the DRL agent to generate service slices and allocation results of shared area RBs

[0098] With the help of the prediction model, the local DRL model training process is summarized as Algorithm 1. At each time step t, the traffic prediction value X at the current moment is n and the predicted value X′ for the next time step n Integrate into state n,e and s' n,eIn this way, the algorithm can foresee the future traffic change trend when making decisions. By using future traffic predictions, the algorithm can respond to upcoming traffic fluctuations in advance and allocate base station resources more efficiently to avoid resource overload or idleness (see lines 3-4). n The greedy strategy selects action a n,e , with probability ε n Random Selection Otherwise select (See lines 6-9). After each action is taken, the state s' is updated by receiving the new traffic prediction value n,e , so that the next decision can be based on the latest traffic information (see lines 10-11). It should be noted that the window optimizer and DRL model directly determine the resource slices on base station n, and the result of the traffic prediction model reflects the "duration of the state", optimizes the state vector required by DRL, and improves the benefits of each DRL model decision.

[0099]

[0100] 4. Optimization of Dual Federated Learning with Cloud-Edge Collaboration

[0101] As an enhancement of the edge-side model, this section develops a cloud-edge collaborative dual federated learning framework. Figure 6 As shown in the figure, the framework contains two sets of federated learning workflows, one for task traffic prediction and the other for resource slicing. On the one hand, the prediction model enhanced by the federation optimizes the decision of the local DRL model. On the other hand, the federation for local DRL further improves the decision performance and scalability of the system. Considering the differences and complexity of the data of adjacent base stations in the road network, it is deployed on the cloud controller, and the multi-head attention mechanism is used to enhance the generalization and adaptability of the global model (including the prediction and decision models).

[0102] 4.1 Federated Model Training for Traffic Prediction

[0103] The cloud-based centralized controller uses a serial federation mode

[26] to sequentially receive the parameters uploaded by the local models, aggregate them, and update the global model. Due to differences in data volume, data distribution, and initialization model quality, the contributions of model parameters uploaded by each base station to the construction of the global model vary greatly. The Transformer encoder (TE) and decoder (TD)

[27] are integrated on the cloud-based controller to optimize the parameters uploaded during the aggregation process, such as Figure 6 As shown. TE uploads the parameters of base station n Encoded into an input sequence, which is stacked over multiple layers to generate Q n , K n, V n , and adapted to the multi-head attention mechanism in Transformer.

[0104] The global prediction model parameters are initialized as is used by base station n for local training and is updated as Defining the similarity function in Under the multi-head attention mechanism, the weight of this local prediction model is calculated as

[0105]

[0106] Substituting the similarity function into formula (13), the attention weight of the prediction model is optimized as

[0107]

[0108] Using the attention weights, the global prediction model parameters are weighted and updated as

[0109]

[0110] In this way, the training parameters of the stacked LSTM model of each base station are fed back to the global model in sequence, so that the global model is continuously optimized, and the optimized parameters are fed back to the local model to improve the performance and adaptability of the local model.

[0111] 4.2 Prediction-enhanced Federated DRL Model Training

[0112] Base Station n The local DRL model parameters at iteration round e are expressed as The global decision model parameters of the base station are initialized as The model uses Perform local training to obtain updated local model parameters Under the attention mechanism, the weight of base station n is calculated as

[0113]

[0114] Substituting equation (16), the global decision model parameters are weighted and updated as

[0115]

[0116] Under the optimization of federated learning, the results of the slice window optimizer on the base station are further improved. After receiving the optimized prediction result X′ n and After that, the global decision model parameter aggregation process is summarized as Algorithm 2. Each base station is updated sequentially, and the global model gradually receives updates from each base station (see lines 4-11). After each base station is trained locally, the parameters Through the multi-head attention mechanism and the previous aggregation parameters Perform weighted updates to generate new aggregation parameters (see lines 7-10). The parameters of the multi-head attention mechanism are aggregated after each round of training. Used to update the global Q network (See line 12.) The specific content of line (6) of Algorithm 2 refers to the local parameter update process of Algorithm 1.

[0117]

[0118] In the serial federation mode, the model update gradually integrates the information of each base station, and uses the multi-head attention mechanism to dynamically adjust the weight of each base station, so that the model can better adapt to diverse environments. The parameter update algorithm of the global prediction model can improve the prediction effect of the local prediction model, thereby improving the decision-making performance of the local decision model and further optimizing the results of the experiment.

[0119] 5 Experimental design and results analysis

[0120] For performance evaluation, we conducted simulation experiments on a high-performance server. The server is equipped with an Intel Core i9-14900K processor, 64GB DDR5 5200MHz memory, 4TB PCIe 4.0 solid-state drive, ASUS PRIMEZ790-PWIFID5 motherboard, and two Gigabyte RTX 4090 24GB WindForce graphics cards. To simulate the collaboration between nodes, the data and models of each ground base station and cloud controller are encapsulated in independent Docker containers. The ground base station container runs LSTM, slice window optimizer, and DRL models. The cloud container runs the Transformer model for global optimization. These containers run on the same physical server. The containers interact with each other through a virtual network to simulate message passing and collaboration in a real network environment. Using Docker Compose to dynamically increase base station nodes, the effect of multi-base station collaboration can be easily tested. This setting allows us to flexibly adjust the number of nodes and simulate a multi-base station road network.

[0121] In order to observe the contribution of different strategies in the proposed scheme to the overall performance, the proposed scheme is divided into three categories as shown in Table 3. Proposed-1 uses a multi-head attention method under a fixed slice window; Proposed-2 uses a multi-head attention method under an overall adjusted slice window; Proposed-3 uses a multi-head attention method under a personalized slice window. The ten methods listed in Table 2 are selected as benchmark methods. Baseline-1, 2, and 3 use a personalized slice window adjustment method; Baseline-4 uses a multi-head attention method based on the DRL model only; Baseline-5, 6, and 7 use a fixed slice window adjustment method; Baseline-8, 9, and 10 use an overall adjusted slice window adjustment method. Based on the PEMS dataset, this paper selects multiple nodes with large differences in traffic flow as experimental objects, simulating the vehicle task flow achievement in a real environment. The experimental parameter configuration is shown in Table 4.

[0122] Table 2

[0123]

[0124] Table 3

[0125]

[0126] Table 4 Experimental parameter configuration table

[0127]

[0128] 5.1 Reward Convergence Analysis

[0129] The first experiment evaluates the convergence of the proposed method under different learning rates by calculating the average cumulative reward of the policy network over multiple time windows. Sampling larger batches of tuples from the experience replay pool can ensure the stability of learning. First, set the sampling batch size to 2048.

[0130] As shown in Figure 7(a), when the learning rate γ is set to 0.005, the convergence curve grows slowly, and the policy network converges to stability after 600 updates; when γ = 0.01, the average cumulative reward converges to about 180 relatively quickly, and then slowly converges to 200; when γ is set to 0.025, the reward converges quickly to about 200 and tends to be stable. This shows that the size of γ affects the convergence of the method proposed in this paper. When γ is set to 0.025, the method can balance the convergence speed and stability, and the average cumulative reward can be stabilized at about 200 after 200 updates. Therefore, γ = 0.025 is taken in subsequent experiments. Figure 7(b) is the probability density curve corresponding to the three learning rates.

[0131] 5.2 Analysis of Slice Window Strategy

[0132] Generally, increasing the number of RBs helps improve the quality expectations of slice performance isolation, but the actual effect is largely determined by the resource allocation strategy. In many cases, once the resource allocation of a slice does not match the actual resource demand, the overall performance isolation effect will be destroyed.

[0133] Next, we further investigate the impact of different slice window division strategies on slice performance isolation during the morning peak hours. Figure 4(a) to Figure 4(c) As shown in Figure 1, the task traffic received by adjacent base stations in the same period varies greatly. The static strategy divides the time axis into two slice windows evenly. The overall dynamic

[31] strategy divides the time axis into three uneven slice windows by capturing the fluctuation of the number of tasks. Specifically, the size of the slice window gradually decreases as the number of tasks reached by the vehicle increases. Figure 8(a) to Figure 8(c) , the static strategy evenly divides the morning peak period into two slice windows. Figure 9(a) to Figure 9(c) ,The overall dynamic strategy divides the morning peak period into three slice windows of gradually decreasing length.

[0134] In contrast, the present invention obtains a fitting function by analyzing the relationship between the length of the slice window and the fluctuation size of the task flow, and the fitting function is used to predict the length of the next slice window T n (h+1) , the number of slice windows of different base stations can be dynamically adjusted during the morning peak hours. Figure 4(a) to Figure 4(c) , Figure 10(a) to Figure 10(c) The corresponding slice window size division selections made by the three base stations during the morning peak period according to the fitting function, and the corresponding slice allocation strategy performance. It can be seen that the growth trend of the task traffic received by base station 1 remains unchanged, so it is divided into two slice windows according to the fitting function and the growth of task traffic, and the slice allocation strategy in the corresponding window is also reflected through the performance isolation expectation. It can be seen that with the growth of task traffic, the performance isolation expectation is decreasing, but the window length is adjusted accordingly, so that the performance isolation expectation is always maintained above 86%. The task traffic of base station 2 has an obvious upward trend in the early morning peak, and then rises slowly as a whole under violent fluctuations. Therefore, according to the fitting function and the fluctuation of task traffic, the morning peak period is divided into three slice windows, and the window length decreases successively. It can be seen that the slice performance isolation expectation changes significantly during the window conversion, but remains at a high level overall. The task traffic of base station 3 is more violent than that of base station 1 and more gentle than that of base station 2. There is an obvious upward fluctuation in the middle, so it is divided into three slice windows to adapt to the task reception under different traffic.

[0135] From the above results, it can be seen that the proposed method can adjust the slice window length more obviously during the peak period of task traffic; when the task traffic fluctuates smoothly, the slice window length can be appropriately increased according to the adjustment of the slice window optimizer, thereby reducing the number of slice divisions and saving the overall overhead of the base station while ensuring the expected performance isolation quality.

[0136] 5.3 Slice Performance Isolation Analysis

[0137] This group of experiments investigated the impact of the total number of ground base station RBs on the expected quality of slice performance isolation, and tested 5 base stations. The number of physical RBs for all ground base stations is determined by the prediction results of the stacked LSTM model. The number of vehicle arrival tasks during the peak period of a certain day is selected and combined with the historical slice window size to allocate physical RBs and slice windows to the local base station. Figure 11(a) to Figure 11(c) Three base stations were selected as experimental objects. Due to the complex road network structure, the task traffic received by different base stations varies greatly (e.g. Figures 4(a) to 4(c) ), so a step-by-step increase is adopted when unifying the number of RBs. The impact of five different total numbers of RBs on the performance isolation quality expectations of base station slices was tested from 145 to 342. It can be seen that when the number of base station RBs is much smaller or much larger than the task traffic received by the base station, the performance isolation quality expectations of the slices will be constant at 0 or 1. When the total number of RBs is at a specific value, the performance of different base stations with different total numbers of RBs can be seen. When the total number of RBs is 310, the difference in performance isolation quality expectations of base stations 1, 2, and 3 can be clearly seen. In B n =310, the performance isolation quality expectations of the methods used by base station 1 are all above 0.8, while those of base station 2 are between 0.1 and 0.2, and those of base station 3 fluctuate between 0.6 and 0.8. The huge gap in the vehicle task traffic received by the three base stations leads to this phenomenon.

[0138] 5.4 Decision-making efficiency analysis

[0139] By observing the impact of different window partitioning strategies (static, overall dynamic or dynamic window) on the decision efficiency of training over a long time scale. Decision efficiency refers to the number of times resource reallocation is triggered within a given period of time. The number here is equivalent to the number of slice windows, which is determined by the slice window optimizer. Fig.12As shown, under the same performance isolation effect, the long-term decision cost of the dynamic window is lower than that of the static window. Although the decision cost of the dynamic window may be higher than that of the static window in a specific period of time (such as when the task traffic in the coverage area fluctuates greatly), from the perspective of long-term optimization (such as within a day), the decision cost of the proposed dynamic window division strategy is lower than that of the static window. Compared with Proposed-1, which adopts a static window strategy, Proposed-3 has a performance improvement of (8.33%-17.39%) in decision cost. Compared with Proposed-2, which adopts an overall dynamic window strategy, Proposed-3 has a performance improvement of (8.33%-19.04%) in decision cost. Compared with Baseline-4, which adopts a personalized window strategy, Proposed-3 has a performance improvement of (13.63%-19.79%) in decision cost. By Fig.12 It can be seen from the results that personalized window strategy can enhance the flexibility of RAN slices and balance control overhead and service provision.

[0140] 6. Conclusion

[0141] This paper proposes a dual-federation-based IoV resource slicing model. Simulation results show that compared with the baseline method, the proposed solution improves the performance isolation effect of slicing. The scenario described in this paper is replicable and universal, and is not limited by the number of base stations and vehicle traffic differences. In addition, the proposed slicing scheme is scalable and can be further used to support task offloading, content distribution, and data collection.

[0142] References

[0143] [1]Wang J,Fu T,Shangguan Q.Wide-area vehicle trajectory data based on advanced tracking and trajectory splicing technologies:Potentials intransportation research[J].Accident Analysis&Prevention,2023,186:107044.

[0144] [2]N.Zhang,S.Zhang,P.Yang,O.Alhussein,W.Zhuang,and X.S.Shen,“Softwaredefined space-air-ground integrated vehicular networks:Challenges andsolutions,”IEEE Commun.Mag.,vol.55,no.7,pp.101–109,2017.

[0145] [3]Pilz J,Mehlhose M,Wirth T,et al.A Tactile Internet demonstration:1ms ultra low delay for wireless communications towards 5G[C] / / 2016IEEEConference on Computer Communications Workshops(INFOCOM WKSHPS).IEEE,2016:862-863.

[0146] [4]Wan L,Guo Z,Wu Y,et al.4G\ / 5G spectrum sharing:efficient 5Gdeployment to serve enhanced mobile broadband and internet of thingsapplications[J].ieee vehicular technology magazine,2018,13(4):28-39.

[0147] [5]Li Y.Deep reinforcement learning:An overview[J].arXiv preprintarXiv:1701.07274,2017.

[0148] [6]Li L,Fan Y,Tse M,et al.A review of applications in federatedlearning[J].Computers&Industrial Engineering,2020,149:106854.

[0149] [7]Zhou L,Chen B,Liu H,et al.Personalized sliding windowrecommendation algorithm based on sequence alignment[J].Entropy,2022,24(11):1662.

[0150] [8]N.Zhang,S.Zhang,P.Yang,O.Alhussein,W.Zhuang,and X.S.Shen,“Softwaredefined space-air-ground integrated vehicular networks:Challenges andsolutions,”IEEE Commun.Mag.,vol.55,no.7,pp.101–109,2017.

[0151] [9]J.Li,W.Shi,P.Yang,Q.Ye,X.S.Shen,X.Li,and J.Rao,“A hierarchicalsoft RAN slicing framework for differentiated service provisioning,”IEEEWireless Commun.,vol.27,no.6,pp.90–97,2020.

[0152]

[10] Richart M,Baliosian J,Serrat J,et al.Resource slicing in virtualwireless networks:Asurvey[J].IEEE Transactions on Network and ServiceManagement,2016,13(3):462-476.

[0153]

[11] Li R,Zhao Z,Sun Q,et al.Deep reinforcement learning for resourcemanagement in network slicing[J].IEEE Access,2018,6:74429-74441.

[0154]

[12] Ito K,Hirakawa G,Arai Y,et al.A road condition monitoring systemusing various sensor data in vehicle-to-vehicle communication environment[J].International Journal of Space-Based and Situated Computing,2016,6(1):21-30.

[0155]

[13] Treiber M,Kesting A.Traffic flow dynamics[J].Traffic FlowDynamics:Data,Models and Simulation,Springer-Verlag Berlin Heidelberg,2013,227:228.

[0156]

[14] Hebert L,Golab L,Poupart P,et al.Fedformer:Contextual federationwith attention in reinforcement learning[J].arXiv preprint arXiv:2205.13697,2022.

[0157]

[15] Tao C,Gao S,Shang M,et al.Get The Point of My Utterance!LearningTowards Effective Responses with Multi-Head Attention Mechanism[C] / / IJCAI.2018:4418-4424.

[0158]

[16] Wang H,Zhang X,Xia Y,et al.An intelligent blockchain-based accesscontrol framework with federated learning for genome-wide association studies[J].Computer Standards&Interfaces,2023,84:103694.

[0159]

[17] Han B,Gopalakrishnan V,Ji L,et al.Network functionvirtualization:Challenges and opportunities for innovations[J].IEEEcommunications magazine,2015,53(2):90-97.

[0160]

[18] Kreutz D,Ramos F M V,Verissimo P E,et al.Software-definednetworking:Acomprehensive survey[J].Proceedings of the IEEE,2014,103(1):14-76.

[0161]

[19] Wang J,Peng B,Zhang X.Using a stacked residual LSTM model forsentiment intensity prediction[J].Neurocomputing,2018,322:93-101.

[0162]

[20] Werbos P J.Backpropagation through time:what it does and how todo it[J].Proceedings of the IEEE,1990,78(10):1550-1560.

[0163]

[21] Gower R M,Loizou N,Qian X,et al.SGD:General analysis and improvedrates[C] / / International conference on machine learning.PMLR,2019:5200-5209.

[0164]

[22] Morgan J A,Tatar J F.Calculation of the residual sum of squaresfor all possible regressions[J].Technometrics,1972,14(2):317-325.

[0165]

[23] Filar J,Vrieze K.Competitive Markov decision processes[M].Springer Science&Business Media,2012.

[0166]

[24] Du Y,Chen J,Zhao C,et al.Comfortable and energy-efficient speedcontrol of autonomous vehicles on rough pavements using deep reinforcementlearning[J].Transportation Research Part C:Emerging Technologies,2022,134:103489.

[0167]

[25] Barron E N,Ishii H.The Bellman equation for minimizing themaximum cost[J].NONLINEAR ANAL.THEORY METHODS APPLIC.,1989,13(9):1067-1090.

[0168]

[26] Su Z,Wang Y,Luan T H,et al.Secure and efficient federatedlearning for smart grid with edge-cloud collaboration[J].IEEE Transactions onIndustrial Informatics,2021,18(2):1333-1344.

[0169]

[27] Zeyer A,Bahar P,Irie K,et al.A comparison of transformer and lstmencoder decoder models for asr[C] / / 2019 IEEE Automatic Speech Recognition andUnderstanding Workshop(ASRU).IEEE,2019:8-15.

[0170]

[28] Korel B.Computation of dynamic program slices for unstructuredprograms[J].IEEE Transactions on Software Engineering,1997,23(1):17-34.

[0171]

[29] Xu Q,Jeon H,Kim K,et al.Warped-slicer:Efficient intra-SM slicingthrough dynamic resource partitioning for GPU multiprogramming[J].ACM SIGARCHComputer Architecture News,2016,44(3):230-242.

[0172]

[30] Yu Q,Wei N,Mi Y.Improvement of federal average algorithm based onCRITIC weighting method[C] / / 2023 2nd International Conference on ArtificialIntelligence and Intelligent Information Processing(AIIIP).IEEE,2023:164-168.

[0173]

[31] Shen H,Tian Y,Wang T,et al.Slicing-based task offloading inspace-air-ground integrated vehicular networks[J].IEEE Transactions on MobileComputing,2023,23(5):4009-4024.

Claims

1. A vehicle networking resource slicing method based on Transformer dual federated learning. In the vehicle networking system, the physical resources of the base station are abstracted into a virtual resource pool and managed by the controller. According to the terminal request information fed back by each base station, the controller adjusts the slice resource allocation; its characteristics are The physical resource blocks (RBs) on each base station are virtualized into multiple slices and a resource sharing area. When the resources of a slice are insufficient, the resources in the sharing area are temporarily occupied. The controller is a two-layer controller structure, in which a centralized controller and a local controller are deployed on the cloud and edge side respectively; The centralized controller integrates a global model, which includes a global traffic prediction model and a global resource slicing decision model; the centralized controller performs base station resource slicing through the global model; A local controller connected to a neighboring base station is integrated with a local model, which includes a local traffic prediction model, a slice window optimizer, and a local resource slice decision model; The local controller makes resource slicing decisions based on the local model; The slicing steps include: (1) Using a window-adaptive segmentation method to segment the physical resources of each base station; (ii) Optimize decision-making results through local model training and dual federated model methods.

2. The vehicle network resource slicing method based on Transformer dual federated learning according to claim 1 is characterized in that In step (1), soft indicators are used to measure the slice isolation quality, and the slice isolation quality maximization problem is modeled as a joint optimization problem of slice window adjustment, resource slicing and resource sharing area setting, specifically: For base station n, the number of slice windows is H n , the time slots included in the slice window h are T n (h) At the end of the slice window h-1, the local controller on base station n re-determines T n (h) and the number of serving slices M n ; The total number of resource blocks (RBs) held by base station n is denoted as B n ; In slice window h, the number of RBs allocated to slice m and shared area are expressed as and satisfy At the beginning of slice window h, the resources of each slice and the resources of the shared area are redivided; within window h, the number of RBs held by each slice and the shared area remains unchanged; Performance isolation requires that the overload of one slice does not affect other slices; if the actual resource demand of slice m in time slot t is The number of RBs occupied by the slice in the resource sharing area on base station n is calculated as Assume that at the beginning of window h, the number of RBs reserved by base station n for the shared area is The performance isolation quality of base station n at time slot t is normalized to exist That is, when all shared areas cannot meet resource requirements, the slice performance isolation is violated. At this time, ρ n,t =0; That is, if the shared area is sufficient to cope with resource pressure, The resource blocks in the shared area will be temporarily occupied, ρ n,t ∈[0,1), which means that the performance isolation is "discounted" at this time; when When slice m can meet the current demand without occupying the resources in the shared area, then ρ n,t =1; For base station n, given by slice window h The slice performance isolation quality maximization problem is modeled as Constraint (1) ensures that the resource blocks allocated to the slice and shared area do not exceed the total number of resource blocks held by the base station; Extending to multiple consecutive slice windows, the long-term optimization problem of the performance isolation quality of base station n is modeled as question The essence of this is how to adjust the slice window length and allocate resources between slices and shared areas to maximize the long-term performance isolation quality.

3. The vehicle network resource slicing method based on Transformer dual federated learning according to claim 1 is characterized in that In step (2), the local model training is: The traffic prediction model of the local controller of base station n predicts the task traffic of the base station, and the prediction result is expressed as X n ;X n Input to the slice window optimizer to determine the time slot of the slice window h That is, the length of the slice window; based on X n and The local resource slice decision model determines the resource allocation of slices and shared areas, and the corresponding decision vector is expressed as For local traffic prediction models: The historical traffic flow data on base station n is used to train the local traffic flow prediction model based on stacked LSTM to predict the traffic flow in the future. In the training process of the local traffic flow prediction model, the training round o set and cardinality are represented as and The local parameters of the model are represented as At the beginning of training, base station n initializes the local parameters of the stacked LSTM Assume that the number of stacked LSTM layers is G; the input of the first layer is the original data sequence X = [x1, x2, ..., x O ], the output is in Represents the oth training round vector The hidden state of the first layer; the output of the Gth layer is The local traffic prediction model is denoted as X(·); the number of resource blocks required for slice m on base station n in window h is predicted as According to the predicted value output by the model during training and the target value X, the loss function is described as The gradient of the back propagation process is calculated using the time series back propagation algorithm; about the local parameters The gradient of is calculated as Using the stochastic gradient descent algorithm SGD, Updated to where η n is the learning rate; For the slice window optimizer: A slice window optimizer is set for each base station; each optimizer receives the prediction result X n After that, bring it in Get the corresponding slice window length result For the resource slicing decision model: The long-term optimization problem of performance isolation quality is simplified to a short-term optimization under discrete windows, which is a Markov decision process MDP problem with a finite time domain; in this problem, resource slicing refers to a local controller dividing the resources held by base station n into M under a given window length. n slices and a shared area to maximize the process of isolating the quality of the slice performance; The output of the local traffic prediction model and window optimizer is fed into the DRL model to help it generate the best resource slicing decision; The resource slicing decision model of base station n is abstracted as a DRL agent; under the slicing window h, the set and cardinality of the model iteration rounds are represented as ε n and E n ; Iteration round e∈ε n The receiving state of the environment is denoted by s n,e , the resource allocation action for slices and shared areas is recorded as a n,e , the reward given by the environment is recorded as r n,e ; The model is in state s n,e Under the condition of n,e+1 |s n,e ,{a n,1 ,a n,2 ,...,a n,e }), the environment state is updated to s by the corresponding local controller n,e+1 ; Expressions related to training include: State space: resource slices on base station n depend on X n and In the iteration round e, the corresponding state vector is expressed as Action space: In iteration round e, the allocation action made by the agent when slicing resources is recorded as In the formula is the decision set of iteration round e, i.e., the local controller allocates slice resource blocks to the base station according to the action; the decision variable of each action is 0, 1 or 2, which is determined by the current state; Reward: The total reward value in iteration round e is recorded as Among them, ρ n,t is used to quantify the reward obtained by completing the task in time slot t; In the state space, the agent estimates the reward for each action and stores it in the Q table; the action value function is recorded as The maximum reward for each state in the Q table represents the maximum possible reward in the future. By querying the Q table, the agent determines the action with the maximum reward in each state. Substituting the Bellman equation into the following formula, the vector in the Q table is expressed as Where γ n is the learning rate, ν is the greedy probability; After the local model is trained, the window optimizer and the local resource slicing decision model will eventually play a role. n and Assist the DRL agent to generate service slices and allocation results of shared area RBs.

4. The vehicle network resource slicing method based on Transformer dual federated learning according to claim 3 is characterized in that In step (2), 2.2) dual federation model training: 2.2.1) Federated model training for traffic prediction The centralized controller uses the serial federation mode to sequentially receive the parameters uploaded by the local model, aggregate them, and update the global model; The centralized controller integrates the Transformer encoder TE and decoder TD to optimize the uploaded parameters during the aggregation process; TE uploads the parameters of base station n Encoded into an input sequence, which is stacked over multiple layers to generate Q n , K n , V n , and adapt to the multi-head attention mechanism in Transformer; The global traffic prediction model parameters are initialized as It is used by base station n to train the local prediction model and is updated as Defining the similarity function in Under the multi-head attention mechanism, the weight of the local traffic prediction model is calculated as Substituting the similarity function into the above formula, the attention weight of the prediction model is optimized as Using the attention weights, the global prediction model parameters are weighted and updated as Then, the training parameters of the local traffic prediction model of each base station are fed back to the global model in sequence, so that the global model is continuously optimized, and the optimized parameters are fed back to the local model; 2.2.2) Prediction-enhanced federated resource slice decision model training: The parameters of the local resource slicing decision model of base station n in iteration round e are expressed as The global resource slice decision model parameters of the base station in the system are initialized as Local resource slicing decision model usage Perform local training to obtain updated local model parameters Under the attention mechanism, the weight of base station n is calculated as The global resource slicing decision model parameters are weighted and updated as After receiving the optimized prediction result X′ n and After that, the global resource slice decision model parameter aggregation process is as follows: each base station is updated in sequence, and the global resource slice decision model gradually receives updates from each base station; after the local model training of each base station is completed, the parameters Through the multi-head attention mechanism and the previous aggregation parameters Perform weighted updates to generate new aggregation parameters; the parameters of the multi-head attention mechanism are aggregated after each round of training, and the final aggregation parameters Used to update the global Q network 5. The vehicle network resource slicing method based on Transformer dual federated learning according to claim 1 is characterized in that With the help of the local traffic prediction model, the training process of the local resource slicing decision model is as follows: At each time step t, the current traffic forecast value X n and the predicted value X′ for the next time step n Integrate into state n,e and s' n,e In this way, the DRL model can foresee the future traffic change trend when making decisions; By using future traffic predictions, the local resource slicing decision model can react to upcoming traffic fluctuations in advance and allocate base station resources efficiently; According to ε n The greedy strategy selects action a n,e , with probability ε n Random Selection Otherwise select After each action is taken, the state s' is updated by receiving the new traffic prediction value n,e .

Images