Dynamic data synchronization method in air-ground integrated network enabled by digital twin

Abstract

The present invention proposes a method for dynamic data synchronization in an air-ground integrated network enabled by digital twins, which mainly solves the problem that the existing technology cannot transmit data in remote areas, resulting in low quality of vehicle network services. Its implementation scheme is: 1) Constructing a heterogeneous network model enabled by digital twins; 2) Calculating the data transmission rate and delay of vehicles accessing different network links; 3) Establishing a heterogeneous network selection problem for the vehicle digital twin based on the vehicle's transmission data cost and delay; 4) Using a DQN-based method to obtain the optimal decision for the heterogeneous network selection problem, so that the vehicle can access a set of optimal networks for data synchronization according to the decision. By providing vehicle users with heterogeneous network link access decisions, the present invention can complete data synchronization with maximum benefits while fully considering the user's own data synchronization requirements and the status of heterogeneous network resources, thereby improving the quality of vehicle network services and meeting the diverse business needs of users, and can be used in intelligent transportation systems.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking, and in particular relates to a dynamic data synchronization method which can be used in an intelligent transportation system. Background Art

[0002] The combination of the sixth-generation integrated air-ground-space network and the Internet of Vehicles can provide ubiquitous in-vehicle services for vehicles. However, with the access of a large number of devices, each vehicle will generate a large amount of private data, such as operating status and driving habits, and the existing integrated air-ground-space network cannot efficiently manage the data with diverse characteristics in the network. On the other hand, the services and applications in the vehicle require a large number of nodes to occupy a large amount of resources for frequent data interaction to formulate optimization plans and decisions, such as traffic management and resource allocation, thereby reducing the utilization rate of resources in the network.

[0003] Digital twins, as a technology that maps the state and parameters of physical entities to digital networks, are expected to solve the above problems in the integrated air-space-ground network. Specifically, by establishing a digital twin for the vehicle, the twin can map the vehicle's private data so that the vehicle data can be efficiently managed and analyzed while ensuring driving safety. In addition, in the digital twin network, the twin of the vehicle and the twin of the infrastructure can make interactive decisions across time, thereby efficiently formulating optimization plans for the nodes in the physical network. Based on the above advantages, the integrated air-space-ground network enabled by digital twins has received widespread attention from academia and industry.

[0004] In the air-ground integrated network enabled by digital twins, vehicles need to comprehensively consider the data synchronization requirements and the resource status of the available network to update the new parameters and status generated by driving and interaction to their twins deployed in the cloud in real time. In this process, the vehicle may be covered by multiple networks at the same time, such as satellites, drones, and CBSs. Therefore, the vehicle needs to choose the best network access according to its preferences to complete data synchronization. On the other hand, as the vehicle moves at high speed, the vehicle may switch from the current network to other networks. Therefore, how to continuously access a set of optimal networks to complete data synchronization and maximize its benefits during the process of driving from the starting point to the end point has become a key challenge that needs to be solved.

[0005] The patent with application publication number CN202010310406.5 discloses "a method and system for selecting heterogeneous Internet of Vehicles networks based on multi-constraint utility functions". It first processes the vehicle request information and network status information uploaded to the data center server, constructs a network attribute parameter initialization decision matrix, and uses constraint functions for parameter reduction processing. Then, a comprehensive utility function model is established to make multi-attribute decisions on the set of alternative networks from subjective and objective perspectives to obtain the maximum network comprehensive utility value. Finally, the switching threshold is used to reduce unnecessary network switching decisions to ensure the optimal network selection mechanism.

[0006] Although this method can effectively reduce the number of network switching and balance the network load, it still has the following shortcomings:

[0007] 1) This method only considers roadside units of various types of networks as heterogeneous vehicle networks, without considering actual road conditions. When vehicles are traveling in remote areas without ground signal coverage, they cannot access the network to complete data transmission.

[0008] 2) This method is aimed at static access and switching decisions, does not consider interactive decisions across time, and cannot realize dynamic network access and business needs for vehicle users.

[0009] 3) This method only considers the performance indicators of the vehicle network, such as network load and signal strength, and does not consider preferences from the user's perspective, which can easily lead to a decline in the quality of vehicle network services. Summary of the invention

[0010] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a dynamic data synchronization method in an integrated air-space-ground network enabled by digital twins to solve the problem of being unable to transmit data in remote areas, provide dynamic network link access decisions for vehicle users, improve the quality of vehicle network services, and meet the diverse business needs of users.

[0011] To achieve the above object, the technical solution adopted by the present invention includes the following steps:

[0012] (1) Building a digital twin-enabled air-ground integrated network model:

[0013] Construct a DT system consisting of N vehicles, M network access facilities, and N vehicle digital twins. i and M network access facility digital twins DT j The digital twin of the integrated air-space-ground network system is enabled by the network access facility, each of which consists of a cellular base station b, a drone u and a satellite s.

[0014] (2) Calculate the data transmission rate and delay when the vehicle accesses different network links:

[0015] 2a) Calculate the transmission rate r of vehicle i accessing cellular base station b respectively i,b , the transmission rate r of vehicle i accessing drone u i,u , the transmission rate r of drone u accessing the control center c u,c , the transmission rate r of vehicle i accessing satellite s i,s , the transmission rate r of satellite s accessing ground station g s,g ;

[0016] 2b) According to the data size D that the vehicle needs to synchronize in each time slot t i (t), calculate the data transmission delay T of the three transmission links from vehicle i to the cellular base station b, the control center c, and the ground station respectively i,b (t), T i,c (t), T i,g (t);

[0017] (3) Heterogeneous network selection problem for establishing vehicle digital twins:

[0018] 3a) The driving time of vehicle i from the starting point to the end point is divided into W time slots. In each time slot t, vehicle i is connected to the available network link m. i (t) Select a network to connect to and select the data size D to be synchronized i (t);

[0019] 3b) Using the matrix L = [l ij ] represents the connection relationship between vehicle i and network access facility j. If vehicle i is connected to network facility j, then l ij =1, otherwise l ij =0;

[0020] 3c) Cost of transmitting data according to the vehicle ij (t) and time delay T ij (t) The heterogeneous network selection problem of vehicle i is expressed as:

[0021]

[0022]

[0023]

[0024] Among them, ij (t) = φ j ·T ij (t),φ j represents the unit time access cost of network access facility j, represents the weight factor, l ij Represents a Boolean variable, which can only have two states: 0 and 1;

[0025] (4) Using the DQN-based method to obtain the optimal strategy for heterogeneous network selection problem:

[0026] (4a) Model the heterogeneous network selection problem as a Markov decision process, that is, define its state space, action space and reward:

[0027] State space: including vehicle position loc i (t), data synchronization delay T ij (t), and the remaining data volume DR that vehicle user i needs to synchronize i (t), expressed as S(t) = {loc i (t),T ij (t),DR i (t)};

[0028] Action space: It includes the vehicle’s choice of currently accessible network links and the amount of data to be transmitted. The twin action space of vehicle i is represented by A(t) = {L i (t),D i (t)}, where L i (t)∈m i (t), D i (t)∈{0,1,2}, where 0, 1, and 2 represent no data transmission, 1M data transmission, and 2M data transmission, respectively;

[0029] Reward: refers to the reward that vehicle i will receive from the environment after performing the action at time slot t: R(t) = λ1r(t) - λ2p(t),

[0030] in, is the reward function, is the penalty factor, D i represents the total amount of data to be synchronized for vehicle i, λ1 and λ2 are the scaling factors of reward and penalty respectively;

[0031] (4b) DQN is used to find the optimal solution for the Markov decision process. That is, according to the defined state space, action space and reward, the DQN algorithm is used to make decisions on the link access and synchronization data amount of vehicle i in each time slot to maximize the action utility at each moment.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] 1. Realize real-time grasp of global information

[0034] The air-ground-space integrated network framework proposed in the present invention takes into account two types of digital twins, namely the digital twin of the vehicle and the digital twin of the network infrastructure, to obtain global information about the vehicle and the driving environment, and to achieve real-time control of user needs, road environment, and channel conditions. Therefore, when the vehicle has a data synchronization requirement, the digital twin of the vehicle can make the optimal decision on dynamic network link access and data transmission volume during the mobile process based on the global information in the cloud, and then send the decision to the corresponding vehicle, achieving real-time control of global information.

[0035] 2. Improved service quality of heterogeneous networks

[0036] The present invention establishes an optimal link access model in heterogeneous networks. Since the user's preference for latency and cost is taken into account from the user's perspective, and the dynamics of the wireless network, the mobility of the vehicle and the characteristics of the heterogeneous access link are comprehensively considered, the vehicle data synchronization optimization problem is modeled as a Markov decision process MDP, and the reward of the MDP is designed according to the user's preference, thereby improving user satisfaction.

[0037] 3. Optimize the data synchronization process

[0038] The present invention solves the MDP problem by designing a solution based on the deep Q-network DQN, making decisions on the link access and synchronization data volume of the vehicle in each time slot, thereby making the optimal link access decision for each vehicle during the journey, realizing cross-time interactive decision-making, completing data synchronization and maximizing benefits. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of the implementation of the present invention;

[0040] Figure 2 It is a schematic diagram of the air-ground integrated network architecture enabled by digital twins in the present invention;

[0041] Figure 3 This is the optimal data synchronization architecture diagram based on DQN in the present invention;

[0042] Figure 4 It is a schematic diagram of the normalized loss function of the deep neural network training process in the present invention;

[0043] Figure 5 It is a comparison diagram of simulation results of vehicle rewards obtained by changing the amount of synchronized data in the present invention;

[0044] Figure 6 It is a comparison chart of simulation results of vehicle rewards obtained by changing the base station density in the present invention. DETAILED DESCRIPTION

[0045] The embodiments and effects of the present invention are further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0046] Reference Figure 1 , the implementation steps of this example are as follows.

[0047] Step 1: Build an integrated air-space-ground network model enabled by digital twins.

[0048] Reference Figure 2 The construction of the space-ground integrated network in this step includes N vehicles, M network access facilities, and the digital twins DT of N vehicles. i , Digital twins DT of M network access facilities i DT j (t) = {C j ,φ j ,loc j},in:

[0049] Each network access facility consists of a cellular base station b, a drone u, and a satellite s;

[0050] Digital twin DT of each vehicle i It is expressed as:

[0051] Digital twin DT of each network access facility i Expressed as: DT j (t) = {C j ,φ j ,loc j}DT i , where is the operating status of vehicle i at time t, such as transmission power, amount of data to be synchronized, and accessible network at time t; i and E i Respectively represent the departure and destination of the vehicle, is the preference of vehicle i for delay and cost, loc i (t) is the position of vehicle i, C j and φ j denote the communication capacity and unit bandwidth resource cost of infrastructure j, loc j is the location of infrastructure j.

[0052] Step 2: Calculate the data transmission rate and delay when the vehicle accesses different network links.

[0053] 2.1) Calculate the transmission rate r of vehicle i accessing cellular base station b i,b :

[0054]

[0055] Among them, B b is the bandwidth allocated by base station b to vehicle i, γ i,b is the signal-to-noise ratio of vehicle i accessing base station b, h b is the antenna gain of the base station, P i represents the transmission power of vehicle i, d i,b is the distance between vehicle i and base station b, L0 and α are the unit distance path loss and path loss index, respectively, σ 2 is the Gaussian white noise power;

[0056] 2.2) Calculate the transmission rate r of vehicle i accessing drone u i,u :

[0057]

[0058] Among them, B u is the bandwidth allocated by UAV u to vehicle i, h u is the antenna gain of the UAV, d i,u is the distance between vehicle i and drone u;

[0059] 2.3) Calculate the transmission rate r of vehicle i accessing satellite s i,s :

[0060]

[0061] Among them, B s represents the bandwidth allocated by the satellite to the vehicle, N0 is the noise power spectral density, is the vehicle's transmission power gain, is the free space loss, represents rain attenuation, is the satellite’s antenna gain, represents the maximum antenna gain of the satellite, θ i is the off-axis angle of vehicle i relative to the maximum radiation of the satellite beam, θ 3dB is the half-power angle of the transmitting antenna, c and f are the speed of light and the communication center frequency, respectively, and d i,s is the distance between the vehicle user and the satellite, is a constant, η is the rainfall intensity, ρ and β are two different frequency correlation coefficients, and the calculation formula is:

[0062]

[0063]

[0064] in, is the vehicle's path elevation angle, ρ is the polarization tilt angle, K H and K V is the regression coefficient, whose value is related to the communication center frequency and can be obtained from the known ITU-R P.838 Report table.

[0065] Report Form ITU-R P.838

[0066]

[0067] 2.4) Calculate the transmission rate r of drone u accessing the control center c u,c :

[0068]

[0069] Among them, B c represents the bandwidth allocated by the control center to the UAV, P u is the transmission power of UAV u, h c is the antenna gain of the controller c, d u,c is the distance between the UAV and the control center, L0 is the path loss per unit distance, σ 2 is the Gaussian white noise power;

[0070] 2.5) Calculate the transmission rate r of satellite s accessing ground station g s,g :

[0071]

[0072] Among them, B g represents the bandwidth allocated to the satellite by the ground station, P s is the satellite's transmission power, and are the satellite’s transmission power gain and the ground station’s receiving power gain, and Represent free space loss and rain attenuation respectively, and the formula is as follows:

[0073]

[0074]

[0075] In the formula, is a constant, η is the rainfall intensity;

[0076] 2.6) According to the data size D that the vehicle needs to synchronize in each time slot t i (t) Calculate the data transmission delay T of the three transmission links from vehicle i to the cellular base station b, the control center c, and the ground station. i,b (t), T i,c (t), T i,g (t):

[0077]

[0078]

[0079]

[0080] Step 3, establish the heterogeneous network selection problem of the vehicle digital twin.

[0081] In order to enable the vehicle to select the best network access during driving and maximize benefits, this step constructs the heterogeneous network access selection problem of the vehicle during movement by comprehensively considering the vehicle user's preference for cost and latency. The specific implementation is as follows:

[0082] 3.1) Divide the driving time of vehicle i from the starting point to the end point into W time slots. In each time slot t, vehicle i has an available network link m. i (t) Select a network to connect to and select the data size D to be synchronized i (t);

[0083] 3.2) Using the matrix L = [l ij ] represents the connection relationship between vehicle i and network access facility j. If vehicle i is connected to network facility j, then l ij =1, otherwise l ij =0;

[0084] 3.3) Cost of transmitting data according to the vehicle ij (t) and time delay T ij (t) The heterogeneous network selection problem of vehicle i is expressed as:

[0085]

[0086]

[0087]

[0088] Among them, ij (t) = φ j ·T ij (t),φ j represents the unit time access cost of network access facility j, represents the weight factor, l ij Represents a Boolean variable, which can only have two states: 0 and 1, which means that the vehicle can only access one network in each time slot.

[0089] Step 4: Use the DQN-based method to obtain the optimal strategy for the heterogeneous network selection problem.

[0090] In order to achieve dynamic decision-making across time, this step uses the DQN algorithm to make decisions on the link access and synchronization data volume of vehicle i in each time slot to maximize the benefits. The specific implementation is as follows:

[0091] (4.1) The heterogeneous network selection problem is modeled as a Markov decision process, that is, its state space, action space and reward are defined:

[0092] Define the state space: including the vehicle position loc i (t), data synchronization delay T ij (t), and the remaining data volume DR that vehicle user i needs to synchronize i (t), expressed as S(t) = {loc i (t),T ij (t),DR i (t)};

[0093] Define the action space: Since the digital twin of the vehicle selects the currently accessible network link and determines the amount of data to be transmitted in each time slot according to the current channel state and vehicle user preference, the action space of the twin of vehicle i can be expressed as A(t) = {L i (t),D i (t)}, where L i (t)∈m i (t), D i (t)∈{0,1,2}, where 0, 1, and 2 represent no data transmission, 1M data transmission, and 2M data transmission, respectively;

[0094] Reward: refers to the reward that vehicle i will get from the environment after performing an action at time slot t: R(t) = λ1r(t) - λ2p(t), where Reward: refers to the reward that vehicle i will get from the environment after performing an action at time slot t:

[0095] R(t)=λ1r(t)-λ2p(t)

[0096] in, is the reward function, which is set according to the constructed network access selection problem;

[0097] is a penalty factor, which is set to address the problem that when a vehicle reaches its destination and fails to complete data synchronization, the vehicle may need to stop and continue to access the network, resulting in a poor experience, so as to encourage the vehicle to complete data synchronization before reaching the destination;

[0098] D i represents the total amount of data to be synchronized for vehicle i, λ1 and λ2 are the scaling factors of reward and penalty respectively;

[0099] (4.2) Use DQN to find the optimal solution for the Markov decision process:

[0100] Reference Figure 3 ,This step is to use the DQN algorithm to make decisions on the link access and synchronization data volume of vehicle i in each time slot based on the defined state space, action space and reward, so as to maximize the action utility at each moment. The specific implementation is as follows:

[0101] This step uses the DQN algorithm to make decisions on the link access and synchronization data volume of vehicle i in each time slot based on the defined state space, action space and reward, so as to maximize the action utility at each moment. The specific implementation is as follows:

[0102] (4.2.1) The action utility function Q is iteratively updated. In each iteration, the update process of the Q function value is:

[0103]

[0104] Where α∈[0,1] is the learning rate, S(t), A(t) and R(t) represent the state, action and reward at time t, respectively, 0<δ<1 is the discount factor, S(t+1) and A(t+1) represent the state and action at the next moment, respectively;

[0105] (4.2.2) A deep neural network with weights {θ} is used as a function approximator to fit the Q function, that is, Q(S(t), A(t))≈Q'(S(t), A(t); θ), so that the predicted Q'(S(t), A(t); θ) is infinitely close to the actual Q tar :

[0106] During the learning process, the O interactions stored in the replay buffer of DQN are used to improve learning efficiency. Each interaction is represented as {S(t), A(t), R(t), S(t+1)};

[0107] During the update process, a small batch of interactions is randomly sampled in the replay buffer and the weights θ of the Q network are trained to minimize the loss function of the deep neural network:

[0108] Loss(θ)=E(Q tar -Q'(S(t),A(t);θ)) 2

[0109] in,

[0110] (4.2.3) Using the weight θ that minimizes the loss function of the deep neural network, the action with the greatest utility in state S(t) is:

[0111]

[0112] Among them, Q'(S(t), A(t); θ) is the prediction result of the neural network output after training, a opt (t) is the optimal link access and synchronization data volume action decision at time t;

[0113] According to the decision, vehicle i continuously accesses a set of optimal networks to complete data synchronization and maximize its benefits.

[0114] The technical effects of the present invention are further described below in conjunction with simulation experiments.

[0115] 1. Simulation conditions

[0116] A ground network consisting of 10 base stations, 6 UAVs, and 1 low-Earth satellite is set up on a road with a length of 15 km from the starting point to the end point, and their coverage radii are 0.5 km, 1 km, and 15 km respectively.

[0117] The platform for the simulation experiment is: Windows 10 operating system and Python 3.7.

[0118] The simulation experiment parameter settings are shown in Table 1:

[0119] Table 1

[0120]

[0121] Among them, h b and h u Represent the antenna gains of the base station and the UAV respectively; B b , B u and B s represent the bandwidth allocated by the base station to the vehicle, by the drone to the vehicle, and by the satellite to the vehicle; B c and B g They represent the bandwidth allocated by the control center to the UAV and the bandwidth allocated by the ground station to the satellite respectively; L0 represents the path loss per unit distance; α represents the path loss exponent; P i ,P u and P s Respectively represent the transmission power of vehicles, drones and satellites; σ 2 represents the Gaussian white noise power transmitted by the base station and the UAV; N0 represents the noise power spectral density transmitted by the satellite; is the vehicle’s transmission power gain; represents the maximum antenna gain of the satellite; f represents the communication center frequency; η is the rainfall intensity.

[0122] 2. Simulation content

[0123] In the above scenario, the rewards obtained by the vehicles of the present invention and the existing solutions FSS and RSS are simulated respectively, where:

[0124] The FSS means that the vehicle connects to the base station in the first 1 / 3 of the journey, connects to the UAV in the middle 1 / 3 of the journey, and connects to the satellite in the last 1 / 3 of the journey.

[0125] The RSS refers to the random selection of access links by the vehicle in the action space at each time slot.

[0126] Simulation 1: In the above scenario, the normalized loss function of the deep neural network DNN training process is obtained. The results are as follows: Figure 4 .from Figure 4 It can be seen that after a certain number of iterations, the loss function tends to zero, indicating that the DQN-based data synchronization scheme proposed in the present invention can reach a convergence state, that is, the optimal access strategy for each time slot is found, which illustrates the effectiveness of the scheme.

[0127] Simulation 2: In this scenario, by changing the amount of data that the vehicle needs to synchronize, the rewards obtained by the vehicle of the present invention and the existing solutions FSS and RSS are simulated respectively. The results are as follows: Figure 5 .

[0128] from Figure 5 It can be seen that as the amount of synchronized data increases, the rewards obtained by the vehicle in all schemes first increase and then decrease. This is because as the amount of data increases, the data transmitted before the vehicle reaches the destination will also increase, thereby increasing the reward of the vehicle. However, when the amount of synchronized data is large, the data may not be fully transmitted, resulting in penalties.

[0129] from Figure 5 It can also be seen that the present invention can obtain the highest reward. This is because: the present invention can not only select the best network for the vehicle to obtain a higher reward, but also transmit as much data as possible to avoid penalties.

[0130] Simulation 3: In the above scenario, by changing the number of cellular base stations in the network, the rewards obtained by the vehicles of the present invention and the existing solutions FSS and RSS are simulated respectively. The results are as follows: Figure 6 .

[0131] from Figure 6 It can be seen that as the number of base stations increases, the rewards for vehicles in all scenarios are on the rise. In particular, when the number of base stations reaches 6, the reward value tends to stabilize. There are two reasons for this: first, as the number of base stations increases, selecting base stations can achieve low-cost and low-latency data transmission, thereby obtaining higher rewards; second, when the number of base stations is 6, the network coverage of the entire road is almost achieved. At this time, increasing the number of base stations will not affect the vehicle's route selection.

[0132] from Figure 6 It can also be seen that the present invention obtains the highest reward.

[0133] In summary, compared with the above-mentioned existing FSS and RSS methods, the present invention can bring higher rewards to the vehicle.

Claims

1. A method for synchronizing dynamic data in an air-ground-space integrated network enabled by digital twins, characterized in that: The steps include: (1) Building a digital twin-enabled air-ground integrated network model: Build a DT system consisting of N vehicles, M network access facilities, and N vehicle digital twins i and M network access facility digital twins DT j The digital twin of the integrated air-space-ground network system is enabled by the network access facility, each of which consists of a cellular base station b, a drone u and a satellite s. (2) Calculate the data transmission rate and delay when the vehicle accesses different network links: 2a) Calculate the transmission rate r of vehicle i accessing cellular base station b respectively i,b , the transmission rate r of vehicle i accessing drone u i,u , the transmission rate r of drone u accessing the control center c u,c , the transmission rate r of vehicle i accessing satellite s i,s , the transmission rate r of satellite s accessing ground station g s,g ; 2b) According to the data size D that the vehicle needs to synchronize in each time slot t i (t), calculate the data transmission delay T of the three transmission links from vehicle i to the cellular base station b, the control center c, and the ground station respectively i,b (t), T i,c (t), T i,g (t); (3) Heterogeneous network selection problem for establishing vehicle digital twins: 3a) The driving time of vehicle i from the starting point to the end point is divided into W time slots. In each time slot t, vehicle i is connected to the available network link m. i (t) Select a network to connect to and select the data size D to be synchronized i (t); 3b) Using the matrix L = [l ij ] represents the connection relationship between vehicle i and network access facility j. If vehicle i is connected to network facility j, then l ij =1, otherwise l ij =0; 3c) Cost of transmitting data according to the vehicle ij (t) and time delay T ij (t) The heterogeneous network selection problem of vehicle i is expressed as: Among them, ij (t) = φ j ·T ij (t),φ j represents the unit time access cost of network access facility j, represents the weight factor, l ij Represents a Boolean variable, which can only have two states: 0 and 1; (4) Using the DQN-based method to obtain the optimal strategy for heterogeneous network selection problem: (4a) Model the heterogeneous network selection problem as a Markov decision process, that is, define its state space, action space and reward: State space: including vehicle position loc i (t), data synchronization delay T ij (t), and the remaining data volume DR that vehicle user i needs to synchronize i (t), expressed as S(t) = {loc i (t),T ij (t),DR i (t)}; Action space: It includes the vehicle’s choice of currently accessible network links and the amount of data to be transmitted. The twin action space of vehicle i is represented by A(t) = {L i (t),D i (t)}, where L i (t)∈m i (t), D i (t)∈{0,1,2}, where 0, 1, and 2 represent no data transmission, 1M data transmission, and 2M data transmission, respectively; Reward: refers to the reward that vehicle i will receive from the environment after performing the action at time slot t: R(t) = λ1r(t) - λ2p(t), in, is the reward function, is the penalty factor, D i represents the total amount of data to be synchronized for vehicle i, λ1 and λ2 are the scaling factors of reward and penalty respectively; (4b) DQN is used to find the optimal solution for the Markov decision process. That is, according to the defined state space, action space and reward, the DQN algorithm is used to make decisions on the link access and synchronization data amount of vehicle i in each time slot to maximize the action utility at each moment.

2. The method according to claim 1, characterized in that The vehicle digital twin DT in step (1) i and network access facility digital twin DT j , respectively expressed as follows: DT j (t)={C j ,φ j ,loc j } in, is the operating status of the i-th vehicle at time t, including: transmission power, amount of data to be synchronized, and accessible network at time t; i and E i Respectively represent the departure and destination of the vehicle; is the preference of vehicle i for delay and cost, loc i (t) is the position of vehicle i, C j and φ j They represent the communication capacity and the cost of unit bandwidth resources, loc j is the location of infrastructure j.

3. The method according to claim 1, characterized in that In step (2a), the transmission rate r of vehicle i accessing cellular base station b is calculated as i,b and the transmission rate r of the access drone u i,u , the formula is as follows: Among them, B b is the bandwidth allocated by base station b to vehicle i, γ i,b is the signal-to-noise ratio of vehicle i accessing base station b, h b is the antenna gain of the base station, P i represents the transmission power of vehicle i, d i,b is the distance between vehicle i and base station b, L0 and α are the unit distance path loss and path loss index, respectively, σ 2 is the Gaussian white noise power, B u is the bandwidth allocated by UAV u to vehicle i, h u is the antenna gain of the UAV, d i,u is the distance between vehicle i and drone u.

4. The method according to claim 1, characterized in that In step (2a), the transmission rate r of vehicle i accessing satellite s is calculated as i,s , the formula is as follows: Among them, B s represents the bandwidth allocated by the satellite to the vehicle, N0 is the noise power spectral density, is the vehicle's transmission power gain, is the free space loss, represents rain attenuation, is the satellite’s antenna gain, represents the maximum antenna gain of the satellite, θ i is the off-axis angle of vehicle i relative to the maximum radiation of the satellite beam, θ 3dB is the half-power angle of the transmitting antenna, c and f are the speed of light and the communication center frequency, respectively, and d i,s is the distance between the vehicle user and the satellite, is a constant, η is the rainfall intensity, and ρ and β are two different frequency-dependent coefficients.

5. The method according to claim 1, characterized in that In step (2a), calculate the transmission rate r of drone u accessing the control center c u,c , the formula is as follows: Among them, B c represents the bandwidth allocated by the control center to the UAV, P u is the transmission power of UAV u, h c is the antenna gain of the controller c, d u,c is the distance between the UAV and the control center, L0 is the path loss per unit distance, σ 2 is the Gaussian white noise power.

6. The method according to claim 1, characterized in that In step (2a), the transmission rate r of satellite s accessing ground station g is calculated s,g , the formula is as follows: Among them, B g represents the bandwidth allocated to the satellite by the ground station, P s is the satellite's transmission power, and are the satellite’s transmission power gain and the ground station’s receiving power gain, and Represent free space loss and rain attenuation respectively, and the formula is as follows: is a constant, η is the rainfall intensity, and ρ and β are two different frequency-dependent coefficients.

7. The method according to claim 1, characterized in that In step (2b), the data size D to be synchronized by the vehicle in each time slot t is i (t) Calculate the data transmission delay T of the three transmission links from vehicle i to the cellular base station b, the control center c, and the ground station. i,b (t), T i,c (t), T i,g (t), the formula is as follows:

8. The method according to claim 1, characterized in that In (4b), according to the defined state space, action space and reward, the DQN algorithm is used to make decisions on the link access and synchronization data amount of vehicle i in each time slot, which is implemented as follows: (4b1) The action utility function Q is iteratively updated. In each iteration, the update process of the Q function value is: Where α∈[0,1] is the learning rate, S(t), A(t) and R(t) represent the state, action and reward at time t, respectively, 0<δ<1 is the discount factor, S(t+1) and A(t+1) represent the state and action at the next moment, respectively; (4b2) A deep neural network with weights {θ} is used as a function approximator to fit the Q function, that is, Q(S(t), A(t))≈Q'(S(t), A(t); θ), so that the predicted Q'(S(t), A(t); θ) is infinitely close to the actual Q tar , the weights θ are trained to minimize the loss function of the deep neural network: Loss(θ)=E(Q tar -Q'(S(t),A(t);θ)) 2 in, (4b3) Using the weight θ that minimizes the loss function of the deep neural network, the action with the greatest utility in state S(t) is: Among them, Q'(S(t), A(t); θ) is the prediction result of the neural network output after training, a opt (t) is the optimal link access and synchronization data volume action decision at time t.

Images