An Information Perception Optimization Method for Unmanned Aerial Vehicle Assisted Vehicle Digital Twin

By using drones as mobile relays in a drone-assisted vehicle digital twin system, and combining Markov decision processes and near-end policy optimization algorithms, the drone flight trajectory and data processing are optimized, solving the problems of vehicle mobility and resource waste, and achieving low-cost, high-efficiency data transmission and synchronous mapping.

CN117155451BActive Publication Date: 2026-11-10ZHENGZHOU UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310950396.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-31
Publication Date
2026-11-10
Estimated Expiration
2043-07-31

AI Technical Summary

Technical Problem

Existing technologies in drone-assisted vehicle digital twin systems fail to effectively consider the high mobility and heterogeneous functions of vehicles, resulting in high data transmission latency and error rates, as well as serious resource waste, and an inability to effectively optimize the age of error information and energy consumption within the system.

Method used

By using drones as mobile relays, a communication system model is established from the vehicle to the drone to the digital twin control unit. By combining Markov decision process and near-end policy optimization algorithm (PPO), the drone flight trajectory and data processing strategy are optimized, important information is dynamically updated, and transmission costs and redundancy are reduced.

Benefits of technology

Under the energy constraints of drones, the system effectively reduced the total age of error messages and transmission costs, optimized data transmission efficiency, reduced resource waste, and improved the system's synchronization mapping capability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117155451B_ABST
    Figure CN117155451B_ABST
Patent Text Reader

Abstract

This invention discloses an information perception optimization method for drone-assisted vehicle digital twins. Autonomous vehicles are equipped with virtual sensors and data acquisition sensors, and driving decisions are trained in a real-world environment. Autonomous vehicles with varying levels of intelligence collect and process data on unexpected road conditions. This data is relayed in real-time to a metaverse control center via a drone, which then selects whether to process the data and dynamically updates the data in the virtual world of the metaverse to aid in the training of the autonomous vehicles. To reduce data transmission costs and redundancy, a metric similar to the age of error messages is used to measure the importance of information. Furthermore, drones are used as relays and for data processing, ensuring effective data transmission within the system despite limited drone flight energy, thus reducing transmission costs while simultaneously meeting the requirements for synchronous mapping in digital twins.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) assisted vehicle equipment, specifically to an information perception optimization method for a digital twin of an UAV assisted vehicle. Background Technology

[0002] The development of 5G and 6G has accelerated the realization of emerging applications such as augmented reality, virtual reality, mixed reality, and digital twins. Simultaneously, it has become possible for autonomous vehicles to utilize digital twin technology to assist in real-time decision-making. A key challenge for these emerging applications is the need to collect current environmental data using sensors, transmit this data to cloud servers in real time, and then provide timely feedback and decisions to the physical world. This data is time-sensitive. For example, in a digital twin-assisted autonomous vehicle environment, the vehicle constantly perceives its surroundings, transmits information to a central control center to form a digital twin world, assists in decision-making, and then feeds back to the vehicle.

[0003] However, constantly updating data to the digital twin leads to unnecessary expenses and resource waste. Error information age is an improvement on information age, which is the time elapsed since the timestamp of successful data packet transmission from the sender to the receiver. It measures the staleness of information but doesn't consider the importance of the information's content. Error information age, on the other hand, takes into account the updating of fresh information. The receiver estimates the receiver's state at every moment, and the penalty value only accumulates when the estimated state differs from the actual state. Meanwhile, drones have the advantages of flexible movement, reduced signal interference at high altitudes, and wide coverage. Using drones as mobile relays to dispatch vehicles transmitting data to the digital twin can effectively reduce transmission latency and error rates.

[0004] Existing technical solutions mostly consider only the transmission of state update packets from fixed nodes, neglecting node mobility, heterogeneous functions, and information importance metrics. This patented technology balances the error information age of each vehicle within the system with the energy consumption of the UAV, jointly optimizing the UAV's flight trajectory and scheduling vehicle data updates and data processing to minimize the total error information age of the system under UAV energy constraints. We propose modeling the problem as a Markov decision process and propose a proximal policy optimization algorithm to obtain a strategy that minimizes the long-term total error age of the system. Summary of the Invention

[0005] The purpose of this invention is to provide an information perception optimization method for drone-assisted vehicle digital twins. Autonomous vehicles are equipped with virtual sensors and data acquisition sensors, and driving decisions are trained in real-world environments. Autonomous vehicles with varying levels of intelligence collect and process data on unexpected road conditions. This data is relayed in real-time to a metaverse control center via a drone, which selects whether to process the data and dynamically updates the data in the virtual world of the metaverse to assist in training the autonomous vehicles. To reduce data transmission costs and redundancy, a metric similar to the age of error messages is used to measure the importance of information. Furthermore, drones are used as relays and for data processing, enabling effective data transmission within the system even with limited drone flight energy, thus reducing transmission costs and simultaneously meeting the requirements for synchronous mapping of digital twins.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] A method for optimizing information perception in a digital twin of a drone-assisted vehicle includes the following steps:

[0008] Step 1: Establish a communication system model from vehicle to drone to digital twin control unit;

[0009] Step 1.1: Input the coordinate range of the target area, divide the area into equally sized cells, and represent the center position of each cell as (x, y). Input the autonomous vehicle N, and represent the position of the autonomous vehicle in time slot t within the area as L. n t = (x n y n The location of the UAV in time slot t within the region is represented by L. u t = (x u y u The digital twin control unit is located at fixed coordinates (x, y). c y c );

[0010] Step 1.2: The N vehicles have different levels of intelligence. Some have semantic extraction modules, which extract important information and compress the data. Autonomous vehicles with semantic extraction capabilities are represented as λn = 1, while those without data processing capabilities are represented as λn = 0. The drone carries its own semantic extraction module. The drone chooses whether to process the data, ρ t =1 indicates that the drone performs semantic extraction, ρ t =0 indicates no processing. Time is divided into discrete, fixed-time timestamps τ. Each τ includes two sub-time slots: the first slot is the time spent transmitting data from the vehicle to the drone, and the second slot is the transmission time from the drone to the digital twin control unit and the time spent by the drone choosing whether to perform semantic extraction.

[0011] Step 1.3: The channel gain between the vehicle and the UAV is guided by the line-of-sight link, and its calculation formula is as follows:

[0012] ;

[0013] Where β0 is the channel gain at a reference distance of 1 meter, d u,n It is the distance between the drone and vehicle n;

[0014] Step 2: Calculate the energy consumed by the UAV at different stages, including the energy consumed during flight and hovering, the energy consumed by the UAV using the semantic extraction module, and the energy consumed by the UAV transmitting data to the digital twin control unit. The maximum speed of the UAV is limited to moving to a maximum of one adjacent grid per time slot. Optimize the UAV's flight trajectory to minimize the total objective AOII under limited UAV energy.

[0015] Step 2.1: The movement direction of the drone is defined as five types: hovering in place, east, west, south, and north, which are represented as follows:

[0016] ,

[0017] This leads to the formula for the flight energy of the drone:

[0018] ,

[0019] Where E m E represents the energy consumed by the drone as it moves east, south, west, and north. h This indicates the energy consumed by the drone while hovering;

[0020] Step 2.2: Uplink data transmission from the vehicle to the drone, obtaining the data transmission rate formula for the vehicle in time slot t:

[0021] ,

[0022] Where W and σ 2 These are the channel bandwidth and noise power from vehicle n to the drone, respectively, P n Let n be the data transmission power of vehicle n, and then the formula for the data transmission time from vehicle n to the UAV is obtained, which is expressed as:

[0023] ,

[0024] κ n t α represents the size of the data after semantic extraction for vehicle n. n tκ represents the data size of vehicle n without semantic extraction. When λn = 1, vehicle n has semantic extraction capabilities, and the transmitted data size is κ. n t ;

[0025] Step 2.3: Obtain the remaining time Tu→c=τ-Tn→u for transmitting data from the UAV to the digital twin control unit. Further, obtain the formula for calculating the UAV's data transmission rate:

[0026] ,

[0027] The transmission power P of the drone u t The calculation is based on Shannon's formula and the formula in step 2.2:

[0028] ,

[0029] in The channel gain from the UAV to the digital twin control unit is represented by the formula above. Based on step 2.2 and the formula above, the formula for calculating the energy consumed from the UAV to the digital twin control center is as follows:

[0030] ,

[0031] Where б is the unit energy, and the energy is discretized here;

[0032] Step 3: Establish an error information age metric (AOII) for vehicle information collection;

[0033] Step 3.1: Regarding vehicles, each vehicle encounters different road conditions. The digital twin control center maintains the digital world of the entire area and provides the digital world of the area where the vehicle is located based on the vehicle's real-time location. The road conditions in the digital world of vehicle n at time slot t are defined as follows: , For the actual road conditions of the vehicle at time slot t, when and When the values ​​are unequal, it indicates that vehicle n has encountered unexpected road conditions and needs to update its data to the digital twin control center. The digital twin control center monitors all vehicle requests, and the drone ultimately decides which vehicle to transmit data to. Assuming that the vehicle only retains the latest transmitted data, it will overwrite the previous data when new road conditions occur. The task processing instruction function formula is expressed as:

[0034] ,

[0035] in It records the vehicle's cumulative tasks, when Y n tWhen Y = 0, it indicates that the vehicle currently has no data transmission task. n t When Y > 0, it indicates that the vehicle has a transmission task. n t When the value is greater than 1, the vehicle only retains the latest data;

[0036] Step 3.2: Similar to AoII (Age of Incorrect Information), the concept of new information updates is considered. That is, when the estimated process state perfectly matches the actual process state, the information is not penalized. Conversely, when the information is in an incorrect state, the penalty increases over time. The timestamp of the last transmission task generated by vehicle n is represented as:

[0037] ,

[0038] The formula for calculating the Vehicle New Information Update (AOII) metric is:

[0039] ,

[0040] Step 4: Based on the Proximal Policy Optimization (PPO) algorithm, the process is modeled as a single-agent Markov decision process, in which the UAV acts as an agent and interacts with the environment through a policy to obtain rewards. In each time slot, a tuple of the Markov decision process is obtained, which includes state S, action A, reward C and transition probability Pr. When the UAV's energy is limited, the UAV acts as a mobile relay with data processing capabilities. According to a policy, it selects a direction of movement and immediately transmits the data collected by the vehicle to the digital twin control unit. The UAV's policy is related to the vehicle's high mobility, the occurrence of sudden road conditions, and the UAV's remaining energy.

[0041] Step 4.1: Set the system's state space S:

[0042] The system includes the state space of the drone and the vehicle. The drone's state includes its position and remaining energy. The formula for calculating the remaining energy is as follows:

[0043] ,

[0044] That is, the state space of the drone is Furthermore, the state of vehicle n captures the vehicle's position and the current AOII, denoted as... Therefore, the system state can be represented as The system state S at time slot t,

[0045] Step 4.2: Set up the drone's motion space A:

[0046] The UAV obtains the observation value S in time slot t. t Then, action A is determined based on strategy π. t The drone's actions include its direction of movement, selection of vehicles for data update scheduling, and choice of whether to use a semantic extraction module.

[0047] Action space is represented as ;

[0048] Step 4.2: Set the drone's reward function C:

[0049] When the drone is in state S t Make a decision action A t Afterwards, the system will receive an instantaneous reward C, which is defined as:

[0050] After weighing the AOII of the UAV in the system and the UAV energy consumption, the UAV flight trajectory, UAV data update scheduling and data processing were jointly optimized. The overall goal of the system is to minimize the total AOII.

[0051] Step 5: Build and train the PPO network model;

[0052] Step 5.1: Set up the initial environment. Input a region divided into equal-sized cells and define the coordinates. Set N vehicles to move within the region according to the Markov movement model and define their movement speeds. Input the coordinates of the digital twin control unit, the drone's flight speed, maximum energy starting position, and the location of the charging pile. Based on the initial parameters, form the drone's state space, the vehicle's state space, and the drone's motion space.

[0053] Step 5.2: Initialize the neural network parameters θ and φ, using the actor network Π θ Approximate strategy Π, using a critic network V φ Approximate state value function, when the drone observes the state S t Enter into Π θ When sampling an action A, a normal distribution corresponding to the action probability distribution is obtained. t You will then receive an instant reward C. t and the next state S t+1 Stored as a sequence [(S t A t C t S t+1 ), ...] are stored in memory M, and then S is stored in memory M. t+1 Input to actor network Π θ Generate the next sequence, and store the generated sequences into M. The goal of the drone is to obtain the system's long-term maximum reward. Train an optimal policy to meet the objective requirements;

[0054] Step 5.3: Calculate the gradient of J(θ) according to the policy gradient theorem, and replace the expected return with the advantage function. The gradient formula is:

[0055] ;

[0056] Where ξ(S,A;π) is the dominance function, expressed in detail as follows:

[0057] ,

[0058] Then, the important trimming sampling theorem and the pruning substitution objective function are introduced, and their formula is expressed as:

[0059] ,

[0060] Importance sampling function is ,in It's the old strategy, the strategy from before the last update; the old strategy is used for sampling. It is a function that limits the magnitude of policy updates, and its calculation formula is:

[0061] ;

[0062] Step 5.4: Calculate the critic network update method, using the critic network V. φ The approximate value function, defined as the loss function used to update the parameters, is as follows:

[0063] ,

[0064] Where Q(S,A;π) is approximately the action value function, expressed using the Bellman update equation and the temporal difference algorithm as follows: ;

[0065] Step 5.5: Based on the storage sequence in Step 5.3, store the state S from the last step. T The input is fed into the critic network to obtain the state value V. φ (S T ), calculate the discount reward according to step 5.4, and obtain the reward value Q = [Q(0),Q(1),...,Q(T)] for each time slot. Then, input all the stored states S in M ​​into the critic network to obtain the V of all states. φ Put Q and V φ Substitute this into the advantage function in step 5.3;

[0066] Step 5.6: Based on the formula in Step 5.5 and the loss function in Step 5.4, calculate the gradient of the loss function and then backpropagate to update the parameters φ of the critic network.

[0067] ,

[0068] Input all states S stored in M ​​into the new actor network and the old actor network to obtain two normal distributions. Input the actions A stored in M ​​into the two normal distributions to obtain the probability corresponding to each action. Substitute these probabilities into the importance sampling formula in step 5.3 to calculate the pruning / substitution target and the objective function of importance sampling in step 5.3, and backpropagate them to the new actor network.

[0069] ;

[0070] After iterating through and updating the storage in M, the update process ends, and the parameters in the new actor are updated to the parameters in the old actor. Return to step 5.2 and sample using the old actor network, repeating steps 5.2 through 5.6 until the parameters θ and φ converge.

[0071] The beneficial effects of this invention are: This patent takes into account the high mobility and heterogeneous functions of vehicles, and uses dynamic and uncertain position information as a better optimization target for solving the state space;

[0072] This patent takes into account the measurement of information importance; in building a virtual world on the digital twin, it is not necessary to update all data at all times, reducing data update costs and redundancy, and only selecting important information to be updated;

[0073] This patent balances the error information age of each vehicle in the system with the energy consumption of the drone, jointly optimizes the drone's flight trajectory, and schedules vehicle data updates and data processing to minimize the total error information age of the system under drone energy constraints.

[0074] This patent addresses the problem of huge state and action spaces by employing a proximal optimization strategy to solve the modeled Markov decision process, thus solving the problem of slow convergence speed in traditional reinforcement learning.

[0075] This patent takes into account the movement of the vehicle in the simulation experiment and the generation of sudden road conditions to transmit data scheduling, which is closer to the dynamic scene in reality.

[0076] This patent takes into account the global optimization problem of the system and focuses more on minimizing the overall AOII performance of all vehicles in the system. Attached Figure Description

[0077] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0078] Figure 1 This is a system model diagram of the present invention;

[0079] Figure 2 This is a flowchart of the present invention;

[0080] Figure 3 This is a flowchart of the PPO algorithm of the present invention. Detailed Implementation

[0081] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0082] like Figure 1-3 As shown, a method for optimizing information perception in a drone-assisted vehicle digital twin includes the following steps:

[0083] Step 1: Establish a communication system model from vehicle to drone to digital twin control unit;

[0084] Step 1.1: Input the coordinate range of the target area, divide the area into equally sized cells, and represent the center position of each cell as (x, y). Input the autonomous vehicle N, and represent the position of the autonomous vehicle in time slot t within the area as L. n t = (x n y n The location of the UAV in time slot t within the region is represented by L. u t = (x u y u The digital twin control unit is located at fixed coordinates (x, y). c y c );

[0085] Step 1.2: The N vehicles have different levels of intelligence. Some have semantic extraction modules, which extract important information and compress the data. Autonomous vehicles with semantic extraction capabilities are represented as λn = 1, while those without data processing capabilities are represented as λn = 0. The drone carries its own semantic extraction module. The drone chooses whether to process the data, ρ t =1 indicates that the drone performs semantic extraction, ρ t =0 indicates no processing. Time is divided into discrete, fixed-time timestamps τ. Each τ includes two sub-time slots: the first slot is the time spent transmitting data from the vehicle to the drone, and the second slot is the transmission time from the drone to the digital twin control unit and the time spent by the drone choosing whether to perform semantic extraction.

[0086] Step 1.3: The channel gain between the vehicle and the UAV is guided by the line-of-sight link, and its calculation formula is as follows:

[0087] ;

[0088] Where β0 is the channel gain at a reference distance of 1 meter, d u,n It is the distance between the drone and vehicle n;

[0089] Step 2: Calculate the energy consumed by the UAV at different stages, including the energy consumed during flight and hovering, the energy consumed by the UAV using the semantic extraction module, and the energy consumed by the UAV transmitting data to the digital twin control unit. The maximum speed of the UAV is limited to moving to a maximum of one adjacent grid per time slot. Optimize the UAV's flight trajectory to minimize the total objective AOII under limited UAV energy.

[0090] Step 2.1: The movement direction of the drone is defined as five types: hovering in place, east, west, south, and north, which are represented as follows:

[0091] ,

[0092] This leads to the formula for the flight energy of the drone:

[0093] ,

[0094] Where E m E represents the energy consumed by the drone as it moves east, south, west, and north. h This indicates the energy consumed by the drone while hovering;

[0095] Step 2.2: Uplink data transmission from the vehicle to the drone, obtaining the data transmission rate formula for the vehicle in time slot t:

[0096] ,

[0097] Where W and σ 2 These are the channel bandwidth and noise power from vehicle n to the drone, respectively, P n Let n be the data transmission power of vehicle n, and then the formula for the data transmission time from vehicle n to the UAV is obtained, which is expressed as:

[0098] ,

[0099] κ n t α represents the size of the data after semantic extraction for vehicle n. n t κ represents the data size of vehicle n without semantic extraction. When λn = 1, vehicle n has semantic extraction capabilities, and the transmitted data size is κ.n t ;

[0100] Step 2.3: Obtain the remaining time Tu→c=τ-Tn→u for transmitting data from the UAV to the digital twin control unit. Further, obtain the formula for calculating the UAV's data transmission rate:

[0101] ,

[0102] The transmission power P of the drone u t The calculation is based on Shannon's formula and the formula in step 2.2:

[0103] ,

[0104] in The channel gain from the UAV to the digital twin control unit is represented by the formula above. Based on step 2.2 and the formula above, the formula for calculating the energy consumed from the UAV to the digital twin control center is as follows:

[0105] ,

[0106] Where б is the unit energy, and the energy is discretized here;

[0107] Step 3: Establish an error information age metric (AOII) for vehicle information collection;

[0108] Step 3.1: Regarding vehicles, each vehicle encounters different road conditions. The digital twin control center maintains the digital world of the entire area and provides the digital world of the area where the vehicle is located based on the vehicle's real-time location. The road conditions in the digital world of vehicle n at time slot t are defined as follows: , For the actual road conditions of the vehicle at time slot t, when and When the values ​​are unequal, it indicates that vehicle n has encountered unexpected road conditions and needs to update its data to the digital twin control center. The digital twin control center monitors all vehicle requests, and the drone ultimately decides which vehicle to transmit data to. Assuming that the vehicle only retains the latest transmitted data, it will overwrite the previous data when new road conditions occur. The task processing instruction function formula is expressed as:

[0109] ,

[0110] in It records the vehicle's cumulative tasks, when Y n t When Y = 0, it indicates that the vehicle currently has no data transmission task. n tWhen Y > 0, it indicates that the vehicle has a transmission task. n t When the value is greater than 1, the vehicle only retains the latest data;

[0111] Step 3.2: Similar to AoII (Age of Incorrect Information), the concept of new information updates is considered. That is, when the estimated process state perfectly matches the actual process state, the information is not penalized. Conversely, when the information is in an incorrect state, the penalty increases over time. The timestamp of the last transmission task generated by vehicle n is represented as:

[0112] ,

[0113] The formula for calculating the Vehicle New Information Update (AOII) metric is:

[0114] ,

[0115] Step 4: Based on the Proximal Policy Optimization (PPO) algorithm, the process is modeled as a single-agent Markov decision process, in which the UAV acts as an agent and interacts with the environment through a policy to obtain rewards. In each time slot, a tuple of the Markov decision process is obtained, which includes state S, action A, reward C and transition probability Pr. When the UAV's energy is limited, the UAV acts as a mobile relay with data processing capabilities. According to a policy, it selects a direction of movement and immediately transmits the data collected by the vehicle to the digital twin control unit. The UAV's policy is related to the vehicle's high mobility, the occurrence of sudden road conditions, and the UAV's remaining energy.

[0116] Step 4.1: Set the system's state space S:

[0117] The system includes the state space of the drone and the vehicle. The drone's state includes its position and remaining energy. The formula for calculating the remaining energy is as follows:

[0118] ,

[0119] That is, the state space of the drone is Furthermore, the state of vehicle n captures the vehicle's position and the current AOII, denoted as... Therefore, the system state can be represented as The system state S at time slot t,

[0120] Step 4.2: Set up the drone's motion space A:

[0121] The UAV obtains the observation value S in time slot t. t Then, action A is determined based on strategy π. tThe drone's actions include its direction of movement, selection of vehicles for data update scheduling, and choice of whether to use a semantic extraction module.

[0122] Action space is represented as ;

[0123] Step 4.2: Set the drone's reward function C:

[0124] When the drone is in state S t Make a decision action A t Afterwards, the system will receive an instantaneous reward C, which is defined as:

[0125] After weighing the AOII of the UAV in the system and the UAV energy consumption, the UAV flight trajectory, UAV data update scheduling and data processing were jointly optimized. The overall goal of the system is to minimize the total AOII.

[0126] Step 5: Build and train the PPO network model;

[0127] Step 5.1: Set up the initial environment. Input a region divided into equal-sized cells and define the coordinates. Set N vehicles to move within the region according to the Markov movement model and define their movement speeds. Input the coordinates of the digital twin control unit, the drone's flight speed, maximum energy starting position, and the location of the charging pile. Based on the initial parameters, form the drone's state space, the vehicle's state space, and the drone's motion space.

[0128] Step 5.2: Initialize the neural network parameters θ and φ, using the actor network Π θ Approximate strategy Π, using a critic network V φ Approximate state value function, when the drone observes the state S t Enter into Π θ When sampling an action A, a normal distribution corresponding to the action probability distribution is obtained. t You will then receive an instant reward C. t and the next state S t+1 Stored as a sequence [(S t A t C t S t+1 ), ...] are stored in memory M, and then S is stored in memory M. t+1 Input to actor network Π θ Generate the next sequence, and store the generated sequences into M. The goal of the drone is to obtain the system's long-term maximum reward. Train an optimal policy to meet the objective requirements;

[0129] Step 5.3: Calculate the gradient of J(θ) according to the policy gradient theorem, and replace the expected return with the advantage function. The gradient formula is:

[0130] ;

[0131] Where ξ(S,A;π) is the dominance function, expressed in detail as follows:

[0132] ,

[0133] Then, the important trimming sampling theorem and the pruning substitution objective function are introduced, and their formula is expressed as:

[0134] ,

[0135] Importance sampling function is ,in It's the old strategy, the strategy from before the last update; the old strategy is used for sampling. It is a function that limits the magnitude of policy updates, and its calculation formula is:

[0136] ;

[0137] Step 5.4: Calculate the critic network update method, using the critic network V. φ The approximate value function, defined as the loss function used to update the parameters, is as follows:

[0138] ,

[0139] Where Q(S,A;π) is approximately the action value function, expressed using the Bellman update equation and the temporal difference algorithm as follows: ;

[0140] Step 5.5: Based on the storage sequence in Step 5.3, store the state S from the last step. T The input is fed into the critic network to obtain the state value V. φ (S T ), calculate the discount reward according to step 5.4, and obtain the reward value Q = [Q(0),Q(1),...,Q(T)] for each time slot. Then, input all the stored states S in M ​​into the critic network to obtain the V of all states. φ Put Q and V φ Substitute this into the advantage function in step 5.3;

[0141] Step 5.6: Based on the formula in Step 5.5 and the loss function in Step 5.4, calculate the gradient of the loss function and then backpropagate to update the parameters φ of the critic network.

[0142] ,

[0143] Input all states S stored in M ​​into the new actor network and the old actor network to obtain two normal distributions. Input the actions A stored in M ​​into the two normal distributions to obtain the probability corresponding to each action. Substitute these probabilities into the importance sampling formula in step 5.3 to calculate the pruning / substitution target and the objective function of importance sampling in step 5.3, and backpropagate them to the new actor network.

[0144] ;

[0145] After iterating through and updating the storage in M, the update process ends, and the parameters in the new actor are updated to the parameters in the old actor. Return to step 5.2 and sample using the old actor network, repeating steps 5.2 through 5.6 until the parameters θ and φ converge.

[0146] The working principle of this invention: This patent takes into account the high mobility and heterogeneous functions of vehicles, and uses dynamic and uncertain position information as a better optimization target for solving the state space;

[0147] This patent takes into account the measurement of information importance; in building a virtual world on the digital twin, it is not necessary to update all data at all times, reducing data update costs and redundancy, and only selecting important information to be updated;

[0148] This patent balances the error information age of each vehicle in the system with the energy consumption of the drone, jointly optimizes the drone's flight trajectory, and schedules vehicle data updates and data processing to minimize the total error information age of the system under drone energy constraints.

[0149] This patent addresses the problem of huge state and action spaces by employing a proximal optimization strategy to solve the modeled Markov decision process, thus solving the problem of slow convergence speed in traditional reinforcement learning.

[0150] This patent takes into account the movement of the vehicle in the simulation experiment and the generation of sudden road conditions to transmit data scheduling, which is closer to the dynamic scene in reality.

[0151] This patent takes into account the global optimization problem of the system and focuses more on minimizing the overall AOII performance of all vehicles in the system.

[0152] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for optimizing information perception in a digital twin of an unmanned aerial vehicle (UAV) assisted vehicle, characterized in that, Includes the following steps: Step 1: Establish a communication system model from vehicle to drone to digital twin control unit; Step 1.1: Input the coordinate range of the target area, divide the area into equally sized cells, and represent the center position of each cell as (x, y). Input the autonomous vehicle N, and represent the position of the autonomous vehicle in time slot t within the area as L. n t =(x n y n The location of the UAV in time slot t within the region is represented by L. u t = (x u y u The digital twin control unit is located at fixed coordinates (x, y). c y c ); Step 1.2: The N vehicles have different levels of intelligence. Some have semantic extraction modules, which extract important information and compress the data. Autonomous vehicles with semantic extraction capabilities are represented as λn = 1, while those without data processing capabilities are represented as λn = 0. The drone carries its own semantic extraction module. The drone chooses whether to process the data, ρ t =1 indicates that the drone performs semantic extraction, ρ t =0 indicates no processing. Time is divided into discrete, fixed-time timestamps τ. Each τ includes two sub-time slots: the first slot is the time spent transmitting data from the vehicle to the drone, and the second slot is the transmission time from the drone to the digital twin control unit and the time spent by the drone choosing whether to perform semantic extraction. Step 1.3: The channel gain between the vehicle and the UAV is guided by the line-of-sight link, and its calculation formula is as follows: ; Where β0 is the channel gain at a reference distance of 1 meter, d u,n It is the distance between the drone and vehicle n; Step 2: Calculate the energy consumed by the UAV at different stages, including the energy consumed during flight and hovering, the energy consumed by the UAV using the semantic extraction module, and the energy consumed by the UAV transmitting data to the digital twin control unit. The maximum speed of the UAV is limited to moving to a maximum of one adjacent grid per time slot. Optimize the UAV's flight trajectory to minimize the total objective AOII under limited UAV energy. Step 2.1: The movement direction of the drone is defined as five types: hovering in place, east, west, south, and north, which are represented as follows: , This leads to the formula for the flight energy of the drone: , Where E m E represents the energy consumed by the drone as it moves east, south, west, and north. h This indicates the energy consumed by the drone while hovering; Step 2.2: Uplink data transmission from the vehicle to the drone, obtaining the data transmission rate formula for the vehicle in time slot t: , Where W and σ 2 These are the channel bandwidth and noise power from vehicle n to the drone, respectively, P n Let n be the data transmission power of vehicle n, and then the formula for the data transmission time from vehicle n to the UAV is obtained, which is expressed as: , κ n t α represents the size of the data after semantic extraction for vehicle n. n t κ represents the data size of vehicle n without semantic extraction. When λn = 1, vehicle n has semantic extraction capabilities, and the transmitted data size is κ. n t ; Step 2.3: Obtain the remaining time Tu→c=τ-Tn→u for transmitting data from the UAV to the digital twin control unit. Further, obtain the formula for calculating the UAV's data transmission rate: , The transmission power P of the drone u t The calculation is based on Shannon's formula and the formula in step 2.2: , in The channel gain from the UAV to the digital twin control unit is represented by the formula above. Based on step 2.2 and the formula above, the formula for calculating the energy consumed from the UAV to the digital twin control center is as follows: , Where б is the unit energy, and the energy is discretized here; Step 3: Establish an error information age metric (AOII) for vehicle information collection; Step 3.1: Regarding vehicles, each vehicle encounters different road conditions. The digital twin control center maintains the digital world of the entire area and provides the digital world of the area where the vehicle is located based on the vehicle's real-time location. The road conditions in the digital world of vehicle n at time slot t are defined as follows: , For the actual road conditions of the vehicle at time slot t, when and When the values ​​are unequal, it indicates that vehicle n has encountered unexpected road conditions and needs to update its data to the digital twin control center. The digital twin control center monitors all vehicle requests, and the drone ultimately decides which vehicle to transmit data to. Assuming that the vehicle only retains the latest transmitted data, it will overwrite the previous data when new road conditions occur. The task processing instruction function formula is expressed as: , in It records the vehicle's cumulative tasks, when Y n t When Y = 0, it indicates that the vehicle currently has no data transmission task. n t When Y > 0, it indicates that the vehicle has a transmission task. n t When the time limit is 1, only the latest data is retained for the vehicle. Step 3.2: Similar to AoII (Age of Incorrect Information), the concept of new information updates is considered. That is, when the estimated process state perfectly matches the actual process state, the information is not penalized. Conversely, when the information is in an incorrect state, the penalty increases over time. The timestamp of the last transmission task generated by vehicle n is represented as: , The formula for calculating the Vehicle New Information Update (AOII) metric is: , Step 4: Based on the Proximal Policy Optimization (PPO) algorithm, the process is modeled as a single-agent Markov decision process, in which the UAV acts as an agent and interacts with the environment through a policy to obtain rewards. In each time slot, a tuple of the Markov decision process is obtained, which includes state S, action A, reward C and transition probability Pr. When the UAV's energy is limited, the UAV acts as a mobile relay with data processing capabilities. According to a policy, it selects a direction of movement and immediately transmits the data collected by the vehicle to the digital twin control unit. The UAV's policy is related to the vehicle's high mobility, the occurrence of sudden road conditions, and the UAV's remaining energy. Step 4.1: Set the system's state space S: The system includes the state space of the drone and the vehicle. The drone's state includes its position and remaining energy. The formula for calculating the remaining energy is as follows: , That is, the state space of the drone is Furthermore, the state of vehicle n captures the vehicle's position and the current AOII, denoted as... Therefore, the system state can be represented as The system state S at time slot t, Step 4.2: Set up the drone's motion space A: The UAV obtains the observation value S in time slot t. t Then, action A is determined based on strategy π. t The drone's actions include its direction of movement, selection of vehicles for data update scheduling, and choice of whether to use a semantic extraction module. Action space is represented as ; Step 4.3: Set the drone's reward function C: When the drone is in state S t Make a decision action A t Afterwards, the system will receive an instantaneous reward C, which is defined as: After weighing the AOII of the UAV in the system and the UAV energy consumption, the UAV flight trajectory, UAV data update scheduling and data processing were jointly optimized. The overall goal of the system is to minimize the total AOII. Step 5: Build and train the PPO network model; Step 5.1: Set up the initial environment. Input a region divided into equal-sized cells and define the coordinates. Set N vehicles to move within the region according to the Markov movement model and define their movement speeds. Input the coordinates of the digital twin control unit, the drone's flight speed, maximum energy starting position, and the location of the charging pile. Based on the initial parameters, form the drone's state space, the vehicle's state space, and the drone's motion space. Step 5.2: Initialize the neural network parameters θ and φ, using the actor network Π θ Approximate strategy Π, using a critic network V φ Approximate state value function, when the drone observes the state S t Enter into Π θ When sampling an action A, a normal distribution corresponding to the action probability distribution is obtained. t You will then receive an instant reward C. t and the next state S t+1 Stored as a sequence [(S t A t C t S t+1 ), ...] are stored in memory M, and then S is stored in memory M. t+1 Input to actor network Π θ Generate the next sequence, and store the generated sequences into M. The goal of the drone is to obtain the system's long-term maximum reward. Train an optimal policy to meet the objective requirements; Step 5.3: Calculate the gradient of J(θ) according to the policy gradient theorem, and replace the expected return with the advantage function. The gradient formula is: ; Where ξ(S,A;π) is the dominance function, expressed in detail as follows: , Then, the important trimming sampling theorem and the pruning substitution objective function are introduced, and their formula is expressed as: , Importance sampling function is ,in It's the old strategy, the strategy from before the last update; the old strategy is used for sampling. It is a function that limits the magnitude of policy updates, and its calculation formula is: ; Step 5.4: Calculate the critic network update method, using the critic network V. φ The approximate value function, defined as the loss function used to update the parameters, is as follows: , Where Q(S,A;π) is approximately the action value function, expressed using the Bellman update equation and the temporal difference algorithm as follows: ; Step 5.5: Based on the storage sequence in Step 5.3, store the state S from the last step. T The input is fed into the critic network to obtain the state value V. φ (S T ), calculate the discount reward according to step 5.4, and obtain the reward value Q = [Q(0),Q(1),...,Q(T)] for each time slot. Then, input all the stored states S in M ​​into the critic network to obtain the V of all states. φ Put Q and V φ Substitute this into the advantage function in step 5.3; Step 5.6: Based on the formula in Step 5.5 and the loss function in Step 5.4, calculate the gradient of the loss function and then backpropagate to update the parameters φ of the critic network. , Input all states S stored in M ​​into the new actor network and the old actor network to obtain two normal distributions. Input the actions A stored in M ​​into the two normal distributions to obtain the probability corresponding to each action. Substitute these probabilities into the importance sampling formula in step 5.3 to calculate the pruning / substitution target and the objective function of importance sampling in step 5.3, and backpropagate them to the new actor network. ; After iterating through and updating the storage in M, the update process ends, and the parameters in the new actor are updated to the parameters in the old actor. Return to step 5.2 and sample using the old actor network, repeating steps 5.2 through 5.6 until the parameters θ and φ converge.