A method for task offloading and transmission power allocation in Internet of Vehicles based on digital twin

Abstract

The present invention discloses a method for unloading tasks and allocating transmission power of Internet of Vehicles based on digital twins, comprising the following steps: Step 1, collecting the positions of vehicles and base stations, and constructing Internet of Vehicles and its digital twin network; Step 2, constructing an optimization function according to the digital twin network; Step 3, constructing a deep deterministic policy gradient model according to the optimization function, and obtaining the selected unloading device, task unloading ratio and allocated transmission power under the condition of minimizing the total delay of the system. The present invention has the characteristics of ensuring the timely completion of tasks, maximizing the overall throughput of the system, and reducing transmission delay.

Description

Technical Field

[0001] The present invention relates to the field of vehicle networking technology, and more specifically, to a vehicle networking task unloading and transmission power allocation method based on digital twins. Background Art

[0002] The Internet of Vehicles (IoV) technology revolution is reshaping the transportation system and expanding into smart cities, smart transportation and autonomous driving. However, vehicle networks are facing multiple challenges in terms of enhancing computing power, strictly controlling latency and optimizing service quality to support a heterogeneous, mission-critical and infotainment-integrated ecosystem. Currently, a series of critical and extremely latency-sensitive tasks are becoming increasingly important, including key operations such as real-time environmental monitoring, privacy-safe computing and collision avoidance. In a dynamic and complex vehicle communication environment, transmission latency is not only directly related to the real-time nature of task processing, but may also fluctuate significantly due to changes in channel conditions, thus affecting user experience and the overall efficiency of the system.

[0003] When facing computing-intensive applications, relying solely on the vehicle's own processing capabilities is still insufficient, becoming a major bottleneck restricting its performance. Mobile Edge Computing deploys MEC servers around the vehicle, allowing the vehicle to offload some or all computing tasks to the MEC server, effectively reducing the vehicle's own computing burden, thereby reducing power consumption, shortening task processing time, meeting the vehicle's demand for high-performance computing capabilities, and improving the experience quality of in-vehicle applications. At the same time, the vehicle can also offload computing tasks to other vehicles to reduce the offloading burden and reduce latency.

[0004] Digital Twin Network is an emerging network used to create virtual twin models of physical objects, which reflects the real-time status and behavior of physical objects, and promotes seamless mapping, iterative operations and deep interaction between virtual and physical spaces. Through DT modeling, communication, computing, data processing and other technologies, DTN provides unprecedented comprehensive perspectives and deep insights for feature mining and future state prediction of physical vehicles and their polymorphic environments, and realizes the co-evolution between physical and virtual spaces.

[0005] At present, in the scheme of DT-assisted vehicle task offloading, the offloading method adopted is mainly binary offloading, which has only a single optimization goal and ignores partial offloading. In addition, the DT model reflects the state and behavior of physical objects with high accuracy, but there are still deviations, such as sensor noise, model simplification, data loss, etc., which may lead to decision errors or performance degradation in practical applications. Therefore, it is necessary to strengthen the monitoring, evaluation and correction of DT deviations to improve the accuracy and reliability of offloading decisions.

[0006] Therefore, dynamic allocation of transmission power becomes a crucial link. Intelligently adjusting the transmission power can more flexibly respond to changes in channel conditions and reduce transmission delays caused by channel interference or poor channel quality. Summary of the invention

[0007] The purpose of this invention is to design and develop a method for vehicle network task offloading and transmission power allocation based on digital twins, and to solve the optimization problem of vehicle network joint task offloading and transmission power allocation assisted by digital twins based on deep reinforcement learning technology, so as to minimize the total delay of the system.

[0008] The technical solution provided by the present invention is:

[0009] A method for vehicle networking task offloading and transmission power allocation based on digital twins includes the following steps:

[0010] Step 1: Collect the locations of vehicles and base stations to build the Internet of Vehicles and its digital twin network;

[0011] Step 2: Construct an optimization function based on the digital twin network:

[0012] ;

[0013] ;

[0014] In the formula, is the total delay in the offloading process according to the task offloading ratio, is the total number of mission vehicles, is the total number of service vehicles, is the total number of vehicles in a collaborative group, is the number of base stations in the digital twin network, is the maximum tolerable delay, is the task offloading ratio, and , For the Service vehicles Available computing resources, For the Base Stations Available computing resources, is the transmission power of the vehicle, and , is the maximum constraint on vehicle transmission power, For the Service vehicles The maximum constraint of available computing resources is For the Base Stations The maximum constraint of available computing resources is is the number of task vehicles that need to be unloaded;

[0015] Step 3: construct a deep deterministic policy gradient model based on the optimization function to obtain the selected offloading device, task offloading ratio and allocated transmission power under the condition of minimizing the total delay of the system.

[0016] Preferably, the digital twin network includes Base stations, each equipped with a MEC server. Base stations Cooperation groups, each of which consists of a base station and its range The vehicles in the coordination group include the mission vehicle and service vehicles , Mission Vehicle Optional offloading of tasks to base station or service vehicle , the task is ;

[0017] in, , , , , is the input data size of the task, is the current channel gain of the task, To complete the task The amount of CPU cycles required.

[0018] Preferably, the total delay in the unloading process according to the task unloading ratio satisfies:

[0019] ;

[0020] In the formula, For the Mission Vehicles Communication time uploaded during the task offloading phase, For the Mission Vehicles Select the time delay for unloading the task. The actual time of local execution.

[0021] Preferably, the Mission Vehicles The communication time for uploading during the task offloading phase meets the following requirements:

[0022] ;

[0023] In the formula, is the transmission rate.

[0024] Preferably, the Mission Vehicles Select the time delay when the task is to be uninstalled:

[0025] ;

[0026] In the formula, For the task from Mission Vehicles Uninstall to Service vehicles The actual execution time, For the task from Mission Vehicles Uninstall to Base Stations The actual execution time.

[0027] Preferably, the actual time of the local execution satisfies:

[0028] ;

[0029] ;

[0030] ;

[0031] In the formula, is the local execution time before considering the deviation, For the Mission Vehicles The deviation of the execution time delay of the task, The amount of CPU cycles required to complete a task, For the Mission Vehicles The computing power of For the Mission Vehicles The deviation of the estimated available computing power in DT from the physically actual computing power.

[0032] Preferably, the task starts from Mission Vehicles Uninstall to Service vehicles The actual execution time meets:

[0033] ;

[0034] ;

[0035] ;

[0036] In the formula, Before considering the deviation Mission Vehicles Select Offload task to Service vehicles Delays in processing, For the Service vehicles Available computing resources, For the Service vehicles The estimated calculation delay deviation in DT is, For the Service vehicles The deviation of the estimated available computing power in DT from the physically actual computing power.

[0037] Preferably, the task starts from Mission Vehicles Uninstall to Base Stations The actual execution time meets:

[0038] ;

[0039] ;

[0040] ;

[0041] In the formula, Before considering the deviation Mission Vehicles Offload the task to Base Stations Delays in processing, For the Base Stations Available computing resources, For the Base Stations The estimated calculation delay deviation in DT is, For the Base Stations The deviation of the estimated available computing power in DT from the physically actual computing power.

[0042] Preferably, the deep deterministic policy gradient model includes:

[0043] The state space is: ;

[0044] The action space is: ;

[0045] The reward function is: ;

[0046] in, Mission vehicle Select to Server To offload tasks, ,when When the mission vehicle Offload the task to the base station; when For other values, the mission vehicle Offload tasks to service vehicle.

[0047] Preferably, the training process of the deep deterministic policy gradient model includes the following steps:

[0048] Step 1: Establish a participant network, a critic network, a target participant network, and a target critic network;

[0049] Step 2: Get the current time from the environment Status Enter the participant network and get Action after time update , the state at the next moment and the current moment reward ;

[0050] Among them, the updated action at the current moment satisfies:

[0051] ;

[0052] In the formula, for The action after the moment is updated, are the parameters of the participant network, For a participant network in a given state The original action generated by is random noise;

[0053] Step 3: Stored in the experience pool;

[0054] Step 4: Randomly select a group from the experience pool The state of the moment and Moment of action Input the critic network and obtain the expected return value;

[0055] Step 5: Change the status Input the target participant network to obtain the corresponding action After that, the status and actions Input the target critic network to get the predicted target return value:

[0056] ;

[0057] In the formula, The current moment taken from the experience pool Rewards, is the discount factor, are the parameters of the target participant network, For the target participant network in state The output action is: are the parameters of the target critic network, is the predicted return value;

[0058] Step 6: Update the parameters of the critic network by minimizing the loss value.

[0059] Wherein, the loss value satisfies:

[0060] ;

[0061] In the formula, Loss value, is the size of each small batch, The current moment taken from the experience pool status, The current moment taken from the experience pool Actions, are the parameters of the current critic network, is the expected reward value of the corresponding action;

[0062] Step 7: Update the parameters of the participant network by gradient ascent:

[0063] ;

[0064] In the formula, is the policy gradient when the participant network parameters are updated, For the participant network about parameters The gradient of , which indicates how the generated action changes when the parameters are adjusted, For the Critic network about the action The gradient of represents the sensitivity of Q value to action in the current state;

[0065] Step 8: Update the parameters of the target participant network and the target critic network:

[0066] ;

[0067] ;

[0068] In the formula, are the parameters of the current critic network, are the parameters of the target critic network, are the parameters of the current participant network, are the parameters of the target participant network, is the scaling factor for soft update, which makes the update of the target network smoother and more stable.

[0069] The beneficial effects of the present invention are:

[0070] The present invention designs and develops a method for task offloading and transmission power allocation for Internet of Vehicles based on digital twins. Firstly, DTN is used to create a twin model of physical entities. Taking into account the difference between the real value of some entity states and the estimated value in DT, the impact of this difference on the offloading strategy is explored; secondly, the dynamic channel conditions in the communication process and the delay constraints of the tasks are comprehensively considered, and the transmission power is dynamically adjusted to maximize the overall throughput of the system while ensuring the timely completion of tasks, and to intelligently allocate transmission power and reduce transmission delay; finally, the deep deterministic policy gradient (DDPG) algorithm is used to perform task offloading. Compared with traditional task offloading strategies, the present invention, with the assistance of the digital twin network, can adjust the offloading strategy and transmission power allocation of each task in real time according to the current system state and environmental changes to minimize the total system delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a schematic diagram of the scenario of vehicle edge computing assisted by the digital twin described in the present invention.

[0072] Figure 2 Schematic diagram of the algorithm flow of DDPG described in the present invention.

[0073] Figure 3 Parameter setting diagram of the simulation experiment described in the present invention.

[0074] Figure 4 Schematic diagram of total calculation delay comparison curve for various schemes with different numbers of motor vehicles according to the present invention.

[0075] Figure 5 Schematic diagram of the total calculated delay comparison curve for different deviation values ​​of the DTDP of the present invention.

[0076] Figure 6 This is a schematic diagram of the total calculation delay comparison curve of each scheme described in the present invention without considering the influence of deviation.

[0077] Figure 7 This is a schematic diagram of the total calculated delay comparison curve of each scheme with a deviation of 0.1G described in the present invention.

[0078] Figure 8 This is a schematic diagram of the total calculated delay comparison curve of each solution with a deviation of 0.15G described in the present invention.

[0079] Fig. 9 This is a schematic diagram of the total calculated delay comparison curve of each solution with a deviation of 0.2G described in the present invention.

[0080] Fig.10 This is a schematic diagram of the task completion rate comparison curve of each solution without considering the influence of deviation described in the present invention.

[0081] Fig.11 This is a schematic diagram of the task completion rate comparison curve of each solution with a deviation of 0.1G described in the present invention.

[0082] Fig.12 This is a schematic diagram of the task completion rate comparison curve of each solution with a deviation of 0.15G described in the present invention.

[0083] Fig.13 This is a schematic diagram of the task completion rate comparison curve of each solution with a deviation of 0.2G described in the present invention.

[0084] Fig.14 It is a schematic diagram of the comparison curve of the delay costs generated by different schemes under different deviation values ​​described in the present invention. DETAILED DESCRIPTION

[0085] The present invention is further described in detail below so that those skilled in the art can implement it according to the description.

[0086] The present invention provides a method for vehicle networking task offloading and transmission power allocation based on digital twin, which specifically includes:

[0087] Step 1: Figure 1 As shown, the locations of vehicles and base stations are collected to build the Internet of Vehicles and its digital twin network;

[0088] The Internet of Vehicles consists of several base stations equipped with MEC servers and several vehicles;

[0089] The digital twin network includes Base Stations , each base station is equipped with a MEC server, Base stations can form Cooperation groups, each group consists of a base station and its range The vehicles in the coordination group include the mission vehicle , and service vehicles , service vehicles Can handle mission vehicles mission, i.e. mission vehicle Can decide to offload tasks to base station or service vehicle , the task is represented as ,in is the input data size of the task, is the current channel gain of the task, which obeys the Rayleigh distribution. This is because in a dynamic channel environment, the channel state will change with time and space. The Rayleigh distribution can simulate this dynamic change to a certain extent through its random change characteristics. It is to complete the task The amount of CPU cycles required, is the maximum tolerable delay;

[0090] Step 2: Construct an optimization function, including:

[0091] The digital twin (DT) of a base station is a dynamic replica of its physical entity, which updates its status by constantly interacting with the base station itself and the actual network environment. However, the DT cannot accurately reflect the real-time status of the base station. This incomplete reflection is reflected in some parameters in the DT, such as available computing resources, which may deviate from the corresponding parameters of the actual status of the base station. This deviation can be positive or negative, indicating that the parameters in the DT may be higher or lower than the actual value. Furthermore, the system also considers the digital twins of physical entities such as vehicles. However, similar to the DT network of the base station, some information in the DT of these physical entities, such as the available computing resources of the vehicle, may also deviate from its corresponding actual value. This deviation is caused by various uncertainties and imperfections that the DT may encounter during the update process. It is an inherent characteristic of DT technology. Therefore, when designing and using DT-based systems, this deviation needs to be considered and managed to ensure the accuracy and reliability of the system.

[0092] Therefore, considering the deviation between the digital twin layer and the physical entity layer, the execution delay calculation of the task includes:

[0093] (1) When a vehicle on the road performs local tasks, it means that these tasks are processed using the vehicle's own computing resources. Its execution process does not rely on the support of other vehicles on the road and the mobile edge computing (MEC) server, maintaining independence and autonomy. The local calculation formula is as follows:

[0094] ;

[0095] In the formula, is the local execution time before considering the deviation, The amount of CPU cycles required to complete a task, For the Mission Vehicles computing power;

[0096] However, since there is a slight deviation in computing power between the physical entity layer and the digital twin layer, the deviation in latency is calculated using the following formula:

[0097] ;

[0098] In the formula, For the Mission Vehicles The deviation of the execution time delay of the task, For the Mission Vehicles The deviation between the estimated available computing power in DT and the physically actual computing power;

[0099] Therefore, the actual time for local execution is:

[0100]

[0101] In the formula, The actual time of local execution;

[0102] Since the vehicle The proportion of unloading is , then the proportion of local processing is , therefore, the actual time of local execution can be further written as:

[0103]

[0104] In the formula, is the task offloading ratio, and .

[0105] When offloading vehicle tasks in the vehicle edge computing network, the uplink communication delay increases in the offloading phase, and the downlink communication delay increases in the result download phase. Since the downlink communication delay is very small and can be ignored, the upload communication time can be expressed as:

[0106] ;

[0107] In the formula, is the transmission rate and satisfies:

[0108] ;

[0109] In the formula, is the bandwidth, is the transmission power of the vehicle, and , is the maximum constraint on vehicle transmission power, is the background noise power between the vehicle and MEC.

[0110] (2) When Mission Vehicles Select Offload task to Service vehicles The estimated time delay for processing is:

[0111] ;

[0112] In the formula, Before considering the deviation Mission Vehicles Select Offload task to Service vehicles Delays in processing, For the Service vehicles The computing resources available;

[0113] And the Service vehicles The estimated calculation delay deviation in DT is:

[0114] ;

[0115] In the formula, For the Service vehicles The estimated calculation delay deviation in DT is, For the Service vehicles The deviation between the estimated available computing power in DT and the physically actual computing power;

[0116] Therefore, the task from Mission Vehicles Uninstall to Service vehicles The actual execution time is:

[0117] ;

[0118] In the formula, For the task from Mission Vehicles Uninstall to Service vehicles The actual execution time of

[0119] (3) When Mission Vehicles Select Offload task to Base Stations The estimated time delay for processing is:

[0120] ;

[0121] In the formula, is the first Mission Vehicles Offload the task to Base Stations Delays in processing, For the Base Stations The computing resources available;

[0122] And the Base Stations The estimated calculation delay deviation in DT is:

[0123] ;

[0124] In the formula, For the Base Stations The estimated calculation delay deviation in DT is, For the Base Stations The deviation between the estimated available computing power in DT and the physically actual computing power;

[0125] Therefore, the task from Mission Vehicles Uninstall to Base Stations The actual execution time is:

[0126] ;

[0127] In the formula, For the task from Mission Vehicles Uninstall to Base Stations The actual execution time of

[0128] Therefore, the Mission Vehicles When you choose to uninstall a task, the time delay is:

[0129] ;

[0130] Based on the above formula, the calculation formula for the total delay in the unloading process according to the task unloading ratio can be obtained:

[0131] ;

[0132] In the formula, is the total delay during the offloading process according to the task offloading ratio;

[0133] According to the above formula, when , it means that the task has not been offloaded, that is, it is all executed locally. .and Indicates that all tasks are uninstalled, then .

[0134] In summary, the above formula can be used to combine the latency of three strategies: local computing, partial offloading computing, and full offloading computing, and the offloading latency It can be used to test the user's satisfaction during task offloading. Generally speaking, the user's satisfaction increases as the offloading delay decreases. Therefore, the optimization function is:

[0135] ;

[0136] ;

[0137] In the formula, For the Service vehicles The maximum constraint of available computing resources is For the Base Stations The maximum constraint of available computing resources is is the number of task vehicles that need to be unloaded.

[0138] The above constraints represent: Select in the collaborative group , to help it perform calculations; the completion time of each task should not exceed its maximum time delay; each You can choose to perform calculations locally or offload tasks at any ratio; The resources of other vehicles or base stations are positive, but should not exceed their total computing capacity; The transmission power of is positive and does not exceed its maximum power; the base station is All resources should not be allocated more than their total computing capacity.

[0139] In this embodiment, It is 0.5G. It's 10G.

[0140] Step 3: Figure 2 As shown, a Deep Deterministic Policy Gradient (DDPG) model is constructed according to the optimization function to obtain the selected offloading device, task offloading ratio and allocated transmission power under minimizing the total delay of the system;

[0141] State space: The state space is a comprehensive understanding of the overall situation of the system, including the number of vehicles in the collaborative group, the size of each task, the computing resource requirements, the task deadline and the state information, namely:

[0142] ;

[0143] Action space: The action space represents the agent's strategy, that is, each vehicle automatically chooses whether to unload, to which target, and the unloading ratio, where Including local computing, partial offloading and full offloading, such offloading in any proportion optimizes the total delay to a great extent. In addition, there is a close interaction between the dynamic channel state and the transmission rate. The change of the channel state will directly affect the level and stability of the transmission rate. The transmission rate needs to be adaptively adjusted according to the change of the channel state to ensure the efficiency and reliability of data transmission. Therefore, the present invention provides the action of task allocation ratio and allocated transmission power to further minimize the delay, namely:

[0144] ;

[0145] In the formula, express Select to Server Uninstall, where Indicates base stations and service vehicles ,when When , it means that the task vehicle unloads the task to the base station. Other values ​​indicate that the task vehicle unloads the task to the service vehicle. .

[0146] Reward: The reward function reflects the benefit of the action strategy taken to the system, based on the current system state and the selected action , the reward is calculated, and the reward function should be associated with the goal, that is:

[0147] ;

[0148] Here, the constant 1 plays a role in regulating and balancing the size and range of the reward in the reward function, indicating that the lower the delay, the greater the reward.

[0149] In summary, the three components of DRL have been constructed: state space, action space, and reward function, and the task offloading strategy has been introduced.

[0150] First, use the current actor network to interact with the environment to collect data for training. Status , this state Contains all necessary information about the current configuration of the environment, which is used as input to the participant network, which then responds to the current state of the environment. Output an action at the current moment , this action is selected in the continuous action space, and the action Execute into the environment, the environment is based on this action and the current state Update and return the status at the next moment and the corresponding reward at the current moment , the experience replay memory technology is used to eliminate the time correlation between data. During the training process, the four-tuple generated by each round of interaction Stored in the experience pool, the pool acts as a buffer to allow subsequent training to randomly extract samples from it, thereby promoting the model to learn a more generalized strategy.

[0151] Next, the samples stored in the experience pool are used to update the parameters of the critic network. This process is the core part of the Critic network training. A batch of experience samples are randomly drawn from the experience pool. The current state of each sample and actions As the input of the Critic network, it calculates the expected return (Q value) of the corresponding action, called At the same time, in order to train the Critic network, a target reward value is needed to guide the network update. This target reward value is calculated through the target network (Critic's target network and Actor's target network). Specifically, Input into the target network of Actor to get the state of the next moment The best action under (or approximately optimal action), then and Input it into the target network of Critic to get the target reward value, recorded as , the calculation formula is as follows:

[0152] ;

[0153] In the formula, The current moment taken from the experience pool Rewards, is the discount factor, which plays an important role in balancing current rewards and future rewards. are the parameters of the target participant network, It indicates that the target participant network is in the next state The output action is: are the parameters of the target critic network, Refers to the target reward value network. When calculating the target Q value, DDPG uses The output of the network is used instead of directly using the output of the main Q network, which makes the calculation of the target Q value more stable, thereby improving the training effect. is the predicted return value, in the best case equal , so the gradient descent method is used to update the parameters of the current critic network Specifically, by comparing the network predictions and target Q value , a loss value (usually mean square error) can be calculated, and then the back propagation algorithm is used to update the parameters of the evaluation network of the Critic network to minimize this loss value. The formula is as follows:

[0154] ;

[0155] In the formula, the loss value is is the size of each small batch, The current moment taken from the experience pool status, The current moment taken from the experience pool Actions, are the parameters of the current critic network.

[0156] In the DDPG algorithm, the goal of the participant network is to learn a strategy that maximizes the expected return (i.e., Q value) starting from a given state. To achieve this, the parameters of the participant network need to be updated according to the Q value gradient provided by the Critic network. Since the goal is to maximize the Q value, the gradient ascent method is used to update the parameters of the participant network. The gradient ascent method is opposite to the gradient descent method. It updates the parameters along the gradient direction of the function (i.e., the direction in which the function value increases fastest). The specific update method is as follows:

[0157] ;

[0158] In the formula, is the policy gradient when the participant network parameters are updated, For the participant network about parameters The gradient of , which indicates how the generated action changes when the parameters are adjusted, For the Critic network about the action The gradient of represents the sensitivity of the Q value to the action in the current state.

[0159] In addition, as a deep reinforcement learning algorithm, DDPG's Actor network will output a deterministic action in a given state, which helps to stabilize the strategy and improve performance in the later stage of training. However, in the early and middle stages of training, the deterministic strategy may limit the agent's full exploration of the environment, causing the agent to fall into a local optimal solution. In order to overcome this limitation, DDPG adds noise to the actions output by the Actor network. By adding strategies such as Ornstein-Uhlenbeck noise to the actions output by the Actor network, the DDPG algorithm can find a better balance between exploration and utilization. This balance enables the agent to continuously try new actions during training, discover better strategies, and ultimately improve its performance on the target task. At the same time, the smoothness of the noise also helps to maintain the stability and coherence of the exploration process. Therefore, the updated action satisfies:

[0160] ;

[0161] In the formula, for The action after the moment is updated (i.e. ), are the parameters of the participant network, For a participant network in a given state The original action generated by is random noise;

[0162] Finally, the parameters of the target participant network and the target critic network are updated. The parameters of the participant network and the critic network are used to perform soft target updates on the target network. Through the update, the rapid changes of the target value during the learning process are suppressed and the stability of learning is improved. The soft target update is performed as follows:

[0163] ;

[0164] ;

[0165] In the formula, are the parameters of the current critic network, are the parameters of the target critic network, are the parameters of the current participant network, are the parameters of the target participant network, is the scaling factor for soft update, which makes the update of the target network smoother and more stable.

[0166] In order to conduct simulation experiments, a simulation platform was generated in Python 3.8, with a Core i7 processor and 32GB of memory. The experiments were implemented using libraries such as numpy, matplotlib, and pandas. In the channel model, it is assumed that the transmission bandwidth of the channel is W = 10 MHZ and the background noise power density is According to the task model, assuming that the data size of the task is (0.8, 1.2) MB and the amount of CPU cycles required to execute the task is (0.1, 1.0) G, the total computational delay of 9-21 vehicles in the collaborative group is first studied. Then, taking 15 vehicles in the collaborative group as an example, the impact of deviation on each performance is further studied. The Actor network and Critic network of DDPG used in the simulation are composed of four two-layer fully connected neural networks, with 500 and 300 neurons in the two hidden layers respectively. The learning rate of the Actor network is , the learning rate of the Critic network is , the experience pool cache size is 6400, the batch size is 64, and the remaining parameters are recorded in Figure 3 middle.

[0167] like Figure 4 As shown in the figure, the total computing delay under different numbers of motor vehicles is compared. It can be seen that the size of the total computing delay increases with the increase in the number of motor vehicles. However, the total computing delay of the autonomous dynamic partial offloading scheme DTDP (Digital twinning considers deviating task offloading and transmission power allocation) described in the present invention is always the lowest compared with the three schemes of no transmission power allocation offloading (NP-DTDP), local offloading (DTDP-LOCAL) and DQN (Deep Q Network). This is because as the number of motor vehicles increases, the number of tasks that need to be processed also increases, so the total computing delay shows an upward trend. The scheme described in the present invention can dynamically allocate transmission power and unload in proportion, so that the total computing delay is lower than the other three schemes, and the performance is optimal.

[0168] exist Figure 5In the experimental results shown, the influence of different DT deviation values ​​is taken into account, and the schemes with deviations of 0G, 0.1G, 0.15G and 0.2G are considered. By comparison, it can be seen that the corresponding actual calculated total delay is negatively correlated with the DT evaluation deviation. When the deviation value is larger, the difference between the calculated actual delay and the delay without considering the deviation value is larger. This means that in practical applications, if the existence of DT deviation is ignored and only the idealized calculation model is relied upon to evaluate system performance, it may lead to misjudgment of system capabilities, thereby affecting subsequent decision-making and optimization strategies. Therefore, it is very important to consider the impact of DT deviation on calculation delay.

[0169] The proposed DTDP scheme is compared with three schemes: No Transmission Power Distribution Offloading (NP-DTDP), Local Offloading (DTDP-LOCAL) and DQN (Deep Q Network). Figure 6 In the experimental results shown, the ideal conditions without considering the influence of deviation are evaluated. The experimental data clearly show that the proposed scheme ND-DTDP (Nodeviation-DTDP) shows significant advantages in reducing computational delay. Compared with the other three comparison methods, it has the lowest delay. As the number of iterations gradually increases, all schemes show a trend of decreasing computational delay and finally stabilize after 800 iterations. It is worth noting that due to the limitations of vehicle computing resources and the underutilization of server computing resources, the DTDP-LOCAL scheme presents the highest computational delay. On the contrary, the ND-DTDP scheme stands out with its unique advantages, that is, on the basis of using digital twin modeling, the transmission power allocation is dynamically adjusted according to the real-time channel state. This strategy enables the ND-DTDP scheme to use computing resources more effectively, thereby maximizing the utilization of computing resources and minimizing computing delay. At the same time, by comparing with the NP-DTDP scheme, the importance of transmission power allocation can be seen. Therefore, throughout the experiment, the ND-DTDP scheme always maintains the lowest computational delay, and the delay level of the DQN method is still higher than that of the unbiased scheme, which further highlights the superiority of the method described in the present invention in optimizing computational delay.

[0170] The experimental data in Figure 7 , 8, 9, clearly reveal that under different DT deviation settings, namely 0.1G, 0.15G and 0.2G respectively, the solution proposed in the present invention shows significantly better performance than other existing solutions in reducing the actual total calculation delay. Under the same or similar operating environment and parameter configuration, the solution can more efficiently allocate and utilize computing resources, reduce unnecessary waiting time and processing delays, and thus achieve a significant reduction in the total calculation delay overall. The above not only verifies the rationality and scientificity of the design of the solution, but also demonstrates its efficiency and practicality in processing related tasks.

[0171] like Fig.10 , 11 As shown in Figures 12 and 13, the experimental results show that the scheme proposed in the present invention is superior to other schemes in terms of task completion rate, which can reach more than 90%. As one of the key indicators for measuring system performance, the task completion rate directly reflects the ability of the system to ensure that tasks are completed on time. This scheme effectively reduces the average delay and maximum delay of task processing by optimizing the task scheduling strategy and enhancing the system's adaptability to dynamic changes. In addition, it also means that the method described in the present invention can quickly mobilize resources within a strict time frame and flexibly respond to various task requirements, reflecting the system's strong ability to ensure that tasks are completed on time and with quality. In addition, it can be observed that for the DTDP scheme, the task completion rate increases slightly with the increase of deviation. This is because the larger the deviation, the smaller the delay of the actual calculation, the greater the reward, and the better the performance of DTDP.

[0172] like Fig.14 As shown, it shows the cost of different schemes under different deviation values. It can be seen that the scheme described in the present invention has the lowest cost compared with other schemes. This is because the scheme described in the present invention has the smallest calculation delay compared with other schemes, so the cost caused by the delay is the smallest, and the cost decreases with the increase of the deviation value. This is because the larger the deviation value, the lower the actual calculated delay, and the smaller the cost.

[0173] The present invention designs and develops a method for task offloading and transmission power allocation for Internet of Vehicles based on digital twins. Firstly, DTN is used to create a twin model of physical entities. Taking into account the difference between the real value of some entity states and the estimated value in DT, the impact of this difference on the offloading strategy is explored; secondly, the dynamic channel conditions in the communication process and the delay constraints of the tasks are comprehensively considered, and the transmission power is dynamically adjusted to maximize the overall throughput of the system while ensuring the timely completion of tasks, and to intelligently allocate transmission power and reduce transmission delay; finally, the deep deterministic policy gradient (DDPG) algorithm is used to perform task offloading. Compared with traditional task offloading strategies, the present invention, with the assistance of the digital twin network, can adjust the offloading strategy and transmission power allocation of each task in real time according to the current system state and environmental changes to minimize the total system delay.

[0174] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the embodiments shown and described herein.

Claims

1. A method for vehicle networking task offloading and transmission power allocation based on digital twin, characterized in that: The steps include: Step 1: Collect the locations of vehicles and base stations to build the Internet of Vehicles and its digital twin network; Step 2: Construct an optimization function based on the digital twin network: ； ； In the formula, is the total delay in the offloading process according to the task offloading ratio, The total number of mission vehicles, is the total number of service vehicles, is the total number of vehicles in a collaborative group, is the number of base stations in the digital twin network, is the maximum tolerable delay, is the task offloading ratio, and , For the Service vehicles Available computing resources, For the Base Stations Available computing resources, is the transmission power of the vehicle, and , is the maximum constraint on vehicle transmission power, For the Service vehicles The maximum constraint of available computing resources is For the Base Stations The maximum constraint of available computing resources is The number of mission vehicles that need to be unloaded; Step 3: construct a deep deterministic policy gradient model according to the optimization function to obtain the selected offloading device, task offloading ratio and allocated transmission power under the condition of minimizing the total delay of the system; The training process of the deep deterministic policy gradient model includes the following steps: Step 1: Establish a participant network, a critic network, a target participant network, and a target critic network; Step 2: Get the current time from the environment Status Enter the participant network and get Action after time update , the state at the next moment and the current moment reward ; Among them, the updated action at the current moment satisfies: ； In the formula, for The action after the moment is updated, are the parameters of the participant network, For a participant network in a given state The original action generated by is random noise; Step 3: Stored in the experience pool; Step 4: Randomly select a group from the experience pool The state of the moment and Moment of action Input the critic network and obtain the expected return value; Step 5: Change the status Input the target participant network to obtain the corresponding action After that, the status and actions Input the target critic network to get the predicted target return value: ； In the formula, The current moment taken from the experience pool Rewards, is the discount factor, are the parameters of the target participant network, For the target participant network in state The output action is: are the parameters of the target critic network, is the predicted return value; Step 6: Update the parameters of the critic network by minimizing the loss value. Wherein, the loss value satisfies: ； In the formula, Loss value, is the size of each small batch, The current moment taken from the experience pool status, The current moment taken from the experience pool Actions, are the parameters of the current critic network, is the expected reward value of the corresponding action; Step 7: Update the parameters of the participant network by gradient ascent: ； In the formula, is the policy gradient when the participant network parameters are updated, For the participant network about parameters The gradient of , which indicates how the generated action changes when the parameters are adjusted, For the Critic network about the action The gradient of indicates that in the current state, The sensitivity of the value to the action; Step 8: Update the parameters of the target participant network and the target critic network: ； ； In the formula, are the parameters of the current critic network, are the parameters of the target critic network, are the parameters of the current participant network, are the parameters of the target participant network, is the scaling factor for soft update, which makes the update of the target network smoother and more stable.

2. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 1, characterized in that: The digital twin network includes Base stations, each equipped with a MEC server. Base stations Cooperation groups, each of which consists of a base station and its range The vehicles in the coordination group include the mission vehicle and service vehicles , Mission Vehicle Optional offloading of tasks to base station or service vehicle , the task is ; in, , , , , is the input data size of the task, is the current channel gain of the task, To complete the task The amount of CPU cycles required.

3. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 2, characterized in that: The total delay in the unloading process according to the task unloading ratio is satisfied: ； In the formula, For the Mission Vehicles Communication time uploaded during the task offloading phase, For the Mission Vehicles Select the time delay for unloading the task. The actual time of local execution.

4. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 3 is characterized in that: The said Mission Vehicles The communication time for uploading during the task offloading phase meets the following requirements: ； In the formula, is the transmission rate.

5. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 4, characterized in that: The said Mission Vehicles Select the time delay when the task is uninstalled: ； In the formula, For the task from Mission Vehicles Uninstall to Service vehicles The actual execution time, For the task from Mission Vehicles Uninstall to Base Stations The actual execution time.

6. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 5, characterized in that: The actual time of the local execution satisfies: ； ； ； In the formula, is the local execution time before considering the deviation, For the Mission Vehicles The deviation of the execution time delay of the task, The amount of CPU cycles required to complete a task, For the Mission Vehicles The computing power of For the Mission Vehicles The deviation of the estimated available computing power in DT from the physically actual computing power.

7. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 6, characterized in that: The task starts from Mission Vehicles Uninstall to Service vehicles The actual execution time meets: ； ； ； In the formula, is the first Mission Vehicles Select Offload task to Service vehicles Delays in processing, For the Service vehicles Available computing resources, For the Service vehicles The estimated calculation delay deviation in DT is, For the Service vehicles The deviation of the estimated available computing power in DT from the physically actual computing power.

8. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 7, characterized in that: The task starts from Mission Vehicles Uninstall to Base Stations The actual execution time meets: ； ； ； In the formula, Before considering the deviation Mission Vehicles Offload the task to Base Stations Delays in processing, For the Base Stations Available computing resources, For the Base Stations The estimated calculation delay deviation in DT is, For the Base Stations The deviation of the estimated available computing power in DT from the physically actual computing power.

9. The method for vehicle networking task offloading and transmission power allocation based on digital twin as claimed in claim 8, characterized in that: The deep deterministic policy gradient model includes: The state space is: ; The action space is: ; The reward function is: ; in, Mission vehicle Select to Server To offload tasks, ,when When the mission vehicle Offload the task to the base station; when For other values, the mission vehicle Offload tasks to service vehicle.

Images