Construction and resource allocation method of digital twin in internet of vehicles

Abstract

The application relates to a method for constructing and allocating resources of a digital twin in a vehicle Internet, and belongs to the technical field of the vehicle Internet. The method is characterized in that: an accurate digital twin construction scheme based on group learning is proposed to construct an accurate digital twin; based on the constructed digital twin, a task division cooperative processing mechanism is used to unload part of the computing tasks of the vehicle to roadside units or resource idle vehicles for processing; the system performance, the unloading decision and the computing resources of the auxiliary vehicle and the roadside unit are jointly optimized, so that the digital twin assisted edge computing system serves to minimize the weighted sum of the total time delay and the computing energy consumption of the vehicle computing task processing; a multi-agent classification proximal optimization strategy algorithm assisted by the digital twin is used to solve the established joint optimization problem, and the optimal unloading decision is obtained. The application can realize the optimization of the synchronization time delay and the task unloading of the digital twin in the vehicle Internet, and further improve the overall performance of the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking and relates to a method for constructing a digital twin and allocating resources in a vehicle networking. Background Art

[0002] The Internet of Vehicles (IoVs) and digital twins (DTs) are key technologies in intelligent transportation systems. The IoV leverages the internet and communication technologies to connect vehicles, roads, and infrastructure, enabling information sharing and exchange, thereby improving the efficiency and safety of transportation systems. Digital twins are considered a promising technology that can support a variety of new in-vehicle applications for vehicle users, such as autonomous driving, video streaming, path planning, and computation offloading. By creating digital twins of physical entities (vehicles and RSUs) in the IoV, the behavior of each entity in the physical world is mapped to a virtual network (i.e., the DT network) and continuously updated with real-time data collected from sensors. With digital twins, physical entities can be coordinated globally. Digital twins can provide accurate feedback to entities through prediction, estimation, and analysis, thereby improving the quality of service (QoS) of vehicle applications.

[0003] For an ideal digital twin, the physical entity and its digital twin should be perfectly matched. The movement or other behaviors of the entity in the physical and digital spaces are synchronized, so the deviation between them can be negligible or considered a known constant. However, this relies on the assumption of perfect synchronization between the physical and digital spaces. In real-world vehicle networks, due to the high mobility of vehicles, even small initial position or velocity deviations between a vehicle and its digital twin can accumulate into large prediction errors, potentially leading to task offloading failures. Therefore, characterizing the impact of deviations on digital twin performance is crucial, especially in dynamic vehicle networks. Since deviations depend on the frequency of digital twin data updates, increasing the frequency of digital twin data updates can reduce deviations and thus improve digital twin performance. However, this incurs significant communication costs for data exchange between the physical and digital spaces. Therefore, determining an appropriate digital twin update frequency is crucial. Furthermore, due to network complexity (traffic congestion, link bandwidth limitations, and network topology) and the choice of data upload paths, twin data synchronization delays can also affect digital twin construction and, in turn, the subsequent computational offloading results. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method for constructing a digital twin and allocating resources in the Internet of Vehicles, so as to construct an accurate DT so that, with the assistance of the DT, computational offloading and resource allocation can be jointly optimized to minimize system energy consumption and computational delay.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for constructing digital twins and allocating resources in the Internet of Vehicles (IoV) is proposed. This method targets a digital twin-assisted IoV urban intersection scenario, which includes autonomous vehicles, roadside units (ROSs), and base stations. The autonomous vehicles are divided into two categories: mission vehicles and auxiliary vehicles, and both are equipped with mobile edge processors and dual-function radar and communication (DFRC) systems.

[0007] In this Internet of Vehicles scenario, the method proposed by the present invention includes:

[0008] A swarm learning-based solution for building an accurate digital twin is proposed. This approach uses a distributed learning algorithm to predict data upload frequency and waiting time at roadside units, addressing data bias and asynchrony to produce an accurate digital twin.

[0009] Based on the constructed digital twin, the vehicle's computing tasks are partially offloaded to roadside units or idle vehicles through a task division and collaborative processing mechanism.

[0010] Establish a joint optimization problem to jointly optimize system performance, offloading decisions, and the computing resources of auxiliary vehicles and roadside units, so as to minimize the weighted sum of the total latency and computing energy consumption of the digital twin-assisted edge computing system serving the on-board computing task processing;

[0011] The established joint optimization problem is solved through the proximal optimization strategy algorithm of multi-agent classification assisted by digital twins to obtain the optimal unloading decision.

[0012] Furthermore, in the Internet of Vehicles scenario, digital twins of the mission vehicle, auxiliary vehicle and roadside unit are constructed respectively.

[0013] The digital twin of the mission vehicle is represented by: DT k (n)={w k (n),Ω k (n),f k (n)}, DT k (n) represents the digital twin of task vehicle k, w k (n) represents the location information of task vehicle k, Ω k (n) represents its task processing information, f k (n) represents its computing resource information, and n represents the time slot.

[0014] The digital twin of the auxiliary vehicle is represented as: DT j (n)={w j (n),α k,j ,f j (n)}, DT j(n) represents the digital twin of auxiliary vehicle j, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, f j (n) represents the computing resource information of the auxiliary vehicle j, w j (n) indicates its location information.

[0015] The digital twin of the roadside unit is represented by: DT u (n)={w u (n),α k,u ,f u (n)}, DT u (n) represents the digital twin of the roadside unit u, α k,u represents the correlation factor between the task vehicle k and the roadside unit u, f u (n) represents the computing resource information of the drive test unit u, w u (n) indicates its location information.

[0016] Furthermore, when a vehicle sends data to the digital twin, the data is forwarded via the RSU. If the relay node is heavily loaded, the data will wait in the queue for a long time, which will undoubtedly add additional latency to data synchronization. In other words, the latency in the queue will affect the effectiveness of data synchronization. In the data synchronization process of the Internet of Vehicles, in addition to considering the waiting time of the relay node, it is also necessary to consider the adaptive data update frequency, which directly affects the model mapping error and, in turn, affects subsequent task offloading decisions.

[0017] Therefore, this method proposes a swarm learning-based accurate digital twin construction scheme (SL-ADTC). In this scheme, a distributed learning algorithm (SL) is deployed at the DT layer. Its main function is to predict the future waiting time of roadside units deployed along the road based on collected historical data. In this way, when a vehicle sends a data block to the DT, it first obtains the predicted waiting time of different RSUs from the DT layer; then the vehicle selects the appropriate RSU to upload the data.

[0018] The SL-ADTC program includes:

[0019] S1: The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time;

[0020] S2: Virtual distributed nodes (vehicles, RSUs) join SL through blockchain smart contracts to perform local model training;

[0021] S3: The virtual control center assigns the initial global model parameters and local training times to each vehicle and RSU as local model parameters and training indicators respectively;

[0022] S4: Each vehicle and RSU sets its local model parameters to global parameters;

[0023] S5: Each vehicle and RSU is equipped with sensors, radars, and other on-board sensing devices. During operation, a large amount of valuable data can be collected, such as environmental data and vehicle operation data (vehicle speed, location, direction), etc.

[0024] S6: The vehicle and RSU use local model parameters and local data sets to locally update the SL model;

[0025] S7: After multiple local trainings, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation;

[0026] S8: SL completes one iteration of adaptive frequency adjustment and global update of the congestion prediction model, verifies the effectiveness of the parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal RSU to upload data, reducing data upload latency.

[0027] S9: Repeat the above steps to achieve digital twin synchronization, improve the overall system performance, build accurate data twins, and provide effective guarantees for subsequent vehicle task processing.

[0028] Furthermore, with the explosive growth of autonomous vehicles, innovative automotive applications are emerging, bringing new capabilities to the network. This places higher demands on vehicle systems, including task processing accuracy, latency, and system energy consumption. To this end, based on the establishment of accurate data twins, a DT-based Task Partitioning and Co-processing (DT-TPCP) mechanism is proposed. In this mechanism, vehicles decompose compute-intensive tasks into subtasks, offloading some of these tasks to RSUs and vehicles with idle resources. A joint optimization problem is formulated, considering the trade-off between task processing latency and system energy consumption. Weight coefficients are assigned to latency and energy consumption to optimize overall system performance.

[0029] Among them, the task division and collaborative processing mechanism includes: the task vehicle generates a computing task and sends a service request to the control center deployed at the base station through a wireless link; after the control center receives the task request, the decision model in the digital twin layer determines the value of the correlation factor to make an offloading decision; the offloading decision includes local calculation and offloading node auxiliary calculation; based on the offloading decision, the global system delay of all task vehicle service requests and the global energy consumption of the service requests are determined.

[0030] Furthermore, a joint optimization problem is established to jointly optimize system performance, offloading decisions, and the computing resources of auxiliary vehicles and roadside units, so that the weighted sum of the total latency and computing energy consumption of the digital twin auxiliary edge computing system serving the vehicle computing task processing is minimized. The joint optimization problem can be expressed as:

[0031]

[0032] C10 0≤ρ(n)≤1

[0033] C11||a k,j (n)||≤a max ,||v k,j (n)||≤v max

[0034]

[0035] Among them, δ1 and δ2 represent the global system delay and global energy consumption The weighted sum parameter, represents the local computing resource size of task vehicle k, f u represents the computing resource size of the roadside unit u, f j represents the computing resource size of auxiliary vehicle j, ρ represents the task division factor, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, α k,u Represents the correlation factor between task vehicle k and roadside unit u.

[0036] Furthermore, the established joint optimization problem is an NP-hard problem containing nonlinear constraints, which is difficult to solve using traditional optimization algorithms. The DT-assisted multi-agent classification proximal optimization strategy (MACPPO) algorithm is used to solve the above optimization problem.

[0037] The MACPPO algorithm includes:

[0038] S1: Initialize the parameters of each Actor network, initialize the parameters of each Critic network, and initialize the experience pool;

[0039] S2: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment;

[0040] The intelligent agent includes an unloading configuration intelligent agent and an unloading node intelligent agent; the unloading configuration intelligent agent is used to focus on the unloading configuration of the computing task, and the unloading node intelligent agent is used to allocate CPU frequency for the computing task of the task vehicle;

[0041] S3: Based on the current strategy and state, each agent selects an action and executes it;

[0042] S4: All agents interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0043] S5: For each agent, randomly extract a small batch of samples from the experience pool;

[0044] S6: Determine whether the set number of training times is exceeded. If so, stop training; otherwise, proceed to the next step.

[0045] S7: For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network;

[0046] S8: When the training is completed, clear the experience pool.

[0047] The beneficial effects of the present invention are:

[0048] (1) This paper proposes a DT-assisted frequency and latency prediction scheme based on group learning. Based on historical data, real-time training status, and resource conditions, it adaptively selects the optimal data upload path and twin frequency, minimizing the synchronization delay of the digital twin and enabling the construction of an accurate digital twin model. This solves the contradiction between data synchronization, resource allocation, task latency, and system energy consumption in intelligent driving of the Internet of Vehicles.

[0049] (2) The present invention uses a DT-based task division and collaborative processing mechanism to offload some tasks to auxiliary vehicles or roadside units to complete computing services under the intelligent decision-making provided by DT, which can cope with the heavy computing tasks of vehicles and improve task processing efficiency.

[0050] (3) By establishing a multi-objective optimization problem and reasonably allocating computing resources, the optimal balance between task delay and energy consumption is achieved, and the optimization problem is solved by a DT-assisted proximal optimization strategy algorithm for multi-agent classification, thereby achieving the optimal allocation of computing resources.

[0051] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0053] Figure 1 This is the system scenario targeted by the method of the present invention;

[0054] Figure 2 Schematic diagram of the DT-assisted group learning framework;

[0055] Figure 3 Offloading flow charts for computational tasks;

[0056] Figure 4 This is the network structure diagram of the MACPPO algorithm. DETAILED DESCRIPTION

[0057] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0058] The present invention proposes a method for constructing a digital twin and allocating resources in the Internet of Vehicles. By constructing an accurate digital twin, computational offloading and resource allocation are jointly optimized with the assistance of the digital twin, thereby minimizing system energy consumption and computational delay.

[0059] The specific content of this method is described below through an embodiment.

[0060] 1. If Figure 1 The system scenario shown considers a DT-assisted connected vehicle network at an urban intersection. This scenario consists of autonomous vehicles, roadside units, and base stations. The autonomous vehicles are divided into two categories: mission vehicles and auxiliary vehicles. Both are equipped with mobile edge processors and a dual-function radar and communication (DFRC) system. DFRC technology integrates communication and sensing functions into practical devices. In addition to traditional communication transmission functions, it also provides environmental perception capabilities such as video surveillance and radar detection. Using DFRC technology, the mission vehicle can simultaneously perform radar sensing and mission data offload, saving bandwidth resources.

[0061] 2. Targeting Figure 1The system scenario shown in the figure is DT modeled: for the task vehicle k, the virtual twin needs to record its task information and location, which can be characterized as DT k (n)={w k (n),Ω k (n),f k (n)},w k (n) represents the location information of task vehicle k, Ω k (n) represents its task processing information, f k (n) represents its computing resource information, and n represents the time slot.

[0062] For auxiliary vehicles, DT needs to reflect its service scheduling, including resource allocation and motivation status. Therefore, the virtual twin of auxiliary vehicle j can be represented as DT j (n)={w j (n),α k,j ,f j (n)}, f j (n) represents the computing resource information of the auxiliary vehicle j, w j (n) represents its position information, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j.

[0063] Similarly, for each roadside unit, DT also needs to reflect its service scheduling, including resource allocation and motivation status. Therefore, the virtual twin of roadside unit u can be represented as DT u (n)={w u (n),α k,u ,f u (n)}, f u (n) represents the computing resource information of roadside unit u, w u (n) represents its position information, α k,u Represents the correlation factor between task vehicle k and roadside unit u.

[0064] 3. DT synchronization delay modeling:

[0065] The vehicle DT synchronization delay includes the following stages:

[0066] ① The vehicle uploads DT synchronization data to the RSU: Wireless communication is used between the vehicle and the RSU. The maximum uplink information rate between the vehicle and the RSU is:

[0067]

[0068] Where B is the transmission channel bandwidth, p k is the transmission power of the mission vehicle, h ku is the channel gain, N0 is the noise power spectral density, is the transmission path loss. ku is the distance between the task vehicle k and the roadside unit u, γ0 is the attenuation factor. Vehicle k synchronizes DT data Upload to RSU, the transmission delay is recorded as:

[0069]

[0070] ② RSU forwards DT synchronization data to the base station: The transmission between RSU and base station is wired, and the wired transmission delay is related to the amount of data transmitted and the transmission distance. The wired transmission delay between RSU and base station is:

[0071]

[0072] in, is the delay required to transmit a unit bit of data per unit distance, d uBS is the distance between the roadside unit u and the base station, t waiting represents the queue waiting time at roadside unit u.

[0073] ③ The server at the base station processes DT synchronization data: The server's computing resources can be allocated to vehicles and RSUs to maintain their DT information. The server's processing time for vehicle-uploaded data is recorded as:

[0074]

[0075] in, Indicates that when the amount of data transmitted is The amount of calculation required, Indicates the amount of computing resources occupied by the twin corresponding to task vehicle k.

[0076] Therefore, the DT synchronization delay of the vehicle can be calculated as:

[0077]

[0078] 4. DT model accuracy characterization

[0079] The relationship between packet error rate and signal to interference and noise ratio in wireless communication can be described as:

[0080]

[0081] Where, ε k,u (n) represents the interference of the wireless link between the vehicle and the RSU, is a piecewise function, expressed as Therefore, when the interference plus noise is very small, the corresponding packet error rate is also very small; when it exceeds the threshold x0, the packet error rate increases sharply and tends to 1. k,u(n) represents the channel gain at time n.

[0082] DT synchronization mapping is a continuous process. An error in the synchronization data packet within a certain time slot may not cause a serious DT mapping deviation. Therefore, the accuracy of DT mapping should be related to the packet error rate and packet loss rate over a period of time. The DT accuracy is specifically expressed as:

[0083]

[0084] In the formula, γ′0∈[0,1] is the time discount factor, which measures the impact of the packet error rate of the wireless channel in past time slots on the DT mapping accuracy at the current time. In other words, the packet error rate in more distant time slots is considered to have a smaller impact on the DT accuracy at the current time. This reflects the fact that when measuring the DT accuracy of a particular time slot, more emphasis is placed on the wireless channel packet error rate in the most recent time slot. n0 represents the n0th time slot, n represents the nth time slot, and N′ represents the left shift N′ relative to time n.

[0085] 5. A swarm learning-based accurate digital twin construction (SL-ADTC) scheme is proposed to achieve digital twin synchronization. The DT-assisted swarm learning framework is shown in the figure below. Figure 2 The SL-ADTC scheme is as follows:

[0086] (1) The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time;

[0087] (2) Virtual distributed nodes (vehicles, RSUs) join the distributed learning algorithm (SwarmLearning, SL) through blockchain smart contracts to perform local model training;

[0088] (3) The virtual control center assigns the initial global model parameters and local training times to each vehicle and RSU as local model parameters and training indicators, respectively;

[0089] (4) Each vehicle and RSU sets its local model parameters to global parameters;

[0090] (5) Each vehicle and RSU is equipped with sensors, radars and other on-board sensing equipment. During operation, a large amount of valuable data can be collected, such as environmental data, vehicle operation data (vehicle speed, position, direction), etc., and local preprocessing is performed to obtain data samples D in combination with historical data. k ={(x1,y1),(x2,y2),…,(x i ,y i )},(x i ,y i) represents the sample data. The total amount of sample data of all vehicles within the communication range of the base station can be expressed as D = ∑D k .

[0091] The goal of SL is to minimize the weighted average loss:

[0092]

[0093] Where n k =|D k |,,F k (w) represents the average loss of the local node, and K represents the total number of SL nodes participating in learning.

[0094] The local objective of the SL node is to minimize the average loss:

[0095]

[0096] Where, f i (w) is the local loss function, n k represents the total number of samples, Indicates seeking loss.

[0097] (6) The vehicle and RSU use the local model parameters and local data set to locally update the SL model according to the following formula:

[0098]

[0099] Where η is the learning rate, Indicates finding the gradient.

[0100] (7) After τ times of local training, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation;

[0101] (8) When the temporary central node receives the local node parameters, the central node begins to aggregate the model according to the weighted average parameter merging method to obtain the global model parameters:

[0102]

[0103] Where w represents the global model parameter after aggregation, w k Represents the local model parameters of the virtual SL node k. In order to achieve the optimization goal of equation (8), the SL node and the temporary central node will continuously cycle the training process for training updates.

[0104] (9) SL completes one iteration of adaptive frequency adjustment and global update of the congestion prediction model, verifies the effectiveness of the parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal RSU to upload data, reducing the data upload delay.

[0105] (10) Repeat the above steps to achieve digital twin synchronization, improve the overall performance of the system, build accurate data twins, and provide effective guarantees for subsequent vehicle task processing.

[0106] 6. A DT-based task partitioning collaborative processing (DT-TPCP) mechanism is proposed, such as Figure 3 The details are as follows:

[0107] Define the correlation factor α between the task vehicle k and the auxiliary vehicle j k,j , the correlation factor α between the task vehicle k and the roadside unit u k,u , α k,j ,α k,u ∈(0,1). Based on the data of each physical entity recorded in the digital twin model, deep reinforcement learning is used to analyze the task requests and historical data (network traffic, computing resources, etc.) that the task vehicle needs to process. The decision model in the DT layer determines the value of the correlation factor to determine the unloading strategy, as shown below:

[0108] α k,j =0,α k,u =0: the task vehicle processes locally;

[0109] α k,j =0,α k,u =1: RSU assisted parallel processing;

[0110] α k,j =1,α k,u =0: auxiliary vehicle auxiliary parallel processing;

[0111] Not considering α k,j =1,α k,u =1.

[0112] Define a triplet at the beginning of each time slot Among them D k (n) is the data size of the generated computing task, C k (n) is the average number of CPU cycles required to process a unit bit of data, The maximum allowed delay is Z. k (n) = ρ(n)D k (n) tasks are locally calculated, and the other part of the data volume is Z k' (n)=[1-ρ(n)]Dk The task of (n) is offloaded to the auxiliary vehicle or roadside unit for processing. Where ρ(n) is the task division factor, 0<ρ(n)<1.

[0113] 1) Local computing: α k,j =0,α k,u =0

[0114] In the case of local computing, processes 1 and 2 in the system (such as Figure 3 The delay caused by (as shown) is negligible and only includes processes 3 and 4, that is, the local computing time and the decision-making time of the digital twin, which is calculated as follows at time slot n:

[0115]

[0116] Where D k' (n) is the service request data size, C DT (n) represents the computational cost of the neural network module processing 1 bit of data, Indicates the computing resources allocated to the DT layer by the edge server in the control center. Represents local computing resources, Indicates the amount of data uploaded by roadside unit u.

[0117] Therefore, in the case of local computation, the system latency is:

[0118] 2) Roadside unit auxiliary parallel computing: α k,j =0,α k,u =1, which is similar to the local calculation process. Here, the calculation process is ignored and the result is given directly:

[0119]

[0120] Where, Indicates the local computing latency, represents the wireless transmission delay between the task vehicle k and the adjacent roadside unit u', represents the wired transmission delay between RSU u' and RSU u, Represents the task processing delay of roadside unit u.

[0121] 3) Auxiliary vehicle auxiliary parallel computing: α k,j =1,α k,u =0, the calculation results are as follows:

[0122]

[0123] in, represents the wireless transmission delay between task vehicle k and auxiliary vehicle j, represents the computational delay of auxiliary vehicle j

[0124] Therefore, the service delay for any service request is expressed as:

[0125]

[0126] Where, α k,j , α k,u When both take the value of 0, χ(α k,j ,α k,u )=0; otherwise, it is equal to 1.

[0127] Therefore, the system's calculation formula for the global service delay of all mission vehicles in time slot n is:

[0128] 7. Establish an energy consumption model, which is modeled as follows:

[0129] (1) Mission vehicle energy consumption: The effective capacitance coefficient of the CPU of mission vehicle k is κ1.

[0130] At time slot n, the energy consumption of local computation is as follows:

[0131]

[0132] Where, f k (n) represents the computing resources of task vehicle k,

[0133] The transmission energy consumption of task vehicle k is expressed as:

[0134]

[0135] Where, represents the transmission delay between task vehicle k and roadside unit u, represents the transmission power of mission vehicle k

[0136] (2) Energy consumption of auxiliary vehicles and roadside units: The effective capacitance coefficients of the auxiliary vehicle and roadside unit CPU are κ2 and κ3, respectively.

[0137] When auxiliary vehicle j and roadside unit u provide computing services for task vehicle k in time slot n, the computing energy consumption is:

[0138]

[0139] Among them, Z k′ (n) represents part of the task data, f j (n) represents the computing power of the auxiliary vehicle j, f u (n) represents the computing capability of the roadside unit u.

[0140] Control center decision-making energy consumption:

[0141]

[0142] Among them, T twin (n) represents the DT layer decision delay, P c (n) represents the computing power of the control center server in processing service requests.

[0143] In summary, the total system energy consumed to process the computing task on task vehicle k is:

[0144]

[0145] Therefore, the global energy consumption of the system for computing tasks on all task vehicles in time slot n can be expressed as:

[0146]

[0147] Based on the above, the weighted sum of the total latency and computing energy consumption of the digital twin-assisted edge computing system serving on-board computing tasks can be minimized by jointly optimizing system performance, offloading decisions, and the computing resources of auxiliary vehicles and roadside units. The corresponding optimization problem is expressed as follows:

[0148]

[0149] C10 0≤ρ(n)≤1

[0150] C11||a k,j (n)||≤a max ,||v k,j (n)||≤v max

[0151]

[0152] Where δ1 and δ2 are weighted sum parameters of delay and computational energy consumption. U represents the total number of roadside units, J represents the total number of auxiliary vehicles, Indicates the maximum allowed delay. It represents the maximum total system energy consumed to process the computing task on task vehicle k. represents the local computing resource size of task vehicle k, ρ represents the task ratio division factor, a k,j (n) represents the unloading correlation coefficient, a max Indicates the maximum acceleration of the vehicle, v k,j (n) represents the speed of the mission vehicle and auxiliary vehicle, v max Indicates the maximum acceleration of the vehicle, w r (n) and w s (n) The location information of two different cars, d minIndicates the minimum safe distance between vehicles. represents the maximum computing capacity of mission vehicle k, Indicates the maximum computing capacity of the roadside unit u, represents the maximum computing capability of the auxiliary vehicle j.

[0153] Constraint C1 ensures that the task vehicle is associated with at most the next unloading node, and C2 indicates that the tolerable task processing delay does not exceed C3 limits the system energy consumption of the computing task on task vehicle k to no more than C4 limits the mission vehicle's transmission power to no more than C5 limits the computing power of the control center server to no more than C6, C7, C8, and C9 limit the computing resources allocated by each network node to the computing task of task vehicle k. C10 is the computing task division ratio constraint, C11 is the acceleration and speed limit of the task vehicle and auxiliary vehicle, and C12 represents the minimum safe distance between vehicles.

[0154] The optimization model shown in Equation (22) contains nonlinear constraints. This problem is NP-hard and difficult to solve using traditional optimization algorithms. Therefore, a DT-assisted multi-agent classification proximal optimization strategy (MACPPO) algorithm is proposed to solve it. The process is as follows:

[0155] 1) Offloading configuration agent: This type of agent focuses on the offloading configuration of the task, and its index is represented as In order to determine the unloading ratio and the association with the target unloading node, they need to observe the location and mission information of the task vehicle and the location of the unloading node.

[0156] Observations: The following observations were made about the uninstall configuration agent:

[0157]

[0158] Where k represents the mission vehicle number.

[0159] Action: The action of the uninstall configuration agent should represent the decision variables and the uninstall ratio, so it can be defined as:

[0160]

[0161] Among them, ρ k [n] represents the task division factor of task vehicle k at time n.

[0162] Reward: The reward function for the unloading configuration agent needs to include the target and the penalty for not meeting the delay requirement. The energy consumption of each task vehicle and its associated destination unloading node needs to be decomposed. Therefore, the reward of agent l is:

[0163]

[0164] in, represents the average energy consumption of the mission vehicle, Indicates the signal transmission power.

[0165] 2) Unloading node agent: The unloading node needs to allocate CPU frequency for the computing task of the task vehicle. The index set of the unloading node agent is Its elements include:

[0166] Observation: Each unloading node can obtain the location and task information of the task vehicles within its service range. Therefore, the observation value can be expressed as:

[0167]

[0168] in, Indicates the unloading of the task vehicle served by node m.

[0169] Action: The offloading node needs to allocate its own CPU frequency to perform the task of the mission vehicle, so its action is:

[0170]

[0171] like Figure 4 The figure shows the training framework of DT-assisted multi-agent deep reinforcement learning (MADRL). During the training process, the centralized state value function of the u-th agent is defined as:

[0172]

[0173] Among them, s t represents the environmental state at time t, θ u represents the new policy parameters, represents the expectation operation, represents the discount factor, represents the reward function, a t+1 Indicates the action performed at time t+1.

[0174] The action-value function is defined as follows:

[0175]

[0176] Therefore, the advantage function can be obtained as:

[0177]

[0178] By the state value V u (s t ) is estimated to be:

[0179]

[0180] Among them, γ u represents the discount factor, r t+l Indicates reward.

[0181] The advantage function is evaluated using the generalized advantage estimation (GAE). The GAE factor λ controls the trade-off between reward variance and bias, δ t =(r t +γ u V u (s t+1 )-V u (s t+l )) is the time difference error. Therefore, the critic can be updated by tracking the loss function:

[0182]

[0183] Among them, w u Represents the weight parameter, which is used to optimize the performance of the algorithm during the learning process. Represents the estimated value function under the optimized weights.

[0184] For the actor network, a clipping factor is introduced to limit the update rate of the policy, which effectively replaces the calculation of the trust region. On this basis, the loss function of the actor network is expressed as:

[0185]

[0186] Among them, clip(·) represents a function used to limit the input to a certain interval, which is usually used to maintain the stability of policy updates. represents the new policy function, represents the old policy function, ∈ represents the update step threshold, ψS t,u The policy entropy representing the degree of exploration, S t,u represents the state of the environment at time t.

[0187] Combined with the above content, the specific steps of the MACPPO algorithm are as follows:

[0188] S1: Initialize the parameters of each Actor network, initialize the parameters of each Critic network, and initialize the experience pool;

[0189] S2: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment;

[0190] S3: Based on the current strategy and state, each agent selects an action and executes it;

[0191] S4: All agents interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0192] S5: For each agent, randomly extract a small batch of samples from the experience pool;

[0193] S6: Determine whether the set number of training times is exceeded. If so, stop the training; otherwise, continue to execute S7;

[0194] S7: For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network;

[0195] S8: When the training is completed, clear the experience pool.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a digital twin and allocating resources in an Internet of Vehicles, characterized by: A swarm learning-based solution for building an accurate digital twin is proposed. This approach uses a distributed learning algorithm to predict data upload frequency and waiting time at roadside units, addressing data bias and asynchrony to produce an accurate digital twin. Based on the constructed digital twin, the vehicle's computing tasks are partially offloaded to roadside units or idle vehicles through a task division and collaborative processing mechanism. The task division and collaborative processing mechanism includes: The mission vehicle generates computing tasks and sends service requests to the control center deployed at the base station via wireless links; After the control center receives the task request, the decision model in the digital twin layer determines the value of the correlation factor to make an offloading decision; the offloading decision includes local calculation and offloading node auxiliary calculation; Determine the global system delay of all mission vehicle service requests and the global energy consumption of service requests based on the offloading decision; The correlation factor includes the correlation factor α between the task vehicle k and the auxiliary vehicle j k,j and the correlation factor α between the task vehicle k and the roadside unit u k,u According to the correlation factors, the uninstallation decision includes: 1)α k,j =0 and α k,u =0, the task is processed locally in the vehicle; 2)α k,j =0 and α k,u =1, offload some tasks to the roadside unit to assist in parallel processing; 3)α k,j =1 and α k,u =0, offload some tasks to the auxiliary vehicle for parallel processing; The task volume unloaded to the roadside unit or auxiliary vehicle is expressed as: Z k' (n)=[1-ρ(n)]D k (n), ρ(n) represents the task division factor, 0<ρ(n)<1, D k (n) represents the data size of the computing task; A joint optimization problem is established to jointly optimize system performance, offloading decisions, and the computing resources of auxiliary vehicles and roadside units, so that the weighted sum of the total latency and computing energy consumption of the digital twin-assisted edge computing system serving the on-board computing task processing is minimized. The established joint optimization problem is expressed as: C10 0≤ρ(n)≤1 C11||a k,j (n)||≤a max ,||in k,j (n)||≤v max Where δ1 and δ2 represent the global system delay and global energy consumption The weighted sum parameter of ; U represents the total number of roadside units, J represents the total number of auxiliary vehicles, Indicates the maximum allowed delay. represents the maximum energy consumption required to process the computing task on task vehicle k, represents the maximum value of computing resources allocated to the digital twin layer by the edge server of the control center, a k,j 、v k,j Denote the acceleration and speed of the vehicle, respectively, a max 、v max Represent the maximum values ​​of acceleration and velocity respectively; represents the local computing resource size of task vehicle k, ρ represents the task ratio division factor, a k,j (n) represents the unloading correlation coefficient, a max Indicates the maximum acceleration of the vehicle, v k,j (n) represents the speed of the mission vehicle and auxiliary vehicle, v max Indicates the maximum acceleration of the vehicle, w r (n) and w s (n) represents the location information of two different vehicles, d min Indicates the minimum safe distance between vehicles. represents the maximum computing capacity of mission vehicle k, Indicates the maximum computing capacity of the roadside unit u, represents the maximum computing capacity of auxiliary vehicle j; The established joint optimization problem is solved through the proximal optimization strategy algorithm of multi-agent classification assisted by digital twins to obtain the optimal unloading decision.

2. The method according to claim 1, wherein: In the IoV scenario, digital twins of the mission vehicle, auxiliary vehicle, and roadside unit are constructed separately; The digital twin of the mission vehicle is represented by: DT k (n)={w k (n),Ω k (n),f k (n)}, DT k (n) represents the digital twin of task vehicle k, w k (n) represents the location information of task vehicle k, Ω k (n) represents the task processing information of task vehicle k, f k (n) represents the computing resource information of task vehicle k, and n represents the time slot; The digital twin of the auxiliary vehicle is represented as: DT j (n)={w j (n),α k,j ,f j (n)}, DT j (n) represents the digital twin of auxiliary vehicle j, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, f j (n) represents the computing resource information of the auxiliary vehicle j, w j (n) represents the position information of the auxiliary vehicle j; The digital twin of the roadside unit is represented by: DT u (n)={w u (n),α k,u ,f u (n)}, DT u (n) represents the digital twin of the roadside unit u, α k,u represents the correlation factor between the task vehicle k and the roadside unit u, f u (n) represents the computing resource information of the drive test unit u, w u (n) represents the location information of the drive test unit u.

3. The method according to claim 2, wherein: For the constructed digital twin, digital twin synchronization is performed through an accurate digital twin construction scheme based on group learning, which includes the following steps: 1) The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time; 2) Virtual distributed nodes join the distributed learning algorithm through blockchain smart contracts to perform local model training; 3) The virtual control center distributes the initial global model parameters and local training times to each vehicle and roadside unit as local model parameters and training indicators, respectively; 4) Each vehicle and roadside unit sets its local model parameters as global parameters; 5) Each vehicle and roadside unit collects data through sensor equipment, performs local preprocessing, and combines historical data to obtain data sample D k ={(x1,y1),(x2,y2),…,(x i ,y i )},(x i ,y i ) represents sample data; 6) The vehicle and roadside unit use the local model parameters and local data set to locally update the distributed learning algorithm model according to the following formula: Where η represents the learning rate, w k represents the local model parameters of the SL node, represents the function gradient operator, n k represents the total number of samples, f i (w) represents the local loss function, F k (w) represents the average loss of the local node; 7) After τ times of local training, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation; 8) When the temporary central node receives the local node parameters, the central node begins to aggregate the model according to the weighted average parameter merging method to obtain the global model parameters: Where w represents the aggregated global model parameters, and the SL nodes and temporary central nodes continuously cycle through the training process for training updates; 9) The SL node completes one iteration of adaptive frequency adjustment and global update of the congestion prediction model, verifies the effectiveness of the parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal roadside unit to upload data, reducing data upload latency. 10) Repeat the above steps to achieve digital twin synchronization and provide guarantee for vehicle mission processing.

4. The method according to claim 1, wherein: When the unloading decision is processed locally by the task vehicle, the system delay is expressed as: Where, represents the local computation time, represents the digital twin decision time; When the offloading decision is to offload part of the task to the auxiliary vehicle for auxiliary parallel processing, the system delay is expressed as: Where, Indicates the local computing latency, represents the wireless transmission delay between task vehicle k and auxiliary vehicle j, represents the computational delay of auxiliary vehicle j; When the offloading decision is to offload part of the tasks to the roadside unit for auxiliary parallel processing, the system delay is expressed as: Where, represents the wireless transmission delay between the task vehicle k and the adjacent roadside unit u', represents the wired transmission delay between RSU u' and RSU u, represents the task processing delay of roadside unit u; For any service request of a mission vehicle, the system delay is expressed as: In the formula, when α k,j , α k,u When both take the value of 0, χ(α k,j ,α k,u )=0, otherwise equal to 1; The global system delay of service requests of all task vehicles in time slot n is expressed as: K represents the number of mission vehicles.

5. The method according to claim 1, wherein: The global energy consumption includes the energy consumption of mission vehicles, the energy consumption of unloading nodes and the energy consumption of control center decision-making; The energy consumption of the mission vehicle includes local computing energy consumption and transmission energy consumption, which are expressed as: Where, represents the local computing energy consumption, represents the transmission energy consumption, κ1 represents the effective capacitance coefficient of the CPU of the mission vehicle, and f k (n) represents the computing power of task vehicle k, represents the wireless transmission delay between task vehicle k and roadside unit u, represents the transmission power of mission vehicle k; The energy consumption of the offloading node is expressed as: Where κ2 and κ3 represent the effective capacitance coefficients of the CPU of the auxiliary vehicle and the roadside unit, respectively, and Z k′ (n) represents part of the task data, f j (n) represents the computing power of the auxiliary vehicle j, f u (n) represents the computing capacity of the roadside unit u; The decision energy consumption of the control center is expressed as: Where, T twin (n) represents the DT layer decision delay, P c (n) represents the computing power of the control center server in processing service requests; The energy consumption required to process the computing task of task vehicle k is: The global energy consumption required for the computational tasks of all task vehicles in time slot n is expressed as:

6. The method according to claim 1, wherein: The proposed joint optimization problem is solved by a proximal optimization strategy algorithm for multi-agent classification assisted by digital twins. The algorithm includes: 1) Initialize the parameters of each Actor network, initialize the parameters of each Critic network, and initialize the experience pool; 2) In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment; The intelligent agent includes an unloading configuration intelligent agent and an unloading node intelligent agent; the unloading configuration intelligent agent is used to focus on the unloading configuration of the computing task, and the unloading node intelligent agent is used to allocate CPU frequency for the computing task of the task vehicle; 3) Based on the current strategy and state, each agent selects an action and executes it; 4) All agents interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool; 5) For each agent, randomly extract a small batch of samples from the experience pool; 6) Determine whether the set number of training times has been exceeded. If so, stop training; otherwise, proceed to the next step. 7) For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network; 8) When the training is completed, clear the experience pool.

Images