Construction and resource allocation method of digital twinborn in Internet of Vehicles

By adopting an accurate digital twin construction scheme based on group learning and a collaborative processing mechanism for task division in the Internet of Vehicles, combined with the near-end optimization strategy of multi-agent classification, the problems of digital twin deviation and communication costs in the Internet of Vehicles are solved, and the problem of digital twin deviation and communication costs in the Internet of Vehicles is realized. Optimization of system performance and resource allocation reduces data synchronization delay and energy consumption.

CN119012390AActive Publication Date: 2024-11-22CHONGQING UNIV OF POSTS & TELECOMM

Patent Information

Application Number
CN202411091394.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-09
Publication Date
2024-11-22
Estimated Expiration
2044-08-09

AI Technical Summary

Technical Problem

In the Internet of Vehicles, deviations between digital twins will cause task offloading failure, and increasing the frequency of data updates to reduce deviations will increase communication costs. How to ensure system performance while reducing deviations and energy consumption?

Method used

Using an accurate digital twin construction scheme based on group learning, the data upload frequency and waiting time are predicted through distributed learning algorithms, combined with the task division collaborative processing mechanism and the near-end optimization strategy of multi-agent classification, optimize system performance and resource allocation to realize data Minimize synchronization delay and energy consumption.

Benefits of technology

It realizes the precise construction of digital twins in the Internet of Vehicles, reduces data synchronization delay and system energy consumption, improves task processing efficiency and optimization of computing resources, and solves the contradiction between data synchronization and resource allocation in the Internet of Vehicles.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119012390A_ABST
    Figure CN119012390A_ABST
Patent Text Reader

Abstract

The invention relates to a digital twinborn construction and resource allocation method in the Internet of Vehicles, and belongs to the technical field of the Internet of Vehicles. The method is characterized in that a precise digital twinborn construction scheme based on group learning is provided, and precise digital twinborn bodies are constructed; based on the constructed digital twinborn body, unloading a calculation task part of the vehicle to a road side unit or a resource idle vehicle for processing through a task division cooperative processing mechanism; the system performance, the unloading decision and the computing resources of the auxiliary vehicle and the road side unit are jointly optimized, so that the total time delay of the digital twin auxiliary edge computing system serving the vehicle-mounted computing task processing and the weighted sum of the computing energy consumption are minimum; and solving the established joint optimization problem through a digital twinning-assisted multi-agent classification near-end optimization strategy algorithm to obtain an optimal unloading decision. According to the invention, optimization of digital twinborn synchronous time delay and task unloading in the Internet of Vehicles can be realized, so that the overall performance of the system is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking and relates to a method for constructing a digital twin and allocating resources in a vehicle networking. Background Art

[0002] Internet of Vehicles (IoVs) and digital twins (DTs) are key technologies in intelligent transportation systems. IoVs uses the Internet and communication technologies to connect vehicles, roads, and infrastructure to share and exchange information, thereby improving the efficiency and safety of transportation systems. Digital twins are seen as a promising technology that can support a variety of new in-vehicle applications for vehicle users, such as autonomous driving, video streaming, path planning, and computational offloading. By creating digital twins of physical entities (vehicles and RSUs) in the IoV, the behavior of each entity in the physical world is mapped to a virtual network (i.e., DT network) and continuously updated with real-time data collected from sensors. With digital twins, physical entities can be coordinated globally, and digital twins can provide accurate feedback to entities through prediction, estimation, and analysis, thereby improving the quality of service (QoS) of vehicle applications.

[0003] For an ideal digital twin, the physical entity and the digital twin should be perfectly matched. The movement or other behaviors of the entity in the physical world and the digital space are synchronized, so the deviation between them can be ignored or considered as a known constant, but this relies on the assumption that the physical world and the digital space are perfectly synchronized. In actual vehicle networks, due to the high mobility of vehicles, small initial position or velocity deviations between the vehicle and its digital twin can accumulate into large prediction errors, which may lead to task offloading failure. Therefore, it is crucial to characterize the impact of deviations on the performance of digital twins, especially in dynamic vehicle networks. The deviation depends on the value of the digital twin data update frequency, so the deviation can be reduced by increasing the digital twin data update frequency, thereby improving the performance of the digital twin, but this will result in a large communication cost for data exchange between the physical space and the digital space. Therefore, it is crucial to determine a suitable digital twin update frequency. In addition, due to the complexity of the network (traffic congestion, link bandwidth limitations, and network topology) and the choice of the path for data upload, the twin data synchronization delay will also affect the construction of the digital twin, and then affect the subsequent calculation offloading results. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method for constructing a digital twin and allocating resources in a connected vehicle network, so as to construct an accurate DT, so that with the assistance of the DT, computational offloading and resource allocation can be jointly optimized to minimize system energy consumption and computational delay.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for constructing a digital twin and allocating resources in a connected vehicle network. The method is aimed at a digital twin-assisted connected vehicle network urban intersection scenario, which includes autonomous driving vehicles, roadside units and base stations. The autonomous driving vehicles are divided into two categories: task vehicles and auxiliary vehicles, and they are all equipped with mobile edge processors and dual-function radar and communication systems (DFRC) systems.

[0007] In this Internet of Vehicles scenario, the method proposed by the present invention includes:

[0008] A precise digital twin construction scheme based on group learning is proposed. The distributed learning algorithm is used to predict the data upload frequency and the waiting time at the roadside unit, solve the data deviation and asynchrony, and obtain a precise digital twin.

[0009] Based on the constructed digital twin, the vehicle's computing tasks are partially offloaded to the roadside unit or resource-free vehicles for processing through the task division and collaborative processing mechanism;

[0010] Establish a joint optimization problem to jointly optimize system performance, unloading decisions, and computing resources of auxiliary vehicles and roadside units, so that the weighted sum of total latency and computing energy consumption of the digital twin-assisted edge computing system serving the on-board computing task processing is minimized;

[0011] The established joint optimization problem is solved through the proximal optimization strategy algorithm of multi-agent classification assisted by digital twins to obtain the optimal unloading decision.

[0012] Furthermore, in the Internet of Vehicles scenario, digital twins of the mission vehicle, auxiliary vehicle and roadside unit are constructed respectively.

[0013] The digital twin of the mission vehicle is represented by: DT k (n) = {w k (n),Ω k (n),f k (n)}, DT k (n) represents the digital twin of task vehicle k, w k (n) represents the location information of task vehicle k, Ω k (n) represents its task processing information, f k (n) represents its computing resource information, and n represents the time slot.

[0014] The digital twin of the auxiliary vehicle is represented as: DT j (n) = {w j (n),α k,j ,f j (n)}, DT j(n) represents the digital twin of auxiliary vehicle j, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, f j (n) represents the computing resource information of auxiliary vehicle j, w j (n) indicates its location information.

[0015] The digital twin of the roadside unit is represented by: DT u (n) = {w u (n),α k,u ,f u (n)}, DT u (n) represents the digital twin of roadside unit u, α k,u represents the correlation factor between task vehicle k and roadside unit u, f u (n) represents the computing resource information of the drive test unit u, w u (n) indicates its location information.

[0016] Furthermore, when the vehicle sends data to the digital twin, the data will be forwarded through the RSU. If the traffic load of the relay node is large at this time, the data will wait in the queue for a long time, which will undoubtedly add additional delay to the data synchronization. In other words, the delay in the queue will affect the effectiveness of data synchronization. In the data synchronization process of the Internet of Vehicles, in addition to considering the waiting time of the relay node, it is also necessary to consider the adaptive data update frequency, which will directly affect the mapping error of the model and thus affect the subsequent task offloading decision.

[0017] Therefore, this method proposes an accurate digital twin construction scheme based on swarm learning (SL-ADTC), in which a distributed learning algorithm (Swarm Learning, SL) is deployed at the DT layer, and its main function is to predict the future waiting time of roadside units deployed along the road based on the collected historical data. In this way, when a vehicle sends a data block to the DT, it first obtains the predicted waiting time of different RSUs from the DT layer; then the vehicle selects the appropriate RSU to upload the data.

[0018] The SL-ADTC program includes:

[0019] S1: The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time;

[0020] S2: Virtual distributed nodes (vehicles, RSUs) join SL through blockchain smart contracts to perform local model training;

[0021] S3: The virtual control center assigns the initial global model parameters and the number of local training times to each vehicle and RSU as local model parameters and training indicators, respectively;

[0022] S4: Each vehicle and RSU sets its local model parameters to global parameters;

[0023] S5: Each vehicle and RSU is equipped with sensors. Each vehicle and RSU is equipped with sensors, radars and other on-board sensing devices. During operation, a large amount of valuable data can be collected, such as environmental data, vehicle operation data (vehicle speed, location, direction), etc.

[0024] S6: The vehicle and RSU use local model parameters and local data sets to locally update the SL model;

[0025] S7: After multiple local trainings, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation;

[0026] S8: SL completes one iteration of adaptive frequency adjustment and global update of the congestion prediction model, verifies the effectiveness of the parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal RSU to upload data to reduce data upload latency.

[0027] S9: Repeat the above steps to achieve digital twin synchronization, improve the overall performance of the system, build accurate data twins, and provide effective guarantees for subsequent vehicle task processing.

[0028] Furthermore, with the explosive growth of autonomous vehicles, some innovative automotive applications continue to emerge, bringing new functions to the network, which puts higher requirements on vehicle systems, including task processing accuracy, latency, and system energy consumption. To this end, based on the establishment of accurate data twins, a DT-based task partitioning collaborative processing (DT-TPCP) mechanism is proposed, in which the vehicle will decompose the generated computationally intensive tasks into subtasks, some of which will be offloaded to RSUs and vehicles with idle resources for processing. At the same time, a joint optimization problem is formulated, considering the contradiction between task processing delay and system energy consumption, and weight coefficients are assigned to latency and energy consumption, so that the overall performance of the system is optimized.

[0029] Among them, the task division and collaborative processing mechanism includes: the task vehicle generates a computing task and sends a service request to the control center deployed at the base station through a wireless link; after the control center receives the task request, the decision model in the digital twin layer determines the value of the correlation factor to make an unloading decision; the unloading decision includes local calculation and unloading node auxiliary calculation; according to the unloading decision, the global system delay of all task vehicle service requests and the global energy consumption of the service requests are determined.

[0030] Furthermore, a joint optimization problem is established to jointly optimize system performance, unloading decisions, and the computing resources of auxiliary vehicles and roadside units, so that the weighted sum of the total delay and computing energy consumption of the digital twin auxiliary edge computing system serving the on-board computing task processing is minimized. The joint optimization problem can be expressed as:

[0031]

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040]

[0041] C10 0≤ρ(n)≤1

[0042] C11||a k,j (n)||≤a max ,||v k,j (n)‖≤v max

[0043]

[0044] Among them, δ1 and δ2 represent the global system delay and global energy consumption The weighted sum parameter of represents the local computing resource size of task vehicle k, f u represents the computing resource size of the roadside unit u, f j represents the computing resource size of auxiliary vehicle j, ρ represents the task division factor, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, α k,u Represents the association factor between task vehicle k and roadside unit u.

[0045] Furthermore, the established joint optimization problem is an NP-hard problem containing nonlinear constraints, which is difficult to solve using traditional optimization algorithms. The DT-assisted multi-agent classification proximal optimization strategy (MACPPO) algorithm is used to solve the above optimization problem.

[0046] The MACPPO algorithm includes:

[0047] S1: Initialize each Actor network parameter, initialize each Critic network parameter, and initialize the experience pool;

[0048] S2: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment;

[0049] The intelligent agent includes an unloading configuration intelligent agent and an unloading node intelligent agent; the unloading configuration intelligent agent is used to focus on the unloading configuration of the computing task, and the unloading node intelligent agent is used to allocate CPU frequency for the computing task of the task vehicle;

[0050] S3: Based on the current strategy and state, each agent selects an action and executes it;

[0051] S4: All agents interact with the environment to obtain their own rewards and jump to the next state, storing the experience data in the experience replay pool;

[0052] S5: For each agent, randomly extract a small batch of samples from the experience pool;

[0053] S6: Determine whether the set number of training times is exceeded, if so, stop training, otherwise proceed to the next step;

[0054] S7: For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network;

[0055] S8: When the training is completed, clear the experience pool.

[0056] The beneficial effects of the present invention are:

[0057] (1) This paper proposes a DT-assisted frequency and waiting time prediction scheme based on group learning. According to historical data, real-time training status and resource conditions, the optimal data upload path and twin finer frequency are adaptively selected to minimize the synchronization delay of the digital twin, so as to achieve accurate digital twin model construction. It solves the contradiction between data synchronization, resource allocation, task delay and system energy consumption in intelligent driving of Internet of Vehicles.

[0058] (2) The present invention adopts a task division and collaborative processing mechanism based on DT. Under the intelligent decision-making provided by DT, part of the tasks are offloaded to auxiliary vehicles or roadside units to complete computing services, which can cope with the heavy computing tasks of vehicles and improve task processing efficiency.

[0059] (3) By establishing a multi-objective optimization problem and reasonably allocating computing resources, the optimal balance between task delay and energy consumption is achieved. The optimization problem is solved by a DT-assisted proximal optimization strategy algorithm for multi-agent classification, thereby achieving the optimal allocation of computing resources.

[0060] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0062] Figure 1 The system scenario targeted by the method of the present invention;

[0063] Figure 2 Schematic diagram of DT-assisted group learning framework;

[0064] Figure 3 Offload the flow chart for computing tasks;

[0065] Figure 4 This is the network structure diagram of the MACPPO algorithm. DETAILED DESCRIPTION

[0066] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0067] The present invention proposes a method for constructing a digital twin and allocating resources in a connected vehicle network. By constructing an accurate digital twin, computational offloading and resource allocation are jointly optimized with the assistance of the digital twin, so as to minimize system energy consumption and computational delay.

[0068] The specific content of the method is described below through an embodiment.

[0069] 1. If Figure 1 The system scenario shown considers a DT-assisted vehicle-connected network urban intersection scenario, which consists of autonomous vehicles, roadside units, and base stations. Autonomous vehicles are divided into two categories, namely mission vehicles and auxiliary vehicles, and they are all equipped with mobile edge processors and dual-function radar and communication systems (DFRC) systems. DFRC technology integrates communication and sensing functions into practical devices. In addition to traditional communication transmission functions, it also has environmental perception functions such as video monitoring and radar detection. Using DFRC technology, mission vehicles can perform radar sensing and mission data offloading at the same time, saving frequency band resources.

[0070] 2. Targeting Figure 1 The system scenario shown in the figure is DT modeled: for the task vehicle k, the virtual twin needs to record its task information and location, which can be characterized as DT k (n) = {w k (n),Ω k (n),f k (n)},w k (n) represents the location information of task vehicle k, Ω k (n) represents its task processing information, f k (n) represents its computing resource information, and n represents the time slot.

[0071] For auxiliary vehicles, DT needs to reflect its service scheduling, including resource allocation and motivation status. Therefore, the virtual twin of auxiliary vehicle j can be characterized as DT j (n) = {w j (n),α k,j ,f j (n)},f j (n) represents the computing resource information of auxiliary vehicle j, w j (n) represents its position information, α k,j Represents the correlation factor between task vehicle k and auxiliary vehicle j.

[0072] Similarly, for each RSU, DT also needs to reflect its service scheduling, including resource allocation and motivation status. Therefore, the virtual twin of RSU u can be represented as DT u (n) = {w u (n),α k,u ,f u (n)},f u(n) represents the computing resource information of roadside unit u, w u (n) represents its position information, α k,u Represents the association factor between task vehicle k and roadside unit u.

[0073] 3. Perform DT synchronization delay modeling:

[0074] The vehicle DT synchronization delay includes the following stages:

[0075] ① The vehicle uploads DT synchronization data to the RSU: Wireless communication is used between the vehicle and the RSU. The maximum uplink information rate between the vehicle and the RSU is:

[0076]

[0077] Where B is the transmission channel bandwidth, p k is the transmission power of the mission vehicle, h ku is the channel gain, N0 is the noise power spectral density, is the transmission path loss. ku is the distance between the task vehicle k and the roadside unit u, and γ0 is the attenuation factor. Vehicle k synchronizes DT data Upload to RSU, the transmission delay is recorded as:

[0078]

[0079] ②RSU forwards DT synchronization data to the base station: The transmission between RSU and the base station is wired, and the wired transmission delay is related to the amount of data transmitted and the transmission distance. The wired transmission delay between RSU and the base station is:

[0080]

[0081] Where φ is the delay required to transmit a unit bit of data per unit distance, d uBS is the distance between the roadside unit u and the base station, t waiting Represents the queue waiting time at roadside unit u.

[0082] ③ The server of the base station processes DT synchronization data: The computing resources of the server can be allocated to vehicles and RSUs to maintain their DT information. The processing time of the server for the data uploaded by the vehicle is recorded as:

[0083]

[0084] in, Indicates that when the amount of data transmitted is The amount of calculation required, Represents the amount of computing resources occupied by the twin corresponding to task vehicle k.

[0085] Therefore, the DT synchronization delay of the vehicle can be calculated as:

[0086]

[0087] 4. DT model accuracy characterization

[0088] The relationship between packet error rate and signal to interference noise ratio in wireless communication is described as:

[0089]

[0090] In the formula, ε k,u (n) represents the interference of the wireless link between the vehicle and the RSU, and θ(·) is a piecewise function expressed as Therefore, when the interference plus noise is very small, the corresponding packet error rate is also very small; when it exceeds the threshold x0, the packet error rate increases sharply and tends to 1. k,u (n) represents the channel gain at time n.

[0091] DT synchronization mapping is a continuous process. The error of synchronization data packet in a time slot may not cause serious DT mapping deviation. Therefore, the accuracy of DT mapping should be related to the packet error rate and packet loss rate in the past period of time. The DT accuracy is specifically expressed as:

[0092]

[0093] In the formula, γ0′∈[0,1] is the time discount impact factor, which is used to measure the impact of the packet error rate in the past time slots of the wireless channel on the DT mapping accuracy at the current moment, that is, the packet error rate in the more distant time slots is considered to have a smaller impact on the DT accuracy at the current moment. This reflects that when measuring the DT accuracy of a certain time slot, more attention is paid to the wireless channel packet error rate in the recent time slot. n0 represents the n0th moment, n represents the nth moment, and N′ represents the left shift of N′ relative to the moment n.

[0094] 5. A swarm learning-based accurate digital twin construction (SL-ADTC) scheme is proposed to achieve digital twin synchronization. The DT-assisted swarm learning framework is shown in the figure below. Figure 2 The SL-ADTC scheme is as follows:

[0095] (1) The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time;

[0096] (2) Virtual distributed nodes (vehicles, RSUs) join the distributed learning algorithm (SwarmLearning, SL) through blockchain smart contracts to perform local model training;

[0097] (3) The virtual control center assigns the initial global model parameters and the number of local training times to each vehicle and RSU as local model parameters and training indicators, respectively;

[0098] (4) Each vehicle and RSU sets its local model parameters to global parameters;

[0099] (5) Each vehicle and RSU is equipped with sensors, radars, and other on-board sensing devices. During operation, a large amount of valuable data can be collected, such as environmental data, vehicle operation data (vehicle speed, location, direction), etc., and local preprocessing is performed to obtain data samples D in combination with historical data. k ={(x1,y1),(x2,y2),...,(x i ,y i )},(x i ,y i ) represents sample data. The total amount of sample data of all vehicles within the communication range of the base station can be expressed as D = ∑D k .

[0100] The goal of SL is to minimize the weighted average loss:

[0101]

[0102] Where n k =|D k |, F k (w) represents the average loss of the local node, and K represents the total number of SL nodes participating in learning.

[0103] The local objective of the SL node is to minimize the average loss:

[0104]

[0105] In the formula, f i (w) is the local loss function, f i (w) = l[(x i ,y i );w].n k represents the total number of samples, and l(·) represents the loss.

[0106] (6) The vehicle and RSU use the local model parameters and local data set to locally update the SL model according to the following formula:

[0107]

[0108] In the formula, η is the learning rate, It means to find the gradient.

[0109] (7) After τ times of local training, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation;

[0110] (8) When the temporary central node receives the local node parameters, the central node starts to aggregate the model according to the weighted average parameter merging method to obtain the global model parameters:

[0111]

[0112] In the formula, w represents the global model parameter after aggregation, w k represents the local model parameters of the virtual SL node k. In order to achieve the optimization goal of equation (8), the SL node and the temporary central node will continuously cycle the training process for training and updating.

[0113] (9) SL completes one iteration of adaptive frequency adjustment and global update of the congestion prediction model, verifies the effectiveness of the parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal RSU to upload data, reducing the data upload delay.

[0114] (10) Repeat the above steps to achieve digital twin synchronization, improve the overall performance of the system, build accurate data twins, and provide effective guarantees for subsequent vehicle task processing.

[0115] 6. A DT-based task partitioning collaborative processing (DT-TPCP) mechanism is proposed, such as Figure 3 The details are as follows:

[0116] Define the correlation factor α between the task vehicle k and the auxiliary vehicle j k,j , the correlation factor α between task vehicle k and roadside unit u k,u , α k,j ,α k,u ∈(0,1). Based on the data of each physical entity recorded in the digital twin model, deep reinforcement learning is used to analyze the task requests and historical data (network traffic, computing resources, etc.) that the task vehicle needs to process. The decision model in the DT layer determines the value of the correlation factor to determine the unloading strategy, as shown below:

[0117] α k,j =0,α k,u =0: task vehicle local processing;

[0118] α k,j =0,α k,u =1: RSU assisted parallel processing;

[0119] α k,j =1,α k,u =0: auxiliary vehicle auxiliary parallel processing;

[0120] Ignore α k,j =1, α k,u =1.

[0121] Define a triplet at the beginning of each time slot Where D k (n) is the data size of the generated computing task, C k (n) is the average number of CPU cycles required to process a unit bit of data, is the maximum allowed delay. The partial offloading mode is adopted to divide the task into two parts, one of which has a data volume of Z k (n) = ρ(n)D k (n) tasks are locally calculated, and the other part of the data volume is Z k' (n) = [1 - ρ (n)] D k The task of (n) is offloaded to the auxiliary vehicle or roadside unit for processing. Among them, ρ(n) is the task division factor, 0<ρ(n)<1.

[0122] 1) Local computing: α k,j =0,α k,u =0

[0123] In the case of local computing, processes 1 and 2 in the system (such as Figure 3 The delay caused by the time slot n is negligible and only includes processes 3 and 4, namely the local computing time and the decision-making time of the digital twin.

[0124]

[0125]

[0126] Where D k' (n) is the service request data size, C DT (n) represents the computational cost of the neural network module processing 1 bit of data, f k DT Indicates the computing resources allocated to the DT layer by the edge server in the control center. Represents local computing resources, Indicates the amount of data uploaded by roadside unit u.

[0127] Therefore, in the case of local computation, the system latency is:

[0128] 2) Roadside unit auxiliary parallel computing: α k,j =0,α k,u =1, which is similar to the local calculation process. Here, the calculation process is ignored and the result is given directly:

[0129]

[0130] In the formula, Indicates the local computing latency, represents the wireless transmission delay between the task vehicle k and the adjacent roadside unit u', represents the wired transmission delay between RSU u' and RSU u, Represents the task processing delay of roadside unit u.

[0131] 3) Auxiliary vehicle auxiliary parallel computing: α k,j =1,α k,u =0, the calculation results are as follows:

[0132]

[0133] in, represents the wireless transmission delay between task vehicle k and auxiliary vehicle j, represents the computational delay of auxiliary vehicle j

[0134] Therefore, the service delay for any service request is expressed as:

[0135]

[0136] In the formula, α k,j , α k,u When both values ​​are 0, X(α k,j ,α k,u )=0; otherwise, it is equal to 1.

[0137] Therefore, the system's calculation formula for the global service delay of all mission vehicles in time slot n is:

[0138] 7. Establish an energy consumption model, which is modeled as follows:

[0139] (1) Mission vehicle energy consumption: The effective capacitance coefficient of the CPU of mission vehicle k is κ1.

[0140] At time slot n, the energy consumption of local calculation is as follows:

[0141]

[0142] In the formula, f k (n) represents the computing resources of task vehicle k,

[0143] The transmission energy consumption of task vehicle k is expressed as:

[0144]

[0145] In the formula, represents the transmission delay between task vehicle k and roadside unit u, represents the transmission power of mission vehicle k

[0146] (2) Energy consumption of auxiliary vehicles and roadside units: The effective capacitance coefficients of the auxiliary vehicle and roadside unit CPU are κ2 and κ3, respectively.

[0147] When auxiliary vehicle j and roadside unit u provide computing services for task vehicle k in time slot n, the computing energy consumption is:

[0148]

[0149] Among them, Z k′ (n) represents part of the task data, f j (n) represents the computing power of auxiliary vehicle j, f u (n) represents the computing capacity of the roadside unit u.

[0150] Control center decision energy consumption:

[0151]

[0152] Among them, T twin (n) represents the DT layer decision delay, P c (n) represents the computing power of the control center server in processing service requests.

[0153] In summary, the total system energy consumed to process the computing task on task vehicle k is:

[0154]

[0155] Therefore, the global energy consumption of the system for computing tasks on all task vehicles in time slot n can be expressed as:

[0156]

[0157] Based on the above, the weighted sum of total latency and computing energy consumption of the digital twin-assisted edge computing system serving on-board computing tasks can be minimized by jointly optimizing system performance, offloading decisions, and computing resources of auxiliary vehicles and roadside units. The corresponding optimization problem is expressed as follows:

[0158]

[0159]

[0160]

[0161]

[0162]

[0163]

[0164]

[0165]

[0166]

[0167]

[0168] C10 0≤ρn)≤1

[0169] C11||a k,j (n)||≤a max ,||v k,j (n)||≤v max

[0170]

[0171] Where δ1 and δ2 are weighted sum parameters of delay and computational energy consumption. U represents the total number of roadside units, J represents the total number of auxiliary vehicles, Indicates the maximum allowed delay. It represents the maximum value of the total system energy consumed to process the computing task on task vehicle k. represents the local computing resource size of task vehicle k, ρ represents the task ratio division factor, a k,j (n) represents the unloading correlation coefficient, a max Indicates the maximum acceleration of the vehicle, v k,j (n) represents the speed of the mission vehicle and the auxiliary vehicle, v max represents the maximum acceleration of the vehicle, w r (n) and w s (n) The location information of two different cars, d min Indicates the minimum safe distance between vehicles. represents the maximum computing capacity of task vehicle k, represents the maximum computing capacity of the roadside unit u, represents the maximum computing capacity of the auxiliary vehicle j.

[0172] Constraint C1 ensures that the task vehicle is associated with at most the next unloading node, and C2 indicates that the tolerable task processing delay does not exceed C3 limits the system energy consumption of processing the computing tasks on task vehicle k to no more than C4 limits the mission vehicle's transmission power to no more than C5 limits the computing power of the control center server to no more than C6, C7, C8 and C9 respectively limit the computing resources of each network node allocated to the computing task of task vehicle k. C10 is the computing task division ratio constraint, C11 is the acceleration and speed limit of the task vehicle and auxiliary vehicle, and C12 represents the minimum safe distance between vehicles.

[0173] The optimization model shown in formula (22) contains nonlinear constraints. This problem is NP-hard and difficult to solve using traditional optimization algorithms. Therefore, a DT-assisted multi-agent classification proximal optimization strategy (MACPPO) algorithm is proposed to solve it. The process is as follows:

[0174] 1) Offloading configuration agent: This type of agent focuses on the offloading configuration of the task, and its index is represented as To determine the unloading ratio and the association with the target unloading node, they need to observe the location and mission information of the task vehicle and the location of the unloading node.

[0175] Observations: The observations on the uninstall configuration agent are as follows:

[0176]

[0177] Where k represents the mission vehicle number.

[0178] Action: The action of the uninstall configuration agent should represent the decision variables and the uninstall ratio, so it can be defined as:

[0179]

[0180] Among them, ρ k [n] represents the task division factor of task vehicle k at time n.

[0181] Reward: The reward function for the unloading configuration agent needs to include the target and the penalty for not meeting the delay requirement. The energy consumption of each task vehicle and its associated destination unloading node needs to be decomposed. Therefore, the reward for agent l is:

[0182]

[0183] in, represents the average energy consumption of the mission vehicle, Indicates the signal transmission power.

[0184] 2) Unloading node agent: The unloading node needs to allocate CPU frequency for the computing task of the task vehicle. The index set of the unloading node agent is Its elements include:

[0185] Observation: Each unloading node can obtain the location and task information of the task vehicle within its service range. Therefore, the observation value can be expressed as:

[0186]

[0187] in, Represents the unloading of the task vehicle served by node m.

[0188] Action: The unloading node needs to allocate its own CPU frequency to perform the task of the task vehicle, so its action is:

[0189]

[0190] like Figure 4 The figure shows the training framework of DT-assisted multi-agent deep reinforcement learning (MADRL). During the training process, the centralized state value function of the u-th agent is defined as:

[0191]

[0192] Among them, s t represents the environmental state at time t, θ u represents the new policy parameters, represents the expectation operation, represents the discount factor, represents the reward function, a t+1 Indicates the action performed at time t+1.

[0193] The action-value function is defined as follows:

[0194]

[0195] Therefore, the advantage function can be obtained as:

[0196]

[0197] By the state value V u (s t ) is estimated to be:

[0198]

[0199] Among them, γ u represents the discount factor, r t+l Indicates reward.

[0200] The advantage function is evaluated using the generalized advantage estimation (GAE). The GAE factor λ controls the trade-off between reward variance and bias, δ t =(r t +γ u V u (st+1 )-V u (s t+l )) is the time difference error. Therefore, the critic can be updated by tracking the loss function:

[0201]

[0202] Among them, w u Represents the weight parameter, which is used to optimize the performance of the algorithm during the learning process. Represents the estimated value function under the optimized weights.

[0203] For the actor network, a clipping factor is introduced to limit the update rate of the policy, which effectively replaces the calculation of the trust region. On this basis, the loss function of the actor network is expressed as:

[0204]

[0205] Among them, clip(·) represents a function used to limit the input to a certain interval, which is usually used to maintain the stability of policy updates. represents the new policy function, represents the old policy function, ∈ represents the update step threshold, ψS t,u The policy entropy representing the degree of exploration, S t,u represents the state of the environment at time t.

[0206] Combined with the above content, the specific steps of the MACPPO algorithm are as follows:

[0207] S1: Initialize each Actor network parameter, initialize each Critic network parameter, and initialize the experience pool;

[0208] S2: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment;

[0209] S3: Based on the current strategy and state, each agent selects an action and executes it;

[0210] S4: All agents interact with the environment to obtain their own rewards and jump to the next state, storing the experience data in the experience replay pool;

[0211] S5: For each agent, randomly extract a small batch of samples from the experience pool;

[0212] S6: Determine whether the set number of training times is exceeded, if so, stop the training, otherwise continue to execute S7;

[0213] S7: For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network;

[0214] S8: When the training is completed, clear the experience pool.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a digital twin and allocating resources in an Internet of Vehicles, characterized in that: A precise digital twin construction scheme based on group learning is proposed. The distributed learning algorithm is used to predict the data upload frequency and the waiting time at the roadside unit, solve the data deviation and asynchrony, and obtain a precise digital twin. Based on the constructed digital twin, the vehicle's computing tasks are partially offloaded to the roadside unit or resource-free vehicles for processing through the task division and collaborative processing mechanism; Establish a joint optimization problem to jointly optimize system performance, unloading decisions, and computing resources of auxiliary vehicles and roadside units, so that the weighted sum of total latency and computing energy consumption of the digital twin-assisted edge computing system serving the on-board computing task processing is minimized; The established joint optimization problem is solved through the proximal optimization strategy algorithm of multi-agent classification assisted by digital twins to obtain the optimal unloading decision.

2. The method according to claim 1, characterized in that: In the Internet of Vehicles scenario, digital twins of the mission vehicle, auxiliary vehicle, and roadside unit are constructed respectively; The digital twin of the mission vehicle is represented by: DT k (n) = {w k (n),Ω k (n),f k (n)}, DT k (n) represents the digital twin of task vehicle k, w k (n) represents the location information of task vehicle k, Ω k (n) represents the task processing information of task vehicle k, f k (n) represents the computing resource information of task vehicle k, and n represents the time slot. The digital twin of the auxiliary vehicle is represented by: DT j (n) = {w j (n),α k,j ,f j (n)}, DT j (n) represents the digital twin of auxiliary vehicle j, α k,j represents the correlation factor between task vehicle k and auxiliary vehicle j, f j (n) represents the computing resource information of the auxiliary vehicle j, w j (n) represents the position information of the auxiliary vehicle j. The digital twin of the roadside unit is represented by: DT u (n) = {w u (n),α k,u ,f u (n)}, DT u (n) represents the digital twin of roadside unit u, α k,u represents the correlation factor between task vehicle k and roadside unit u, f u (n) represents the computing resource information of the drive test unit u, w u (n) represents the location information of the drive test unit u.

3. The method according to claim 2, characterized in that: For the constructed digital twin, the digital twin synchronization is performed through the accurate digital twin construction scheme based on group learning, including the following steps: 1) The digital twin layer collects various data from the physical network layer and maps the status and actions of physical entities in real time; 2) Virtual distributed nodes join the distributed learning algorithm through blockchain smart contracts to perform local model training; 3) The virtual control center distributes the initial global model parameters and the number of local training times to each vehicle and roadside unit as local model parameters and training indicators respectively; 4) Each vehicle and roadside unit sets its local model parameters as global parameters; 5) Each vehicle and roadside unit collects data through sensor equipment, performs local preprocessing, and combines historical data to obtain data sample D k ={(x1,y1),(x2,y2),...,(x i ,y i )},(x i ,y i ) represents sample data; 6) The vehicle and the roadside unit use the local model parameters and the local data set to locally update the distributed learning algorithm model according to the following formula: In the formula, η represents the learning rate, w k represents the local model parameters of SL node k, represents the function gradient operator, n k represents the total number of samples, f i (w) represents the local loss function, F k (w) represents the average loss of the local node; 7) After τ times of local training, the pre-defined synchronization interval of each node is met, and the SL node uploads the local parameters to the temporary central node for model aggregation; 8) When the temporary central node receives the local node parameters, the central node starts to aggregate the model according to the weighted average parameter merging method to obtain the global model parameters: In the formula, w represents the aggregated global model parameters, and the SL nodes and temporary central nodes continuously cycle the training process for training updates; 9) SL completes one iteration of adaptive frequency adjustment and global update of congestion prediction model, verifies the effectiveness of parameter adjustment strategy, and sends the strategy to each network node. The vehicle selects the optimal roadside unit to upload data, reducing data upload delay; 10) Repeat the above steps to achieve digital twin synchronization and provide guarantee for vehicle task processing.

4. The method according to claim 1, characterized in that: The task division and collaborative processing mechanism includes: The mission vehicle generates computing tasks and sends service requests to the control center deployed at the base station through a wireless link; After the control center receives the task request, the decision model in the digital twin layer determines the value of the correlation factor to make an offloading decision; the offloading decision includes local calculation and offloading node auxiliary calculation; The global system delay of all mission vehicle service requests and the global energy consumption of service requests are determined based on the unloading decision.

5. The method according to claim 4, characterized in that: The association factor includes the association factor α between the task vehicle k and the auxiliary vehicle j k,j and the correlation factor α between the task vehicle k and the roadside unit u k,u ; Based on the correlation factors, the uninstallation decision includes: 1) α k,j = 0 and α k,u =0, the task vehicle is processed locally; 2) α k,j = 0 and α k,u =1, unload some tasks to the roadside unit to assist in parallel processing; 3) α k,j =1 and α k,u = 0, offload some tasks to the auxiliary vehicle to assist in parallel processing; The task volume unloaded to the roadside unit or auxiliary vehicle is expressed as: Z k' (n) = [1 - ρ (n)] D k (n), ρ(n) represents the task division factor, 0<ρ(n)<1, D k (n) represents the data size of the computing task.

6. The method according to claim 4, characterized in that: When the unloading decision is processed locally by the task vehicle, the system delay is expressed as: In the formula, represents the local computation time, represents the digital twin decision time; When the offloading decision is to offload part of the tasks to the auxiliary vehicle for auxiliary parallel processing, the system delay is expressed as: In the formula, Indicates the local computing latency, represents the wireless transmission delay between task vehicle k and auxiliary vehicle j, represents the computational delay of auxiliary vehicle j; When the offloading decision is to offload part of the tasks to the roadside unit for auxiliary parallel processing, the system delay is expressed as: In the formula, represents the wireless transmission delay between the task vehicle k and the adjacent roadside unit u', represents the wired transmission delay between RSU u' and RSU u, represents the task processing delay of roadside unit u; For any service request of the mission vehicle, the system delay is expressed as: In the formula, when α k,j , α k,u When both values ​​are 0, χ(α k,j ,α k,u )=0, otherwise it is equal to 1; The global system delay of service requests of all task vehicles in time slot n is expressed as: K represents the number of mission vehicles.

7. The method according to claim 4, characterized in that: The global energy consumption includes the energy consumption of mission vehicles, the energy consumption of unloading nodes and the decision-making energy consumption of the control center; The energy consumption of the mission vehicle includes local computing energy consumption and transmission energy consumption, which are respectively expressed as: In the formula, represents the local computing energy consumption, represents the transmission energy consumption, κ1 represents the effective capacitance coefficient of the CPU of the mission vehicle, and f k (n) represents the computing power of mission vehicle k, represents the wireless transmission delay between task vehicle k and roadside unit u, represents the transmission power of mission vehicle k; The energy consumption of the offloading node is expressed as: Where κ2 and κ3 represent the CPU effective capacitance coefficients of the auxiliary vehicle and the roadside unit, respectively, and Z k′ (n) represents part of the task data, f j (n) represents the computing power of auxiliary vehicle j, f u (n) represents the computing capacity of the roadside unit u; The decision-making energy consumption of the control center is expressed as: Where, T twin (n) represents the DT layer decision delay, P c (n) represents the computing power of the control center server in processing service requests; Then the energy consumption required to process the computing task of task vehicle k is: The global energy consumption required for the computational tasks of all task vehicles in time slot n is expressed as:

8. The method according to claim 1, characterized in that: The established joint optimization problem is expressed as: C10 0≤ρ(n)≤1 C11||a k,j (n)||≤a max ,||in k,j (n)||≤v max Where δ1 and δ2 represent the global system delay and global energy consumption The weighted sum parameter of ; U represents the total number of roadside units, J represents the total number of auxiliary vehicles, Indicates the maximum allowed delay. represents the maximum energy consumption required to process the computing task on task vehicle k, represents the maximum value of computing resources allocated to the digital twin layer by the edge server of the control center, a k,j 、v k,j Respectively represent the acceleration and speed of the vehicle, a max 、v max Represent the maximum values ​​of acceleration and velocity respectively; represents the local computing resource size of task vehicle k, ρ represents the task ratio division factor, a k,j (n) represents the unloading correlation coefficient, a max Indicates the maximum acceleration of the vehicle, v k,j (n) represents the speed of the mission vehicle and the auxiliary vehicle, v max represents the maximum acceleration of the vehicle, w r (n) and w s (n) represents the location information of two different vehicles, d min Indicates the minimum safe distance of vehicles. represents the maximum computing capacity of task vehicle k, represents the maximum computing capacity of the roadside unit u, represents the maximum computing capacity of the auxiliary vehicle j.

9. The method according to claim 8, characterized in that: The established joint optimization problem is solved by a proximal optimization strategy algorithm for multi-agent classification assisted by digital twins, which includes: 1) Initialize each Actor network parameter, initialize each Critic network parameter, and initialize the experience pool; 2) In each iteration, a random process is initialized for action exploration to obtain the initial observation of each agent, that is, the initial state of the environment; The intelligent agent includes an unloading configuration intelligent agent and an unloading node intelligent agent; the unloading configuration intelligent agent is used to focus on the unloading configuration of the computing task, and the unloading node intelligent agent is used to allocate CPU frequency for the computing task of the task vehicle; 3) Based on the current strategy and state, each agent selects an action and executes it; 4) All agents interact with the environment to obtain their own rewards and jump to the next state, storing the experience data in the experience replay pool; 5) For each agent, randomly select a small batch of samples from the experience pool; 6) Determine whether the set number of training times has been exceeded. If so, stop training; otherwise, proceed to the next step; 7) For each agent, calculate the target state value of the Critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the Actor network; 8) When the training is completed, clear the experience pool.

Citation Information

Patent Citations

  • Perception task processing method and system in digital twin Internet of Vehicles

    CN115086917A

  • Dynamic deployment method for digital twin servers in edge Internet of Vehicles

    CN115209426A

  • Internet of vehicles edge computing unloading multi-objective decision-making method based on digital twinning

    CN116782296A

  • User association, task unloading and resource allocation optimization method based on digital twinning

    CN117858109A

  • Digital twinning-based car networking task unloading method

    CN118075265A

Cited By

  • Multi-layer intelligent agent simulation method and system for vehicle-road cooperative computing

    CN119783385A

  • A multi-layer agent simulation method and system for vehicle-road collaborative computing

    CN119783385B

  • Digital twin assisted vehicle task unloading and computing resource allocation optimization method

    CN120111575A

  • Optimization Method for Vehicle Task Offloading and Computing Resource Allocation Assisted by Digital Twin

    CN120111575B