Digital twinning dynamic deployment and migration optimization method based on vehicle infrastructure collaborative awareness

Abstract

The invention relates to a digital twinning dynamic deployment and migration optimization method based on vehicle-infrastructure collaborative awareness, and belongs to the technical field of Internet of Vehicles and digital twinning. The method comprises the following steps: constructing a digital twin network oriented to the Internet of Vehicles; a three-party utility model driven by heterogeneous sensing resources is constructed, and sensing quality, task completion time and node energy consumption are comprehensively considered; the sensing area overlapping degree of the vehicle and the road side unit is dynamically quantified, and the data acquisition redundancy rate is reduced in combination with a preset threshold value; a parallel pre-migration mechanism is designed according to the computing load, the storage capacity and the communication resource state of the edge server, and independent fragmentation transmission of the digital twin model is achieved through a multi-path concurrent transmission strategy; establishing a joint optimization model with maximized comprehensive utility; a multi-agent near-end strategy optimization algorithm is adopted to realize collaborative optimization of a digital twin deployment strategy, migration path selection and perception task distribution. According to the invention, the real-time performance and reliability of the digital twinning service in the Internet of Vehicles scene are improved.

Description

Technical Field

[0001] The present invention belongs to the technical fields of vehicle networking and digital twin, and relates to a digital twin dynamic deployment and migration optimization method based on vehicle-road collaborative perception. Background Art

[0002] The application of digital twin technology in vehicle networking has formed an infrastructure for virtual-real mapping and real-time interaction, providing technical support for core functions such as traffic simulation and path planning. However, with the increase in the complexity of vehicle networking scenarios, the existing technical system exposes the following key bottlenecks:

[0003] 1) The contradiction between the static resource allocation mechanism and the adaptation of dynamic traffic flow. The current system generally adopts a computational resource allocation strategy with fixed rules and does not establish an elastic resource scheduling model linked to traffic flow fluctuations. For example, when sudden traffic surges occur during peak hours, the computationally statically allocated nodes are prone to service degradation due to overloading, while there are idle computational resources during off-peak hours, resulting in insufficient overall resource utilization. This rigid resource supply mode is difficult to meet the millisecond-level real-time response requirements in vehicle networking scenarios.

[0004] 2) The lack of intelligent optimization in the twin migration process. When vehicles move across domains or network topologies change, the existing migration methods mainly rely on simple proximity principles to select target nodes, without considering multi-dimensional parameters such as path loss and energy consumption costs. Measured data shows that the service interruption time of traditional migration schemes is long, and the additional energy consumption caused by frequent cross-regional migrations accounts for a large proportion. Especially in multi-vehicle collaborative scenarios, the lack of a global optimization algorithm exacerbates the volatility of service quality.

[0005] 3) The lack of heterogeneous terminal collaborative perception ability. Heterogeneous nodes such as vehicle on-board units (OBUs), roadside units (RSUs), and edge servers in vehicle networking form data islands, and the existing system has not established a unified spatio-temporal reference framework. Tests show that due to clock synchronization deviations and coordinate system differences, the fusion error rate of multi-source perception data is high, directly affecting the state update accuracy of the twin. In addition, traditional schemes have not realized the joint modeling of vehicle motion trajectory prediction and roadside facility perception data, resulting in a lag in dynamic traffic situation perception.

[0006] The above technical defects have become the main obstacles restricting the improvement of the efficiency of the vehicle networking digital twin system, and there is an urgent need for a digital twin dynamic deployment and migration method that integrates vehicle-road collaborative perception and reinforcement learning to improve the adaptive ability of the digital twin system. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a digital twin dynamic deployment and migration optimization method based on vehicle-road collaborative perception. By constructing a multi-objective optimization model, it realizes the adaptive deployment and efficient migration of vehicle digital twins in an edge computing environment. This method first establishes a utility function including delay, energy consumption, and model accuracy based on vehicle operating status, roadside device perception data, and network resource status; then uses an improved deep reinforcement learning algorithm to dynamically optimize the construction granularity of digital twins and the resource allocation strategy of edge servers, maximizing the overall system utility under the condition of ensuring service quality.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A digital twin dynamic deployment and migration optimization method based on vehicle-road collaborative perception specifically includes the following steps:

[0010] S1: Construct a digital twin model for the vehicle network;

[0011] S2: Construct a three-party utility model driven by heterogeneous perception resources, comprehensively considering perception quality, task completion time, and node energy consumption;

[0012] S3: Dynamically quantify the overlap degree of the perception areas of vehicles and roadside units, and combine a preset threshold to reduce the data collection redundancy rate;

[0013] S4: According to the computing load, storage capacity, and communication resource status of the edge server, design a parallel pre-migration mechanism, and realize the shard-independent transmission of the digital twin model through a multi-path concurrent transmission strategy;

[0014] S5: Establish a joint optimization model for maximizing comprehensive utility;

[0015] S6: Adopt a multi-agent proximal policy optimization algorithm, and through a centralized training and distributed execution framework, realize the collaborative optimization of digital twin deployment strategies, migration path selection, and perception task allocation.

[0016] Furthermore, in step S1, the constructed digital twin model for the vehicle network includes the following three layers:

[0017] (1) Physical terminal layer, including M ICVs with limited perception and storage resources, where ICV represents an intelligent connected vehicle, and the ICV set is defined as V = {V1, V2,..., V M}; within time slot m, the mth intelligent connected vehicle ICV m communicates with the edge server by accessing a single RSU, and uploads vehicle perception data and status data to the edge server for DT synchronization, where RSU represents a roadside unit and DT represents a digital twin;

[0018] (2) Edge twin layer, including edge servers and RSUs that provide access services and sensing services, and each edge server is connected to all other edge servers; the set of RSUs is denoted as The set of edge servers is denoted as E = {e1, e2, …, e n}, where N represents the number of RSUs and n represents the number of edge servers;

[0019] (3) Cloud control layer: A large-scale cloud DT management platform is maintained in the cloud control layer, which is responsible for integrating the concepts of DT and network slicing, and realizing intelligent cloud resource management and dynamic network scheduling; under this platform, network slicing realizes service-oriented network management through network function virtualization technology, while DT enhances the intelligence of network slicing by constructing a digital model with organizational and customized characteristics and introducing a user-centered perspective; the cloud DT management platform runs by receiving all VDT data from the edge twin layer, which is expressed as:

[0020] DT = {H, Env(t), S(t)}

[0021] In the formula, H and Env(t) respectively represent all historical data and environmental information in the entire traffic scenario; S(t) represents the relevant information of all agents. Assume that the set of all agents that can perform sensing tasks is S, including RSUs and ICVs; VDT represents virtual digital twin.

[0022] Furthermore, in step S2, a three-party utility model driven by heterogeneous sensing resources is constructed, specifically including:

[0023] (1) Construct a sensing satisfaction model: In a vehicle-road collaborative sensing system, the sensing satisfaction of a node not only depends on its sensing quality, but also is affected by the scale of the collected data and the time consumed to complete the sensing task. Specifically, sensing data usually has timeliness, and its value gradually decays over time. Therefore, the length of the task execution time directly determines the timeliness of the data. The faster the task is completed, the higher the effectiveness and practicality of the data. Considering sensing quality, data volume and its timeliness comprehensively, the sensing satisfaction of digital twin can be defined as a weighted combination of these factors, aiming to optimize the execution efficiency of sensing tasks and minimize the impact of time delay on data utility. Therefore, the mathematical description of DT for node sensing satisfaction can be expressed as:

[0024]

[0025] In the formula, Q s represents the sensing satisfaction of ICVs nodes, δ is the satisfaction parameter, J(s, t) represents the Jaccard similarity coefficient, represents the sensing frequency of ICVs nodes, Indicates the residence time of ICVs nodes and the execution time of sensing tasks, which is the time for ICVs nodes to complete the upload of raw data; Indicates the amount of data collected by ICVs nodes; Indicates the time gain of ICVs nodes to complete sensing tasks, that is, the amount of time the node completes the task ahead of the set tolerance delay; Based on the definition of the sensing satisfaction of a single node, the overall sensing satisfaction of all nodes in the system can be expressed as:

[0026]

[0027] In the formula, Q represents the overall sensing satisfaction of all nodes in the system;

[0028] (2) Construct a three-party utility model: Assume that the ICV m 's moving path is represented by a set L m of geographical coordinates, and its total length is |L m |; When the ICV m travels at a speed of v m , the ICV m nodes will perform sensing and data collection tasks within the specified area; The residence time and sensing task execution time of the ICV m nodes are The sensing frequency of the ICV m nodes is f m , then the amount of data that the node can collect within the time period is Based on the standardized deployment architecture of RSU, the RSU sensing decision is based on the set sensing frequency f se , then the amount of data collected by the RSU node is f se t se , where t se represents the sensing time of the RSU,

[0029] Assume that each RSU has bandwidth resources and orthogonally allocates them to the ICVs within its coverage; Let B n,m (t) represent the bandwidth resources allocated to the ICV m by the nth RSU in time slot t, and the transmission rate r m can be expressed as:

[0030]

[0031] In the formula, p m,n (t) and g m,n (t) represent the RSU transmit power and channel gain respectively; σ 2is the power density of the background noise; ICV m The time required for the node to complete the upload of the original data is To ensure that the node can complete the data upload within the set tolerance delay, it is necessary to satisfy All ICV nodes can obtain the rewards provided by the DT perception service slice, and their benefits depend on ICV m The proportion of the data volume collected by the node in the total data volume of all perception nodes, denoted as where R is the reward given by DT to a single vehicle node; on the other hand, the vehicle node needs to bear the costs of perception task execution and original data transmission; let be the cost of unit data collection, then the perception cost is Let c tr be the cost of occupying the communication bandwidth per unit time, then the transmission cost is ICV m The utility U of the node m is the difference between the benefit and the cost:

[0032]

[0033] where, c tr represents the cost of occupying the communication bandwidth per unit time;

[0034] To quantify the comprehensive utility of the RSU, a utility function model is constructed based on economic game theory; this model comprehensively considers multi-dimensional factors such as data transmission benefits, perception costs, digital twin maintenance costs, and task participation risks, and is specifically defined as follows:

[0035]

[0036] In the formula, U n is the utility of the roadside node n; p n is the reward given by the digital twin for each unit of data collected by the roadside node n, then the benefit of the roadside node is p n f se t se ; is the perception cost of unit data; c DT is the cost of maintaining the ICV digital twin in the RSU, then the perception cost of the roadside node is The cost of maintaining the digital twin is x m,n (t) represents the access strategy of ICV m When x m,n (t) = 1, ICV m transmits its perception data to the nth RSU for data fusion, otherwise, x m,n(t)=0; In addition, if the DT requires the RSU to participate in the crowd-sensing task, in addition to the explicit cost, it also needs to bear the implicit risk loss; Let w n be the risk loss per unit time, then the risk loss of the RSU is w n (t se ) 2 ;

[0037] From the overall perspective of the DT system, its revenue comes from the sensing satisfaction of all nodes. However, due to the different coverage capabilities and motion blur degrees of vehicle nodes and roadside nodes for target objects, the sensing satisfaction function can be adjusted as follows:

[0038]

[0039] In the formula, Q req represents the sensing satisfaction function; q m and q n represent the sensing qualities of vehicle node m and roadside node n respectively; δ1 and δ2 represent weight coefficients; T represents the time gain for the node to complete the sensing task; The total cost of the digital twin system is mainly composed of the total rewards provided to all vehicle nodes and roadside nodes. Therefore, the utility of the digital twin system is determined by the difference between the revenue and the cost:

[0040]

[0041] In the formula, U DT (R,p) represents the utility of the digital twin system, p = {p1, p2,..., p N} is the set of sensing task rewards allocated by the DT to the RSU; R represents the reward given by the digital twin to the vehicle node; p n represents the reward given by the digital twin for collecting unit data from roadside node n.

[0042] Furthermore, step S3 specifically includes: In the vehicle-road collaborative sensing system, the collaborative sensing network between the ICV and the RSU constitutes the core support of the data layer. The sensing nodes are usually equipped with a variety of sensors, including but not limited to cameras, ultrasonic sensors, radars, etc., to collect relevant data on the surrounding vehicles and road environment. However, due to the significant differences in the sensing capabilities of the ICV and the RSU, and the high mobility of the vehicle further increasing the complexity of environmental sensing, comprehensively obtaining physical information has become a major challenge.

[0043] To improve the resource utilization efficiency in the vehicle-road collaborative perception scenario and ensure data integrity, it is necessary to reduce the redundancy of perception data and eliminate perception blind spots. Aiming at the problems of overlapping perception areas and insufficient coverage caused by the dynamic movement of ICVs and the fixed deployment of RSUs, an optimization method for perception areas based on the Jaccard similarity coefficient is proposed. Specifically, the perception area of the agent at time slot t is defined as Surf(i,t), and the degree of area overlap between different agents is quantified by PRI, as shown below:

[0044]

[0045] In the formula, J(i,t) is the PRI index; J min and J max are the given minimum and maximum thresholds respectively, ranging between [0,1]; Surf(i,t) represents the perception area of the ICV at time t; Surf(j,t) represents the perception area of the ICV's adjacent one-hop, which can be expressed as:

[0046]

[0047] In the formula, κ1 and κ2 are the perception coefficients of the ICV or RSU respectively; this mechanism can help the system balance the overlap degree of perception areas, ensure the effective execution of perception tasks, avoid duplicate data collection at the same time, reduce the waste of communication and computing resources, and ensure the collection of environmental information without blind spots.

[0048] To improve the accuracy of collaborative perception, the position and attitude of the agent are optimized to enhance the overall perception ability. The sensing accuracy is crucial for the accuracy of the DT model and can be estimated by calculating the number of perceived entities. To ensure the high accuracy of DT, the gap between the actual number of perceived entities and the expected number of perceived entities should be minimized, thereby optimizing the model performance. Assuming that the number of physical entities is known and usually follows a Poisson distribution; at this time, the DT model can use the maximum likelihood function to estimate the number of physical entities; specifically, the maximum likelihood estimation formula is as follows:

[0049]

[0050] In the formula, L(λ) is the maximum likelihood function, o1,o2,…,o k is the number of observations of the agent in each perception; λ is the expected number of accurate perceptions; based on this, DT can estimate the number of physical entities in the perception environment. Using the above maximum likelihood estimation formula, DT can further calculate the accurate number of perceived entities according to the properties of the Poisson distribution:

[0051]

[0052] Perform average calculations based on the observed actual data and predict the expected number of physical entities through the estimation results, thereby achieving a sense of collaboration; the agent can dynamically adjust its own position and posture according to the guidance of the DT model to optimize the collaborative perception effect within the target area. To further improve the perception accuracy, the DT model can be enhanced by means of environment simulation, enabling the agent to dynamically adjust the perception strategy according to the real-time estimation results and optimize the allocation of perception tasks. Based on this, the following collaborative perception model is used to quantify the perception accuracy of DT:

[0053]

[0054] In the formula, represents the perception accuracy function, N u [·] represents the number of physical entities sensed by the ICV or RSU within its sensing area during time slot t; K represents the normalization factor; Surf(t, S(x j , u i )) represents the state S of the agent at time t, where x j and u i are the position and posture of the node respectively; is the minimum acceptable perception quality to ensure the accuracy of DT, used to guarantee the effect of collaborative perception.

[0055] Furthermore, in step S4, the designed parallel pre-migration mechanism includes a VDT deployment model and a VDT migration model, and the construction method is as follows:

[0056] (1) VDT deployment model

[0057] For each ICV m , its DT server can be the local server connected to it or other edge servers connected to the RSU, collectively referred to as the DT server of ICV m ; x m (t, e n ) represents the digital twin placement indicator of ICV m , where x m (t, e n ) = 1 indicates that the VDT of ICV m is deployed on the edge server e n at time slot t, otherwise x m (t, e n ) = 0; VDT represents the virtual digital twin;

[0058] Due to the high-speed mobility of the vehicle, the vehicle may gradually leave the edge server that currently maintains its VDT and approach other edge servers. At this time, it is necessary to adjust the deployment location of the VDT, that is, trigger the DT migration process when the deployment location changes; define the matrix X = [xm (t, e n ) is the deployment strategy matrix of VDT, expressed as:

[0059]

[0060] In the formula, x m (t, e n ) represents the DT migration strategy function;

[0061] When the communication delay between the ICV m and its current VDT hosting server e n exceeds the threshold τ max , or the computing load of the server e n reaches saturation, the system triggers a migration decision; the migration strategy matrix of the entire system is denoted as:

[0062] X′ = (x1′, x2′, …, x′ m )

[0063] Due to the resource-intensive maintenance of digital twins, especially the huge overhead in terms of computing load, transmission delay, and energy consumption, the digital twin of each ICV can only be maintained by one edge server. The placement decision of digital twins imposes the following constraints:

[0064]

[0065] This constraint ensures that the digital twin of each ICV is maintained by only one edge server at any time slot t; in practical applications, the placement strategy of DT is closely related to its maintenance cost; to meet the QoS requirements, DT should be able to dynamically migrate to a suitable edge server according to the moving state of the ICV;

[0066] (2) VDT Migration Model

[0067] Due to the high mobility of vehicles, the ICV may access different RSUs at different time periods. To ensure delay-sensitive services, the DT model needs to be dynamically migrated from the original RSU to the target RSU to ensure the continuity and real-time performance of the system. The migration process of the VDT migration model includes the following three key steps:

[0068] (1) Model Transmission: The original RSU transmits the DT model data to the target RSU through wired communication, and the latter is responsible for receiving, constructing, and updating the DT model to maintain the continuity and real-time response ability of the system;

[0069] (2) State Update: After the ICV enters the coverage area of the target RSU, it will upload the latest state data through wireless communication to ensure that the DT model can be adjusted in real time to adapt to the new operating environment;

[0070] (3) Computing Resource Allocation and Model Update: After receiving data from the original RSU and the ICV, the target RSU allocates computing resources to update the DT model so that it can adapt to environmental changes and continuously provide efficient perception and decision-making support;

[0071] When the digital twin of the ICV migrates from the original server to the target server, migration latency is inevitable, including factors such as backhaul transmission, data processing, and queuing; the migration latency of each ICV's digital twin increases with the size of its digital twin model and the hop distance from the original server to the current server; specifically, the ICV m from server e m to server e n The digital twin migration latency can be expressed as:

[0072]

[0073] In the formula, W m is the size of the digital twin model of the ICV m ; d(m,n) represents the hop distance between server e m and server e n ; ψ(t) is the transmission rate of digital twin migration, representing the latency required to transmit a unit of data along the migration path; is a positive coefficient; if the original server e m is the same as the current server e n , that is, m = n, then the digital twin model of the ICV m has not migrated, so the migration latency is Therefore, the migration latency of the ICV m at time slot t can be expressed as:

[0074]

[0075] In the formula, represents the path with the longest completion time among all paths, that is, the time finally determining the completion of DT migration; P represents the set of paths, represents the migration latency, represents the DT placement decision coefficient at t - 1, represents the current DT placement decision coefficient. This method can ensure that data can be transmitted as soon as possible, rather than being affected by the bandwidth or link state of a single path. In addition, due to competition for server resources, additional computing and storage delays may occur. Therefore, in actual deployment, a resource reservation mechanism can be combined to reduce the resource allocation latency after migration completion and further improve the low-latency characteristics of the system.

[0076] Furthermore, in step S5, a joint optimization model for maximizing the comprehensive utility is established, and the specific objective is as follows:

[0077]

[0078] C3: 0 ≤ v m ≤ v max , m = 1,..., M

[0079]

[0080] C8: Q th ≤ Q req , m = 1,..., M

[0081]

[0082] C12: {x m,n (t), x m (t, e n )} ∈ {0, 1}, m = 1,..., M, n = 1,..., N

[0083] In the formula, the objective function is a problem of maximizing the comprehensive utility, aiming to maximize the overall utility of vehicle nodes, roadside nodes, and digital twins; the weighting coefficients λ1, λ2, and λ3 respectively represent the degree of emphasis on the utility of each part; constraint C1 ensures that the overlap degree of the sensing area is controlled within a reasonable range to avoid data redundancy or blind spots; constraint C2 ensures that the sensing quality of each node exceeds the set threshold to ensure the quality requirements of the system for the sensing data; constraint C3 ensures that the speed v of the vehicle m is within a physically reasonable range, that is, between the maximum speed v max and the minimum speed, in line with traffic rules; constraint C4 means that the total bandwidth allocated by each RSU to vehicle node m cannot exceed the maximum bandwidth B allowed by the system max ; represents the total time slot; constraint C5 limits the number of tasks that each vehicle node n can be allocated, ensuring that the tasks executed by each node within a given time slot t do not exceed the maximum allocation number M max ; constraint C6 ensures that the migration delay T migr (m, t) of the digital twin does not exceed the maximum value T allowed by the system max , and the migration delay is inversely proportional to the bandwidth B n,m (t) and the data volume; β1 represents the migration coefficient; constraint C7 ensures that the VDT of each ICV is maintained by only one edge server in each time slot t; constraint C8 stipulates that the overall sensing satisfaction of this system shall not be lower than the set requirement Q th, ensure the stability and effectiveness of the system; Constraint C9 restricts that the total sum of sensing resources consumed by each node during the sensing process does not exceed F max ; F max represents the maximum sensing resource consumption; Constraint C10 ensures that the channel gain r of each node m meets the minimum requirement r min ; Constraint C11 requires that the total amount of data collected by all nodes meets the data volume D required by the digital twin service; Constraint C12 represents the binary variable of the digital twin deployment and edge selection strategy.

[0084] Furthermore, in step S6, each RSU in the system is regarded as an agent. They observe the local state information and make decisions according to the policy function to maximize their own long-term cumulative discounted reward; specifically, the elements of the partially observable Markov decision process are defined as follows:

[0085] (1) State space The real-time sensing quality information of all vehicles and RSUs, the digital twin deployment location information of ICVs, the vehicle-RSU association information, and the resource utilization conditions of all edge servers;

[0086] (2) Partially observable state Each agent can only observe its own information and the vehicle information associated with it;

[0087] (3) Action space The action space includes the edge selection strategy, the twin deployment strategy, and the resource allocation strategy of the sensing frequency assigned to the sensing task;

[0088] (4) Reward function The reward function is the weighted sum of the sensing utility gain, the migration delay penalty, and the task completion reward.

[0089] Furthermore, in step S6, the optimization problem of the joint optimization model is modeled as a multi-agent deep reinforcement learning algorithm MA-POMDP based on MAPPO; MA-POMD can be represented as a quadruple, denoted as:

[0090]

[0091] In the formula, and respectively represent the joint state space and action space of all agents, and respectively represent the state space and action space of agent n; is the set of state transition probabilities, is the set of agent rewards; Assume that in the t time slot, the global state is The agent executes a joint action Then each agent obtains a reward The environment transitions to a new state with probability

[0092] Furthermore, in step S6, the online optimization algorithm DT-MAPPO based on MAPPO is used to solve the joint optimization problem of the deployment and migration of the digital twin assisted by vehicle-road collaborative perception, specifically including: the policy of agent n is fitted by a DNN with parameter θ n which is called the Actor network; to achieve the collaborative training of agents, a virtual central node is established to deploy a DNN with parameter φ, which is called the Critic network; this central node can be deployed in any edge server and communicate with other nodes; meanwhile, a twin Actor network with the same parameter θ n is created for each agent n at the central node to reduce additional communication overhead;

[0093] Define π n as the policy of agent n, that is, when the observable state is , the probability of selecting action ; approximate the policy function with a DNN, denoted as The joint policy of all agents is denoted as:

[0094] The ultimate goal of MAPPO training is to find the optimal agent joint policy π(a|s) to maximize the discounted reward U t , specifically expressed as:

[0095] U t =R t +γ·R t+1 +…+γ k ·R t+k ,k→∞

[0096] where the discount factor γ∈(0,1] is used to measure the attenuation degree of future rewards in the calculation of cumulative rewards, and R t+k represents the immediate reward obtained at time step t + k; define the global action value function as:

[0097]

[0098] where represents the global action value function of taking a t in state s t ;

[0099] Define the global state value function as:

[0100] ​

[0101] Wherein, represents the global state value function under state s t ;

[0102] Define the advantage function to evaluate the quality of taking actions relative to the average value in a given state, specifically expressed as:

[0103]

[0104] When training MAPPO, the data sampling process can be separated from the optimization strategy; first, the agent interacts with the environment using the current strategy to sample, performs a series of actions, and observes the corresponding rewards and state transitions; the samples are denoted as:

[0105] τ = {s0, a0, r0, s1, a1, …, s T-1 , s T}

[0106] Then, in the training stage, the sampled samples are used to update the Actor network parameters; MAPPO uses the proximal policy optimization method to maximize the cumulative reward, that is, to ensure that the updated policy remains within a certain range of the old policy; each time an update is made, the Actor network updates the network parameters through the policy loss function, and the Actor network loss function L Actor (·) is calculated as:

[0107]

[0108] Wherein, represents the old policy of agent n; represents the new policy of agent n after update; clip(·) is a truncation function used to ensure that the gap between the parameters of the Actor network after update and the old parameters is within a certain range; ε is a hyperparameter used to set the limit range of the clip(·) function; is the estimated value of the advantage function , specifically obtained by the method of Generalized Advantage Estimation (GAE), and is specifically defined as:

[0109]

[0110] Wherein, δ t = r t + γV(s t+1 ) - V(s t ), and V(s t ) represents the evaluation of state s by the Critic networkt Value estimation; λ ∈ [0, 1] is a hyperparameter that balances bias and variance; the Actor network parameters of agent n are updated iteratively using stochastic gradient ascent, expressed as:

[0111]

[0112] where α Actor is the learning rate of the Actor network, is the gradient of the Actor network parameters of agent n; the Critic network updates its network parameters through the global state value loss function; the loss function L Critic (φ) is expressed as:

[0113]

[0114] where V φ (s t ) is the value estimation of the state s t by the Critic network with parameters φ; V j (s(t)) is the target value;

[0115] The Critic network parameters φ are updated iteratively using the stochastic gradient descent method, expressed as:

[0116]

[0117] where α Critic is the learning rate of the Critic network, is the gradient of the Critic network parameters φ.

[0118] The beneficial effects of the present invention are as follows: meeting the requirements of the vehicle-road collaborative perception network for low-latency synchronization of digital twins and efficient utilization of perception data, the present invention significantly reduces the data transmission delay during the migration process and ensures the real-time synchronization of the digital twin and the physical entity through the digital twin parallel pre-migration mechanism and dynamic migration decision factor optimization, combined with the multi-path concurrent transmission technology; at the same time, based on the tripartite utility model driven by heterogeneous perception resources and the improved perception area similarity index, the overlapping degree of the perception area is dynamically quantified, effectively reducing the transmission and computational overhead of redundant perception data and improving the global utilization rate of edge resources; further, through the multi-agent proximal policy optimization algorithm and the centralized training and distributed execution framework, the adaptive decision-making ability of the system in dynamic traffic scenarios is enhanced, and the efficient collaborative optimization of digital twin deployment and migration strategies is realized.

[0119] Other advantages, objectives, and features of the present invention will be set forth to some extent in the following description, and to some extent, will be apparent to those skilled in the art based on an examination of the following, or can be learned from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0120] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0121] Figure 1 is the digital twin network architecture diagram for the vehicle networking constructed by the present invention.

[0122] Figure 2 is the framework diagram of the multi-agent deep reinforcement learning algorithm based on MAPPO proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0123] The following uses specific specific examples to illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention schematically. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0124] Please refer to Figures 1 to 2 , a digital twin dynamic deployment and migration optimization method based on vehicle-road collaborative perception of the present invention specifically includes the following steps:

[0125] S1: Construct a digital twin network for the vehicle networking, as Figure 1 shown.

[0126] S2: Construct a three-party utility model driven by heterogeneous perception resources, comprehensively considering perception quality, task completion time, and node energy consumption;

[0127] S3: Dynamically quantify the overlap degree of the perception areas of vehicles and roadside units, and combine a preset threshold to reduce the data collection redundancy rate;

[0128] S4: According to the computing load, storage capacity, and communication resource status of the edge server, design a parallel pre-migration mechanism, and realize the shard-independent transmission of the digital twin model through a multi-path concurrent transmission strategy;

[0129] S5: Establish a joint optimization model for maximizing comprehensive utility;

[0130] S6: The multi-agent proximal policy optimization algorithm is adopted, and through the centralized training and distributed execution framework, the collaborative optimization of the digital twin deployment strategy, migration path selection, and sensing task allocation is realized.

[0131] In the above step S1, the digital twin network for the vehicle-to-everything network is as Figure 1 shown, mainly including the physical terminal layer, the edge twin layer, and the cloud application layer.

[0132] S11: Physical terminal layer: This layer is mainly composed of M ICVs with limited sensing and storage resources. The ICV set is defined as V = {V1, V2, …, V M}. In the time slot , the ICV m communicates with the edge server by accessing a single RSU, and uploads the vehicle sensing data and status data to the edge server for DT synchronization.

[0133] S12: Edge twin layer: This layer is mainly composed of edge servers and RSUs that provide access services and sensing services. Each edge server is connected to all other edge servers. The RSU set is denoted as The edge server set is denoted as E = {e1, e2, …, e n}.

[0134] S13: Cloud control layer: A large-scale cloud DT management platform is maintained in the cloud layer, which is responsible for integrating the concepts of DT and network slicing, and realizing intelligent cloud resource management and dynamic network scheduling. Under this architecture, network slicing realizes service-oriented network management through network function virtualization technology, while DT constructs a digital model with organizational and customized characteristics and introduces a user-centered perspective to enhance the intelligence of network slicing. The cloud DT runs by receiving the data of all VDTs in the edge twin layer, which is expressed as:

[0135] DT = {H, Env(t), S(t)}

[0136] In the formula, H and Env(t) respectively represent all historical data and environmental information in the entire traffic scenario; S(t) represents the relevant information of all agents. It is assumed that the set of all agents that can perform sensing tasks is S, including roadside nodes and intelligent vehicles.

[0137] In the above step S2, the construction of the heterogeneous sensing resource-driven three-party utility model mainly includes the following steps:

[0138] S21: Construct a perception satisfaction model. In the vehicle-road collaborative perception system, the perception satisfaction of a node not only depends on its perception quality, but is also affected by the scale of the collected data and the time consumed to complete the perception task. Specifically, perception data usually has timeliness, and its value gradually decays over time. Therefore, the length of the task execution time directly determines the timeliness of the data. The faster the task is completed, the higher the effectiveness and practicality of the data. Considering the perception quality, data volume, and its timeliness comprehensively, the perception satisfaction of the digital twin can be defined as a weighted combination of these factors, aiming to optimize the execution efficiency of the perception task and minimize the impact of time delay on data utility. Therefore, the mathematical description of DT for the node perception satisfaction can be expressed as

[0139]

[0140] where δ is the satisfaction parameter; represents the perception frequency of the node; represents the time required for the node to execute the perception task; is the time for the node to complete the upload of the original data; represents the data volume collected by the node; represents the time gain for the node to complete the perception task, that is, the amount of time the node completes the task ahead of the set tolerance delay. Based on the definition of the perception satisfaction of a single node, the overall perception satisfaction of all nodes in the system can be expressed as:

[0141]

[0142] S22: Construct a tripartite utility model. Assume that the ICV m 's moving path is represented by a set L m of geographical coordinates, and its total length is |L m |. When the ICV m travels at a speed of v m , this ICV node will perform perception and data collection tasks within the specified area range. The residence time and the execution time of the perception task of this vehicle node are The perception frequency of the vehicle node is f m , then the data volume that this node can collect within the time period is Based on the standardized deployment architecture of the RSU, its perception decision is based on the set perception frequency f se , then the data volume collected by this node is f se t se , where t se represents the perception time of the RSU,

[0143] Assume that each RSU has Bandwidth resources and orthogonally allocate them to the ICVs within its coverage area. Let B n,m (t) represent the bandwidth resources allocated by the RSUn to the ICV m within time slot t. The transmission rate can be expressed as:

[0144]

[0145] where p m,n (t) and g m,n (t) represent the transmission power and channel gain of the RSU respectively; σ 2 is the power density of the background noise. The time required for the vehicle node to complete the upload of the original data is To ensure that the node can complete the data upload within the set tolerance delay, it is necessary to satisfy All vehicle nodes can obtain the rewards provided by the DT perception service slice, and its income depends on the proportion of the data volume collected by the node in the total data volume of all perception nodes, denoted as where R is the reward given by the DT to a single vehicle node. On the other hand, the vehicle node needs to bear the costs of perception task execution and original data transmission. Let be the cost of unit data collection, then the perception cost is Let be the cost of occupying the communication bandwidth per unit time, then the transmission cost is The utility of the vehicle node is the difference between the income and the cost:

[0146]

[0147] To quantify the comprehensive utility of the RSU, this chapter constructs its utility function model based on economic game theory. This model comprehensively considers multi-dimensional factors such as data transmission income, perception cost, digital twin maintenance cost, and task participation risk, and is specifically defined as follows:

[0148]

[0149] where p n is the reward given by the digital twin for each unit of data collected by the roadside node n, then the income of the roadside node is p n f se t se ; is the perception cost of unit data; c DT is the cost of maintaining the digital twin of the ICV in the RSU, then the perception cost of the roadside node is The cost of maintaining the digital twin is x m,n (t) represents the access strategy of the ICV m , when x m,nWhen (t) = 1, ICV m Transmit its perception data to the RSU n for data fusion; otherwise, x m,n (t) = 0; In addition, if the DT requires the RSU to participate in the crowd-sensing task, in addition to the explicit cost, it also needs to bear the implicit risk loss. Let w n be the risk loss per unit time, then the risk loss of the RSU is w n (t se ) 2 .

[0150] From the overall perspective of the DT system, its revenue comes from the perception satisfaction of all nodes. However, due to the different coverage capabilities and motion blur degrees of vehicle nodes and roadside nodes for target objects, the perception satisfaction function can be adjusted as follows:

[0151]

[0152] In the formula, q m and q n represent the perception quality of vehicle nodes and roadside nodes respectively. The total cost of the digital twin system is mainly composed of the total rewards provided to all vehicle nodes and roadside nodes. Therefore, the utility of the digital twin system is determined by the difference between revenue and cost:

[0153]

[0154] In the formula, p = {p1, p2,..., p N} is the set of perception task rewards allocated by the DT to the RSU.

[0155] In the above step S3, in the vehicle-road collaborative perception system, the collaborative sensing network between the ICV and the RSU constitutes the core support of the data layer. The perception nodes are usually equipped with a variety of sensors, including but not limited to cameras, ultrasonic sensors, radars, etc., to collect relevant data on the surrounding vehicles and road environment. However, due to the significant differences in the perception capabilities of the ICV and the RSU, and the high mobility of the vehicle further increases the complexity of environmental perception, comprehensively obtaining physical information has become a major challenge.

[0156] To improve the resource utilization efficiency and ensure data integrity in the vehicle-road collaborative perception scenario, it is necessary to reduce the redundancy of perception data and eliminate perception blind spots. Aiming at the problems of overlapping perception areas and insufficient coverage caused by the dynamic movement of the ICV and the fixed deployment of the RSU, a perception area optimization method based on the Jaccard similarity coefficient is proposed. Specifically, the perception area of the agent in time slot t is defined as Surf(i, t), and the degree of area overlap between different agents is quantified by PRI, as shown below:

[0157]

[0158] In the formula, J(i, t) is the PRI index; J min and J max are the given minimum and maximum thresholds respectively, with a range between [0, 1]. Surf(i, t) represents the sensing area of the ICV at time t; Surf(j, t) represents the sensing area of its adjacent one-hop, which can be expressed as:

[0159]

[0160] In the formula, κ1 and κ2 are the sensing coefficients of the ICV or RSU respectively. This mechanism can help the system balance the overlap degree of the sensing area, ensure the effective execution of the sensing task, avoid duplicate data collection at the same time, reduce the waste of communication and computing resources, and ensure the collection of environmental information without blind spots.

[0161] To improve the accuracy of collaborative sensing, the position and attitude of the agent are optimized to enhance the overall sensing ability. The sensing accuracy is crucial for the accuracy of the DT model and can be estimated by calculating the number of sensed entities. To ensure the high accuracy of the DT, the gap between the actual number of sensed entities and the expected number of sensed entities should be minimized, thereby optimizing the model performance. Assume that the number of physical entities is known and usually follows a Poisson distribution. At this time, the DT model can use the maximum likelihood function to estimate the number of physical entities. Specifically, the maximum likelihood estimation formula is as follows:

[0162]

[0163] In the formula, o1, o2, …, o k are the number of observations of the agent in each sensing; λ is the expected number of accurate senses. Based on this, the DT can estimate the number of physical entities in the sensing environment. Using the above maximum likelihood estimation formula, the DT can further calculate the accurate number of sensed entities according to the properties of the Poisson distribution:

[0164]

[0165] Based on the average calculation of the observed actual data and predicting the expected number of physical entities through this estimation result, collaborative sensing is realized. The agent can dynamically adjust its own position and attitude according to the guidance of the DT model to optimize the collaborative sensing effect in the target area. To further improve the sensing accuracy, the DT model can be enhanced by means of environment simulation, enabling the agent to dynamically adjust the sensing strategy according to the real-time estimation result and optimize the allocation of sensing tasks. Based on this, the following collaborative sensing model is proposed to quantify the sensing accuracy of the DT:

[0166]

[0167] Wherein, N u [x] represents the number of physical entities sensed by the ICV or RSU within its sensing area during time slot t; K represents the normalization factor; Surf(t, S(x j , u i )) represents the state S of the agent at time t, where x j and u i are the position and attitude of the node respectively; To ensure the minimum acceptable sensing quality of precision DT to guarantee the effect of collaborative sensing.

[0168] In the above step S4, the proposed parallel pre-migration mechanism includes a VDT deployment model and a VDT migration model, which mainly include the following steps:

[0169] S41: VDT deployment model. For each ICV m , its DT server can be the local server connected to it or other edge servers connected to the RSU, which are collectively referred to as the DT server of this ICV. x m (t, e n ) represents the digital twin placement indicator of the ICV m , where x m (t, e n ) = 1 indicates that the VDT of the ICV m is deployed on the edge server e n in time slot t, otherwise x m (t, e n ) = 0.

[0170] Due to the high-speed mobility of the vehicle, the vehicle may gradually leave the current edge server maintaining its VDT and approach other edge servers. At this time, it is necessary to adjust the deployment location of the VDT, that is, trigger the DT migration process when the deployment location changes. Define the matrix X = [x m (t, e n )] as the deployment strategy matrix of the VDT, which is expressed as:

[0171]

[0172] When the communication delay between the ICV m and its current VDT hosting server e n exceeds the threshold τ max , or the computing load of the server e n reaches saturation, the system triggers a migration decision. The migration strategy matrix of the entire system is denoted as:

[0173] X′ = (x1′, x2′,…, x′ m )

[0174] Due to the resource-intensive maintenance of digital twins, especially the huge overhead in terms of computing load, transmission delay, and energy consumption, the digital twin of each ICV can only be maintained by one edge server. The placement decision of the digital twin imposes the following constraints:

[0175]

[0176] This constraint ensures that the digital twin of each ICV is maintained by only one edge server at any time slot t. In practical applications, the placement strategy of DT is closely related to its maintenance cost. To meet the QoS requirements, DT should be able to dynamically migrate to a suitable edge server according to the moving state of the ICV.

[0177] S42: VDT migration model. Due to the high mobility of vehicles, an ICV may access different RSUs at different time periods. To ensure latency-sensitive services, the DT model needs to be dynamically migrated from the original RSU to the target RSU to ensure the continuity and real-time performance of the system. The complete DT model migration process involves the following three key steps:

[0178] (1) Model transmission: The original RSU transmits the DT model data to the target RSU through wired communication, and the latter is responsible for receiving, constructing, and updating the DT model to maintain the continuity and real-time response ability of the system.

[0179] (2) State update: After the ICV enters the coverage area of the target RSU, it uploads the latest state data through wireless communication to ensure that the DT model can be adjusted in real time to adapt to the new operating environment.

[0180] (3) Computing resource allocation and model update: After receiving the data from the original RSU and the ICV, the target RSU allocates computing resources to update the DT model so that it can adapt to environmental changes and continuously provide efficient perception and decision support.

[0181] When the digital twin of the ICV migrates from the original server to the target server, migration delay is inevitable, including factors such as backhaul transmission, data processing, and queuing waiting. The migration delay of the digital twin of each ICV increases with the increase in the size of its digital twin model and the hop distance from the original server to the current server. Specifically, the ICV m from server e m to server e n The digital twin migration delay can be expressed as:

[0182]

[0183] where W m is the ICV mThe size of the digital twin model; d(m,n) represents the hop distance between server e m and server e n The hop distance between them; ψ(t) is the transmission rate of digital twin migration, representing the delay required to transmit a unit of data along the migration path; is a positive coefficient. If the original server e m is the same as the current server e n , that is, m = n, then the digital twin model of ICV m has not migrated, so the migration delay is Thus, the migration delay of ICV m at time slot t can be expressed as:

[0184]

[0185] In the formula, represents the path with the longest completion time among all paths, that is, the time finally determining the completion of DT migration. This method can ensure that data can be transmitted as soon as possible, rather than being affected by the bandwidth or link state of a single path. In addition, due to the competition for server resources, additional computing and storage delays may occur. Therefore, in actual deployment, a resource reservation mechanism can be combined to reduce the resource allocation delay after migration completion, further improving the low-latency characteristics of the system.

[0186] In step S5, for the DT dynamic deployment and task allocation problem in the vehicle-road collaborative perception network, it aims to optimize the migration of the DT model and the perception task allocation to improve the system perception quality, reduce the migration delay, and balance resource utilization. The specific description is as follows:

[0187]

[0188] C3: 0 ≤ v m ≤ v max , m = 1,..., M

[0189]

[0190] C8: Q th ≤ Q req , m = 1,..., M

[0191]

[0192] C12: {x m,n (t), x m (t, e n )} ∈ {0, 1}, m = 1,..., M, n = 1,..., N

[0193] In the formula, the objective function is a comprehensive utility maximization problem, aiming to maximize the overall utility of vehicle nodes, roadside nodes, and digital twins. The weighting coefficients λ1, λ2, and λ3 are used to represent the degree of emphasis on the utility of each part respectively; the constraint C1 ensures that the overlap degree of the sensing area is controlled within a reasonable range to avoid data redundancy or blind spots; the constraint C2 ensures the sensing quality of each node exceeds the set threshold to ensure the quality requirements of the system for the sensed data; the constraint C3 ensures that the speed v of the vehicle m is within a physically reasonable range, that is, between the maximum speed and the minimum speed, and complies with traffic rules; the constraint C4 means that the total bandwidth allocated by each RSU to the vehicle node m cannot exceed the maximum bandwidth B allowed by the system max ; the constraint C5 limits the number of tasks that each vehicle node n can be allocated, ensuring that the tasks executed by each node within a given time slot t do not exceed the maximum allocation number M max ; the constraint C6 ensures that the migration delay T of the digital twin migr (m,t) does not exceed the maximum value T allowed by the system max , and the migration delay is inversely proportional to the bandwidth B n,m (t) and the data volume; the constraint C7 ensures that the VDT of each ICV is maintained by only one edge server in each time slot t; the constraint C8 stipulates that the overall sensing satisfaction of this system shall not be lower than the set requirement Q th , ensuring the stability and effectiveness of the system; the constraint C9 limits the total sensing resources consumed by each node during the sensing process not to exceed F max ; the constraint C10 ensures that the channel gain r of each node m meets the minimum requirement r min ; the constraint C11 requires that the total data volume collected by all nodes meets the data volume D required by the digital twin service; the constraint C12 represents the binary variable of the digital twin deployment and edge selection strategy.

[0194] In step S6, as Figure 2 shown, the specific steps to solve the optimal digital twin dynamic deployment and migration scheme are as follows:

[0195] S61: Regard each RSU in the system as an agent. They observe the local state information and make decisions according to the policy function to maximize their own long-term cumulative discounted reward. The detailed elements of the partially observable Markov decision process are defined as follows:

[0196] (1) State space The real-time sensing quality information of all vehicles and RSUs, the deployment location information of the digital twins of ICVs, the vehicle-RSU association information, and the resource utilization conditions of all edge servers.

[0197] (2) Partially Observable State Each agent can only observe its own information and the vehicle information associated with it.

[0198] (3) Action Space The action space includes an edge selection strategy, a twin deployment strategy, and a resource allocation strategy for the sensing frequency assigned by the sensing task.

[0199] (4) Reward Function The reward function is the weighted sum of the sensing utility gain, the migration delay penalty, and the task completion reward.

[0200] S62: Since the proposed joint optimization problem is an NP-hard problem and contains discrete variables, traditional optimization algorithms cannot be used to solve it. Therefore, a multi-agent deep reinforcement learning algorithm based on MAPPO is proposed to solve this problem. First, the optimization problem is modeled as a MA-POMDP. MA-POMDP can be represented as a quadruple, denoted as:

[0201]

[0202] Where, and represent the joint state space and action space of all agents respectively, and represent the state space and action space of agent n respectively. is the set of state transition probabilities, is the set of agent rewards. Assume that at time slot t, the global state is the agents execute the joint action then each agent obtains the reward the environment transfers to the new state with a probability of .

[0203] S63: A DT-MAPPO online optimization algorithm based on MAPPO is proposed to solve the joint optimization problem of the deployment and migration of the digital twin assisted by vehicle-road collaborative sensing.

[0204] The policy of agent n is fitted by a DNN with parameter θ n , called the Actor network. To achieve the collaborative training of agents, a virtual central node is established to deploy a DNN with parameter φ, called the Critic network. This central node can be deployed in any edge server and communicate with other nodes. At the same time, a twin Actor network with the same parameter θ n is created for each agent n at the central node to reduce the additional communication overhead.

[0205] Define $\pi$ n as the policy of agent $n$, that is, when the observable state is , the probability of choosing action . Approximate the policy function with a DNN, denoted as The joint policy of all agents is denoted as:

[0206]

[0207] The ultimate goal of MAPPO training is to find the optimal joint policy $\pi(a|s)$ of agents to maximize the discounted reward $U$ t , specifically expressed as:

[0208] $U$ t $=R$ t $+\gamma\cdot R$ t+1 $+\cdots+\gamma$ k $\cdot R$ t+k , $k\rightarrow\infty$

[0209] In the formula, the discount factor $\gamma\in(0,1]$ is used to measure the attenuation degree of future rewards in the calculation of cumulative rewards. Define the global action-value function as:

[0210]

[0211] Define the global state-value function as:

[0212]

[0213] Define the advantage function to evaluate how good it is to take an action relative to the average in a given state, specifically expressed as:

[0214]

[0215] When training MAPPO, the data sampling process can be separated from the optimization of the policy. First, the agent interacts with the environment using the current policy to sample, executes a series of actions, and observes the corresponding rewards and state transitions. The sample is denoted as:

[0216] $\tau=\{s_0,a_0,r_0,s_1,a_1,\cdots,s$ T-1 , $s$ T $\}$

[0217] Then, in the training stage, use the sampled samples to update the Actor network parameters. MAPPO uses the proximal policy optimization method to maximize the cumulative reward, that is, to ensure that the updated policy remains within a certain range of the old policy. Each time an update is made, the Actor network updates the network parameters through the policy loss function. The calculation method of the Actor network loss function corresponding to agent $n$ is:

[0218]

[0219] In the formula, represents the old policy of agent n; represents the new policy updated by agent n; clip(·) is a truncation function used to ensure that the gap between the parameters after the update of the Actor network and the old parameters is within a certain range; ε is a hyperparameter used to set the limit range of the clip(·) function; is the estimated value of the advantage function , which is specifically obtained by the method of Generalized Advantage Estimation (GAE), and is specifically defined as:

[0220]

[0221] In the formula, δ t = r t + γV(s t+1 ) - V(s t ); λ ∈ [0, 1] is a hyperparameter that balances bias and variance. The parameters of the Actor network of agent n are updated iteratively by stochastic gradient ascent, expressed as:

[0222]

[0223] In the formula, α Actor is the learning rate of the Actor network; the Critic network updates the network parameters through the global state value loss function. The loss function of the Critic network is expressed as:

[0224]

[0225] The parameters φ of the Critic network are updated iteratively by the stochastic gradient descent method, expressed as:

[0226]

[0227] In the formula, α Critic is the learning rate of the Critic network.

[0228] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not restrictive. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A digital twin dynamic deployment and migration optimization method based on vehicle-road collaborative perception, characterized in that: The method specifically comprises the following steps: S1: Build a digital twin model for the Internet of Vehicles; S2: Construct a tripartite utility model driven by heterogeneous sensing resources, taking into account the sensing quality, task completion time and node energy consumption; S3: Dynamically quantify the overlap between the perception area of ​​the vehicle and the roadside unit, and reduce the data collection redundancy rate by combining the preset threshold; S4: Based on the computing load, storage capacity, and communication resource status of the edge server, a parallel pre-migration mechanism is designed to achieve independent transmission of shards of the digital twin model through a multi-path concurrent transmission strategy; S5: Establish a joint optimization model to maximize comprehensive utility; S6: Adopting a multi-agent proximal strategy optimization algorithm, through centralized training and distributed execution framework, the collaborative optimization of digital twin deployment strategy, migration path selection and perception task allocation is achieved.

2. The digital twin dynamic deployment and migration optimization method according to claim 1 is characterized in that: In step S1, the digital twin model for the Internet of Vehicles is constructed, which includes the following three layers: (1) Physical terminal layer, including M ICVs with limited sensing and storage resources, where ICV represents intelligent connected vehicles and the ICV set is defined as V = {V1, V2, …, V M }; In the time slot The mth intelligent connected vehicle ICV m By accessing a single RSU to communicate with the edge server, the vehicle perception data and status data are uploaded to the edge server for DT synchronization, where RSU stands for roadside unit and DT stands for digital twin; (2) Edge twin layer, including edge servers and RSUs that provide access services and perception services. Each edge server is connected to all other edge servers. The RSU set is denoted as The edge server set is denoted as E = {e1, e2, …, e n }, where N represents the number of RSUs and n represents the number of edge servers; (3) Cloud control layer: A large-scale cloud DT management platform is maintained in the cloud control layer, which is responsible for integrating the concepts of DT and network slicing. The cloud DT management platform operates by receiving data from all VDTs in the edge twin layer as follows: DT={H,Env(t),S(t)} In the formula, H and Env(t) represent all historical data and environmental information in the entire traffic scene respectively; S(t) represents the relevant information of all agents. It is assumed that the set of all agents that can perform perception tasks is S, including RSU and ICV; VDT represents virtual digital twin.

3. The digital twin dynamic deployment and migration optimization method according to claim 2 is characterized in that: In step S2, a tripartite utility model driven by heterogeneous sensing resources is constructed, which specifically includes: (1) Constructing a perception satisfaction model: The mathematical description of DT's node perception satisfaction is expressed as: In the formula, Q s represents the perceived satisfaction of the ICVs node, δ is the satisfaction parameter, J(s,t) represents the Jaccard similarity coefficient, represents the perception frequency of the ICVs node, represents the residence time of the ICVs node and the execution time of the sensing task, It is the time when the ICVs node completes uploading the original data; Indicates the amount of data collected by the ICVs node; It represents the time gain of ICVs node to complete the perception task. Based on the definition of perception satisfaction of a single node, the overall perception satisfaction of all nodes in the system is expressed as: In the formula, Q represents the overall perceived satisfaction of all nodes in the system; (2) Constructing a tripartite utility model: Assuming ICV m The moving path is composed of a set of geographic coordinates L m Indicates that its total length is |L m |; When ICV m At speed v m When driving, ICV m The nodes will perform perception and data collection tasks within the specified area; ICV m The node residence time and the execution time of the perception task are ICV m The node’s perception frequency is f m , then the amount of data collected by the node in the time period is Based on the standardized deployment architecture of RSU, RSU perception decision is based on the set perception frequency f se , then the amount of data collected by the RSU node is f se t se , where t se represents the perception time of RSU, Assume that each RSU has bandwidth resources and allocate them orthogonally to the ICVs within its coverage area; let B n,m (t) indicates the nth RSU is assigned to the ICV in time slot t m Bandwidth resources, transmission rate r m It is expressed as: In the formula, p m,n (t) and g m,n (t) represents RSU transmit power and channel gain respectively; σ 2 is the power density of background noise; ICV m The time required for a node to complete uploading raw data is In order to ensure that the node can complete data upload within the set tolerance delay, it is necessary to meet All ICV nodes can receive rewards provided by DT-aware service slices, and their income depends on ICV m The proportion of the node data collected in the total data of all sensing nodes is set as Where R is the reward given by DT to a single vehicle node; on the other hand, the vehicle node needs to bear the cost of sensing task execution and original data transmission; is the cost of unit data collection, then the perception cost is Let c tr is the cost of occupying communication bandwidth per unit time, then the transmission cost is ICV m The utility of the node U m is the difference between benefit and cost: In order to quantify the comprehensive utility of RSU, a utility function model is constructed based on economic game theory. This model comprehensively considers data transmission benefits, perception costs, digital twin maintenance costs, and task participation risks. The specific definitions are as follows: Where U n is the utility of roadside node n; p n is the reward given by the digital twin to the roadside node n for collecting unit data, then the benefit of the roadside node is p n f se t se ; is the perceived cost per unit data; c DT is the cost of maintaining the ICV digital twin in the RSU, and the perception cost of the roadside node is The cost of maintaining the digital twin is x m,n (t) represents ICV m The access policy of x m,n When (t) = 1, ICV m Transmit its sensing data to the nth RSU for data fusion, otherwise, x m,n (t) = 0; In addition, if DT requires RSU to participate in the group intelligence sensing task, in addition to the explicit cost, it also needs to bear the implicit risk loss; let w n is the risk loss per unit time, then the risk loss of RSU is w n (t se ) 2 ; The perceived satisfaction function is: In the formula, Q req represents the perceived satisfaction function; q m and q n They represent the perception quality of vehicle node m and roadside node n respectively; δ1 and δ2 represent weight coefficients; T represents the time gain of the node to complete the perception task; the utility of the digital twin system is determined by the difference between the benefit and the cost: Where U DT (R,p) represents the utility of the digital twin system, p = {p1,p2,…,p N } is the set of perception task rewards assigned by DT to RSU; R represents the reward given by the digital twin to the vehicle node; p n It represents the reward given by the digital twin to the roadside node n for collecting unit data.

4. The digital twin dynamic deployment and migration optimization method according to claim 3 is characterized in that: Step S3 specifically includes: defining the perception area of ​​the agent in time slot t as Surf(i,t), and quantifying the degree of area overlap between different agents through PRI, as shown below: Where, J(i,t) is the PRI index; J min and J max are the given minimum and maximum thresholds, ranging from [0,1]; Surf(i,t) represents the perception area of ​​ICV at time t; Surf(j,t) represents the perception area of ​​the adjacent hop of ICV, expressed as: Where, κ1 and κ2 are the perception coefficients of ICV or RSU respectively; Assume that the number of physical entities is known and usually obeys a Poisson distribution; in this case, the DT model uses the maximum likelihood function to estimate the number of physical entities; specifically, the maximum likelihood estimation formula is as follows: Where L(λ) is the maximum likelihood function, o1,o2,…,o k is the number of observations made by the agent in each perception; λ is the expected number of accurate perceptions; using the above maximum likelihood estimation formula, DT can further calculate the accurate number of perceived entities based on the properties of the Poisson distribution: Based on the average calculation of the actual observed data, the expected number of physical entities is predicted through the estimated results to achieve collaborative perception; the following collaborative perception model is used to quantify the perception accuracy of DT: In the formula, represents the perceptual accuracy function, N u [·] represents the number of physical entities sensed by the ICV or RSU in its sensing area during time slot t; K represents the normalization factor; Surf(t,S(x j ,u i )) represents the state S of the agent at time t, where x j and u i are the position and posture of the node respectively; To ensure the minimum acceptable perceptual quality of DT.

5. The digital twin dynamic deployment and migration optimization method according to claim 4 is characterized in that: In step S4, the designed parallel pre-migration mechanism includes a VDT deployment model and a VDT migration model, and the construction method is as follows: (1) VDT deployment model For each ICV m , its DT server can be a local server connected to it, or other edge servers connected to RSU, collectively referred to as ICV m DT server; x m (t,e n ) indicates ICV m The digital twin placement indicator of m (t,e n )=1 indicates ICV m The VDT is deployed on the edge server e at time slot t n On the contrary, x m (t,e n )=0; VDT means virtual digital twin; Adjust the deployment location of VDT, that is, trigger the DT migration process when the deployment location changes; define the matrix X = [x m (t,e n )] is the deployment strategy matrix of VDT, expressed as: In the formula, x m (t,e n ) represents the DT migration strategy function; When ICV m With its current VDT hosting server n The communication delay exceeds the threshold τ max , or server e n When the computing load reaches saturation, the system triggers the migration decision; the migration strategy matrix of the entire system is recorded as: X′=(x1′,x2′,…,x′ m ) The digital twin of each ICV can only be maintained by one edge server, and the placement decision of the digital twin imposes the following constraints: This constraint ensures that the digital twin of each ICV is maintained by only one edge server at any time slot t; in practical applications, the placement strategy of DT is closely related to its maintenance cost; in order to meet QoS requirements, DT should be able to dynamically migrate to a suitable edge server according to the mobility status of ICV; (2) VDT Migration Model The migration process of the VDT migration model includes the following three steps: (1) Model transmission: The original RSU transmits the DT model data to the target RSU through wired communication. The target RSU is responsible for receiving, building, and updating the DT model to maintain the continuity and real-time response capability of the system. (2) Status update: After the ICV enters the coverage area of ​​the target RSU, it will upload the latest status data through wireless communication to ensure that the DT model can be adjusted in real time to adapt to the new operating environment; (3) Computing resource allocation and model updating: After receiving data from the original RSU and ICV, the target RSU allocates computing resources to update the DT model so that it can adapt to environmental changes and continue to provide efficient perception and decision support; When the digital twin of an ICV is migrated from the original server to the target server, the migration delay includes backhaul transmission, data processing, and queuing. The migration delay of the digital twin of each ICV increases with the size of its digital twin model and the hop distance from the original server to the current server. Specifically, the ICV m From the server m To server n Digital Twin Migration Delays It is expressed as: Where W m ICV m The size of the digital twin model; d(m,n) represents the server e m and server n ψ(t) is the hop distance between two nodes; ψ(t) is the transmission rate of digital twin migration, which represents the delay required to transmit one unit of data along the migration path; is a positive coefficient; if the original server e m With the current server n The same, that is, m = n, then ICV m of digital twins were not migrated, so the migration delay was Therefore, ICV m The migration delay at time slot t is expressed as: In the formula, It represents the path with the longest completion time among all the paths, which is the final time to determine the completion of DT migration; P represents the path set, Indicates a delay in migration, represents the DT placement decision coefficient at t-1, Indicates the current DT placement decision coefficient.

6. The digital twin dynamic deployment and migration optimization method according to claim 5 is characterized in that: In step S5, a joint optimization model for maximizing comprehensive utility is established, with the specific objectives being: In the formula, the objective function is a comprehensive utility maximization problem, which aims to maximize the overall utility of vehicle nodes, roadside nodes and digital twins; the weighting coefficients λ1, λ2 and λ3 respectively represent the importance of the utility of each part; constraint C1 ensures that the overlap of the perception area is controlled within a reasonable range to avoid data redundancy or blind spots; constraint C2 ensures the perception quality of each node Exceeding the set threshold Ensure the system's quality requirements for perception data; Constraint C3 ensures the vehicle's speed v m Within the physically reasonable range, that is, the maximum speed v max and the minimum speed, which complies with traffic regulations; constraint C4 indicates that the total bandwidth allocated by each RSU to vehicle node m cannot exceed the maximum bandwidth B allowed by the system. max ; represents the total time slot; constraint C5 limits the number of tasks that can be assigned to each vehicle node n, ensuring that the tasks performed by each node in a given time slot t does not exceed the maximum number of assigned tasks M max ; Constraint C6 ensures the migration delay T of the digital twin migr (m,t) will not exceed the maximum value T allowed by the system max , migration delay and bandwidth B n,m (t) is inversely proportional to the amount of data; β1 represents the migration coefficient; Constraint C7 ensures that the VDT of each ICV is maintained by only one edge server in each time slot t; Constraint C8 stipulates that the overall perceived satisfaction of the system must not be lower than the set requirement Q th , ensuring the stability and effectiveness of the system; Constraint C9 limits the total amount of sensing resources consumed by each node during the sensing process to no more than F max ; F max represents the maximum sensing resource consumption; constraint C10 ensures that the channel gain r of each node m Meet the minimum requirements min ; Constraint C11 requires that the total amount of data collected by all nodes meets the data volume D required by the digital twin service; Constraint C12 represents the binary variable of the digital twin deployment and edge selection strategy.

7. The digital twin dynamic deployment and migration optimization method according to claim 6 is characterized in that: In step S6, each RSU in the system is regarded as an intelligent agent, which observes local state information and makes decisions according to the policy function to maximize its long-term cumulative discounted reward; specifically, the partially observable Markov decision process elements are defined as follows: (1) State space Real-time perception quality information of all vehicles and RSUs, deployment location information of digital twins of ICVs, vehicle-RSU association information, and resource utilization of all edge servers; (2) Partially observable states Each agent can only observe its own information and the information of the vehicles associated with it; (3) Action Space The action space includes edge selection strategy, twin deployment strategy, and resource allocation strategy for the sensing frequency assigned to the sensing task; (4) Reward Function The reward function is a weighted sum of perceived utility gain, migration delay penalty, and task completion reward.

8. The digital twin dynamic deployment and migration optimization method according to claim 7 is characterized in that: In step S6, the optimization problem of the joint optimization model is modeled as a multi-agent deep reinforcement learning algorithm MA-POMDP based on MAPPO; MA-POMD is represented as a four-tuple, denoted as: In the formula, and denote the joint state space and action space of all agents respectively, and Represent the state space and action space of agent n respectively; is the set of state transition probabilities, is the set of agent rewards; assuming that in time slot t, the global state is Agents perform joint actions Then each agent receives a reward Environment The probability of transferring to a new state 9. The digital twin dynamic deployment and migration optimization method according to claim 8, characterized in that: In step S6, the MAPPO-based online optimization algorithm DT-MAPPO is used to solve the joint optimization problem of deployment and migration of digital twins assisted by vehicle-road cooperative perception, which specifically includes: the central node is deployed in any edge server and communicates with other nodes; at the same time, a central node with the same parameter θ is created for each intelligent agent n n Twin Actor network; Defining π n is the strategy of agent n, that is, when the observable state is When you select Action The probability of using DNN to approximate the policy function is recorded as The joint strategy of all agents is recorded as: The ultimate goal of MAPPO training is to find the optimal agent joint strategy π(a|s) to maximize the discounted reward U t , specifically expressed as: U t =R t +γ·R t+1 +…+c k ·R t+k ,k→∞ Where, the discount factor γ∈(0,1] is used to measure the degree of decay of future rewards in the calculation of cumulative rewards, R t+k Represents the immediate reward obtained at time step t+k; the global action value function is defined as: In the formula, Indicates that in state s t Next take a t The global action value function of Define the global state value function as: In the formula, Indicates state s t The global state value function under ; Defining the advantage function It is used to evaluate how good an action is in a given state relative to the average value, specifically expressed as: When training MAPPO, the data sampling process is separated from the optimization strategy. First, the agent uses the current strategy to interact with the environment and perform a series of actions to observe the corresponding rewards and state transitions. The samples are recorded as: τ={s0,a0,r0,s1,a1,…,s T-1 ,s T } Then, in the training phase, the sampling samples are used to update the Actor network parameters; MAPPO uses the proximal policy optimization method to maximize the cumulative reward, that is, to ensure that the updated policy is within a certain range from the old policy; each time the Actor network is updated, the network parameters are updated through the policy loss function, and the Actor network loss function L corresponding to the agent n is Actor (·)The calculation method is: In the formula, represents the old strategy of agent n; represents the new strategy of agent n after updating; clip(·) is a truncation function; ε is a hyperparameter used to set the limit range of the clip(·) function; is the advantage function The estimated value of is obtained by the method of generalized advantage estimation, which is specifically defined as: In the formula, δ t =r t +γV(s t+1 )-V(s t ), V(s t ) represents the Critic network's response to state s t The value estimate of ; λ∈[0,1] is a hyperparameter that balances the bias and variance; the Actor network parameters of agent n are updated iteratively using stochastic gradient ascent, expressed as: In the formula, α Actor is the learning rate of the Actor network, is the Actor network parameter gradient of agent n; the Critic network updates the network parameters through the global state value loss function; the loss function L of the Critic network Critic (φ) is expressed as: Where V φ (s t ) is the Critic network with parameter φ for state s t The value of V j (s(t)) is the target value; The critic network parameter φ is iteratively updated using the stochastic gradient descent method, expressed as: In the formula, α Critic is the learning rate of the Critic network, is the gradient of the Critic network parameter φ.

Images