Digital twin model migration system and optimization method for vehicle-mounted edge calculation
By employing a two-layer learning framework and a multi-agent selective parameter sharing mechanism, combined with a dual-timescale optimized data upload and model migration strategy, the accuracy and timeliness issues of digital twin models in vehicle edge computing are resolved, achieving efficient model migration and real-time information maintenance.
Patent Information
- Application Number
- CN202510975120.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-15
- Publication Date
- 2025-10-28
AI Technical Summary
Existing digital twin models in vehicle-mounted edge computing suffer from problems such as large data upload and processing volumes, poor information freshness, and high model migration costs, resulting in unsatisfactory accuracy and timeliness. In particular, distributed learning in highly dynamic scenarios suffers from high training overhead and low learning efficiency.
A two-layer learning framework is adopted, which includes a multi-agent selective parameter sharing mechanism based on spatiotemporal correlation. The spatiotemporal dependencies between agents are captured through a multi-head attention mechanism. Combined with the dual-time scale optimization data upload and model migration strategy, selective parameter sharing and dynamic adjustment strategies are introduced to optimize the twin model migration system.
It significantly reduced system operating costs, improved model convergence speed, reduced communication overhead, and enabled efficient digital twin model migration and real-time information maintenance, thereby enhancing the system's accuracy and timeliness.
Smart Images

Figure CN120849115A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle edge computing technology, and more particularly to the field of vehicle edge computing technology for digital twin model migration, specifically to a digital twin model migration system and optimization method for vehicle edge computing. Background Technology
[0002] Digital twin models have become a key technology driving innovation and transformation in intelligent vehicles. Current applications are mainly concentrated in areas such as R&D simulation, virtual testing, predictive maintenance, and condition monitoring, demonstrating enormous potential. Leading automakers, numerous component suppliers, and technology companies are actively deploying digital twin technology. It is currently at a critical stage of development, moving from proof-of-concept and specific scenario applications to large-scale, deeply integrated deployment. However, the practical application of digital twin models still faces multiple challenges, including model accuracy, data processing, and security standards. In particular, the accuracy and timeliness of models require massive computing power, making this the most pressing issue to be addressed in the application of existing digital twin models in the intelligent vehicle field.
[0003] Specifically, digital twin (DT) technology plays a crucial role in intelligent transportation systems, providing vehicles with real-time digital mapping and data-driven insights. With the development of DT technology, increasing research focuses on integrating DT with vehicular edge computing (VEC) to improve system performance and reliability. This integration not only supports vehicles executing resource-intensive applications at the network edge but also enables data analysis, environmental simulation, and collaborative decision optimization. However, the high mobility of vehicles may lead to frequent switching in the edge access network, increasing the overhead of AoI (Aspect-Oriented Integration) of the digital twin model data. This inevitably results in poor accuracy and timeliness in practical applications of digital twins. Summary of the Invention
[0004] To address the issues of high data upload and processing volume, poor information freshness, and high model migration costs in the field of automotive edge computing, which lead to unsatisfactory accuracy and timeliness of existing digital twin models, this invention first summarizes the limitations of existing technologies regarding digital twin migration for ease of understanding: Existing technologies lack sufficient consideration for optimization at different time scales, and distributed learning in highly dynamic scenarios suffers from high training overhead and low learning efficiency. These limitations make existing methods perform poorly when dealing with the spatiotemporal coupling characteristics and cost optimization issues involved in digital twin migration.
[0005] This application provides a digital twin model migration system and optimization method for vehicle edge computing, which can achieve at least one of the following effects: 1. This invention provides a system based on digital twin model transfer and proposes a two-layer learning framework that includes a mechanism for selective parameter sharing among multiple agents based on spatiotemporal correlation. It captures the spatiotemporal dependencies between agents through a multi-head attention mechanism and shares parameters only among highly correlated agents, thereby accelerating model convergence and reducing communication overhead.
[0006] 2. The present invention also provides a dual-timescale, dual-layer learning method that can adaptively optimize the twin model transfer and data upload strategy at different time scales, effectively reducing the freshness of AoI information in the system and minimizing the model transfer cost.
[0007] To overcome the problems and technical challenges of existing technologies, this invention proposes a distributed deep reinforcement learning framework based on dual-timescale optimization. This framework significantly reduces system operating costs by jointly optimizing data upload and model transfer at different timescales. Simultaneously, it introduces a spatiotemporally dependent parameter sharing mechanism and a dynamic adjustment strategy to improve multi-agent collaborative efficiency and overall system performance, thereby achieving efficient and scalable digital twin transfer. To achieve the above objectives, the technical solution adopted in this application is as follows: This invention provides a digital twin model migration system for vehicle edge computing, comprising: The physical entity layer includes vehicles, roadside units, and sensor networks used to generate real-time environmental data; The digital twin layer, deployed on edge servers, performs the creation, updating, and migration of vehicle digital twin models; The cloud center provides global computing and storage resources; The dual-timescale optimization module includes: Upper-level learning units, in the first cycle on a long timescale Internally, reinforcement learning is used to optimize the transfer decisions of the digital twin model and generate deployment decision variables. , indicating vehicle i The twin model in the time slot t Deployed on an edge server j ; The next learning unit is shorter than the first cycle. The second cycle Internally, reinforcement learning is used to jointly optimize the selection of data upload paths and the allocation of resources. and computational task scheduling; the first cycle For the second cycle multiples of, and > ; The multi-agent collaborative module selectively shares parameters between any two adjacent edge servers based on spatiotemporal correlation; the output of the upper-layer decision serves as the constraint condition for the lower-layer decision, and the real-time state feedback of the lower-layer decision is used to update the upper-layer policy. The objective function is to minimize the sum of the system's long-term average peak AoI and the model migration cost, expressed as: in, This represents the total number of time slots. Let t be the average peak information age of the system within time slot t. This is the weighting factor for cost and AoI. Let t be the total cost of migrating the twin model within time slot t.
[0008] To address the high computational complexity of sequence-to-sequence prediction models, such as RNNs and LSTMs, during iterative processing, this invention employs a multi-head attention mechanism with parallel computing capabilities to extract spatiotemporal dependencies between agents. Simultaneously, to address the issue that existing multi-head self-attention mechanisms' positional encoding methods fail to explicitly retain the timestamp information of each element, potentially leading to the loss of temporal dimension information, we improve the encoding method by combining positional encoding with time-step encoding, thereby more effectively preserving temporal dimension information. Preferably, the multi-agent collaborative module includes: The spatiotemporal feature encoding module is used to extract dynamic features of each edge server agent, including temporal features, spatial features, and interaction features. The temporal features include the deployment status of the twin model and real-time traffic load. The spatial features include geographical location coordinates and network topology connections. The interaction features include historical model migration records and vehicle movement trajectories. The multidimensional edge information of each agent is feature-encoded, and the above features help to comprehensively reflect the spatiotemporal correlation between agents.
[0009] The multi-head attention dependency analysis module calculates the spatiotemporal dependency matrix Z between agents through a multi-head self-attention mechanism, where the elements... , representing an intelligent agent m and n The strength of the dependency is expressed as: Where Q, K, and V represent the query vector, key vector, and value vector, respectively, and the parameters... Let Q and K represent the dimensions of vectors Q and K; Selective parameter sharing module, only when > At that time, intelligent agent m and n Shared parameters.
[0010] This invention also provides an optimization method for performing dual-time hierarchical learning scale optimization on the above-mentioned digital twin model transfer system for vehicle edge computing, comprising the following steps: Step STP100, the upper-level learning phase, in the first cycle of the timescale. Next, according to the status Optimize the twin model transfer strategy to minimize the sum of the long-term average peak AoI and the transfer cost: in, This represents the total number of time slots. The average peak value of the system within time slot t. This is the weighting factor for cost and AoI. The total cost of migrating the twin model within time slot t; Representative vehicle i Digital twin models in time slots t Storage capacity requirements; Representative vehicle i In the time slot t Geographical coordinates; The deployment state of the twin model representing the previous migration cycle (t−1) is denoted as: Forming transfer decisions for higher-level models; Step STP200, the lower-level learning phase, in the second cycle of the timescale. Next, according to the status Optimize data upload and resource allocation strategies; among them, The deployment status of the twin model for the current time slot t; For vehicles i In the time slot t Real-time location; For vehicles i Real-time data update request quadruple: in, Generate a timestamp for the request; For data size; To calculate the strength; To mitigate latency constraints, decisions are made regarding the uploading of data to lower layers. In the step-sharing phase, the spatiotemporal dependency matrix Z between agents is calculated through a multi-head attention mechanism, and selective parameter sharing is achieved based on Sinkhorn-Knopp normalization.
[0011] Preferably, the total cost of twin model migration within time slot t is... The calculation formula is: in, For binary decision variables, To migrate from server j to server The unit distance cost coefficient; Storage size of the twin model of vehicle i at time t, in GB; Migrate server j to server The number of hops for the shortest path.
[0012] Preferably, the average peak value The following calculation formula is used to obtain it: in, Generate a timestamp for the k-th data packet. The previous data packet generated a timestamp; Total end-to-end processing time; AoI is the peak value for a single event. The number of entities processed within time slot t.
[0013] 6. The optimization method according to claim 5, characterized in that the total end-to-end processing time... Including data transmission latency Edge network queuing delay and model update computation latency And their respective calculation methods are as follows: , , Migrate server j to server Shortest path hop count, For network links l The amount of data transmitted, in bits; For link l The transmission rate, measured in bps; λ represents the average queue length on the edge server; λ represents the average task arrival rate. For vehicles i The total number of CPU cycles required for an update request; Edge Server j Assigned to vehicles i The proportion of computing resources; For server j In time t Total computing power, measured in GHz.
[0014] Preferably, the objective function for the transfer decision of the upper-level model is specifically: in, , These are the weighting coefficients. For period t Worst peak AoI; vehicle i The twin model in time t Storage size, in GB; Migrate server j to server The number of hops for the shortest path.
[0015] Preferably, the lower-layer data upload decision includes: Communication link selection, vehicle i Twin model selection of edge servers j Or upload data to the cloud center (Cloud); Resource allocation constraints, satisfy or ; Latency guarantee, end-to-end latency ,in in, The amount of data transmitted via network link l, in bits; The transmission rate of link l, in bps; The total number of CPU cycles required for update requests for vehicle i; The proportion of computing resources allocated by edge server j to vehicle i; This represents the total computing power of server j at time t, in GHz.
[0016] Preferably, it further includes a step for ensuring convergence of the hierarchical learning at two time scales, specifically including: Hypergradient update mechanism, upper-level policy parameters Through the lower-level solution Implicit gradient update: in, f g is the upper-level objective function; g is the lower-level objective function. Let g be the Hessian matrix of the lower-level function with respect to g; Target network soft update, Critic network parameters φ Update according to weight τ; in, This is the soft update coefficient, with a value of 0.001. Convergence is easily controlled; when strong convexity, Lipschitz continuity, and bounded gradient variance are satisfied, the algorithm's convergence rate is... , where T is the total number of training iterations.
[0017] More preferably, it also includes a reward function for temporal coupling between the twin model migration and data upload, the reward function consisting of a lower-level reward function and an upper-level reward function, the lower-level reward function... The calculation formula is: in, Incremental penalty for AoI To calculate the delay cost, For transmission delay costs, , , These are the weighting coefficients; The upper-level reward function The calculation formula is: The total cost of migrating the twin model within time slot t; This represents the worst peak AoI within the period. , These are the weighting coefficients.
[0018] Beneficial effects: 1. This invention provides a system based on digital twin model transfer, which adopts a two-layer learning framework and includes a mechanism for selective parameter sharing among multiple agents based on spatiotemporal correlation. It captures the spatiotemporal dependencies between agents through a multi-head attention mechanism and shares parameters only among highly correlated agents, thereby accelerating model convergence and reducing communication overhead.
[0019] 2. This invention also provides a dual-timescale, dual-layer learning method that can adaptively optimize the twin model transfer and data upload strategy at different time scales, effectively reducing the freshness of AoI information in the system and minimizing model transfer costs.
[0020] 3. The multi-head attention mechanism provided by this invention is used to accurately capture the spatiotemporal correlation between agents. Based on this, parameter sharing is only performed between highly correlated agents to accelerate model convergence and effectively reduce communication overhead between agents.
[0021] 4. In this invention, the upper and lower layers are executed alternately at different time scales. The upper-layer strategy can be reused across multiple cycles, while the lower-layer strategy is dynamically optimized in real time based on the network. This mechanism achieves coordinated and unified dynamic migration of the future-oriented twin model and real-time information maintenance. Furthermore, this invention also sets up a two-layer reward function, which can effectively optimize long-term AoI and twin model migration costs. Attached Figure Description
[0022] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of dual-timescale, dual-layer learning based on the sharing of spatiotemporal dependent correlation parameters.
[0024] Figure 2 This is a schematic diagram of the base station distribution in the Shanghai Telecom base station dataset.
[0025] Figure 3 These are the reward and loss curves of the upper and lower layer networks, varying with the number of iterations across 100 user scenarios.
[0026] Figure 4 This is a schematic diagram of the reward and loss curves of the upper and lower layer networks as the number of iterations changes under different numbers of users.
[0027] Figure 5 This diagram illustrates the changes in system costs under different time periods, user numbers, and computing power conditions.
[0028] Figure 6 This diagram illustrates the changes in information freshness under different time slices, user numbers, and computing power conditions. Detailed Implementation
[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0030] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0031] Example 1: This embodiment provides a digital twin model migration system for vehicular edge computing. To better understand this system and focus on the technical problems solved by this embodiment, before describing this embodiment, we first discuss the technical bottlenecks of existing twin models in the field of vehicular edge computing applications. As is well known, vehicles are in motion during use, and their speed and location are uncertain, which determines the high mobility and randomness of their positions. The high mobility of vehicles may lead to frequent switching in the edge access network, thereby increasing the AoI of the twin model data, for example, when the vehicle moves out of the model maintenance area, i.e., the coverage area of the edge server. Therefore, the problem of digital twin model migration in vehicular edge networks has increasingly attracted the attention of academia and industry. This attention stems from the higher requirements placed on the management, synchronization, and migration of twin models in high-speed scenarios to ensure seamless operation and high-quality performance. In recent years, with the development of deep learning theory, especially the introduction of deep reinforcement learning, significant progress has been made in model migration strategies. Compared with traditional methods, deep learning has gradually become the mainstream technology because it can consider system interaction factors, adapt to dynamic changes, and is applicable to large-scale network environments.
[0032] However, current deep learning-based digital twin transfer methods primarily focus on processing high-dimensional, unstructured data, such as multimodal vehicle user data, automatic feature extraction (e.g., vehicle features, edge service features, and interaction features), and improving system adaptability and real-time decision-making capabilities. These methods often overlook the fundamental differences in optimization cycles between twin model transfer and data uploading, potentially leading to poor performance in terms of information freshness and cost-effectiveness. On one hand, twin model transfer involves significant data transmission and computational resource adjustments; frequent transfers in the short term incur high costs, such as data scheduling, network bandwidth, and energy consumption. On the other hand, vehicle-generated data is dynamic and real-time, requiring rapid uploading to edge servers for processing to maintain model accuracy and timeliness. Therefore, the multi-layered nested optimization problem caused by different optimization cycles, and the temporal coupling required for long-term AoI minimization, have become pressing challenges that need to be addressed.
[0033] Furthermore, in large-scale vehicular edge networks, although distributed learning methods can alleviate the high data aggregation costs and large transmission latency issues faced by centralized learning, many key challenges and limitations still exist. Specifically, significant differences exist among different agents in terms of resource capabilities, experience, and learning efficiency, leading to inconsistencies in training speed and model updates, thereby reducing the coordination and stability of the entire system. Sharing parameters among multiple agents can alleviate these inconsistencies to some extent; however, this approach also incurs high communication overhead. In addition, indiscriminate sharing of parameters may negatively impact the learning process. To promote collaboration among agents, capturing the correlations between them is beneficial. However, in VEC systems, these correlations exhibit complex and coupled spatiotemporal dependencies, mainly stemming from the diverse mobility patterns of vehicle users, such as different driving habits, traffic flow changes caused by various social activities, such as sporting events like football matches, and the diversity of road environments, such as accident-prone areas. Therefore, how to effectively achieve parameter sharing to improve the collaborative efficiency of multiple agents and the overall system performance while reducing parameter sharing costs has become another critical issue that urgently needs to be addressed. To address the aforementioned technical challenges, this embodiment proposes a digital twin model transfer system for vehicle-mounted edge computing, based on a distributed dual-timescale, dual-layer learning framework. In this framework, the upper layer optimizes the digital twin model transfer strategy on a larger timescale, while the lower layer optimizes data upload and resource allocation strategies on a smaller timescale, thereby achieving forward-looking twin model transfer and real-time information maintenance. Specifically, it includes: The physical entity layer includes vehicles, roadside units, and sensor networks used to generate real-time environmental data; it provides the necessary real-time data and status information for the digital twin layer.
[0034] The digital twin layer, deployed on edge servers, performs the creation, updating, and migration of vehicle digital twin models; by adopting a distributed architecture, it can adapt to large-scale vehicle networks, support low-latency data processing, and has good system scalability. The cloud center provides global computing and storage resources for large-scale data analysis, advanced simulation computing, and long-term data storage. The dual-timescale optimization module includes: Upper-level learning units, in the first cycle on a long timescale Internally, reinforcement learning is used to optimize the transfer decisions of the digital twin model and generate deployment decision variables. , indicating vehicle i The twin model in the time slot t Deployed on an edge server j ; The next learning unit is shorter than the first cycle. The second cycle Internally, reinforcement learning is used to jointly optimize the selection of data upload paths and the allocation of resources. and computational task scheduling; the first cycle For the second cycle multiples of, and > ; The multi-agent collaborative module selectively shares parameters between any two adjacent edge servers based on spatiotemporal correlation; the output of the upper-layer decision serves as the constraint condition for the lower-layer decision, and the real-time state feedback of the lower-layer decision is used to update the upper-layer policy. The objective function is to minimize the sum of the system's long-term average peak AoI and the model migration cost, expressed as: in, This represents the total number of time slots. Let t be the average peak information age of the system within time slot t. This is the weighting factor for cost and AoI. Let be the total transfer cost of the twin model within time slot t. A two-layer learning framework is adopted, which includes a mechanism for selective parameter sharing among multiple agents based on spatiotemporal correlation. By capturing the spatiotemporal dependencies between agents through a multi-head attention mechanism, parameters are shared only among highly correlated agents, thereby accelerating model convergence and reducing communication overhead.
[0035] To address the high computational complexity of sequence-to-sequence prediction models, such as RNNs and LSTMs, during iterative processing, this invention employs a multi-head attention mechanism with parallel computing capabilities to extract spatiotemporal dependencies between agents. Simultaneously, to address the issue that existing multi-head self-attention mechanisms' positional encoding methods fail to explicitly retain the timestamp information of each element, potentially leading to the loss of temporal dimension information, we improve the encoding method by combining positional encoding with time-step encoding, thereby more effectively preserving temporal dimension information. In this embodiment, the multi-agent collaborative module includes: The spatiotemporal feature encoding module is used to extract dynamic features of each edge server agent, including temporal features, spatial features, and interaction features. The temporal features include the deployment status of the twin model and real-time traffic load. The spatial features include geographical location coordinates and network topology connections. The interaction features include historical model migration records and vehicle movement trajectories. The multidimensional edge information of each agent is feature-encoded, and the above features help to comprehensively reflect the spatiotemporal correlation between agents.
[0036] The multi-head attention dependency analysis module consists of multiple stacked encoders and decoders. The encoder output is used to assist in the training of the decoder. Each encoder sequentially executes a self-attention module and a feed-forward neural network module, employing residual connections and layer normalization operations. The encoder's role is to learn the dependencies between agents in the spatiotemporal dimension, capture key spatiotemporal patterns, and calculate the spatiotemporal dependency matrix Z between agents through a multi-head self-attention mechanism, where elements... , representing an intelligent agent m and n The strength of the dependency is expressed as: Where Q, K, and V represent the query vector, key vector, and value vector, respectively, and the parameters... Let Q and K represent the dimensions of vectors Q and K; This embodiment utilizes a multi-head self-attention mechanism. Similarity scores can be scaled, which helps generate smoother attention weights and prevents gradient vanishing when processing long sequences. A multi-head self-attention mechanism further enhances this strategy, allowing the model to focus on different parts of the input simultaneously, thus capturing diverse spatiotemporal correlations. Each attention head processes the data independently, and their outputs are then concatenated to form a comprehensive representation of dependencies. To further improve the modeling ability for dynamic interactions and collaborative relationships, the dependency matrix Z is normalized and then input into the feedforward neural network module. In this embodiment, the feedforward neural network module adopts the following structure: it contains two fully connected layers, the first layer using the ReLU activation function and the second layer using a linear activation function. The processing procedure is as follows: · in, and This is the weight matrix. and As a bias term, through a multi-head attention mechanism, this framework can effectively extract complex spatiotemporal dependency features, thereby improving the ability to model dynamic interactions and collaborative relationships.
[0037] Selective parameter sharing module, only when > At that time, intelligent agent m and n Shared parameters. To facilitate understanding of the mechanism of selective parameter sharing, this embodiment will further elaborate on the process of selective parameter sharing from a vector perspective.
[0038] First, the spatiotemporal dependent feature vector output by the encoder above Mapped to a symmetric matrix ,in, Represents intelligent agents j and The spatiotemporal dependency strength between them, and satisfying symmetry. , Indicates the first j Okay, number The linear index of the matrix elements in the column within the vector.
[0039] when When the value is negative, it indicates that the agent... j and The correlation between them is low, so we set their weights to 0. Our goal is to selectively share parameters only among highly correlated agents to accelerate model convergence and reduce communication overhead. Next, we normalize each row and column of the matrix using the Sinkhorn-Knopp method. This method is an iterative algorithm that transforms any non-negative matrix into a doubly random matrix, where the sum of the elements in each row and column is 1, thus satisfying the interpretation requirements of probability weights in parameter sharing while maintaining the relative correlation between agents. The normalization process is shown below: Here, 1 represents an all-one vector, and diag(·) denotes converting the vector into a diagonal matrix. After iteration, the matrix converges into a set of interpretable probability weights, used to guide selective parameter sharing among agents. The detailed process of this method is described in Algorithm 1. Through this mechanism, each agent, when updating its parameters, not only refers to its own gradient information but also incorporates parameter information from other highly correlated agents. This approach not only improves the diversity of training data and distributes exploration costs but also strengthens the collaborative ability among agents. The dual-timescale, dual-layer learning algorithm based on spatiotemporally dependent parameter sharing is as follows: Algorithm 1: A dual-timescale, dual-layer learning algorithm based on sharing spatiotemporal dependent correlation parameters .
[0040] Example 2: The present invention also provides an optimization method for performing dual-time hierarchical learning scale optimization on the digital twin model transfer system for vehicle edge computing provided in Example 1, comprising the following steps: Step STP100, the upper-level learning phase, in the first cycle of the timescale. Next, according to the status Optimize the twin model transfer strategy to minimize the sum of the long-term average peak AoI and the transfer cost: in, This represents the total number of time slots. The average peak value of the system within time slot t. This is the weighting factor for cost and AoI. The total cost of migrating the twin model within time slot t; Representative vehicle i Digital twin models in time slots t Storage capacity requirements; Representative vehicle i In the time slot t Geographical coordinates; The deployment state of the twin model representing the previous migration cycle (t−1) is denoted as: Forming transfer decisions for higher-level models; Step STP200, the lower-level learning phase, in the second cycle of the timescale. Next, according to the status Optimize data upload and resource allocation strategies; among them, The deployment status of the twin model for the current time slot t; For vehicles i In the time slot t Real-time location; For vehicles i Real-time data update request quadruple: in, Generate a timestamp for the request; For data size; To calculate the strength; To mitigate latency constraints, decisions are made regarding the uploading of data to lower layers. In the step-sharing phase, the spatiotemporal dependency matrix Z between agents is calculated through a multi-head attention mechanism, and selective parameter sharing is achieved based on Sinkhorn-Knopp normalization.
[0041] In this embodiment, the total cost of twin model migration within time slot t The calculation formula is: in, For binary decision variables, To migrate from server j to server The unit distance cost coefficient; Storage size of the twin model of vehicle i at time t, in GB; Migrate server j to server The number of hops for the shortest path.
[0042] In this embodiment, the average peak value The following calculation formula is used to obtain it: in, Generate a timestamp for the k-th data packet. The previous data packet generated a timestamp; Total end-to-end processing time; AoI is the peak value for a single event. The number of entities processed within time slot t.
[0043] 6. The optimization method according to claim 5, characterized in that the total end-to-end processing time... Including data transmission latency Edge network queuing delay and model update computation latency And their respective calculation methods are as follows: , , Migrate server j to server Shortest path hop count, For network links l The amount of data transmitted, in bits; For link l The transmission rate, measured in bps; λ represents the average queue length on the edge server; λ represents the average task arrival rate. For vehicles i The total number of CPU cycles required for an update request; Edge Server j Assigned to vehicles i The proportion of computing resources; For server j In time t Total computing power, measured in GHz.
[0044] In this embodiment, the optimization objective function for the upper-level model transfer decision is specifically: in, , These are the weighting coefficients. For period t Worst peak AoI; vehicle i The twin model in timet Storage size, in GB; Migrate server j to server The number of hops for the shortest path.
[0045] In this embodiment, the lower-layer data upload decision includes: Communication link selection, vehicle i Twin model selection of edge servers j Or upload data to the cloud center (Cloud); Resource allocation constraints, satisfy or ; Latency guarantee, end-to-end latency ,in in, The amount of data transmitted via network link l, in bits; The transmission rate of link l, in bps; The total number of CPU cycles required for update requests for vehicle i; The proportion of computing resources allocated by edge server j to vehicle i; This represents the total computing power of server j at time t, in GHz.
[0046] This embodiment also includes a step for ensuring convergence in dual-timescale hierarchical learning, specifically including: Hypergradient update mechanism, upper-level policy parameters Through the lower-level solution Implicit gradient update: in, f g is the upper-level objective function; g is the lower-level objective function. Let g be the Hessian matrix of the lower-level function with respect to g; Target network soft update, Critic network parameters φ Update according to weight τ; in, This is the soft update coefficient, with a value of 0.001. Convergence is easily controlled; when strong convexity, Lipschitz continuity, and bounded gradient variance are satisfied, the algorithm's convergence rate is... , where T is the total number of training iterations.
[0047] In this embodiment, a reward function is also included for temporal coupling between the twin model migration and data upload. This reward function consists of a lower-level reward function and an upper-level reward function. The calculation formula is: in, Incremental penalty for AoI To calculate the delay cost, For transmission delay costs, , , These are the weighting coefficients; The upper-level reward function The calculation formula is: The total cost of migrating the twin model within time slot t; This represents the worst peak AoI within the period. , These are the weighting coefficients.
[0048] Example 3: This embodiment evaluates the system algorithm performance using the system from Embodiment 1 and the algorithm from Embodiment 2 on a Shanghai Telecom base station dataset and a taxi trajectory dataset. The details are as follows: Part 1: Experiment Setup This embodiment uses a Shanghai Telecom base station dataset of over 7.2 million records that accurately reflects the network environment and vehicle mobility patterns. The dataset records the internet access behavior of 9,481 users through 3,233 base stations over a six-month period. The distribution of Shanghai base stations is as follows: Figure 2 As shown. To ensure fairness in the experiment, we selected areas with high user activity ( Figure 4 The area marked in green in (a) is... Figure 4(b) shows a magnified view of a randomly selected 4km × 3km area. The edge server's computing power is based on 10GHz, the base station bandwidth is 10MHz, and the storage capacity is 35GB. The number of vehicles varies between 40 and 100. The size of the digital twin models of the vehicles is uniformly distributed within the range of [0.5, 1]GB. In the two-layer learning architecture, both the upper-layer Actor and Critic contain 3 hidden layers with 1024, 1024, and 512 units respectively, a learning rate of 1e-4, and a batch size of 32. The lower-layer Actor contains 4 hidden layers with 1024, 1024, 512, and 256 units respectively, while the lower-layer Critic has 3 hidden layers with 1024, 1024, and 512 units respectively. The learning rate for both the lower-layer Actor and Critic is 1e-4, and the batch size is 256. Both the upper and lower layers use the Adam optimizer.
[0049] In the multi-head attention mechanism, both the encoder and decoder are two-layer neural network structures. All neural network architectures are implemented using PyTorch, and the experimental parameters are detailed in Table 1 below: Table 1 Experimental Simulation Parameters Part Two: Benchmark Comparison Methods We evaluate our proposed algorithm against several state-of-the-art methods. Note that for fair comparison, we extend all algorithms to run on two time scales. **Collaborative Learning Based on Reward Contribution (CLRC):** CLRC estimates the contribution of each agent to the global reward by calculating an individual baseline for each agent and promotes collaboration among agents based on their respective reward contributions. **Federated Collaborative Model Transfer (FCMM):** FCMM promotes multi-agent collaborative learning through intra-cluster parameter sharing and global asynchronous parameter aggregation. **Collaborative Learning Based on Geographic Proximity (CLGP):** CLGP measures the relevance between agents based on geographic distance and promotes collaborative learning among geographically proximate agents. **Iterative Transfer Algorithm Based on Competition Ratio (IMCR):** Inspired by the literature, IMCR performs model transfer by comparing the difference between the current time slot's AoI and the historical AoI. It should be noted that IMCR is a comparison-based iterative method, not a learning-based method.
[0050] Part Three: Experimental Results 3.1 Rewards and Punishments: Figure 3The reward and loss behavior of the upper and lower layers during training is illustrated. We observe some differences in the reward convergence process among different algorithms, mainly attributed to the random initialization of network parameters, the randomness of the exploration strategy, and the uncertainty introduced by stochastic gradient descent during parameter learning. To mitigate the impact of randomness, we conducted multiple experiments, with shaded areas representing the result intervals. As the number of training iterations increases, our algorithm converges to the highest reward value in both the upper and lower layers, with the upper layer reaching the optimal reward at approximately 160 iterations. This phenomenon demonstrates that the dual-timescale dual-layer optimization mechanism and the parameter sharing mechanism assisted by spatiotemporal dependency can significantly improve system performance. Our proposed algorithm integrates a selective parameter sharing mechanism within the dual-timescale optimization framework, achieving not only faster convergence speed and higher system rewards but also significantly enhancing multi-agent collaborative efficiency and overall system performance.
[0051] We also evaluated the training reward and loss of the algorithm under different user numbers, such as Figure 4 As shown, the system reward decreases with the increase in the number of users. This is mainly due to: intensified resource competition, more complex interaction relationships, and the higher system overhead introduced by new users. Furthermore, the number of iterations required for algorithm convergence also increases with the number of users. This is because more users mean higher system complexity and decision-making dimensions, requiring more rounds of training for the algorithm to learn the optimal strategy. At the same time, the loss during training exhibits some volatility, reflecting that the system needs to adapt to more complex interaction relationships and a higher-dimensional decision space when facing multi-user collaborative optimization problems. Figure 3 and Figure 4 The results validate the robustness and adaptability of our algorithm in handling highly dynamic, multi-user scenarios.
[0052] 3.2 System overhead: Figure 5 The system overhead is illustrated under different time slots, different numbers of users, and different computing capabilities. System overhead includes system AoI (AoI) and twin model migration overhead. For example... Figure 5As shown in (a), the total system overhead fluctuates across different time slots, primarily due to the dynamic changes in traffic demand and vehicle movement. This algorithm demonstrates the best performance in terms of system overhead, with average system costs reduced compared to other algorithms: 20.43% lower than CLRC, 14.58% lower than FCMM, 27.86% lower than CLGP, and 31.75% lower than IMCR. This significant improvement is mainly attributed to the proposed two-layer learning architecture: data upload decision optimization at short timescales ensures real-time data upload and processing, enabling the system to quickly respond to environmental changes and maintain data freshness; while model transfer optimization at long timescales effectively reduces the high costs associated with frequent twin model transfers, thereby reducing overall system overhead. Furthermore, by introducing a selective parameter sharing mechanism, this algorithm also excels in improving multi-agent collaborative efficiency and optimizing global resource utilization, further reducing system operating costs.
[0053] 3.3 Age of Information (AoI): Age of Information (AoI) reflects the real-time nature and freshness of information in the system. This indicator is crucial for latency-sensitive vehicle-to-everything (V2X) edge computing systems. Figure 6 The system's AoI performance is demonstrated under different time slots, user numbers, and computing capabilities. Compared with CLRC, FCMM, CLGP, and IMCR methods, the proposed algorithm reduces AoI by an average of 31.39%, 21.30%, 40.45%, and 44.01%, respectively. This significant improvement is attributed to the lower-level optimization mechanism at short time scales, ensuring real-time data uploads and timely system status updates, enabling the system to quickly acquire the latest information and reduce information lag. This mechanism ensures that the digital twin model accurately reflects the latest state of the vehicle and its environment within each time slot. Furthermore, by modeling the spatiotemporal dependencies between edge nodes, more efficient data and resource sharing can be achieved, thereby reducing redundant processing and transmission, improving information update efficiency, and further reducing the system's AoI.
[0054] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A digital twin model transfer system for vehicle-mounted edge computing, characterized in that, include: The physical entity layer includes vehicles, roadside units, and sensor networks used to generate real-time environmental data; The digital twin layer, deployed on edge servers, performs the creation, updating, and migration of vehicle digital twin models; The cloud center provides global computing and storage resources; The dual-timescale optimization module includes: Upper-level learning units, in the first cycle on a long timescale Internally, reinforcement learning is used to optimize the transfer decisions of the digital twin model and generate deployment decision variables. , indicating vehicle i The twin model in the time slot t Deployed on an edge server j ; The next learning unit is shorter than the first cycle. The second cycle Internally, reinforcement learning is used to jointly optimize the selection of data upload paths and the allocation of resources. and computational task scheduling; the first cycle For the second cycle multiples of, and > ; The multi-agent collaborative module selectively shares parameters between any two adjacent edge servers based on spatiotemporal correlation; the output of the upper-layer decision serves as the constraint condition for the lower-layer decision, and the real-time state feedback of the lower-layer decision is used to update the upper-layer policy. The objective function is to minimize the sum of the system's long-term average peak AoI and the model migration cost, expressed as: in, This represents the total number of time slots. Let t be the average peak information age of the system within time slot t. This is the weighting factor for cost and AoI. Let t be the total cost of migrating the twin model within time slot t.
2. The digital twin model transfer system for vehicle edge computing according to claim 1, characterized in that, The multi-agent collaborative module includes: The spatiotemporal feature encoding module is used to extract dynamic features of each edge server agent, including temporal features, spatial features, and interaction features. The temporal features include the deployment status of the twin model and real-time traffic load. The spatial features include geographic location coordinates and network topology connections. The interaction features include historical model migration records and vehicle movement trajectories. The multi-head attention dependency analysis module calculates the spatiotemporal dependency matrix Z between agents through a multi-head self-attention mechanism, where the elements... , representing an intelligent agent m and n The strength of the dependency is expressed as: Where Q, K, and V represent the query vector, key vector, and value vector, respectively, and the parameters... Let Q and K represent the dimensions of vectors Q and K; Selective parameter sharing module, only when > At that time, intelligent agent m and n Shared parameters.
3. An optimization method for performing dual-time hierarchical learning scale optimization on the digital twin model transfer system for vehicle-mounted edge computing as described in claim 1, characterized in that, Includes the following steps: Step STP100, the upper-level learning phase, in the first cycle of the timescale. Next, according to the status Optimize the twin model transfer strategy to minimize the sum of the long-term average peak AoI and the transfer cost: in, This represents the total number of time slots. The average peak value of the system within time slot t. This is the weighting factor for cost and AoI. The total cost of migrating the twin model within time slot t; Representative vehicle i Digital twin models in time slots t Storage capacity requirements; Representative vehicle i In the time slot t Geographical coordinates; The deployment state of the twin model representing the previous migration cycle (t−1) is denoted as: Forming transfer decisions for higher-level models; Step STP200, the lower-level learning phase, in the second cycle of the timescale. Next, according to the status Optimize data upload and resource allocation strategies; among them, The deployment status of the twin model for the current time slot t; For vehicles i In the time slot t Real-time location; For vehicles i Real-time data update request quadruple: in, Generate a timestamp for the request; For data size; To calculate the strength; To mitigate latency constraints, decisions are made regarding the uploading of data to lower layers. In the step-sharing phase, the spatiotemporal dependency matrix Z between agents is calculated through a multi-head attention mechanism, and selective parameter sharing is achieved based on Sinkhorn-Knopp normalization.
4. The optimization method according to claim 3, characterized in that, Total cost of twin model migration within time slot t The calculation formula is: in, For binary decision variables, To migrate from server j to server The unit distance cost coefficient; Storage size of the twin model of vehicle i at time t, in GB; Migrate server j to server The number of hops for the shortest path.
5. The optimization method according to claim 3, characterized in that, The average peak value The following calculation formula is used to obtain it: in, Generate a timestamp for the k-th data packet. The previous data packet generated a timestamp; Total end-to-end processing time; AoI is the peak value for a single event. The number of entities processed within time slot t.
6. The optimization method according to claim 5, characterized in that, The total end-to-end processing time Including data transmission latency Edge network queuing delay and model update computation latency And their respective calculation methods are as follows: , , Migrate server j to server Shortest path hop count, For network links l The amount of data transmitted, in bits; For link l The transmission rate, measured in bps; This represents the average queue length on the edge server. λ represents the average arrival rate of the task; For vehicles i The total number of CPU cycles required for an update request; Edge Server j Assigned to vehicles i The proportion of computing resources; For server j In time t Total computing power, measured in GHz.
7. The optimization method according to claim 3, characterized in that, The specific objective function for the transfer decision of the upper-level model is as follows: in, , These are the weighting coefficients. For period t Worst peak AoI; vehicle i The twin model in time t Storage size, in GB; Migrate server j to server The number of hops for the shortest path.
8. The optimization method according to claim 3, characterized in that, The lower-level data upload decision includes: Communication link selection, vehicle i Twin model selection of edge servers j Or upload data to the cloud center (Cloud); Resource allocation constraints, satisfy or ; Latency guarantee, end-to-end latency ,in in, The amount of data transmitted via network link l, in bits; The transmission rate of link l, in bps; The total number of CPU cycles required for update requests for vehicle i; The proportion of computing resources allocated by edge server j to vehicle i; This represents the total computing power of server j at time t, in GHz.
9. The optimization method according to claim 3, characterized in that, It also includes steps for convergence guarantees in dual-timescale hierarchical learning, specifically including: Hypergradient update mechanism, upper-level policy parameters Through the lower-level solution Implicit gradient update: in, f g is the upper-level objective function; g is the lower-level objective function. Let g be the Hessian matrix of the lower-level function with respect to g; Target network soft update, Critic network parameters φ Update according to weight τ; in, This is the soft update coefficient, with a value of 0.
001. Convergence is easily controlled; when strong convexity, Lipschitz continuity, and bounded gradient variance are satisfied, the algorithm's convergence rate is... , where T is the total number of training iterations.
10. The optimization method according to claim 3, characterized in that, It also includes a reward function for temporal coupling between twin model migration and data upload, the reward function consisting of a lower-level reward function and an upper-level reward function, the lower-level reward function... The calculation formula is: in, Incremental penalty for AoI To calculate the delay cost, For transmission delay costs, , , These are the weighting coefficients; The upper-level reward function The calculation formula is: The total cost of migrating the twin model within time slot t; This represents the worst peak AoI within the period. , These are the weighting coefficients.
