Collaborative vehicle-mounted edge computing task unloading method driven by digital twin technology

Abstract

The invention discloses a cooperative vehicle-mounted edge computing task unloading method driven by a digital twinning technology, which belongs to the field of vehicle-mounted edge computing and comprises the following steps of: realizing global state perception and prediction by constructing real-time digital twinning virtual mapping of a physical network; and a multi-agent depth deterministic strategy gradient algorithm fused with federated learning is designed to collaboratively optimize the computing resources of the fixed edge server and the dynamic opportunity vehicle. According to the method, on the premise of protecting user privacy, the method can be adaptive to a dynamic environment, intelligent generation of a task unloading decision and elastic scheduling of resources are realized, so that the task completion rate is remarkably improved, and meanwhile, the average time delay and the overall energy consumption of the system are effectively reduced.

Description

Technical Field

[0001] This invention belongs to the field of vehicle-mounted edge computing, and in particular relates to a collaborative vehicle-mounted edge computing task offloading method driven by digital twin technology. Background Technology

[0002] Currently, vehicular edge computing (VEC) effectively alleviates the local resource pressure on intelligent vehicles running computationally intensive and latency-sensitive applications by pushing computing power down to the network edge (such as roadside units). To further optimize resource scheduling in highly dynamic vehicle-to-everything (V2X) environments, digital twin (DT) technology has been introduced to construct a real-time virtual mirror of the physical network, enabling global state perception and prediction beyond line-of-sight. Meanwhile, reinforcement learning algorithms such as Multi-Agent Deep Deterministic Policy Gradient (MADDPG) are being used to solve multi-vehicle collaborative decision-making problems, while federated learning (FL) mechanisms are being explored to protect data privacy in distributed training.

[0003] However, existing technical solutions still face significant challenges. In highly dynamic VEC scenarios, how to deeply integrate DT's real-time perception and prediction capabilities, MADDPG's multi-agent collaborative decision-making capabilities, and FL's privacy protection features to form an efficient, adaptive, and scalable collaborative task offloading framework remains a complex problem. Specifically, traditional MADDPG suffers from slow convergence and decision lag in non-stationary environments, and its centralized training mode is prone to privacy leaks and communication overhead. Furthermore, existing methods are inefficient in collaboratively utilizing heterogeneous computing resource pools composed of fixed edge servers and dynamic opportunistic vehicles, making it difficult to systematically reduce latency and energy consumption while ensuring task completion rates. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a collaborative vehicle-mounted edge computing task offloading method driven by digital twin technology, comprising: Based on the real-time status data of the physical vehicle edge computing network, a virtual mapping network corresponding to the real-time status data is constructed and maintained; The global state information of the physical vehicle edge computing network is obtained based on the virtual mapping network; Based on the virtual mapping network and the global state information, a cooperative task offloading optimization algorithm is executed. The cooperative task offloading optimization algorithm integrates multi-agent reinforcement learning and federated learning mechanisms to generate task offloading decisions for in-vehicle computing tasks. Based on the task offloading decision, computing resources in the physical vehicle edge computing network are scheduled to execute the vehicle computing task.

[0005] Optionally, the process of constructing and maintaining a virtual mapping network corresponding to the real-time status data includes: constructing and maintaining a virtual mapping network containing physical vehicle entities, physical edge server entities, and dynamic relationships between the entities, based on the real-time status data of vehicles and servers in the physical vehicle edge computing network.

[0006] Optionally, the global state information of the physical vehicle edge computing network is obtained based on the virtual mapping network, specifically including: Real-time status monitoring of the physical vehicle-mounted edge computing network is performed based on the virtual mapping network. Based on the monitored real-time status information, the future state of the network is predicted to obtain global status information containing the prediction information.

[0007] Optionally, before executing the cooperative task unloading optimization algorithm, the method further includes: Based on the characteristics of in-vehicle computing tasks, network resource constraints, and quality of service requirements, the unloading and resource allocation process of the in-vehicle computing tasks is modeled as a Markov decision process.

[0008] Optionally, the execution of the collaborative task unloading optimization algorithm specifically includes: A reinforcement learning model based on the multi-agent deep deterministic policy gradient (MADDPG) framework is constructed to solve the Markov decision process.

[0009] Optionally, the construction of the reinforcement learning model based on the multi-agent deep deterministic policy gradient (MADDPG) framework specifically includes: Each user vehicle and / or each computing service provider entity in a physical vehicular edge computing network is defined as an intelligent agent.

[0010] Optionally, the state of the agent in the time slot is determined based on the global state information, which includes: agent location information, computing resource information of associated vehicles and servers, and state information of the task to be unloaded; The agent's actions in a time slot are determined based on the policy network and the current state. The actions include: making decisions on whether to execute the subtask to be unloaded locally, unload it to an edge server, or unload it to an opportunistic vehicle. The agent updates the policy network based on the reward from the environment after performing an action and the next state it transitions to. The reward is calculated based on the total system cost and the constraint satisfaction.

[0011] Optionally, the collaborative task offloading optimization algorithm incorporates a federated learning mechanism, specifically including: Each agent updates the policy network parameters locally using its own experience data; According to a preset period, the policy network parameters of each agent are aggregated to update the global model parameters; Each agent synchronizes or updates its local policy network based on the global model parameters.

[0012] On the other hand, the present invention also provides an electronic device including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0013] On the other hand, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0014] Compared with the prior art, the present invention has the following advantages and technical effects: This invention effectively integrates global state awareness, multi-agent collaborative decision-making, and privacy protection mechanisms by constructing a digital twin-driven collaborative task offloading framework. Utilizing the real-time, predictive network view provided by DT (Digital Transmission Technology), and combining it with a federated learning-enhanced multi-agent deep deterministic policy gradient (FL-MADDPG) algorithm, it achieves dynamic and collaborative optimization allocation of resources between fixed edge servers and opportunistic vehicles. This method can adapt to the highly dynamic characteristics of vehicular networks, significantly accelerate the convergence speed of policies during the learning phase, and intelligently make offloading decisions while protecting the privacy of local vehicle data. Ultimately, the system can significantly reduce the average latency of task execution and the total system energy consumption while maximizing task completion rate, thereby comprehensively improving the overall performance and service quality of the vehicular edge computing system. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the communication framework of the DT-assisted VEC network according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the DT-driven collaborative VEC computation offloading framework according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the calculation and unloading process according to an embodiment of the present invention; Figure 4 This is a schematic diagram of convergence analysis in an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the impact of the number of user vehicles on the average response latency according to an embodiment of the present invention; Figure 6 This is a schematic diagram illustrating the impact of the number of subtasks on the average response latency in an embodiment of the present invention. Figure 7 This is a schematic diagram illustrating the impact of RSU computing power on average response latency in an embodiment of the present invention. Figure 8 This is a schematic diagram illustrating the task completion rate of the algorithm proposed in this embodiment of the invention; Figure 9 This is a schematic diagram illustrating the task completion rate of F-DQN according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the task completion rate of DDPG according to an embodiment of the present invention. Detailed Implementation

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0018] Example 1 This embodiment provides a collaborative vehicle-mounted edge computing task offloading method driven by digital twin technology, including: Based on the real-time status data of the physical vehicle edge computing network, a virtual mapping network corresponding to the real-time status data is constructed and maintained; The global state information of the physical vehicle edge computing network is obtained based on the virtual mapping network; Based on the virtual mapping network and the global state information, a cooperative task offloading optimization algorithm is executed. The cooperative task offloading optimization algorithm integrates multi-agent reinforcement learning and federated learning mechanisms to generate task offloading decisions for in-vehicle computing tasks. Based on the task offloading decision, computing resources in the physical vehicle edge computing network are scheduled to execute the vehicle computing task.

[0019] Digital twin (DT) technology offers a new perspective for intelligent network management. As a digital mirror of physical entities, DT enables real-time data interaction between the physical world and virtual space, allowing for full lifecycle monitoring and prediction of vehicle status, network topology, and resource distribution. It enables beyond-line-of-sight perception and collaborative decision-making, laying a data foundation for mitigating computational latency and service discontinuity. For example... Figure 1As shown, in the DT-assisted VEC architecture, two vehicles beyond the V2V communication range can exchange information and coordinate through an efficient digital link, overcoming the limitations of distance and stability in direct communication between physical vehicles. The real-time status of the physical vehicles is synchronized to the virtual space. DT not only breaks the limitations of the physical field of view and provides a global network view, but also assists vehicles in making optimal unloading decisions in complex hybrid computing environments through advanced simulation in the virtual space, thereby overcoming the decision-making bias caused by information lag in traditional methods.

[0020] In this embodiment, the DT-driven collaborative VEC computation offloading framework is as follows: Figure 2 As shown, this embodiment fully utilizes opportunity vehicles with high-performance computing capabilities but idle resources to provide computational offloading services in collaboration with ES (Elastic Compute Service). It is mainly divided into a vehicle layer, an edge computing layer, and a DT (Data Technology) layer. The vehicle layer primarily refers to intelligent vehicles equipped with sensing, communication, caching, and computing devices, consisting of user vehicles and opportunity vehicles. These vehicles generate various computationally intensive and latency-sensitive onboard computing tasks. In this embodiment, opportunity vehicles are referred to as Opportunity Edge Servers (OES). Due to differences in computing capabilities and task processing requirements among different vehicles, some vehicles, lacking sufficient onboard computing resources, need to offload their onboard computing tasks to the OES or ES for execution via V2V or V2R communication technologies. Definition As a user vehicle with computational unloading requirements, defined As an opportunity vehicle. Assume each user vehicle generates a computational task, which can be divided into... Sub-task, user car The Each subtask is represented as Subtask ,in, Indicates the size of the data. Indicates the number of CPU cycles required. This represents the maximum tolerable delay. Furthermore, this embodiment defines the task offloading decision as a three-dimensional matrix. Each element Subtasks The uninstallation decision, among which, .when When, execute locally; when At that time, execution occurs on either the OES or ES. The edge computing layer consists of multiple RSUs equipped with ES and OES. Each RSU not only has the function of aggregating parameters but also has computing power. Because the distance between powerful RSUs is relatively short, vehicle users can offload tasks to nearby RSUs, thereby meeting low-latency requirements. The set of RSUs is defined as follows: The DT network is deployed on a cloud server. The cloud server can use the DT network to simulate the VEC network topology, monitor the network model in real time, obtain the status of physical entities, and generate real-time network environment information. RSUs and vehicles periodically transmit information to the cloud server. The cloud server with the DT network can provide RSUs and vehicles with their respective status information, assisting in dynamic association between RSUs and vehicles. Based on the collected information, it can further assist in predicting task offloading strategies, avoiding frequent information exchanges between physical entities in the physical network. Furthermore, FL builds a DT model based on physical state operational data. The VEC computation offloading technology allows laggards to offload some computational tasks to trusted ESs and utilize the server's idle computing resources to assist vehicles in model training, thereby mitigating the lagging effect in FL.

[0021] let This represents a dynamic set of associations between ES (Elasticsearch) and vehicles. If a vehicle is connected to ES, it can be recorded in the set. In DT, a virtual object can be represented as: (1) Among them, in time slots , For user vehicles The task set For user vehicles The calculation frequency, This represents the computational frequency of ES. DT utilizes FL to construct the corresponding global model. It continuously interacts with the actual network to maintain consistency.

[0022] Given the high dynamism of VEC networks, maintaining the stability of communication links between vehicles is crucial. Furthermore, not all vehicles on the road are willing to share resources to assist other vehicles in completing onboard computing tasks. Some studies have addressed vehicle mobility issues through clustering methods. As a promising architecture that aggregates multi-scale factors in VEC networks, social networks can extract raw data from vehicles to construct social relationships and further address the challenges of vehicle mobility. In this embodiment, we focus on using an opportunistic vehicle relationship model to cluster vehicles, describing the stability of link connections between vehicles and further improving the efficiency of V2V computation offloading. The social trust value between vehicles is measured by two main indicators: directional similarity and speed similarity. Therefore, speed and directional similarity are used to describe the relationship between available vehicles and task vehicles. (2) in, and These are parameters that are adjusted, representing the weights of directional similarity and velocity similarity, respectively. . The similarity of directions between vehicles is indicated primarily by checking whether the directions of the two vehicles are the same. If the user's vehicle and Their driving directions are similar. ,otherwise, . Speed ​​similarity is indicated primarily by checking whether the speeds of two vehicles are the same, and then setting... . The larger the value, the more likely the vehicle is to be damaged. and vehicles The higher the level of trust between them.

[0023] Each user vehicle has three computation offloading decisions to complete its onboard computing tasks: execution locally, execution on ES, and execution on OES.

[0024] 1) Local computing model: This means the user's vehicle The first Subtask Calculations are performed locally on the vehicle. In this case, task processing latency and energy consumption can be expressed as: (3) (4) in, Indicates user vehicle The computing power. The energy consumption of local execution is proportional to the square of the CPU voltage, and this is observed in the CPU clock frequency. It has a roughly linear relationship with the voltage supply. Therefore, the energy consumption of one CPU cycle can be expressed as: ,in The value depends on the vehicle CPU architecture.

[0025] 2) ES offloading model: This means that when the vehicle's computing resources are insufficient to meet the task requirements, ES assists in computing subtasks. The calculation results will be returned to the user's vehicle. Since the computational results of a task are typically small, the transmission time of the results can be ignored. In this case, the subtask... The processing latency for unloading to Elasticsearch mainly includes upload task latency and execution latency, expressed as follows: (5) in, Indicates ES computing power Subtasks Shared ES The proportion of computing resources, Indicates user vehicle and ES The uplink transmission rate is expressed as: (6) in, Indicates channel bandwidth. Indicates noise power. Indicates vehicle Unload tasks to ES Transmission power at that time Indicates the use of user vehicles Interference caused by other vehicles on the same channel. This represents the path loss index. It is a constant coefficient. Considering path loss and random effects in actual communication transmission, Indicates user vehicle and ES The small-scale fading channel power gain between these parameters means that communication channels are typically modeled as frequency-flat fading channels, i.e. .

[0026] Furthermore, the energy consumption for executing tasks on Elasticsearch (ES) primarily comprises two parts: transmission and computation. Therefore, on ES... Complete the sub-tasks The total energy required is: (7) in, Indicates ES Energy consumption per unit of computing resources.

[0027] 3) OES offloading model: This means that when the vehicle's computing resources are insufficient to meet the task requirements, the vehicle may assist in calculating the user's needs if there are available idle computing resources nearby. The Sub-tasks The calculation results will be returned to the user's vehicle. In this case, subtasks The processing latency for unloading to Elasticsearch mainly includes upload task latency, execution latency, and response waiting latency, expressed as: (8) in, The calculation method can be found in formula (6). To calculate... It is necessary to know in the subtask Previously assigned to the same OES All tasks. (Get) Indicates in subtask Previously assigned to the same OES All tasks, The waiting response delay is Total execution response latency of medium-duration tasks: (9) in, Indicates in Tasks previously assigned to OES Subtasks The computational load.

[0028] In OES Complete the sub-tasks The total energy required is similar to that of offloading to ES, and is expressed as follows: (10) To ensure successful task completion, this embodiment introduces the concept of task completion rate, which depends on the successful execution of the last subtask. If a preceding subtask fails, it means the result was not transmitted before the vehicle leaves the RSU or the opportunity vehicle's communication range. The total number of completed tasks is denoted as: (11) Among them, binary Indicates user vehicle Have all tasks been completed? If so... This indicates that the task has been completed; otherwise, .

[0029] Subsequently, the user vehicle The task completion rate represents the percentage of tasks successfully executed out of the total number of tasks, expressed as: (12) Based on the different task offloading models mentioned above, user vehicles The total delay for completing all subtasks is: (13) Similarly, user vehicles The total energy required to complete the task is expressed as: (14) Due to limited resources in the VEC environment, appropriate task offloading and resource allocation strategies need to be developed to fully utilize the computing resources of ES and OES, thereby reducing energy consumption and latency during task execution, while maximizing task completion rate and improving the vehicle user experience. This can be viewed as a multi-objective joint optimization problem, which can be transformed into a single-objective optimization problem through weighted summation, expressed as: It is composed of the weighted sum of total latency, total energy consumption, and the reciprocal of the task completion rate. Its mathematical expression is as follows: (15) in, , and The weighting factors, representing latency, energy consumption, and task completion rate, are determined using the analytic hierarchy process (AHP) for both qualitative and quantitative analysis. Among these, the constraints... and Involving subtasks The unloading decision constraint ensures that a subtask can only execute on a single processor. This indicates that the computing resources allocated to a task must not exceed the total resource limit; this is a constraint. This indicates that the task execution time must meet the maximum tolerable delay of the task, and the constraint conditions. This indicates the maximum computing resource limits for the user's vehicle, ES, and OES. The problem is a challenging mixed-integer nonlinear programming problem, typically belonging to the NP-hard category. Therefore, this embodiment uses the FL-MADDPG algorithm for solution.

[0030] In the DT-driven collaborative VEC compute offloading scenario, Elasticsearch deployed on RSUs provides compute offloading services to vehicles. Simultaneously, opportunistic self-organizing clouds formed by underutilized opportunistic vehicles within the transportation network provide compute offloading services to user vehicles. The specific compute offloading process is as follows: Figure 3 As shown. In this embodiment, the task offloading problem in the DT-VEC dynamic environment is solved using the FL-MADDPG algorithm. First, the task offloading process is modeled as an MDP. Second, a MADDPG-based algorithm framework is designed to solve the problem of convergence difficulty of traditional single-agent reinforcement learning algorithms in dynamic environments. Finally, to synchronize the running time of all participants' training cycles, an FL mechanism is introduced during training. Participating nodes are selected through random selection, and the aggregation period is divided into local aggregation period and global aggregation period. The task offloading decision can benefit from combining global information aggregated from neighboring twins and historical information uploaded by vehicles to adapt to highly dynamic environments and process large amounts of data in a short time, ensuring the accuracy and timeliness of vehicle decisions.

[0031] The state space, action space, and reward function for the computational offloading problem in joint V2R and V2V are designed as follows.

[0032] (1) State Space: The state space needs to be determined by observing the entire system after DT-assisted clustering, including the number of vehicles in the aggregation group, the task status of user vehicles, the computing resource scale of ES and OES, and the location information of vehicles, etc. Therefore, the system state can be represented as: (16) in, Indicates user vehicle In time Location information, , and These represent the computing capabilities of the user vehicle, ES, and OES, respectively.

[0033] (2) Action Space: In the current time slot, the unloading decision mainly includes the unloading decisions for all subtasks. This represents three unloading decisions. (In time...) The Action Space (ES) and Operational Execution Space (OES) determine the task execution method and allocate computing and communication resources to the user vehicle. The expression indicates whether the user vehicle chooses to execute the command locally or offload it to ES or OES for execution. (17) (3) Reward Function: The reward function of MADDPG is considered a guide for algorithm learning, and the goal of MADDPG is to maximize the cumulative reward; therefore, the goal and the expected reward should be consistent. To maximize the system cost ( The reward function must establish a reasonable potential relationship with the system cost and also consider penalties for not meeting resource constraints and latency tolerance. Therefore, the reward function can be expressed as: (18) in, This indicates that if the constraint condition If the condition is not met, the value is -1; otherwise, the value is 0. , , , , , These are experimental parameters.

[0034] The MADDPG algorithm is a combination of Actor-Critic and Off-policy algorithms, employing techniques from both to optimize cumulative rewards. The Actor network and Critic network each consist of their respective training and target networks. The task offloading algorithm based on FL-MADDPG is shown in Algorithm 1.

[0035] First, this embodiment utilizes the current Actor network to interact with the environment, obtaining the state space from the DT (Data Dependency Table) and using it as input to the Critic network. This network is responsible for adjusting the current state... Select current action Used to generate interactive effects with the environment and Then the quadruple Store in the experience pool.

[0036] Then, the Critic network is updated using DQN and double-Q learning methods, employing two neural networks to compute... It is generated based on the next state and the next action. Among them, the target The formula for calculating the value is: (19) in, The true value equal Therefore, gradient descent is used to update the current Critic network parameters. ,make Closer to Its update formula is: (20) in, and These represent the decay factor and the size of the mini-batch, respectively.

[0037] Next, this embodiment updates the parameters of the current Actor network. Because the actions output by the Actor network will affect the current Critic network... The value, therefore, is updated by updating the parameters of the current Actor network. To maximize The output parameter is updated using the gradient ascent method. This update primarily involves modifying the Actor network. The action parameters and update strategies include: (twenty one) Finally, the bi-target network is updated: (twenty two) in, This is the update speed of the bi-target network, and it is generally taken as a small value, such as 0.01.

[0038] Clearly, from the perspective of maximizing task efficiency, each agent will tend to schedule subtasks to the Elasticsearch (ES), even if the current subtask has a high latency tolerance. This can cause subsequent subtasks with lower latency tolerance to fail to be offloaded due to excessively long task queues at the edge and insufficient computing resources, thus reducing task offloading efficiency. However, if newly connected nodes lack a well-trained learning model, they may make inefficient task offloading decisions, thereby reducing the overall network performance. The FL-MADDPG algorithm allows newly connected vehicles to obtain a well-trained global model through a cloud server. This model is collaboratively trained by distributed vehicles and coordinated through the cloud server, thereby optimizing the learning effect. Therefore, to obtain the maximum offloading efficiency, agents need to cooperate to find the optimal task offloading strategy. To achieve cooperation between agents, FL is introduced into the offline learning environment of the MADDPG algorithm framework, specifically in the following form: (twenty three) in, For the aggregated global model, The size of the experience replay cache pool for each participating vehicle. The sum of the experience replay cache pool capacities of the vehicle sets participating in the aggregation. For agent policy network weights.

[0039] The specific algorithms include: Algorithm 1: Requirement: DT-assisted VEC network; Ensure: (Ensure) offloading decisions; 1: Initialize the Actor network and Critic Network with and .

[0040] 2: Initialize the target Actor network parameters using ; 3: Initialize the target Critic network parameters ; 4: Initialize the replay buffer pool Small batch size ; 5: Initialization and ; 6: for episode do: 7: Reset the DT simulation environment and observe the initial state. ; 8: for to do: 9: for vehicle do: 10: The vehicle's perception of the environment and its own task requirements are obtained. ; 11: Select Action Using Actor Networks ; 12: Perform the action And receive a reward ; 13: Observe the next state ; 14: if If it is not full then: 15: Empirical samples Store in the experience replay pool middle; 16: else: 17: From A certain number of empirical samples were drawn from the middle. And randomly select a size of Small batches; 18: Through the formula Calculation target value; 19: Update the Actor network policy using the sampled policy gradient (formula) ); 20: Update the target Actor-Critic network; 21: end if; 22: end for; 23: end for; 24: if do: 25: According to the formula Execute model aggregation; 26: for vehicle do: 27: ; 28: end for; 29: end if; To illustrate the efficiency of the proposed solution, this embodiment compares the proposed solution MADDPG with the four benchmark solutions listed below.

[0041] 1) Local Computing on the Vehicle (LC): Onboard computing tasks are executed locally by the user's autonomous vehicle.

[0042] 2) Fixed Edge Server only Algorithm (FESO): When the LC latency is greater than the latency tolerance of the on-board computing task, the computing task will be offloaded to the MEC edge server to meet the maximum tolerable latency of the computing task.

[0043] 3) F-DQN: Federated learning, as a distributed machine learning method, enables agents to acquire knowledge from each other. F-DQN introduces DNN to enhance reinforcement learning, enabling agents to formulate optimal strategies in complex environments.

[0044] 4) Deep Deterministic Policy Gradient (DDPG): This algorithm adds an Actor-Critic structure to DQN to obtain the optimal unloading strategy and uses the Actor network to obtain deterministic actions.

[0045] This embodiment first demonstrates the training and convergence process of the proposed algorithm. Figure 4 The average reward changes during training of the proposed MADDPG algorithm, F-DQN, and DDPG algorithms are shown. In the simulation, the agent underwent 6000 training rounds. In the initial training cycles, the agent explores the complex vehicle environment and collects various state transitions to fill the experience replay buffer; therefore, no network training is performed during this stage, resulting in significant fluctuations in the algorithm's average reward. As the number of training cycles increases, the agent explores the state space and action strategies as much as possible, gradually adjusting and updating network parameters to optimize the digital twin data update frequency selection strategy. Therefore, both the proposed method and the comparison methods maintain good incentive effects, and the agent's average reward begins to gradually increase. Data shows that the F-DQN algorithm shows positive results in the early stages of training, but may lose reliability later due to over-focusing on task parameter optimization. The DDPG algorithm, sensitive to learning and exploration rates, exhibits large fluctuations in its average reward value. The proposed FMADDPG algorithm demonstrates lower variance and faster convergence speed. The MADDPG algorithm performs model aggregation after a certain number of training cycles, making it easier to escape local optima, resulting in a more stable upward trend in its average reward. Due to the non-stationary nature of the vehicle-to-everything (V2X) environment, the cumulative reward will fluctuate slightly around the average value in subsequent training cycles.

[0046] The comparison results of average response time include: Number of user vehicles: The average response latency of the task reflects the degree to which the edge device improves the service quality of in-vehicle applications, and comprehensively measures the service quality of task offloading. Figure 5 As the number of user vehicles in the VEC environment increases, the workload of tasks with different needs rises. Limited system resources need to be allocated to more vehicles, leading to a decrease in expected bandwidth and an increase in average waiting time, both resulting in increased uplink and computation latency, thus increasing task delay. Since the algorithm is based on generating the minimum unloading cost, it outperforms the other four schemes, demonstrating that utilizing idle vehicle resources significantly reduces average latency and employs a more optimized task processing method, thereby effectively improving system execution efficiency and reducing resource contention. For the DDPG unloading scheme, the unloading latency increases significantly when the number of vehicles is large because the distributed unloading decisions of each vehicle agent cannot observe the behavior of other agents, lacking a further way to achieve distributed agent cooperation. It is worth noting that the F-DQN algorithm performs worse than the method proposed in this embodiment due to the need to synchronously upload the local model.

[0047] The impact of the number of user vehicles on average response latency: Subtask data size: In Figure 6 The paper primarily compares the average latency of five schemes for task completion at different task scales. When the input data for the onboard computing task increases, no computational offloading occurs for the user's autonomous vehicle, so the LC algorithm has a constant and the largest average response latency. The average latency of the other algorithms increases with the increase of task input data. This is because, when each edge computing node executes a task, on the one hand, data transmission latency increases, and on the other hand, the demand for computing and communication resources increases with the task scale. By introducing a federated learning (FL) mechanism, the proposed algorithm, compared to other algorithms, can make optimal offloading decisions based on the size of the task input data under different task distribution conditions, maintaining high task execution efficiency. Simultaneously, it utilizes the opportunity autonomous vehicle with abundant computing resources to alleviate the computational load on the fixed edge server, enhancing the computing processing capabilities of the scalable edge computing architecture. Since the DDPG algorithm obtains the optimal action when the task scale increases and the system environment becomes more complex, its computational cost is consistently significantly lower than other schemes, thus exhibiting superior performance. In contrast, F-DQN uses a synchronous offloading mechanism, resulting in significant latency.

[0048] The impact of the number and size of subtasks on average response latency: RSU's computing power: Figure 7It is evident that as RSU computing power increases, the average latency of all baseline schemes, except for the local execution scheme, decreases. This is because fully local computation is independent of RSU computing power and depends only on the computing power of the local vehicle. Therefore, the trend of LC remains unchanged. Furthermore, the trend indicates that when RSU computing power reaches a high level, task waiting time becomes negligible, resulting in the lowest average latency for offloading tasks to the RSU. Notably, compared to DDPG and F-DQN, the proposed algorithm still maintains a low average task completion latency. The fundamental reason is that the proposed algorithm, through an improved DDPG algorithm and joint learning mechanism, employs a reasonable task offloading strategy to transfer onboard computing tasks to nearby idle opportunity vehicles, reducing the load on the MEC edge server. This allows user vehicles to make fuller use of server-side resources, providing reliable computing support in different environments, reducing vehicle waiting time and task execution time, thereby reducing average latency.

[0049] To evaluate the execution efficiency of the proposed algorithm, this embodiment analyzes the task completion rate of each algorithm under different system computing resource conditions, such as... Figures 8-10 As shown in the diagram. The Z-coordinate represents the task completion rate, while the X and Y coordinates represent the computing power of the MEC edge server and the opportunistic vehicle, respectively. From... Figures 8-10 It can be seen that when the computing resources of the MEC edge server in the system decrease, the insufficient resources prevent some tasks from being offloaded to the MEC server, and the task completion rate begins to decline linearly, significantly impacting the system. However, experiments show that compared with DDPG and F-DQN, the proposed algorithm can still maintain a higher task completion rate. This is because the DDPG algorithm causes the backlog of the MEC edge server task execution queue to continue to grow and exceed the maximum queue capacity, resulting in the MEC edge server queue being overloaded and unstable. While F-DQN can ensure queue reliability, the utilization rate of the task execution queue is significantly low due to insufficient uploaded training data. The method in this embodiment can dynamically adjust the offloading decision according to the fluctuation of resource status, thereby improving the execution efficiency of computing tasks.

[0050] On the other hand, this embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0051] On the other hand, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method.

[0052] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A collaborative vehicle-mounted edge computing task offloading method driven by digital twin technology, characterized in that, include: Based on the real-time status data of the physical vehicle edge computing network, a virtual mapping network corresponding to the real-time status data is constructed and maintained; The global state information of the physical vehicle edge computing network is obtained based on the virtual mapping network; Based on the virtual mapping network and the global state information, a cooperative task offloading optimization algorithm is executed. The cooperative task offloading optimization algorithm integrates multi-agent reinforcement learning and federated learning mechanisms to generate task offloading decisions for in-vehicle computing tasks. Based on the task offloading decision, computing resources in the physical vehicle edge computing network are scheduled to execute the vehicle computing task.

2. The method according to claim 1, characterized in that, The process of constructing and maintaining a virtual mapping network corresponding to the real-time status data includes: constructing and maintaining a virtual mapping network containing physical vehicle entities, physical edge server entities, and dynamic relationships between the entities, based on the real-time status data of vehicles and servers in the physical vehicle edge computing network.

3. The method according to claim 2, characterized in that, The global state information of the physical vehicle edge computing network is obtained based on the virtual mapping network, specifically including: Real-time status monitoring of the physical vehicle-mounted edge computing network is performed based on the virtual mapping network. Based on the monitored real-time status information, the future state of the network is predicted to obtain global status information containing the prediction information.

4. The method according to claim 1, characterized in that, Before executing the collaborative task unloading optimization algorithm, the following steps are also included: Based on the characteristics of in-vehicle computing tasks, network resource constraints, and quality of service requirements, the unloading and resource allocation process of the in-vehicle computing tasks is modeled as a Markov decision process.

5. The method according to claim 4, characterized in that, The optimization algorithm for unloading collaborative tasks specifically includes: A reinforcement learning model based on the multi-agent deep deterministic policy gradient (MADDPG) framework is constructed to solve the Markov decision process.

6. The method according to claim 5, characterized in that, The construction of the reinforcement learning model based on the multi-agent deep deterministic policy gradient (MADDPG) framework specifically includes: Each user vehicle and / or each computing service provider entity in a physical vehicular edge computing network is defined as an intelligent agent.

7. The method according to claim 1, characterized in that, The state of the agent in the time slot is determined based on the global state information, which includes: agent location information, computing resource information of associated vehicles and servers, and state information of tasks to be unloaded. The agent's actions in a time slot are determined based on the policy network and the current state. The actions include: making decisions on whether to execute the subtask to be unloaded locally, unload it to an edge server, or unload it to an opportunistic vehicle. The agent updates the policy network based on the reward from the environment after performing an action and the next state it transitions to. The reward is calculated based on the total system cost and the constraint satisfaction.

8. The method according to claim 1, characterized in that, The collaborative task offloading optimization algorithm incorporates a federated learning mechanism, specifically including: Each agent updates the policy network parameters locally using its own experience data; According to a preset period, the policy network parameters of each agent are aggregated to update the global model parameters; Each agent synchronizes or updates its local policy network based on the global model parameters.

9. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, When the processor executes the computing program, it implements the method of any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1-8.

Images