Computing offloading algorithm for intelligent assisted driving switching scenario

Abstract

The application claims a vehicle-to-everything cooperative perception computing offloading algorithm for handover scenarios, belonging to the field of mobile communication technology and computer network technology, and is used to solve the problem of cooperative perception service interruption or increased processing delay caused by network handover when vehicles cross the coverage boundary of RSU. First, considering dynamic factors such as task computing amount, node real-time processing capacity and communication link state, three offloading modes are designed: RSU edge offloading, auxiliary vehicle cooperative computing and request vehicle local computing. Second, the system optimization goal is modeled as maximizing the average service quality of perception tasks, and is formalized as a combination optimization problem with resource constraints. Third, due to the high dynamicity and uncertainty of the vehicle-to-everything environment, the decision-making process is converted into a Markov decision process, and the Rainbow DQN algorithm is used to solve it to generate the optimal offloading strategy. Finally, vehicles dynamically select computing nodes to execute perception tasks according to the generated strategy. The algorithm can significantly reduce the service interruption probability and average processing delay during the handover process, improve the task completion rate and overall system service quality, and ensure the continuity and real-time performance of cooperative perception services in high-speed mobile scenarios.

Description

Technical Field

[0001] This invention belongs to the field of mobile communications, specifically to the computational offloading algorithm for intelligent assisted driving in the Internet of Vehicles. Background Technology

[0002] With the rapid development of intelligent transportation systems and vehicle-to-everything (V2X) technology, vehicle cooperative perception has become a key means to improve vehicle driving safety and environmental perception capabilities. In V2X architectures empowered by mobile edge computing, vehicles, limited by the perception range and accuracy of their own sensors, often rely on perception data provided by nearby auxiliary vehicles to achieve a comprehensive understanding of complex traffic environments. Furthermore, due to limited computing resources, vehicles can upload perception data to roadside units (RSUs) for computational offloading. RSUs, deployed along roads and equipped with edge servers, provide powerful computing support for real-time processing of perception data. However, vehicle mobility introduces a significant challenge to computational offloading: when a vehicle crosses the coverage boundary of its current serving RSU and enters the coverage area of ​​an adjacent RSU, a network connection switch is necessary. Without a reasonable mobility management mechanism, this switchover can lead to temporary interruptions or significant delays in the vehicle's reception of perception data. If blind spots exist on the road that the vehicle cannot perceive, its driving safety will be seriously threatened. Therefore, designing a computation offloading mechanism adapted to vehicle mobility at critical moments when vehicles cross RSU coverage boundaries and switch networks, so as to ensure that vehicles can obtain the required results with low latency to maintain high QoS, has become one of the core issues that current intelligent assisted driving systems urgently need to solve. Existing literature has addressed vehicle mobility and RSU handover issues in computational offloading. The paper [WANGH, LV T, LIN Z, et al. Energy-delay minimization of task migration based on game theory in MEC-assisted vehicular networks[J]. IEEE Transactions on Vehicular Technology, 2022, 71(8): 8175-8188.] proposes a game theory-based task migration strategy. By modeling vehicles as rational participants, the strategy involves making decisions among various offloading options such as local processing, vehicle-to-vehicle, vehicle-to-infrastructure, and task migration to minimize system energy consumption and latency. Distance constraints are introduced for vehicle mobility to ensure migration feasibility. The paper [MALEKI H, BAŞARAN M, DURAK-ATA L. Handover-enabled dynamic computation offloading for vehicular edge computing networks[J]. IEEE Transactions on Vehicular Technology, 2023, 72(7): 9394-9405.] designed a dynamic offloading mechanism that supports handover. It uses deep reinforcement learning to dynamically determine which edge node to offload part or all of the task to when a vehicle passes through the coverage area of ​​multiple edge servers, thereby adapting to rapid changes in network topology while optimizing both latency and energy consumption. The paper [LING C, ZHANG W, HE H, et al. Qos and fairness oriented dynamic computation offloading in the internet of vehicles based on estimatetime of arrival[J]. IEEE Transactions on Vehicular Technology, 2024, 73(7):10554-10571.] integrates the "estimated time of arrival" service into the computational offloading architecture. It dynamically selects the optimal RSU for offloading by predicting the future location of vehicles, and takes into account both service quality and fairness of response among RSUs. It proposes two algorithms based on dynamic programming and greedy strategy.The literature [FAN W, ZHANG Y, ZHOU G, et al. Deep reinforcement learning-based task offloading for vehicular edge computing with flexible RSU-RSU cooperation[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(7): 7712-7725.] allows tasks to be offloaded to any suitable RSU and the results are directly transmitted back to the RSU in the current vehicle area, avoiding result reception failure due to vehicle movement. At the same time, a hybrid solution framework combining reinforcement learning algorithm and numerical optimization subroutine is designed to jointly optimize offloading decision, resource allocation and transmission power. However, the aforementioned studies only model vehicle mobility within a general computational offloading framework, failing to address the specific application scenario of cooperative vehicle perception. Unlike general computational tasks, cooperative perception tasks have extremely high timeliness requirements: perception data is typically only valid for a very short time; if transmission or processing delays are too large, its value for assisted driving decisions will significantly decrease or even become completely ineffective. Furthermore, since cooperative perception usually occurs between adjacent vehicles or within a local area, it often does not involve crossing multiple RSUs (Roadside Units). Therefore, RSU handover mechanisms designed to support long-distance movement are not applicable in such scenarios. Based on the above analysis, this invention proposes a computational offloading algorithm for handover scenarios in intelligent assisted driving, reducing the probability of service interruption and average processing latency during the handover process, and improving task completion rate and overall system service quality. Summary of the Invention

[0003] This invention aims to solve the problems of the prior art and proposes a computational unloading algorithm for switching scenarios in intelligent assisted driving. The technical solution of this invention is as follows: A computational offloading algorithm for switching scenarios in intelligent assisted driving includes the following steps: First, the RSU collects global state parameter information within the system, including the characteristics of each perception task, the available computing power of the RSU, and the local computing power of the vehicle nodes. Second, the RSU inputs the collected state information into a pre-trained Rainbow DQN model for online inference and decision-making. This model outputs the optimal joint decision scheme. Finally, each execution node in the system performs corresponding computation and transmission operations according to the decision instructions issued by the RSU. If the decision is RSU edge offloading, the auxiliary vehicle uploads data to the RSU, and the RSUs collaborate through the backend backhaul network to complete data forwarding and computation. The RSU specified in the decision then sends the result back to the requesting vehicle. If the decision is auxiliary vehicle collaborative computation or requesting vehicle local computation, the raw data or processing results are directly transmitted via the V2V link. Furthermore, for ease of subsequent description, we will first introduce the network scenario of this algorithm. Consider a typical RSU handover scenario in urban streets: during the execution of the vehicle perception task, the vehicle starts from the coverage area of ​​the current RSU, and its travel path can only reach the coverage area of ​​the next adjacent RSU at most. Therefore, the system scenario focuses only on two key RSUs—the currently serving RSU (denoted as RSU 1). ) and target switching RSU (denoted as Each RSU is equipped with an independent edge server, which can provide computational offloading services for vehicle perception data fusion. Assume there are a total of [number missing] in this scenario. The vehicle is about to be switched, denoted as any one of the vehicles They all traveled at a constant speed along the road, from The coverage area towards The coverage area moves. Vehicles After acquiring perception data from vehicle sensors such as cameras and LiDAR, obstructions in the road can be identified, thus revealing multiple blind spots for the vehicle. To compensate for these blind spots, the vehicle relies on perception data from other surrounding vehicles (referred to as "assistant vehicles"). Assume that in this scenario, there are a total of... Vehicles, recorded as any one of the vehicles All located in or Within its coverage area. Furthermore, vehicles Transmit information about its own blind spots to back, Can be with Collaboration, for each vehicle A list of auxiliary vehicles that can compensate for its perception blind spots is generated. (Auxiliary vehicles) The raw perception data collected (such as point clouds, images, etc.) needs to undergo computational processing (such as object detection, semantic segmentation, or data fusion) before it can be requested by the vehicle. Effectively utilize it to compensate for its perception blind spots. Therefore, the vehicle... Requesting auxiliary vehicles The task of processing perceived data is represented as Represent all perception tasks within the system as a set The features of each task can be represented by a quadruple. Description, in which This represents the amount of raw sensory data. This indicates the amount of data representing the processing result. This indicates the number of CPU cycles required to complete the task. This indicates the maximum allowable processing latency for the task; exceeding this threshold will cause the perceived data to become outdated and invalid. To improve resource utilization and real-time response, the system will allocate processing time for each task based on the current network status and node capabilities. Make the optimal processing decision. Specifically, the task... You can choose one of the following three execution methods: (1) RSU edge unloading: auxiliary vehicle The task Offloaded to via V2I link or The processing utilizes the computing resources of the edge server, and the results are then transmitted back via a V2I link. Because edge servers have strong computing power, this mode can significantly reduce task processing latency; however, its performance is limited by the current load level of the RSU and the duration of the connection between the vehicle and the RSU. (2) Assist vehicle cooperative calculation: Assist vehicle Complete the task using its own computing resources The process is performed, and the results are sent to the requesting vehicle via a V2V link. This mode requires no infrastructure, has low communication latency, and is suitable for scenarios with high RSU load; however, its feasibility depends on… Computing power and its relationship with Stability of V2V links between them. (3) Request local vehicle calculation: auxiliary vehicle The raw sensing data is transmitted to the requesting vehicle via a V2V link. Then utilize The vehicle's computing resources complete the task. This mode avoids result transmission failures due to vehicle mobility, but due to the limited computing power of the onboard processor, it is only suitable for tasks with small computational loads. Furthermore, to further improve the efficiency of computing resource utilization and service quality, and They possess collaborative scheduling capabilities, enabling them to flexibly select specific RSUs to execute computational tasks based on global state information such as computational load and vehicle movement status. To support this cross-RSU collaborative scheduling mechanism, and Efficient data exchange is achieved between them via a backhaul network (such as fiber optic). This backhaul link supports the following two data transmission modes: (1) Original data transmission: If the vehicle If the currently connected RSU is not the same as the RSU performing the computational offloading task, the current RSU can forward the received raw sensing data to the other RSU. For example, if the auxiliary vehicle... Currently connected to However, system decisions are made by Execute the task The calculation, then The received raw sensing data is forwarded to the backhaul network. . (2) Result data transmission: If the vehicle The required perception data has been calculated and unloaded by the RSU, but the RSU currently connected to the vehicle is inconsistent with it. In this case, another RSU can forward the resulting data to the current RSU. For example, if the task... Already The above process has been completed, but a request has been made for the vehicle. It has now moved to Within its coverage area, then The processing results can be sent to via the backhaul network. Then by The results are transmitted via a V2I link to . Furthermore, the task will be completed using the RSU edge offloading method. Total delay is defined as It consists of the following parts: auxiliary vehicles Delay in transmitting raw data to its connected RSU Delay in the transmission of raw data between RSUs The latency of RSU task computation The delay in transmitting calculation results between RSUs The calculation results are sent back to the vehicle by the RSU. latency The following sections will introduce how to calculate the delay for each part. Further introduction and The calculation method is as follows. In the vehicle-to-everything (V2X) system considered in this invention, the data transmission rate of all wireless links is modeled based on Shannon's formula. Let the channel bandwidth between any pair of communication nodes be... The transmission power is The channel gain is The noise power is Then its maximum transmission rate It can be represented as: Furthermore, based on the different roles of the communicating parties, this formula is specifically applied to three types of links: V2I uplink, V2I downlink, and V2V link. Corresponding to the above three types of links, the vehicle... The transfer rate to RSU is expressed as RSU to vehicle The transmission rate is expressed as ,vehicle To the vehicle The transmission rate is expressed as .vehicle Raw data transmission delay to RSU and RSU to vehicle Calculation result return delay They can be obtained from the following formulas: Further introduction and The calculation method is as follows. Since not all tasks involve forwarding operations between RSUs, it is first necessary to determine whether the task requires data transmission between RSUs. This invention makes this determination based on the following three factors: (1) Auxiliary vehicle The connected RSU is denoted as , 0 represents Connected to , 1 represents Connected to (2) Task The specific execution of RSU is denoted as , A value of 0 indicates a task. Depend on implement, A value of 1 indicates a task. Depend on Execution; (3) The final RSU of the returned task result is denoted as , A value of 0 indicates a task. The results data are from Transmitted to vehicle , A value of 1 indicates a task. The results data are from Transmitted to vehicle . Furthermore, the task Whether the raw or resulting data needs to be transferred between RSUs is denoted as a variable. and When the variable value is 1, it means that data needs to be transferred between RSUs; otherwise, it does not. To represent the XOR operation, then and The calculation method is as follows: Furthermore, to improve the accuracy of latency modeling, this invention considers the potential queuing phenomenon that may occur when RSUs transmit data through the backhaul network, and introduces a queue-based transmission model for the backhaul network between RSUs. For tasks requiring data transmission between RSUs, and Both consist of queuing delay and the data transmission delay of the task itself. Assume that each RSU's backhaul interface maintains a first-come, first-served queue of data packets to be forwarded. When data needs to be transmitted across RSUs, it must first queue in the sending RSU's backhaul queue and wait for the link to become idle before it can be sent. Definition , Tasks When data is ready to be forwarded, the original data set and the result data set waiting to be sent are returned to the queue. Assume... and The maximum transmission rate of the wired connection between them is Then the task The queuing delay for data transmission between RSUs is: Furthermore, The total delay of transmitting raw and result data between RSUs can be calculated by the following formula: Further introduction The calculation method is as follows. Since dynamically allocating precise computing resource ratios to each task introduces significant scheduling overhead and management complexity, this invention adopts a queue-based sequential execution mode for task processing modeling to reduce system burden and improve deployment feasibility. Specifically, when a task... After being unloaded to the target RSU, the task's data is temporarily stored in the RSU's memory buffer and then enters the task processing queue. The target RSU then processes the tasks in the queue sequentially according to a first-come, first-served principle. Definition For the task The set of tasks awaiting processing when unloaded to the target RSU. Assume the maximum computing power the RSU can provide is... Then the task The processing delay at RSU is the sum of the queuing delay and its own computation delay, which can be calculated by the following formula: Furthermore, the task is completed using the RSU edge offloading method. Total latency It can be calculated using the following formula: Furthermore, to fully consider vehicle mobility, the system needs to compare vehicles. and The remaining communication time and the total delay to complete the task will determine the final decision. still Return the calculation results This is to avoid service delays or failures caused by connection interruptions during the handover process. and The remaining communication time is denoted as ,like Greater than Total delay for obtaining calculation results ,illustrate You can drive out The calculation results should be obtained within the communication range, therefore, it should be... Return to This restriction can be expressed as the following constraint: Furthermore, if Less than Total delay for obtaining calculation results ,illustrate Unable to drive out The calculation results should be obtained within the communication range, therefore, it should be... Return to This restriction can be expressed as the following constraint: Furthermore, the task will be completed using a collaborative computing mode assisted by the vehicle. The total delay is denoted as It consists of two parts: the task is performed in the auxiliary vehicle. computational latency Calculation results from Return to V2V transmission latency .Will Recorded as vehicles The maximum computing power that can be provided, and the computation latency. The calculation method is as follows: Furthermore, vehicles To the vehicle The result is data transmission delay for: in For vehicles To the vehicle The transmission rate can be calculated using formula (1). In summary, the task is completed using an auxiliary vehicle cooperative computing mode. Total latency for: Furthermore, due to the high mobility of vehicles during operation, auxiliary vehicles... With the requested vehicle The constantly changing relative positions between the vehicles may cause the V2V link to disconnect before task processing is complete, resulting in failure to transmit results back. To ensure reliable task delivery, it is essential to guarantee that both vehicles remain within effective communication range when task processing is complete. and The remaining communication time is denoted as The above constraints can be expressed as: Furthermore, it will request the vehicle to complete the task in its local computing mode. The total delay is denoted as It consists of two parts: auxiliary vehicles. Transmit raw sensor data to the requesting vehicle latency ,vehicle Latency of processing tasks on the local computing unit .vehicle To the vehicle raw data transmission delay for: in For vehicles To the vehicle The transmission rate can be calculated using formula (1). Recorded as vehicles The maximum computing power that can be provided, and the computation latency. The calculation method is as follows: Furthermore, the task is completed using a request-based local vehicle computation mode. Total latency for: Furthermore, due to the high mobility of vehicles during operation, auxiliary vehicles... With the requested vehicle The relative positions between them are constantly changing to ensure the mission For successful execution, it must be ensured that both vehicles remain within effective V2V communication range until the initial data transmission is complete. and The remaining communication time is denoted as The above constraints can be expressed as: Furthermore, to achieve efficient computational offloading for intelligent assisted driving, this section formally constructs an optimization problem. The core objective is to maximize the overall QoS of the system by selecting the optimal execution strategy for each perception data processing task in dynamic scenarios where the vehicle undergoes RSU (Roadside Request) switching. For each requesting vehicle... With auxiliary vehicles Matching generated tasks Its unloading decision is determined by a discrete decision variable. It means that when At that time, the representative will Unload to RSU for processing; when At that time, it represents the auxiliary vehicle Collaborative computing; when At that time, it represents the vehicle Perform local computation. (Task) The final actual total processing latency It can be calculated using the following formula: in This is an indicator variable; its value is 1 when the condition in parentheses is true and 0 when the condition is false. This is to quantify each task. To improve service quality, this invention introduces a QoS metric closely related to task latency. When the actual total processing latency of a task... Less than the maximum latency tolerated by the task hour, The smaller the value, the more timely the task is completed, and the higher the QoS; when the actual total processing time of the task... Greater than the maximum latency tolerated by the task When this condition is met, the QoS value is 0. This is expressed by the following formula: Furthermore, to fairly measure overall performance, this invention uses the arithmetic mean of the QoS of all tasks as the system-level QoS, and sets the optimization objective as maximizing the system-level QoS, jointly optimizing the offloading decision for each task. For tasks that are unloaded at the RSU edge, an RSU execution decision must also be formulated. and RSU backhaul decision The total number of tasks in the system is expressed as The optimization problem can be represented as: Among them, constraints C1, C2, and C3 specify the range of values ​​for each decision variable; constraints C4 and C5 indicate that during RSU edge computation, the system will... and The remaining communication time will ultimately determine the outcome. still The calculation results are sent back; constraint C6 means that when the auxiliary vehicle is performing collaborative calculations, the task completion delay must not exceed the duration of the V2V link between the two vehicles; constraint C7 means that when requesting the vehicle to perform local calculations, the transmission delay of the original perception data must not exceed the duration of the V2V link between the two vehicles. Furthermore, due to the dynamic environment and complex constraints of this problem, traditional optimization methods, such as linear programming and heuristic algorithms, struggle to solve efficiently in real-time mobile scenarios. Therefore, this invention proposes a deep reinforcement learning algorithm based on Rainbow DQN to address this problem. Rainbow DQN, as an improved version of DQN, integrates several key technologies, effectively handling high-dimensional state spaces, providing a more stable training process, and achieving superior decision-making performance. The Markov decision process will be constructed below, comprising a state space, action space, and reward function. Definitions... The state vector at time t is The specific meanings of each dimension are as follows: (1) This represents the feature vector of the current set of tasks to be decided, containing each task. Quadruple information This refers to the amount of raw data, the amount of processed result data, the number of CPU cycles required, and the maximum tolerable latency. (2) This represents the RSU resource state vector, containing the maximum computing resources that the current serving RSU and the target RSU can provide. Task processing queue Backhaul link queue and Maximum transmission rate . (3) Represents the vehicle state vector, containing the requesting vehicle. and Remaining communication time Requesting vehicles and auxiliary vehicles Remaining V2V communication time Assist vehicle computing resources Request vehicle local computing resources . (4) This represents the communication channel state vector, which includes the V2I link transmission rate. and V2V link transmission rate It is calculated in real time using the Shannon formula. Furthermore, the action space corresponds to the set of decision variables in the optimization problem, and is defined as follows: The action at a given moment is ,in This represents the unloading decision vector for each task; This indicates the RSU execution decision for each task; This indicates the RSU feedback decision for each task. Note that this only applies when... The value is 1, which means the task is... When unloading to RSU is performed, and Only then will it be effective; when When it is 2 or 3, and No need to consider it. Furthermore, the reward function directly reflects the quality of system decisions and must be closely related to the system's QoS indicators. Based on the optimization objective defined above, The reward function at time step is designed as follows: The reward function ensures that when the task completion delay is less than the maximum tolerable delay, the smaller the delay, the higher the reward; when the task completion delay exceeds the maximum tolerable delay, the reward is 0. Furthermore, DQN is an algorithm that combines deep neural networks with Q-Learning in traditional reinforcement learning. It can be used to solve decision problems in high-dimensional state spaces, but it has some significant drawbacks, such as poor learning stability in complex tasks, slow convergence speed, and limited performance. Rainbow DQN, by integrating six targeted improvement techniques, addresses many of the pain points of DQN in one go, significantly improving the algorithm's learning ability and generalization performance in complex environments. It integrates the following six techniques on the basis of the original DQN: (1) Double DQN: Standard DQN uses the same Q-network to select actions and evaluate values ​​when calculating the target Q-value, which can easily overestimate the true Q-value. Double DQN separates action selection and value evaluation, using different networks to accomplish these two tasks, which can alleviate the overestimation problem and improve the target Q-value. Calculated using the following formula: in It is a discount factor. and These represent the target network and the main network, respectively. (2) Prioritized Experience Replay (PER): Standard DQN uniformly and randomly samples a batch of experiences from the replay buffer during training, which may result in low training efficiency. PER measures the importance of each experience based on its temporal difference error and adjusts the sampling probability accordingly, enabling the agent to replay important experiences more frequently and significantly improving sample utilization efficiency. (3) Dueling Network: To improve the accuracy of policy evaluation, Dueling Network decomposes the Q-network into state value functions. and dominance function The two are then combined using the following formula to obtain the final Q value: The advantage of this improvement is that it can accurately estimate the value of all actions even when they are similar in certain states. This enables the network to converge quickly and improves its generalization ability. (4) Multi-step Returns: Standard DQN uses single-step returns, which may have problems such as high bias and slow learning. Therefore, multi-step returns can be used to replace single-step returns. Target value of step return The calculation method has been updated (while also applying Double DQN): (5) Distributional RL: This improvement no longer estimates the expectation of Q-values, but models their complete distribution. For example, C51 discretizes the Q-value distribution into 51 atoms, and the network outputs the probability of each atom, preserving uncertainty information and improving robustness and policy quality. (6) Noisy Nets: This improvement transforms the weight parameters of the neural network into noisy random variables, thereby driving the randomness of the strategy through noise and replacing the traditional manually designed exploration mechanism such as ε-greedy. Furthermore, a computational unloading algorithm based on Rainbow DQN is proposed, employing an offline training and online inference deployment architecture to meet the low latency requirements of intelligent assisted driving scenarios. Specifically, in the offline phase, the algorithm fully trains the Rainbow DQN model using a large amount of simulated typical switching scenario data, enabling it to learn the optimal unloading decision strategy under different dynamic environments. After training, the model parameters are fixed and deployed to the RSU. In the actual operation phase, the RSU only needs to perform one forward inference based on the current observation state through the pre-trained lightweight neural network to output the optimal unloading action within milliseconds, without the need for online iterative solution. This mechanism can significantly reduce decision latency, thereby ensuring the continuity and reliability of cooperative perception services in vehicle movement and switching scenarios. Furthermore, each execution node in the system performs corresponding calculation and transmission operations based on the decision instructions issued by the RSU, requests the vehicle to obtain the processing results of the collaborative perception task, and then makes the final assisted driving decision in combination with the vehicle's own driving information. The advantages and beneficial effects of this invention are as follows: 1. A dynamic offloading strategy was constructed that integrates three modes: local computation of requesting vehicles, collaborative computation of assisted vehicles, and RSU edge offloading. The remaining communication time constraint was introduced to effectively reduce the risk of data transmission failure during the handover process caused by vehicle movement and ensure the continuity of collaborative perception services. 2. A computational offloading algorithm based on Rainbow DQN deep reinforcement learning was designed. By jointly optimizing the offloading decision, RSU execution node selection and result return node allocation, the system significantly improved the task service quality in vehicle movement switching scenarios, effectively reduced the average task completion latency and improved the task completion rate. Attached Figure Description Figure 1 This is a system scene diagram; Figure 2 A graph showing the relationship between average task QoS and the number of tasks; Figure 3 A graph showing the relationship between average task QoS and average vehicle speed; Figure 4 This is a graph showing the relationship between average task completion time and the number of tasks. Figure 5 A graph showing the relationship between average task completion time and average vehicle speed; Figure 6 A graph showing the relationship between average task completion rate and the number of tasks; Figure 7 A graph showing the relationship between average task completion rate and average vehicle speed; Figure 8 The graph shows the convergence performance of the algorithm. Figure 9 This is a schematic diagram illustrating the computational unloading algorithm flow for switching scenarios in intelligent assisted driving. Detailed Implementation

[0004] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention. The technical solution of the present invention is as follows: In the Internet of Vehicles (IoV) integrating mobile edge computing, a dynamic offloading strategy is constructed that integrates three modes: requesting vehicle local computing, assisting vehicle collaborative computing, and RSU edge offloading. Remaining communication time constraints are introduced to reduce the risk of data transmission failure. At the same time, a computation offloading algorithm based on Rainbow DQN is proposed. By jointly optimizing the offloading decision, RSU execution node selection, and result return node allocation, the overall service quality of the IoV system under switching scenarios is improved. The computational unloading algorithm for switching scenarios in intelligent assisted driving proposed in this invention includes the following steps: Step 1: When a vehicle generates a collaborative perception task request, the RSU collects global state parameter information within the system, including the characteristics of the task to be processed, the RSU's own resource status, and the dynamic status of the vehicle node. Step 2: The RSU inputs the collected state information into the pre-trained Rainbow DQN model for online inference and decision-making. The model outputs the optimal joint decision scheme, including offloading decisions for each perception task, RSU execution node selection, and result feedback node allocation. Step 3: Each execution node in the system performs corresponding calculation and transmission operations based on the decision instructions issued by the RSU. After requesting the vehicle to obtain the processing results of the cooperative perception task, it makes the final assisted driving decision based on the vehicle's own driving information. To evaluate the computational offloading algorithm proposed in this invention, simulation code was written in Python and trained and tested using a custom vehicle-to-everything (V2X) dynamic environment simulator. The simulation scenario was set as a 600-meter-long one-way urban main road, with two adjacent RSUs (denoted as RSUs) deployed along the road. and Each RSU has a coverage radius of 300 meters. The scenario includes a requesting vehicle and an auxiliary vehicle moving at a constant speed. The requesting vehicle starts from... Coverage area heading towards Coverage area, simulating a typical handover process. In the experimental design of this invention, the main algorithm performance metrics examined are: average task QoS, average task completion latency, and average task completion rate. Average task QoS is the arithmetic mean of the QoS values ​​of all tasks; this metric is the core optimization objective of this invention and directly reflects the overall service quality of the system. Average task completion latency is the average of the actual total latency of all tasks from start to finish; this metric provides a more detailed description of the algorithm's real-time performance. Task completion rate is the proportion of tasks successfully completed within the maximum tolerable latency out of the total number of tasks, used to measure service reliability. To verify the performance of the algorithm proposed in this invention, other algorithms were introduced for comparative analysis: (1) Random selection algorithm: For each task, one of three modes is randomly selected: local computing, auxiliary vehicle computing or RSU unloading. (2) Greedy selection algorithm: For each task, calculate the estimated latency under the three execution modes of local computing, auxiliary vehicle computing, and unloading to the current RSU, and select the mode with the smallest estimated latency. (3) DQN-based calculation unloading algorithm: In the unloading mode designed in this invention, the classic DQN algorithm is used to solve the unloading decision. (4) The computation offloading algorithm based on Double DQN, namely the algorithm proposed in the literature [MALEKI H, BAŞARAN M, DURAK-ATA L. Handover-enabled dynamic computation offloading for vehicular edge computing networks[J]. IEEE Transactions on Vehicular Technology, 2023, 72(7): 9394-9405.]: This algorithm stipulates that the target vehicle can only offload the computation task to the auxiliary vehicle or the current edge server. When a switch occurs, a new offloading target will be selected, and the remaining part of the task will be offloaded. If there is no suitable offloading target, the computation will be performed locally until the task is fully executed. The decision on computation offloading is based on the Double DQN algorithm. In the above comparison algorithms, algorithm (1) uses random selection for decision-making, which can be used as a performance lower bound when compared with other algorithms to highlight the necessity of intelligent decision-making. Algorithm (2) represents an intuitive but short-sighted heuristic strategy that ignores the impact of vehicle mobility. Compared with it, it can show the performance advantage of the present invention in switching scenarios. Algorithm (3) uses the most traditional DQN algorithm to solve each decision. Compared with it, it can verify the performance gains brought by the various improvements in Rainbow DQN. Algorithm (4) uses a more passive strategy to deal with the RSU switching problem. Compared with it, it can show the flexibility of the unloading mode of the present invention. Figure 2 The relationship between average task QoS and the number of tasks is illustrated. When the number of tasks is small, system resources are sufficient, and all algorithms can achieve high QoS. As the number of tasks increases to 6, the system load increases significantly, the RSU queue length increases, leading to increased latency for some tasks, and the QoS of all algorithms shows a downward trend. However, the QoS of the algorithm in this invention decreases the most gradually, dropping from an initial 0.95 to about 0.7, still maintaining a high level. In contrast, the performance of the random selection algorithm deteriorates rapidly, with the QoS dropping from 0.68 to below 0.3, highlighting the inefficiency of blind decision-making under resource constraints. Although the greedy selection algorithm is superior to the random algorithm, it lacks consideration of future states and is prone to getting trapped in local optima during scenario switching. Its QoS is close to 0.7 when the number of tasks reaches 4, and continues to decline rapidly thereafter. Although the DQN and DDQN algorithms introduce reinforcement learning mechanisms and have certain adaptability, they still suffer from slow convergence and low exploration efficiency in complex dynamic environments, especially exhibiting significant performance bottlenecks under high loads. The algorithm of this invention maintains the highest average task QoS regardless of the number of tasks, and its performance decline trend is the most gradual, demonstrating excellent robustness and dynamic adaptability. Figure 3 The relationship between average task QoS and average vehicle speed is demonstrated. As vehicle speed increases, the dwell time within the RSU coverage area decreases, and V2V link stability declines, leading to a higher risk of task interruption and communication uncertainty. The QoS of all algorithms shows a downward trend. The random selection algorithm consistently performs the worst, with its QoS plummeting to only 0.34 at 80 km / h, highlighting the infeasibility of strategy-less offloading in highly dynamic vehicular networks. While the greedy selection algorithm approaches optimal performance at low speeds due to ample communication windows, its QoS drops sharply above 60 km / h because its short-sighted decision-making fails to consider potential RSU switching or link interruptions during task execution, leading to task failure. The DQN algorithm partially captures vehicle movement patterns through reinforcement learning and outperforms the greedy selection algorithm in high-speed scenarios, but it is still limited by overestimation of the value function and insufficient exploration capabilities. Because the offloading strategy of the DDQN algorithm is restricted to the current RSU or nearby auxiliary vehicles, it cannot support cross-RSU data transmission, and its QoS also significantly decreases at high speeds. In comparison, the algorithm of this invention maintains the highest QoS level across all speed ranges, significantly outperforming other methods. Figure 4 The relationship between average task completion latency and the number of tasks is shown. As the number of system tasks increases, the average task completion latency of each algorithm shows an upward trend. The random selection algorithm, by completely ignoring the system state, easily concentrates multiple tasks on already heavily loaded resource nodes, exacerbating resource bottlenecks. With 6 tasks, the average latency reaches as high as 480ms, becoming the performance lower limit. While the greedy selection algorithm performs reasonably well in low-load scenarios due to its instantaneous optimal decision-making, in high-concurrency scenarios, it lacks prediction of vehicle mobility and future resource competition, frequently getting trapped in local optima, causing task queuing and backlog. Its latency rapidly climbs to 410ms with 6 tasks. The DQN algorithm effectively alleviates the short-sighted decision-making problem by learning the long-term system state, and its latency control is superior to the greedy selection algorithm; however, the overestimation tendency of its Q-value function still leads to occasional suboptimal unloading choices. While DDQN mitigates the overestimation problem by decoupling action selection and value assessment, its offloading mechanism is limited by "switching and reselecting" and does not support cross-RSU data transmission. This results in insufficient flexibility in intensive task scenarios, making it difficult to cope with frequent network switching and resource fluctuations. In contrast, the algorithm of this invention maintains the lowest and most stable latency growth curve even under high load. Figure 5The relationship between average task completion latency and average vehicle speed is demonstrated. As the average vehicle speed increases, the average task completion latency of all algorithms shows an upward trend. The random selection algorithm, which completely ignores the influence of network state and mobility, experiences a sharp increase in latency with speed, reaching as high as 360ms at 80km / h, highlighting the necessity of intelligent decision-making. While the greedy selection algorithm performs reasonably well at low speeds, its short-sighted selection of RSUs or auxiliary vehicles about to disconnect at high speeds leads to a rapid deterioration in latency. Although the DQN algorithm can partially learn vehicle movement patterns, it still struggles to effectively control latency at high speeds; the DDQN algorithm, due to its passive switching mechanism, is prone to "task dangling" in frequently switching scenarios, resulting in limited performance improvement. In contrast, the algorithm of this invention maintains the lowest and most stable latency growth curve across different speeds. Figure 6 and Figure 7 The impact of the number of tasks and average vehicle speed on the average task completion rate is shown separately. Overall, the average task completion rate of all algorithms decreases with increasing task quantity or average vehicle speed. The random selection algorithm, which completely ignores environmental conditions and relies solely on random node selection, is prone to resource mismatch and communication failures in highly dynamic or high-load scenarios, resulting in the lowest completion rate. While the greedy selection algorithm performs reasonably well under simple conditions, it is limited to immediate latency optimization and lacks prediction of subsequent state evolution, making it prone to suboptimal decisions in complex scenarios, leading to task timeouts or abandonment. Although DQN and DDQN algorithms possess a basic long-term optimization framework, they lack sufficient integration of key enhancement technologies from the Rainbow series, resulting in significant shortcomings in policy stability and generalization ability, making it difficult to guarantee highly reliable service. In contrast, the algorithm of this invention consistently maintains the highest task completion rate and exhibits the slowest performance degradation. Figure 8 This paper demonstrates the cumulative reward trends of the proposed algorithm, DQN, and DDQN during training rounds, reflecting the convergence and learning stability of each algorithm. Specifically, in the early stages of training, all three algorithms are in the exploration phase, with significant fluctuations in cumulative rewards. However, the proposed algorithm quickly surpasses the other two, indicating that its integration of noisy networks, priority experience replay, and distribution modeling effectively improves exploration efficiency and sample utilization, enabling faster discovery of high-reward strategies. In contrast, DQN and DDQN, lacking these enhancement mechanisms, exhibit relatively slow and unstable learning processes. During the mid-training phase, the cumulative reward of the algorithm in this invention steadily increased with relatively small fluctuations, indicating that its strategy tended to be stable and possessed good convergence. While DQN and DDQN also showed an upward trend, their growth rates lagged significantly, and their maximum values ​​were much lower than those of the algorithm in this invention, indicating that their learning ability was limited and they failed to fully explore the potential optimal strategies in the environment. In summary, the algorithm in this invention, with its multi-dimensional algorithmic improvements, demonstrated faster convergence speed, higher cumulative reward, stronger stability, and generalization ability during training, verifying its superiority in complex vehicle network offloading decision-making tasks. The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices. Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined in this invention, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves. It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The above embodiments should be understood as illustrative only and not as limiting the scope of protection of the present invention. After reading the description of the present invention, those skilled in the art can make various alterations or modifications to the present invention, and these equivalent changes and modifications also fall within the scope defined by the claims of the present invention.

Claims

1. A computational unloading algorithm for switching scenarios in intelligent assisted driving, characterized in that, Includes the following steps:

101. The RSU collects global state parameter information within the system, including the characteristics of each perception task, the available computing power of the RSU, the local computing power of the vehicle nodes, and the remaining communication time between the vehicle and the RSU or between vehicles.

102. Input the state information collected in step 101 into the pre-trained Rainbow DQN model to perform online reasoning and decision-making, and output the optimal joint decision-making scheme. The joint decision-making scheme includes the unloading decision for each perception task, the RSU execution node selection decision, and the result feedback node allocation decision.

103. Each execution node in the system performs corresponding calculation and transmission operations according to the decision instructions issued by the RSU: If the decision is to offload the RSU at the edge, the auxiliary vehicle uploads the data to the RSU, and the RSUs work together through the back-end backhaul network to complete the data forwarding and calculation, and the RSU specified by the decision sends the result back to the requesting vehicle; if the decision is to assist the vehicle in collaborative calculation or request the vehicle to perform local calculation, the raw data or processing result is directly transmitted through the V2V link.

104. After requesting the vehicle to obtain the processing results of the collaborative perception task, make the final assisted driving decision based on the vehicle's own driving information.

2. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, Network scenarios specifically include:

201. The system focuses on two key RSUs: the current service RSU (denoted as RSU 201). ) and target switching RSU (denoted as Each RSU is equipped with an independent edge server; 202. The system contains The vehicle requesting a switchover is denoted as... ,as well as Auxiliary vehicles, denoted as ; 203. Request for vehicle Transmit information about its own blind spots to back, and Collaborate to match each requesting vehicle with a list of auxiliary vehicles that can fill its perception blind spots; 204. Request vehicle Requesting auxiliary vehicles The task of processing perceived data is represented as All perceptual tasks are represented as a set. Task characteristics are composed of quadruplets The descriptions represent the amount of raw sensing data, the amount of processed result data, the number of CPU cycles required to complete the task, and the maximum allowable processing latency for the task.

3. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, The decision to unload perceived data includes three execution methods:

301. RSU Edge Unloading: Assistance Vehicle The task Offloaded to via V2I link or The processing utilizes the computing resources of the edge server, and the result is then transmitted back to the requesting vehicle via a V2I link. ; 302. Assisted vehicle cooperative calculation: Assisted vehicle Complete the task using its own computing resources The process is performed, and the results are sent to the requesting vehicle via a V2V link. ; 303. Request local vehicle calculation: auxiliary vehicle The raw sensing data is transmitted to the requesting vehicle via a V2V link. Then utilize The vehicle's computing resources complete the task. The processing.

4. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, RSU edge offloading mode supports cross-RSU cooperative scheduling, specifically including:

401. Raw data transmission: If the auxiliary vehicle The currently connected RSU is not the same as the RSU that is performing the computation offloading task. The currently connected RSU will forward the received raw sensing data to another RSU through the backend backhaul network.

402. Result Data Transmission: If the task The process has already been completed at one of the RSUs, but a request for vehicle relocation has been made. At this point, the vehicle has moved into the coverage area of ​​another RSU. The RSU that has completed processing sends the result data through the backend backhaul network to the RSU currently connected to the requesting vehicle, which then transmits it to... ; 403. Whether data needs to be transmitted across RSUs is determined based on the following three factors: (1) Auxiliary vehicles The connected RSU is denoted as , 0 represents Connected to , 1 represents Connected to (2) Task The specific execution of RSU is denoted as , A value of 0 indicates a task. Depend on implement, A value of 1 indicates a task. Depend on Execution; (3) The final RSU of the returned task result is denoted as , A value of 0 indicates a task The results data are from Transmitted to vehicle , A value of 1 indicates a task. The results data are from Transmitted to vehicle ; 404, Task Whether the raw or resulting data needs to be transferred between RSUs is denoted as a variable. and When the variable value is 1, it means that data needs to be transferred between RSUs; otherwise, it does not. To represent the XOR operation, then and The calculation method is as follows: ； 。 5. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, The specific delay calculation methods for the three execution modes are as follows:

501. Total latency calculation in RSU edge offload mode: when task When selecting RSU edge unloading, the total latency is... It consists of five parts: auxiliary vehicle Delay in transmitting raw data to its connected RSU Delay in the transmission of raw data between RSUs The latency of RSU task computation The delay in transmitting calculation results between RSUs The calculation results are sent back to the vehicle by the RSU. latency ; The wireless transmission delay is calculated based on the transmission rate using Shannon's formula: ； ； Channel bandwidth is The transmission power is The channel gain is The noise power is The vehicle arrived RSU transmission rate RSU to vehicle transmission rate It can be calculated using Shannon's formula: ； Inter-RSU transmission delay includes queuing delay and transmission delay: ； ； The queuing delay is calculated as follows: ； RSU computation latency uses a queue-based sequential execution model: ； The final total delay is expressed as: ； 502. Total latency calculation in assisted vehicle collaborative computing mode: when the task When selecting auxiliary vehicle cooperative computing, the total latency It consists of two parts: the mission is carried out in the auxiliary vehicle. computational latency and the calculation results from Return to the requesting vehicle V2V transmission latency The calculation formula is as follows: ； ； ； in To assist vehicles The maximum computing power; among which The V2V transmission rate between the two workshops; 503. Request total latency calculation in vehicle local computing mode: when task When the option is to request local vehicle calculation, the total latency is... It consists of two parts: auxiliary vehicle Transmit raw sensor data to the requesting vehicle latency and vehicles Latency of processing tasks on the local computing unit The calculation formula is as follows: ； ； ； in To request a vehicle The maximum computing power.

6. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, The joint decision-making scheme must meet the following constraints:

601. RSU Edge Unloading Constraint: The system needs to compare vehicles. and The remaining communication time and the total delay to complete the task will determine the final decision. still Return the calculation results This is to avoid service delays or failures caused by connection interruptions during the handover process. and The remaining communication time is denoted as ,like Greater than Total delay for obtaining calculation results ,illustrate You can drive out The calculation results should be obtained within the communication range, therefore, it should be... Return to This can be expressed as the following constraint: ； like Less than Total delay for obtaining calculation results ,illustrate Unable to drive out The calculation results should be obtained within the communication range, therefore, it should be... Return to This can be expressed as the following constraint: ； 602. Assisted vehicle collaborative computation constraints: Ensure task completion delay No more than the requested vehicle With auxiliary vehicles Remaining V2V communication time between : ； 603. Request vehicle local computation constraints: Guarantee the transmission delay of raw sensing data. No more than the requested vehicle With auxiliary vehicles Remaining V2V communication time between : 。 7. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, The model for the optimization problem specifically includes:

701. Define the task Unloading decision variables These represent RSU edge offloading, auxiliary vehicle collaborative computing, and requesting vehicle local computing, respectively.

702. Define RSU execution decision variables for tasks involving RSU edge offloading. and RSU backpropagation decision variables ; 703. Constructing QoS Indicators : When the actual total processing time of the task Less than the maximum latency tolerated by the task hour, The smaller the value, the more timely the task is completed, and the higher the QoS; when the actual total processing time of the task... Greater than the maximum latency tolerated by the task When QoS is 0, it is expressed as follows: ； 704. The optimization objective is set to maximize system-level QoS, i.e., the arithmetic mean of the QoS of all tasks. The optimization jointly optimizes offload decisions, RSU execution decisions, and RSU return decisions. The optimization problem is expressed as: ； Among them, constraints C1, C2, and C3 specify the range of values ​​for each decision variable; constraints C4 and C5 indicate that during RSU edge computation, the system will... and The remaining communication time will ultimately determine the outcome. still The calculation results are sent back; constraint C6 means that when the auxiliary vehicle is performing collaborative calculations, the task completion delay must not exceed the duration of the V2V link between the two vehicles; constraint C7 means that when requesting the vehicle to perform local calculations, the transmission delay of the original perception data must not exceed the duration of the V2V link between the two vehicles.

8. The algorithm for calculating unloading in switching scenarios for intelligent assisted driving according to claim 1, characterized in that, The Rainbow DQN model used specifically includes:

801. State Space: Definition The state vector at time t is The specific meanings of each dimension are as follows: (1) This represents the feature vector of the current set of tasks to be decided, containing each task. Quadruple information That is, the amount of raw data, the amount of processed result data, the number of CPU cycles required, and the maximum tolerable latency; (2) This represents the RSU resource state vector, containing the maximum computing resources that the current serving RSU and the target RSU can provide. Task processing queue Backhaul link queue and Maximum transmission rate ; (3) Represents the vehicle state vector, containing the requesting vehicle. and Remaining communication time Requesting vehicles and auxiliary vehicles Remaining V2V communication time Assist vehicle computing resources Request vehicle local computing resources ; (4) This represents the communication channel state vector, which includes the V2I link transmission rate. and V2V link transmission rate It is calculated in real time using the Shannon formula; 802. The action space corresponds to the set of decision variables in the optimization problem, defined as follows: The action at a given moment is ,in This represents the unloading decision vector for each task; This indicates the RSU execution decision for each task; This indicates the RSU feedback decision for each task. Note that it only applies when... The value is 1, which means the task is... When unloading to RSU is performed, and Only then will it be effective; when When it is 2 or 3, and No need to consider; 803. The reward function directly reflects the quality of system decisions and needs to be closely related to the system's QoS indicators. The reward function at time step is designed as follows: ； 804. Algorithm Architecture: Integrates six improved techniques: Double DQN, Priority Experience Replay (PER), Dueling Network, Multi-step Returns, Distributional RL, and Noisy Nets; 805. Deployment method: The architecture of offline training and online inference is adopted. The model is trained using simulation data in the offline stage. In the actual operation stage, the RSU performs forward inference based on the current observation state through the pre-trained model and outputs the optimal action.

Images