QoS-Aware Service Migration in IoV Using CVX Deep Reinforcement Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the context of IoV systems, the challenge lies in optimizing service migration and resource allocation in MEC-based systems to ensure uninterrupted QoS due to vehicle mobility and limited communication coverage, while existing methods fail to effectively address the complex interplay of migration delays, communication delays, and computation delays.
Innovation Solution
A QoS-aware service migration method using CVX deep reinforcement learning (SeMiR) is proposed, which decouples the optimization problem of service migration and resource allocation into two sub-problems. An actor-critic-based asynchronous-update deep reinforcement learning method is designed for optimal service migration, and an optimal resource allocation method based on convex optimization is derived to achieve optimal resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If service instances are migrated frequently to follow vehicle movement, then QoS is improved, but system complexity and migration overhead increase
Solution Approach 1:
The patent changes the decision parameter from binary (migrate/not migrate) to probabilistic (migration probability), allowing the system to adaptively adjust migration behavior based on learned patterns of vehicle mobility and MEC resource availability, thereby improving QoS while controlling complexity
Solution Approach 2:
The patent implements feedback mechanisms where the DRL agent continuously observes system state (vehicle location, task queue, resource usage) and adjusts migration decisions based on cumulative reward signals, enabling the system to learn optimal migration strategies that balance QoS improvement against migration overhead
2Manufacturing precision
If classic optimization theories are used for joint optimization of service migration and resource allocation, then optimality is improved, but computational cost increases due to numerous iterations
Solution Approach 1:
The patent replaces classic optimization theories (mathematical programming) with deep reinforcement learning (data-driven learning), substituting iterative mathematical solvers with neural network-based policy learning that converges faster in dynamic environments while achieving near-optimal performance
Solution Approach 2:
The patent performs preliminary action by pre-training the DRL model offline to learn optimal migration and resource allocation policies, so that during online operation, decisions can be made quickly based on the pre-learned policy without requiring real-time iterative optimization
3Stability of the object's composition
If value-based DRL is used for service migration, then learning stability is improved, but learning efficiency decreases due to huge decision-making space
Solution Approach 1:
The patent segments the huge decision-making space by decomposing the joint optimization problem into two separate DRL tasks: one for service migration decisions and another for resource allocation decisions. This segmentation reduces the action space of each individual DRL agent, improving learning efficiency while maintaining stability through coordinated training
Data Source
AI summary
A QoS-aware service migration method in an IoV system based on CVX deep reinforcement learning is provided. Firstly, the optimization problem of service migration and resource allocation is decoupled into two sub-problems; then, a new actor-critic-based asynchronous-update deep reinforcement learning method is designed to explore the optimal service migration, where a delayed-update actor makes decisions of service migration and a one-step-update critic evaluates the decisions to guide the direction of the policy update; and on this basis, an optimal resource allocation method for an MEC server based on convex optimization is proposed to achieve the optimal resource allocation. According to the QoS-aware service migration method, extensive experiments are conducted based on a real-world testbed and a dataset of city vehicle trajectories, thereby verifying the feasibility and effectiveness of SeMiR. Compared with benchmark methods, the SeMiR has the best performance of service migration under various scenarios.