Method and system for vehicle-mounted MEC network service migration based on multi-unmanned aerial vehicle cooperation, electronic equipment and medium

Abstract

The application discloses a vehicle-mounted MEC network service migration method and system based on multi-unmanned aerial vehicle cooperation, electronic equipment and medium, and the method is as follows: S1: a mobile edge computing multi-unmanned aerial vehicle cooperative network model of Internet of Vehicles is constructed, a plurality of unmanned aerial vehicles carrying mobile edge computing resources are deployed in a set area, the unmanned aerial vehicles are used as air mobile edge computing nodes, and task access, task unloading and computing processing services are provided for ground vehicle users; S2: a target optimization model for reducing system time delay is established; S3: based on the target optimization model, a migration cost virtual queue is used to process long-term constraints, and the optimization target is converted into a single-time-slot optimization problem through Lyapunov optimization; S4: a vehicle short-time trajectory prediction model is constructed to obtain predicted position information of a vehicle user in a future set time slot; S5: a multi-agent partially observable decision model is constructed; and S6: a MAPPO algorithm is used to solve the single-time-slot optimization problem, and a probability distribution of service migration decisions and unmanned aerial vehicle trajectory control actions is output.

Description

Technical Field

[0001] This invention belongs to the field of UAV-assisted mobile edge computing technology, and particularly relates to a method, system, electronic device and storage medium for migrating vehicle-mounted MEC network services based on multi-UAV collaboration. Background Technology

[0002] Mobile edge computing (MEC) reduces task transmission latency and improves service response efficiency by deploying computing, storage, and network service capabilities at the network edge closer to the user. In connected vehicle scenarios, vehicle users move at high speeds, their locations change dynamically, and computing tasks arrive randomly. Services such as collaborative perception, video analytics, high-precision map updates, and intelligent driving assistance have high requirements for low latency and service continuity. Due to the limited computing power of in-vehicle terminals, relying solely on local vehicle processing is insufficient to meet actual business needs.

[0003] Existing vehicle-to-everything (V2X) mobile edge computing systems mostly rely on roadside units or cellular base stations to provide edge services. However, fixed infrastructure suffers from issues such as fixed deployment locations, limited coverage, and uneven load distribution. To improve service flexibility, drones equipped with mobile edge computing resources can serve as aerial edge servers, providing vehicle users with wireless access, compute offloading, and service migration support.

[0004] However, the high-speed movement of vehicles leads to continuous changes in the link quality and access relationship between vehicles and drones, easily causing problems such as frequent service migrations, delayed trajectory adjustments, and unstable service decisions. Furthermore, in multi-drone scenarios, service areas overlap, load coupling occurs, and migration paths are interconnected. Service migration decisions and drone trajectory control influence each other, making it difficult to achieve globally coordinated low-latency strategies by optimizing task offloading or trajectory individually. In addition, frequent service migrations incur additional communication overhead, migration latency, and resource consumption, requiring a long-term trade-off between reducing system latency and mitigating migration costs.

[0005] Therefore, how to comprehensively consider short-term vehicle trajectory changes, UAV trajectory control, service migration overhead, long-term migration cost constraints, and multi-UAV collaborative decision-making, and develop a service migration and trajectory control method that adapts to dynamic vehicle networking scenarios has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] To address the technical challenges of high-speed vehicle user movement, frequent service node changes, dynamic adjustment of drone coverage, and high difficulty in multi-drone collaborative control in vehicle-to-everything (V2X) mobile edge computing environments, this invention proposes a method, system, electronic device, and medium for migrating services in vehicle-mounted mobile edge computing (MEC) networks based on multi-drone collaboration.

[0007] This invention combines short-term vehicle trajectory prediction, service migration cost constraints, multi-UAV collaborative decision-making, and near-end policy optimization mechanisms. By estimating the future position change trend of vehicles, UAVs can adjust their service migration and trajectory control strategies in advance. At the same time, it uses a virtual queue of migration costs to constrain long-term migration overhead and achieves multi-UAV probabilistic policy collaborative training through the Multi-Agent Near-End Policy Optimization (MAPPO) algorithm, thereby improving the stability of service migration decisions and trajectory control in dynamic vehicle-to-everything (V2X) scenarios.

[0008] The present invention adopts the following technical solution:

[0009] A method for migrating vehicular MEC network services based on multi-UAV collaboration includes the following steps:

[0010] S1: Construct a multi-drone collaborative network model for vehicle-to-everything (V2X) mobile edge computing. Deploy multiple drones equipped with mobile edge computing resources in target road areas or urban hotspots. Use drones as aerial mobile edge computing nodes to provide ground vehicle users with task access, task offloading, and computing processing services. Within each time slot, vehicle users access the corresponding local drone and generate computing tasks based on their location and communication link status. The drones are interconnected through aerial links, forming a collaborative network that supports data forwarding and service status migration between drones. When a vehicle user's location changes, link quality deteriorates, or the original service drone is no longer suitable for providing services, the vehicle user's service status or computing tasks are migrated to a new service drone through inter-drone links to maintain the continuity of vehicle user services.

[0011] S2: Based on the multi-UAV collaborative network model of vehicle-to-everything (V2X) mobile edge computing in step S1, establish a target optimization model for reducing system latency. The target optimization model aims to minimize the total system latency and uses the UAV's motion trajectory and migration decision as optimization variables. According to the process of vehicle user task generation, task unloading, UAV computing processing, service node switching, and UAV location update, establish a latency model for the V2X mobile edge computing system.

[0012] S3: Based on the objective optimization model of step S2, the long-term constraints are handled by the migration cost virtual queue, and the optimization objective is transformed into a single-slot optimization problem through Lyapunov optimization.

[0013] S4: Construct a short-term trajectory prediction model for vehicles using a Long Short-Term Memory (LSTM) network. Take the historical location sequence of vehicle users as input to obtain the predicted location information of vehicle users in future set time slots. Use the predicted location information to assist subsequent service migration decisions and UAV trajectory control.

[0014] S5: Based on the drone and user status obtained in step S1, the optimization problem P2 obtained in step S3, and the vehicle user predicted location information obtained in step S4, construct a multi-agent partially observable Markov decision model.

[0015] S6: Solve the single-slot optimization problem in step S3 using the Multi-Agent Proximal Policy Optimization (MAPPO) algorithm, so that each UAV agent outputs the probability distribution of service migration decisions and UAV trajectory control actions based on the local observation decision model constructed in step S5; preferably, policy training is completed through a centralized value network, advantage function estimation and pruning policy update mechanism.

[0016] Furthermore, in step S1, the vehicle-to-everything (V2X) mobile edge computing multi-UAV collaborative network model includes a discrete time-slot operation mechanism, a vehicle user access mechanism, a UAV interconnection mechanism, and a service migration mechanism. Specifically, it operates according to discrete time slots. Within each time slot, a vehicle user generates a computing task and accesses the corresponding UAV based on its location and communication link status. UAVs maintain interconnection through air links. When a vehicle user's location changes or the original serving UAV is no longer suitable to continue providing services, the service status or computing task can be migrated through the inter-UAV link to maintain the continuity of vehicle user services.

[0017] Furthermore, in step S2, the total latency generated by the vehicle user within a time slot consists of communication transmission latency, UAV computation latency, and service migration latency. With minimizing the total system latency as the optimization objective, the vehicle user's service migration decision and the UAV trajectory control variables are used as joint optimization variables to establish a joint optimization problem for multi-UAV service migration and trajectory control in vehicle-to-everything (V2X) mobile edge computing.

[0018] Meanwhile, to avoid additional communication overhead and system resource consumption caused by frequent switching of vehicle user service nodes, service migration cost constraints are added to the optimization model, so that the system can control service migration overhead while reducing task processing latency.

[0019] The initial target optimization model is established as follows:

[0020]

[0021] Furthermore, in step S3, to address the difficulty in directly constraining service migration costs during long-term operation, a virtual migration cost queue is introduced to record the cumulative migration cost status of the system within consecutive time slots. When the service migration cost is high in a certain time slot, the corresponding virtual queue length increases; when the migration cost is lower than the set budget, the virtual queue is released. In this way, the long-term average migration cost constraint is transformed into a virtual queue stability constraint. A Lyapunov drift plus penalty function is constructed to transform the original long-term optimization problem into a time-slot-by-time optimization problem. Thus, the system only needs to make decisions based on the current vehicle state, drone state, and virtual queue state in each time slot to constrain migration costs and reduce the total system latency in the long term. The original long-term optimization problem (i.e., the initial objective optimization model P1) can be equivalently transformed into the following time-slot-by-time optimization problem (P2):

[0022]

[0023] Furthermore, in step S4, to improve the foresight of service migration and trajectory control decisions, a short-term vehicle trajectory prediction model is constructed. This model takes the location sequence of vehicle users over several past time slots as input, extracts the short-term motion change characteristics of the vehicle users, and outputs the predicted position of the vehicle users in the next time slot or several future time slots. The prediction results are used to assist subsequent UAVs in determining the potential access relationships, service migration needs, and trajectory adjustment directions of vehicle users.

[0024] Further, in step S5, the multi-agent partially observable Markov decision model includes local observations, global state, actions, and a reward function. Specifically, the local observations of each UAV include the UAV's position, the user's predicted position, and the signal-to-noise ratio; the system's global state consists of the local observations of all UAVs; actions include service migration decisions and changes in UAV position; and the reward function is constructed based on system latency and migration cost, used to guide the UAV agents to learn cooperative strategies with low latency and low migration cost.

[0025] Furthermore, in step S6, a multi-agent proximal policy optimization algorithm is used to train and solve the above-mentioned time-slot optimization problem. Each UAV agent is configured with a policy network, which takes local observations as input and outputs the probability distribution of service migration decisions and trajectory control actions.

[0026] This invention also discloses a vehicle-mounted MEC network service migration system based on multi-UAV collaboration, used to execute the above method, comprising the following modules:

[0027] The module for constructing a multi-drone collaborative network model for mobile edge computing in the Internet of Vehicles (IoV) involves building such a model. Multiple drones equipped with mobile edge computing resources are deployed within a defined area, acting as aerial mobile edge computing nodes to provide ground vehicle users with task access, task offloading, and computational processing services. Within each time slot, vehicle users access the corresponding local drone and generate computational tasks based on their location and communication link status. The drones remain interconnected via aerial links, forming a collaborative network that supports data forwarding and service state migration between drones.

[0028] Target optimization model establishment module: Based on the aforementioned multi-UAV collaborative network model for vehicle-to-everything (V2X) mobile edge computing, a target optimization model for reducing system latency is established. The target optimization model aims to minimize the total system latency, with the UAV's motion trajectory and migration decision as optimization variables. A latency model for the V2X mobile edge computing system is established based on vehicle user task generation, task unloading, UAV computation processing, service node switching, and UAV location updates.

[0029] Optimization Target Transformation Module: Based on the aforementioned target optimization model, the long-term constraints are handled using a migration cost virtual queue, and the optimization target is transformed into a single-slot optimization problem through Lyapunov optimization;

[0030] Vehicle short-term trajectory prediction model construction module: The vehicle short-term trajectory prediction model is constructed using the Long Short-Term Memory Network (LSTM). The historical location sequence of the vehicle user is used as input to obtain the predicted location information of the vehicle user in the future set time slot.

[0031] Multi-agent partially observable Markov decision model construction module: Based on the state of the drone and user, the optimization problem P2, and the vehicle user's predicted location information, a multi-agent partially observable Markov decision model is constructed.

[0032] The optimization problem-solving module utilizes the MAPPO algorithm, a multi-agent proximal policy optimization algorithm, to solve single-slot optimization problems. This allows each UAV agent to output the probability distribution of service migration decisions and UAV trajectory control actions based on the local observation decision model.

[0033] The present invention also discloses an electronic device, including a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute the above-described method or system.

[0034] The present invention also discloses a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the above-described method or system.

[0035] Compared with the prior art, the present invention has the following significant technical advantages:

[0036] This invention, through a unified design of vehicle trajectory prediction, migration cost constraints, and multi-UAV probabilistic strategy collaborative learning, can reduce the total system latency, reduce service migration overhead, and improve the stability and continuity of multi-UAV collaborative services in dynamic vehicle-to-everything (V2X) mobile edge computing scenarios. Attached Figure Description

[0037] The present invention will now be further described with reference to the accompanying drawings.

[0038] Figure 1 This is a flowchart of a method for migrating vehicle-mounted MEC network services based on multi-UAV collaboration, provided by a preferred embodiment of the present invention.

[0039] Figure 2 This is an application scenario diagram of an embodiment of the present invention.

[0040] Figure 3 This is a comparison diagram of the system reward convergence between the preferred embodiment of the present invention and the method used in the prior art.

[0041] Figure 4 This is a block diagram of a vehicle-mounted MEC network service migration system based on multi-UAV collaboration, provided by a preferred embodiment of the present invention. Detailed Implementation

[0042] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0043] like Figure 1 As shown in the figure, this embodiment presents a method for migrating vehicular MEC network services based on multi-UAV collaboration, the specific implementation of which includes the following steps:

[0044] S1: Construct a collaborative network model for vehicle-to-everything (V2X) mobile edge computing and multiple drones, specifically as follows: Figure 2As shown, multiple drones equipped with mobile edge computing resources are deployed in target road areas or urban hotspots, serving as aerial mobile edge computing nodes to provide ground vehicle users with task access, task offloading, and computing processing services. Within each time slot, vehicle users access the corresponding local drone and generate computing tasks based on their location and communication link status. The drones remain interconnected via aerial links, forming a collaborative network that supports data forwarding and service status migration between drones. When a vehicle user's location changes, link quality deteriorates, or the original serving drone is no longer suitable for providing services, the vehicle user's service status or computing tasks are migrated to a new serving drone via inter-drone links to maintain service continuity.

[0045] More specifically, in step S1 of this embodiment, the establishment of the vehicle-to-everything (V2X) mobile edge computing multi-UAV collaborative network model is as follows:

[0046] As shown in Figure 2, this embodiment considers a scenario of vehicle-to-everything (V2X) mobile edge computing multi-UAV service migration and trajectory control with trajectory prediction assistance. Multiple UAVs equipped with mobile edge computing nodes are deployed within the target city road area. These UAVs possess both wireless access capabilities and edge computing capabilities, providing task offloading, computational processing, and service migration support for ground-based mobile vehicle users.

[0047] This system adopts a discrete time slot operation mechanism. The positions of vehicle users and drones are updated at the beginning of each time slot and remain unchanged within that time slot.

[0048] Vehicle User Access Mechanism: During the vehicle user access process, the vehicle user generates a computing task at the beginning of each time slot and accesses the corresponding drone based on its current location and wireless link status. The drone currently directly connected to by the vehicle user is called the local drone, and the drone responsible for performing the vehicle user's tasks or maintaining its service status is called the service drone.

[0049] Drone interconnection mechanism: Drones are interconnected through an air link. When the local drone currently connected to by the vehicle user is different from the service drone, the task data can be uploaded to the local drone first, and then forwarded to the service drone for processing through a multi-hop link between drones.

[0050] To ensure the continuity of vehicle user services, the system has a service migration mechanism. When the movement of a vehicle user causes a degradation in the quality of the original service drone link, incompatibility of the service range, or excessive load, the system can migrate the vehicle user's task context, service instance, or related configuration information to a new service drone to ensure the continuity of vehicle user services.

[0051] Specifically, the following models are involved:

[0052] 1.1 Delay Model

[0053] This embodiment uses vehicle user task processing latency as a service quality evaluation indicator. Vehicle user In the time slot The total latency consists of communication latency, computation latency, and service migration latency, and can be expressed as:

[0054] in, These represent migration delay, communication delay, and computation delay, respectively.

[0055] 1.2 Migration Delay

[0056] When vehicle users When the service drone changes in adjacent time slots, a service state transition is required. Let... This represents the hop distance between the old drone and the new drone in the service migration path; A positive coefficient, representing the additional transmission delay per hop; To serve migration path bandwidth; The data size of the service instance. Migration latency can be defined as:

[0057]

[0058] like This indicates that no service migration has occurred and the migration delay is 0.

[0059] 1.3 Communication Delay

[0060] The vehicle user and the drone use orthogonal frequency division multiplexing (OFDM) for task uploading. Define the vehicle user. The signal-to-noise ratio between the local drone and the local drone is:

[0061]

[0062] in, Indicates user To local drones The power of the launch, The power of Gaussian white noise. For time slots drones With users The channel gain between them, whose value varies with communication distance, is defined using the free-space path loss model as:

[0063] in, This represents the channel gain at a reference distance of 1 meter. Indicates drone In the time slot Two-dimensional position coordinates, Indicates user At the two-dimensional location of this time slot, This indicates the fixed flight altitude of the drone.

[0064] According to Shannon's formula, users In the time slot To drones The uplink data transmission rate can be expressed as:

[0065]

[0066] in, Channel bandwidth, representing the user With drones The bandwidth of the communication link between them.

[0067] In the time slot ,user Unload mission data to the local drone The access latency is:

[0068]

[0069] in, user In the time slot The amount of data generated by the task This refers to the uplink transmission rate.

[0070] When the local drone and the service drone are inconsistent, mission data needs to be forwarded to the service drone via the inter-drone link. Let the number of hops between the local drone and the service drone be . The bandwidth of the backhaul link between drones is The additional return delay factor for each hop is Then the return transmission delay is:

[0071]

[0072] like This indicates that no service migration has occurred and the return delay is 0.

[0073] Communication latency consists of two parts: the access latency between the user and the local drone, and the return latency between the local drone and the service drone. Therefore, vehicle users... In the time slot The communication delay is:

[0074]

[0075] 1.4 Calculation Delay

[0076] vehicle users The task computation density is denoted as This represents the number of CPU cycles required to process each bit of task data. (Vehicle user) In the time slot The total number of CPU cycles required for the task is The computing resources required to serve drones are Then vehicle user The computation delay can be expressed as:

[0077]

[0078] in, Indicates service drones In the time slot The computational load of the tasks already undertaken.

[0079] 1.5 Migration Cost Model

[0080] Service migration can improve service continuity for vehicle users, but frequent migrations incur additional communication overhead, service state transmission overhead, and computational resource scheduling overhead. Therefore, this embodiment further establishes a migration cost model to limit frequent migrations during long-term system operation. Let the unit cost of completing one service migration be... Then vehicle user In the time slot The migration cost is defined as:

[0081]

[0082] in, As a service migration indicator variable, when users When the service drone remains unchanged, ,on the contrary, .

[0083] To suppress frequent service migrations during long-term system operation, a long-term average migration cost constraint is set:

[0084]

[0085] in, This represents the upper limit of the long-term average migration cost allowed by the system.

[0086] S2: Based on the multi-UAV collaborative network model of vehicle-to-everything (V2X) mobile edge computing in step S1, establish a target optimization model for reducing system latency. The target optimization model aims to minimize the total system latency, with the UAV's motion trajectory and migration decision as optimization variables. Based on the process of vehicle user task generation, task unloading, UAV computation processing, service node switching, and UAV location update, establish a latency model for the V2X mobile edge computing system.

[0087] Specifically, in step S2 of this embodiment, a total system latency model is established from three aspects: service migration latency, communication latency, and computation latency. The goal of this invention is to minimize the total user latency of the system while ensuring that the system migration cost does not exceed a threshold, and to minimize the drone's movement trajectory. and migration decision as optimization variables The initial target optimization model is established as follows:

[0088]

[0089] in, Indicates a collection of drones. Indicates the number of drones; Represents a set of vehicle users. Indicates the number of vehicle users; Represents a set of discrete time slots; Indicates vehicle user In the time slot Service migration decisions or service drone selection decisions; and They represent drones In the time slot Two-dimensional position coordinates; These refer to service drones and local drones, respectively. Indicates vehicle user In the time slot Total task processing latency; Indicates time slot The total system migration cost; This represents the upper limit of the long-term average migration cost allowed by the system; Indicates drone With drones In the time slot The distance; Indicates the minimum safe distance between drones; Indicates drone In the time slot The distance traveled; This represents the maximum allowable movement distance of a drone within a single time slot. Constraint C1 limits the drone selection range, meaning that service drones, local drones, and user service decisions should all be determined from the currently available drone set to avoid invalid or unexecutable service allocation results. Constraint C2 limits the average migration cost during long-term system operation to ensure it does not exceed a pre-set threshold, thereby reducing additional communication overhead, energy consumption, and resource usage caused by frequent service migrations, and ensuring the stability and feasibility of system operation; C3... ,in, and Two different drones and In the time slot The location coordinates, this constraint ensures that in any time slot This constraint is used to ensure that any two different drones always maintain a safe distance of no less than a preset safety distance, so as to avoid drones getting too close and affecting flight safety; in C4 For drones In the time slot The distance traveled, this constraint is used to limit the maximum distance the drone can travel between two consecutive time slots to no more than a threshold. This avoids situations in the model where there is a large instantaneous movement that does not match the actual flight capability.

[0090] S3: Based on the objective optimization model in step S2, the long-term constraints are handled using a migration cost virtual queue, and the optimization objective is transformed into a single-slot optimization problem through Lyapunov optimization; specifically as follows:

[0091] To address the difficulty of directly handling the long-term average migration cost constraint in problem P1, this embodiment introduces the Lyapunov optimization method, transforming the long-term constraint across time slots into an optimization objective that can be solved independently within each time slot. Specifically, for each UAV... Construct a virtual queue for migration costs This is used to characterize the cumulative deviation of the drone from migration cost constraints during long-term operation. The virtual queue is updated as follows:

[0092]

[0093] in, Indicates time slot By drone A set of users who provide services or are responsible for decision-making. Indicates time slot Chinese users In drones The migration costs generated by the side This represents the system's allowed average migration cost budget. Indicates drone In the time slot The length of the virtual queue. The larger the virtual queue value, the more likely the long-term migration cost constraint for that drone is to be overcome.

[0094] To satisfy the long-term average migration cost constraint, the virtual queue needs to remain stable during long-term operation. Therefore, the Lyapunov function is defined as follows:

[0095]

[0096] in, This represents the queue state vector composed of all virtual queues of drones. Further, the single-slot Lyapunov drift is defined as:

[0097]

[0098] Based on this, system delay is introduced as a penalty term to construct a drift plus penalty function:

[0099]

[0100] in, This indicates the total number of vehicle users. Indicates the vehicle user number. The Lyapunov penalty factor is used to adjust the trade-off between system latency and migration cost. Indicates user In the time slot The total latency. By minimizing the drift penalty function, it is possible to achieve the desired latency. While meeting long-term migration cost constraints, we should minimize the total system latency as much as possible.

[0101] For any time slot The drift plus penalty function has the following upper bound:

[0102]

[0103] in, This represents the drift plus penalty function; Indicates drone In the time slot Migration cost and virtual queue length; This represents the queue state vector composed of all virtual queues of drones; Indicates vehicle user In drones The migration costs incurred on the side; This represents the system's allowed long-term average migration cost budget; This represents the Lyapunov penalty factor, used to adjust the trade-off between system latency and migration costs; Indicates vehicle user In the time slot Total task processing latency; Represents the mathematical expectation; Represents a finite constant that is independent of the current optimization variable.

[0104]

[0105] in, This represents a finite upper bound on migration costs, used to ensure that the constant term in the Lyapunov drift upper bound is bounded.

[0106] Due to the above formula as well as Since the service migration decision and UAV trajectory decision are independent of the current time slot, they can be omitted during time-slot optimization without affecting optimality. Therefore, the original long-run optimization problem (P1) can be equivalently transformed into the following time-slot optimization problem (P2):

[0107]

[0108] in, Indicates drone In the time slot Migration cost and virtual queue length; Indicates vehicle user In drones The migration costs incurred on the side; This represents the Lyapunov penalty factor, used to adjust the trade-off between system latency and migration costs; Indicates vehicle user In the time slot Total task processing latency.

[0109] That is, within each time slot, the system jointly optimizes service migration decisions based on the current virtual queue status, user status, and drone location. and drone trajectories This enables low-latency online control under conditions where long-term migration costs are limited.

[0110] S4: A short-term vehicle trajectory prediction model is constructed using a Long Short-Term Memory (LSTM) network. The historical location sequence of the vehicle user is used as input to obtain the predicted location information of the vehicle user in a future set time slot. This predicted location information is then used to assist in subsequent service migration decisions and UAV trajectory control; specifically as follows:

[0111] To enhance the ability of service migration decisions to perceive vehicle mobility, this embodiment constructs a vehicle trajectory prediction model based on a stacked long short-term memory network to predict the future location of vehicles. Details are as follows:

[0112] Assume user vehicle In the time slot The motion state is represented as:

[0113] , ,

[0114] in, This indicates the length of the vehicle's historical trajectory input sequence, i.e., the number of historical time slots used for trajectory prediction. Representing users respectively In the time slot Two-dimensional coordinates, and Representing users respectively In the time slot Given the velocity, direction, and acceleration, the corresponding predicted output is expressed as:

[0115]

[0116] in, In time slot user Predicted coordinates.

[0117] in, Representing users respectively Two-dimensional coordinates for the predicted next time slot.

[0118] S5: Based on the drone and user status obtained in step S1, the optimization problem P2 obtained in step S3, and the vehicle user predicted location information obtained in step S4, construct a multi-agent partially observable Markov decision model.

[0119] Specifically, in step S5 of this embodiment, a partially observable Markov decision process for trajectory prediction assistance is constructed. Since each UAV can only obtain local state information within its coverage area or neighborhood, this embodiment treats each UAV as an agent and models the service migration and trajectory control process as a partially observable multi-agent decision problem.

[0120] 1) Observation

[0121] In the time slot drones The local observation is represented as:

[0122]

[0123] in, For drones In the time slot Location coordinates, For users In the time slot The predicted location coordinates of the next time slot. For users With drones In the time slot signal-to-noise ratio, For drones The collection of vehicles covered.

[0124] 2) Status

[0125] The system's global state is jointly constituted by the observation information from all UAVs, that is:

[0126]

[0127] in, Indicates a collection of drones. This indicates the number of drones.

[0128] 3) Actions

[0129] At every moment drones Select an action from its action space according to the current strategy. This is used to update its service targets and locations. This action can be represented as:

[0130]

[0131] in, In time slot drones Service migration decisions covering all vehicles For drones In the time slot The change in position.

[0132] 4) Reward Function

[0133] Based on the time-slot optimization objective obtained in step S3, the reward function is defined as:

[0134]

[0135] in, Indicates drone In the time slot The rewards received Indicates drone In the time slot The set of users of the service.

[0136] This reward function is used to guide drones to reduce system latency while minimizing service migration overhead.

[0137] S6: The Multi-Agent Proximal Policy Optimization (MAPPO) algorithm is used to solve the time-slot-wise deterministic optimization problem P2 obtained in step S3. This allows each UAV agent to output the probability distribution of service migration decisions and UAV trajectory control actions based on the local observation decision model constructed in step S5. Policy training is then completed through a centralized value network, advantage function estimation, and a pruning policy update mechanism. Details are as follows:

[0138] This embodiment employs a multi-agent near-field policy optimization algorithm to solve the service migration decision and UAV trajectory control problems. Each UAV... It is treated as an intelligent agent and configured with a corresponding policy network. The policy network uses the drone as an example. Using local observations as input, the output is the probability distribution of the UAV's actions in the current time slot.

[0139] drones The policy network is represented as:

[0140]

[0141] in, Indicates drone The strategy network parameters, Indicates drone In the time slot Local observations Indicates drone In the time slot The action chosen.

[0142] The drone obtains its current action by sampling the probability distribution output by the policy network:

[0143]

[0144] All drones in the time slot The combined action is represented as:

[0145]

[0146] Among them, joint actions It also determines the service migration results for vehicle users and the trajectory control results for drones.

[0147] During the training phase, a centralized value network is constructed to estimate the value of the current state based on the global state of the system. The centralized value network is represented as follows: ,in, Represents the parameters of the value network. This represents the system's global state. This global state is only used for value assessment during the training phase; during actual execution, each UAV generates actions based solely on its own local observations.

[0148] During training, the system collects trajectory data generated by the interaction between multiple UAVs and the environment. This trajectory data is represented as follows:

[0149]

[0150] in, Indicates time slot The global reward function. Based on the slot-by-slot optimization objective obtained in step S3, the reward function is defined as:

[0151]

[0152] The reward function takes into account both the virtual queue of migration costs and the latency of user task processing, making the drone agent more inclined to choose service migration and trajectory control actions with low latency and low migration costs during the learning process.

[0153] Calculate the time-series difference error based on the centralized value network:

[0154]

[0155] in, This represents the discount factor.

[0156] Further calculation of the advantage function:

[0157]

[0158] in, This represents a discount factor in reinforcement learning, used to measure the impact of future rewards on the current policy update. This represents the time step accumulation index in the advantage estimation. Represents the advantage estimation parameters. It is used to measure the relative merit of the current action compared to the average value of the current state.

[0159] To limit the magnitude of policy updates, define the probability ratio between the current policy and the old policy:

[0160]

[0161] in, This represents the ratio of the probability of the new strategy to the probability of the old strategy for drone v in time slot t. Indicates the current policy network parameters. This represents the policy network parameters before the update.

[0162] Based on the probability ratio and the dominance function, construct the objective function for the pruning strategy:

[0163]

[0164] in, For trimming parameters, This represents the pruning function, used to limit the magnitude of change between the old and new policies, preventing large single policy updates from causing training instability.

[0165] The value network loss function is defined as:

[0166]

[0167] in, Indicates from time slot The initial cumulative return from the discount is defined as:

[0168]

[0169] In each training round, the parameters of each UAV policy network are updated by maximizing the objective function of the pruning policy. And update the parameters of the centralized value network by minimizing the value network loss function. After multiple rounds of trajectory acquisition and network parameter updates, a converged multi-UAV service migration and trajectory control strategy was obtained.

[0170] After training, the system enters the online execution phase. At this point, each drone only needs to rely on its own local observations. By outputting the action probability distribution through the policy network and sampling the service migration decision and trajectory control actions, centralized training and distributed execution of multi-UAV systems can be achieved.

[0171] The existing DQN benchmark schemes are compared with the method proposed in this invention, and the performance is evaluated using numerical results:

[0172] DQN: Each drone independently learns its own task scheduling and movement strategies using a deep Q network, and there is no explicit collaboration between agents.

[0173] Simulation results are as follows Figure 3 Compared to the baseline scheme, the method proposed in this invention ultimately achieves a superior system reward. The MAPPO method of this invention exhibits a faster convergence speed in the early stages of training and a more stable overall training process. In contrast, existing DQN-based methods ultimately converge to a significantly lower reward level and exhibit more pronounced fluctuations during training.

[0174] In summary, this invention discloses a service migration method for vehicular MEC networks based on multi-UAV collaboration. The general steps include: S1: establishing a multi-UAV collaborative service network model for vehicular network mobile edge computing; S2: establishing a joint optimization model with the goal of reducing total system latency and service migration decision and UAV trajectory control as optimization variables; S3: introducing a virtual queue for migration costs and transforming the long-term average migration cost constraint into a time-slot optimization problem through Lyapunov optimization; S4: constructing a short-term trajectory prediction model for vehicles using a Long Short-Term Memory (LSTM) network and obtaining the predicted location information of vehicle users in future time slots using the historical location sequence of vehicle users as model input; S5: constructing a multi-agent partially observable Markov decision process by combining UAV location, predicted user location, and signal-to-noise ratio; S6: solving the time-slot optimization problem using a multi-agent near-end policy optimization algorithm, enabling UAV agents to generate service migration and trajectory control action probability distributions based on local observations, and completing policy training through a centralized value network, advantage function estimation, and pruning policy update mechanism. This invention, through a unified design of vehicle trajectory prediction, migration cost constraints, and multi-UAV probabilistic strategy collaborative learning, can reduce the total system latency, reduce service migration overhead, and improve the stability and continuity of multi-UAV collaborative services in dynamic vehicle-to-everything (V2X) mobile edge computing scenarios.

[0175] like Figure 4 As shown, this embodiment discloses a vehicle-mounted MEC network service migration system based on multi-UAV collaboration, used to execute the above method, including the following modules:

[0176] The module for constructing a multi-drone collaborative network model for mobile edge computing in the Internet of Vehicles (IoV) involves building such a model. Multiple drones equipped with mobile edge computing resources are deployed within a defined area, acting as aerial mobile edge computing nodes to provide ground vehicle users with task access, task offloading, and computational processing services. Within each time slot, vehicle users access the corresponding local drone and generate computational tasks based on their location and communication link status. The drones remain interconnected via aerial links, forming a collaborative network that supports data forwarding and service state migration between drones.

[0177] Target optimization model establishment module: Based on the aforementioned multi-UAV collaborative network model for vehicle-to-everything (V2X) mobile edge computing, a target optimization model for reducing system latency is established. The target optimization model aims to minimize the total system latency, with the UAV's motion trajectory and migration decision as optimization variables. A latency model for the V2X mobile edge computing system is established based on vehicle user task generation, task unloading, UAV computation processing, service node switching, and UAV location updates.

[0178] Optimization Target Transformation Module: Based on the aforementioned target optimization model, the long-term constraints are handled using a migration cost virtual queue, and the optimization target is transformed into a single-slot optimization problem through Lyapunov optimization;

[0179] Vehicle short-term trajectory prediction model construction module: The vehicle short-term trajectory prediction model is constructed using the Long Short-Term Memory Network (LSTM). The historical location sequence of the vehicle user is used as input to obtain the predicted location information of the vehicle user in the future set time slot.

[0180] Multi-agent partially observable Markov decision model construction module: Based on the state of the drone and user, the optimization problem P2, and the vehicle user's predicted location information, a multi-agent partially observable Markov decision model is constructed.

[0181] The optimization problem-solving module utilizes the MAPPO algorithm, a multi-agent proximal policy optimization algorithm, to solve single-slot optimization problems. This allows each UAV agent to output the probability distribution of service migration decisions and UAV trajectory control actions based on the local observation decision model.

[0182] Other aspects of this embodiment can be found in the above method embodiments.

[0183] A preferred embodiment of the present invention also discloses an electronic device, including a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the above-described method or system.

[0184] A preferred embodiment of the present invention also discloses a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute the above-described method or system.

[0185] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for migrating vehicular MEC network services based on multi-UAV collaboration, characterized in that, Includes the following steps: S1: Construct a multi-drone collaborative network model for vehicle-to-everything (V2X) mobile edge computing. Deploy multiple drones equipped with mobile edge computing resources within a designated area. Use the drones as aerial mobile edge computing nodes to provide task access, task offloading, and computing processing services to ground vehicle users. Within each time slot, vehicle users access the corresponding local drone and generate computing tasks based on their location and communication link status. The drones are interconnected through aerial links, forming a collaborative network that supports data forwarding and service state migration between drones. S2: Based on the multi-UAV collaborative network model of vehicle-to-everything (V2X) mobile edge computing in step S1, establish a target optimization model for reducing system latency. The target optimization model aims to minimize the total system latency and uses the UAV's motion trajectory and migration decision as optimization variables. Based on vehicle user task generation, task unloading, UAV computing processing, service node switching, and UAV location updates, establish a latency model for the V2X mobile edge computing system. S3: Based on the objective optimization model of step S2, the long-term constraints are handled by the migration cost virtual queue, and the optimization objective is transformed into a single-slot optimization problem through Lyapunov optimization. S4: Use the Long Short-Term Memory (LSTM) network to build a short-term trajectory prediction model for vehicles. Take the historical location sequence of vehicle users as input to obtain the predicted location information of vehicle users in the future set time slots. S5: Based on the drone and user status obtained in step S1, the optimization problem obtained in step S3, and the vehicle user predicted location information obtained in step S4, construct a multi-agent partially observable Markov decision model. S6: The MAPPO algorithm is used to solve the single time slot optimization problem in step S3 by optimizing the multi-agent near-end strategy, so that each UAV agent outputs the probability distribution of service migration decision and UAV trajectory control action according to the local observation decision model constructed in step S5.

2. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 1, characterized in that, In step S1, the vehicle-to-everything (V2X) mobile edge computing multi-drone collaborative network model operates in discrete time slots. In each time slot, vehicle users generate computing tasks and access the corresponding drones according to their location and communication link status. The drones are interconnected through air links. When the location of a vehicle user changes or the original service drone is no longer suitable to continue providing services, the service status or computing task is migrated through the inter-drone link to maintain the continuity of vehicle user services.

3. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 2, characterized in that, In step S2, the total latency generated by the vehicle user in a time slot consists of communication transmission latency, UAV computing latency, and service migration latency. With minimizing the total system latency as the optimization objective, the service migration decision of the vehicle user and the motion trajectory control variables of the UAV are used as joint optimization variables to establish a joint optimization problem of multi-UAV service migration and trajectory control in vehicle-to-everything (V2X) mobile edge computing. At the same time, service migration cost constraints are added to the target optimization model; The initial target optimization model is established as follows: 。 4. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 3, characterized in that, In step S3, a virtual migration cost queue is introduced to record the cumulative migration cost status of the system in consecutive time slots; a Lyapunov drift plus penalty function is constructed to transform the original long-run optimization problem into a time-slot-by-time optimization problem; thus, in each time slot, decisions only need to be made based on the current vehicle state, UAV state, and virtual queue state to constrain migration costs and reduce the total system latency in the long run; the original long-run optimization problem P1 is equivalently transformed into the following time-slot-by-time optimization problem P2: 。 5. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 4, characterized in that, In step S4, the vehicle short-term trajectory prediction model takes the location sequence of the vehicle user in the past several time slots as input, extracts the short-term motion change characteristics of the vehicle user, and outputs the predicted position of the vehicle user in the next time slot or in the future several time slots.

6. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 5, characterized in that, In step S5, the multi-agent partially observable Markov decision model includes local observations, global state, actions, and a reward function. The local observations of each drone include the drone's position, the user-predicted position, and the signal-to-noise ratio. The global state consists of the local observations of all drones. Actions include service migration decisions and changes in drone position. The reward function is constructed based on system latency and migration cost, and is used to guide the drone agents to learn cooperative strategies with low latency and low migration cost.

7. The method for migrating vehicular MEC network services based on multi-UAV collaboration as described in claim 6, characterized in that, In step S6, a multi-agent near-end policy optimization algorithm is used to train and solve the time-slot optimization problem; each UAV agent is configured with a policy network, which takes local observations as input and outputs the probability distribution of service migration decisions and trajectory control actions.

8. A vehicle-mounted MEC network service migration system based on multi-UAV collaboration, used to perform the method as described in any one of claims 1-7, characterized in that, Includes the following modules: The module for constructing a multi-drone collaborative network model for mobile edge computing in the Internet of Vehicles (IoV) involves building such a model. Multiple drones equipped with mobile edge computing resources are deployed within a defined area, acting as aerial mobile edge computing nodes to provide task access, task offloading, and computational processing services to ground vehicle users. Within each time slot, vehicle users access the corresponding local drone and generate computational tasks based on their location and communication link status. The drones remain interconnected via aerial links, forming a collaborative network that supports data forwarding and service state migration between drones. Target optimization model establishment module: Based on the aforementioned multi-UAV collaborative network model for vehicle-to-everything (V2X) mobile edge computing, a target optimization model for reducing system latency is established. The target optimization model aims to minimize the total system latency, with the UAV's motion trajectory and migration decision as optimization variables. A latency model for the V2X mobile edge computing system is established based on vehicle user task generation, task unloading, UAV computation processing, service node switching, and UAV location updates. Optimization Target Transformation Module: Based on the aforementioned target optimization model, the long-term constraints are handled using a migration cost virtual queue, and the optimization target is transformed into a single-slot optimization problem through Lyapunov optimization; Vehicle short-term trajectory prediction model construction module: The vehicle short-term trajectory prediction model is constructed using the Long Short-Term Memory Network (LSTM). The historical location sequence of the vehicle user is used as input to obtain the predicted location information of the vehicle user in the future set time slot. Multi-agent partially observable decision model construction module: Based on the state of the UAV and user, the optimization problem, and the predicted location information of the vehicle user, a multi-agent partially observable Markov decision model is constructed. The optimization problem-solving module utilizes the MAPPO algorithm, a multi-agent proximal policy optimization algorithm, to solve single-slot optimization problems. This allows each UAV agent to output the probability distribution of service migration decisions and UAV trajectory control actions based on the local observation decision model.

9. An electronic device, characterized in that, The system includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to execute the method as claimed in any one of claims 1-7 or the system as claimed in claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the method of any one of claims 1-7 or the system of claim 8.

Images