Unmanned aerial vehicle edge intelligence service migration and resource allocation method and device

Abstract

This invention provides a method and apparatus for migrating and allocating resources for edge intelligent services on unmanned aerial vehicles (UAVs). The method includes steps of large-time-slice service migration planning and small-time-slice resource allocation. The large time-slice includes multiple small time-slices. In the large-time-slice service migration planning step: a macro-state input vector, a first starting coordinate vector, and a service deployment vector are constructed to form a first state vector, which is input into a first reinforcement learning model. The first reinforcement learning model outputs a first action vector, which includes a data migration vector, comprising adapter migration data and base model migration data. In the small-time-slice resource allocation step: a second state vector is constructed, and a second reinforcement learning model outputs a second action vector, which includes the allocation of resources in the server cluster. This solution ensures real-time service response through small-time-slice resource allocation and avoids unnecessary frequent migrations through large time-slices.

Description

Technical Field

[0001] This invention relates to the field of mobile communication technology, and in particular to a method and apparatus for migrating and allocating edge intelligent services and resources for unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rise of the low-altitude economy, drone applications are evolving from simple aerial photography and mapping to complex tasks such as urban security and disaster relief. To cope with dynamic environments, drones need to be equipped with intelligent services built on deep neural networks (DNNs). These intelligent services not only include inference logic code, but more importantly, they contain massive model parameter files (such as the weight matrices of Transformer or CNN networks, typically reaching gigabyte levels) and specific deep learning inference frameworks. Due to the limited energy consumption and computing power of airborne hardware, the mainstream architecture adopts a mobile edge computing model. This means that the aforementioned intelligent agent model containing all parameters is deployed on a ground-based edge server, receiving drone perception data via a wireless network, performing inference calculations, and returning control commands. This architecture offloads computationally intensive tasks.

[0003] During long-distance drone flights, as the drone leaves the coverage area of ​​the current base station, the aforementioned intelligent services must be migrated from the source edge server to the target edge server to maintain low latency and service continuity. The closest existing implementation typically employs general virtual machine and container online migration technology. The specific process involves the system treating the entire container environment running the intelligent agent (including the base model and adapter) as an indivisible monolithic object and packaging it as a whole. This general migration solution has significant technical flaws when handling AI services: unlike ordinary lightweight applications such as web services, the core carrier of AI services is a massive amount of model parameters. Existing solutions ignore the structured decomposition of AI model parameters, such as their potential to be divided into general feature extraction layers and task-specific layers, and still use a "full-package transmission" mode. This results in each migration requiring the transmission of redundant GB-level data through limited backhaul links, causing excessively long service interruptions and potentially leading to collisions due to loss of control during handover intervals. It also results in a significant waste of edge network bandwidth resources. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a method for migrating and allocating resources for edge intelligent services of unmanned aerial vehicles (UAVs) to eliminate or improve one or more defects existing in the prior art.

[0005] One aspect of the present invention provides a method for migrating and allocating edge intelligent services and resources for unmanned aerial vehicles (UAVs). The method includes steps such as large time-slice service migration planning and small time-slice resource allocation, wherein the large time-slice comprises multiple small time-slices. In the steps of the large time-slice service migration planning: For each adapter in the server cluster, calculate the popularity ratio and construct a macro-state input vector based on the popularity ratio of each adapter. The starting geographic coordinates of each drone in the drone cluster are constructed based on the server connected to at the start time of the current time slice, and a first starting coordinate vector is constructed based on the starting geographic coordinates of each drone. Based on the deployment location of each base model throughout the server cluster, a service deployment vector is constructed; The macroscopic state input vector, the first starting coordinate vector, and the service deployment vector are used to construct a first state vector. The first state vector is input into a first reinforcement learning model. The first reinforcement learning model outputs a first action vector, which includes a service migration node vector and a data migration vector. The data migration vector includes adapter migration data and base model migration data. In the step of allocating small time-slice resources: A second state vector is constructed based on the small-scale fading channel gain, the coordinates of the UAV in a small time slice, the computing resources required for each UAV's processing task, and the service migration node vector of the previous large time slice. The second state vector is input into the second reinforcement learning model, and the second reinforcement learning model outputs a second action vector, which includes the resource allocation in the server cluster.

[0006] By adopting the above scheme, this scheme simplifies the migration object from a "full model" to "lightweight adapter parameters" by pre-setting a shared base model on the edge side. Only differentiated data during agent operation is transmitted. If there is no lack of base model migration data, the amount of base model migration data in the data migration vector is 0. To address the system instability caused by existing single-scale decision-making, this scheme further constructs a dual-time-scale optimization framework. By decoupling low-frequency migration decisions from high-frequency resource allocation decisions in time and using a super-gradient-based reinforcement learning algorithm to handle hierarchical dependencies, this scheme can ensure millisecond-level real-time service response of UAVs through small time-slice resource allocation, and avoid unnecessary frequent migrations over long periods through large time-slice service migration planning, thus achieving the optimal balance of long-term system benefits.

[0007] In some embodiments of the present invention, in the step of calculating the popularity ratio for each adapter in the server cluster, the popularity ratio corresponding to the adapter is calculated using the following formula: in, Indicates adapter k The popularity ratio Indicates adapter k The number of times drones were used in the previous large time slice, U This indicates the total number of drones.

[0008] In some embodiments of the present invention, in the step of constructing a service deployment vector based on the deployment location of each base model in the entire server cluster, a binary deployment matrix is ​​constructed based on the server location where each base model is deployed, and the service deployment vector is obtained by expanding the binary deployment matrix.

[0009] In some embodiments of the present invention, both the first reinforcement learning model and the second reinforcement learning model employ a DQN policy network.

[0010] In some embodiments of the present invention, the steps of the large time-slice service migration planning further include training a first reinforcement learning model. In the first reinforcement learning model training step, migration delay is calculated based on a global data migration vector; data freshness is calculated based on the updated data sent by the UAV to the connected server; and a first reward function is calculated based on the migration delay and data freshness.

[0011] In some embodiments of the present invention, in the step of calculating the first reward function based on the migration delay and data freshness, the first reward function is calculated using the following formula: in, Indicates a large time slice The first reward function, Indicates a large time slice The sum of the migration delays of each drone. The weight parameters represent the migration delay. U This indicates the total number of drones. Indicates drone In a short time slice Data freshness, The weighting parameter represents the freshness of the data.

[0012] In some embodiments of the present invention, in the step of calculating migration delay based on global data migration vector, it is determined whether data migration exists for each UAV based on the service migration node vector. If data migration exists for a UAV, the migration delay corresponding to each UAV is calculated using the following formula: in, Indicates a large time slice drones The corresponding migration delay, Indicates the drone migration start server To the migration destination server Number of jumps between; drones The server to be migrated to Total amount of data; This indicates the data transmission speed of the wired channel between servers.

[0013] In some embodiments of the present invention, in the step of calculating data freshness based on the updated data sent by the drone to the connected server, the data freshness is calculated using the following formula: in, Indicates drone In a short time slice Data freshness; Indicates drone In a short time slice Total time consumed for updating data; Indicates drone In a short time slice Data freshness; The length of a small time slice.

[0014] In some embodiments of the present invention, in the step of calculating the total time consumed for updating data for each time slice of each UAV, the sum of the wireless transmission delay, wired transmission delay and calculation delay of the UAV time slice is calculated as the total time consumed for updating data. The wireless transmission delay is calculated using the following formula: The wired transmission delay is calculated using the following formula: The computation delay is calculated using the following formula: in, Indicates that the drone is in a small time slice wireless transmission latency, Indicates that the drone is in a small time slice Wired transmission latency, Indicates that the drone is in a small time slice computational delay, Indicates drone Update data size, Indicates drone The wireless transmission rate with the connected migration start server, Indicates the drone migration start server To the migration destination server Number of jumps between, This indicates the data transmission speed of the wired channel between servers. Indicates the server currently connected to the drone. The calculation proportion allocated in the previous large time slice, Indicates the server currently connected to the drone. Total computational load.

[0015] In some embodiments of the present invention, in the step of constructing a second state vector based on small-scale fading channel gain, the coordinates of the UAV in a small time slice, the computing resources required for the processing task of each UAV, and the service migration node vector of the previous large time slice, a gain vector is constructed based on the small-scale fading gain of each UAV and the server connected in the previous small time slice, a server connection vector is constructed based on the server connected to the UAV in the previous small time slice, and a resource demand vector is constructed based on the computing resources required for the processing task of each UAV. The gain vector, the server connection vector, the resource demand vector, and the service migration node vector of the previous large time slice are combined to construct the second state vector.

[0016] In some embodiments of the present invention, the second action vector is composed of a communication bandwidth allocation matrix and a computing resource allocation matrix. The communication bandwidth allocation matrix is ​​configured with a bandwidth allocation ratio value for each channel in the server cluster, and the computing resource allocation matrix is ​​configured with a computing ratio value for each server in the server cluster.

[0017] A second aspect of the present invention also provides an apparatus for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs). The apparatus includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the apparatus implements the steps of the method described above.

[0018] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned UAV edge intelligent service migration and resource allocation method.

[0019] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the text, or may be learned by practice of the invention. The objects and other advantages of the invention will become apparent from the description and the accompanying drawings.

[0020] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0021] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0022] Figure 1 This is a schematic diagram illustrating one embodiment of the UAV edge intelligent service migration and resource allocation method of the present invention; Figure 2 This is a schematic diagram illustrating the implementation scenario of this solution; Figure 3 This is a schematic diagram of the processing flow of this solution; Figure 4 This is a schematic diagram of the spatial decoupling migration mechanism based on LoRA in this scheme; Figure 5 This is a diagram illustrating the time delay of this solution; Figure 6 This is a diagram illustrating the age changes in the information provided in this plan; Figure 7 This is a schematic diagram of the joint processing of the first reinforcement learning model and the first reinforcement learning model in this scheme. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0024] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0025] While fine-tuning techniques such as Low-Rank Adaptation (LoRA) have emerged in the field of large models, existing migration architectures have not yet applied such techniques to data reduction in service space migration.

[0026] Furthermore, besides the aforementioned spatial data transmission issues, existing UAV edge service migration schemes also suffer from severe scale mismatch defects in their temporal decision-making mechanisms. In practical UAV edge networks, service migration decisions are typically macroscopic, long-cycle behaviors, and frequent migrations lead to high migration costs. In contrast, communication resource allocation and UAV flight control are microscopic, short-cycle behaviors, requiring millisecond-level responses to channel changes. However, existing optimization frameworks, such as traditional single-timescale Markov decision processes, often ignore this hierarchical temporal dependency. Existing technologies often combine low-frequency migration decisions with high-frequency resource allocation decisions in a single optimization problem for joint solution.

[0027] Existing service migration solutions typically treat the agent service as an indivisible monolithic object, employing a full migration model. This means that regardless of whether the target server already has the necessary basic operating environment, the system forcibly transfers the operating system image, dependency libraries, and the entire gigabyte-scale AI model parameters. This approach ignores the decoupling characteristic of the AI ​​model parameters themselves—the "general-purpose base + task-specific adapter"—leading to the repeated transmission of a large amount of redundant data in the backhaul link. This not only results in extremely high migration latency, severely impacting the safety of real-time tasks such as drone obstacle avoidance, but also leads to the excessive consumption of limited bandwidth resources in edge networks.

[0028] Furthermore, existing work often overlooks the significant differences in time scale between low-frequency service migration decisions and high-frequency resource allocation decisions, typically placing these two disparate decision variables within the same single-time-scale optimization framework for joint solution. This approach has two major fatal flaws: first, it leads to an exponential expansion of the state space and action space, causing a dimensionality explosion that makes it difficult for optimization algorithms to converge quickly in large-scale networks; second, due to the lack of a long-term planning perspective, the system is highly susceptible to getting trapped in local optima, such as frequently triggering high-cost migration operations to meet instantaneous communication quality fluctuations, resulting in a significant decrease in overall system energy efficiency and service stability.

[0029] In view of the above-mentioned deficiencies in the existing technology, the purpose of this invention is to provide a method for migrating intelligent agent services in UAV edge networks based on spatiotemporal decoupling, mainly to solve the following technical problems: To address the issue of high overhead in existing full migration methods, this invention proposes a modular migration mechanism based on LoRA. By pre-setting a shared base model at the edge, the migration object is simplified from a "full model" to "lightweight adapter parameters," transmitting only the differentiated data during agent runtime. This method aims to eliminate redundant data transmission and spatially decouple the black-box agent service model, thereby significantly reducing migration bandwidth consumption and service interruption duration.

[0030] To address the system instability caused by existing single-scale decision-making, this invention constructs a dual-time-scale optimization framework based on Stackelberg game theory. By decoupling low-frequency migration decisions from high-frequency resource allocation decisions in time, and utilizing a super-gradient-based reinforcement learning algorithm to handle hierarchical dependencies, this method aims to achieve an optimal balance between ensuring millisecond-level real-time service response from UAVs and avoiding unnecessary frequent migrations over long periods, thus maximizing the long-term benefits of the system.

[0031] Figure 2 The diagram illustrates a scenario for migrating drone edge intelligent services. The system architecture is logically divided into a cloud control layer, an edge computing layer, and a user terminal layer. Each layer works collaboratively through wired or wireless links to support the operation of mobile edge intelligent services for drones.

[0032] A cloud server is deployed at the top level of the system. As the macro-control and training center of the entire network, the cloud server possesses massive computing and storage resources. In this embodiment, the cloud server is primarily responsible for pre-training large-scale deep neural networks and extracting "base models" (i.e., the orange cubes in the diagram) with general feature capabilities using knowledge distillation techniques. During the system initialization phase, the cloud server distributes and deploys these base models to all edge servers in the network via the core network, thereby achieving full network pre-deployment of the base models at the edge. This pre-deployment mechanism forms the basis for subsequent module reuse.

[0033] At the ground infrastructure layer, the system comprises edge servers distributed across different areas. Each edge server is equipped with corresponding computing and bandwidth resources to support intelligent services for drone unloading. High-speed backhaul links are established between adjacent edge servers via wired connections, forming the physical channel for data interaction. Furthermore, since each edge server pre-stores the same base model, there is no need to repeatedly transmit this base model data via the backhaul link during service migration interactions between servers.

[0034] At the user layer, the system includes multiple drone terminals. The drones establish communication links with edge servers within their current coverage area via wireless connections and upload data, sending images or video streams collected by onboard sensors to the edge for real-time inference. In this embodiment, the intelligent services requested by the drones adopt a modular structure, consisting of a general-purpose base model deployed at the edge and task-specific adapter modules (i.e., the colored squares in the diagram).

[0035] In actual operation, the drone will move randomly within the area, such as... Figure 2The black dashed single-arrow line indicates the drone's trajectory. Drone 1 is moving from the coverage area of ​​edge server 1 to the coverage area of ​​edge server 3. As drone 1 gradually flies away from edge server 1 and enters the coverage area of ​​edge server 3, the system will trigger the service migration mechanism. At this time, the source node sends only the differentiated adapter modules required for drone 1 to run to the target node via a wired backhaul link. After receiving the adapter modules, the target node loads and assembles them with the locally pre-stored base model, quickly reconstructing intelligent service 1 for drone 1. This process avoids the transmission of all model parameters, significantly reducing migration overhead.

[0036] like Figure 1 and 3 As shown, this invention proposes a method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs). The method includes steps such as large-time-slice service migration planning and small-time-slice resource allocation, wherein the large time slice comprises multiple small time slices. In the steps of the large time-slice service migration planning: Step S110: Calculate the popularity ratio for each adapter in the server cluster, and construct a macro-state input vector based on the popularity ratio of each adapter. Step S120: Construct the starting geographic coordinates of each drone in the drone cluster to the server connected to at the start time of the current time slice, and construct the first starting coordinate vector based on the starting geographic coordinates of each drone. Step S130: Based on the deployment location of each base model in the entire server cluster, construct the service deployment vector; Step S140: Construct a first state vector from the macroscopic state input vector, the first starting coordinate vector, and the service deployment vector; input the first state vector into the first reinforcement learning model; the first reinforcement learning model outputs a first action vector; the first action vector includes a service migration node vector and a data migration vector; the data migration vector includes adapter migration data and base model migration data. In specific implementation, the first initial coordinate vector is defined as a vector with dimension 1. real matrix (where This represents the total number of drones in the system, where 3 indicates that the drones' position coordinates are in three dimensions. ; In practical implementation, the service deployment vector is defined as a dimension of binary matrix ( This represents the total number of drones in the system. This is because there is a one-to-one correspondence between intelligent services and drones; each drone is configured with one intelligent service to provide intelligent decision-making. (representing the total number of base stations in the system), denoted as .

[0037] To address the issues of backhaul link congestion and excessively long service interruption times caused by "full model migration" in existing technologies, a modular decoupling and differentiated migration mechanism based on LoRA is proposed. This mechanism achieves efficient migration by "trading storage for bandwidth" by physically decoupling the massive intelligent service into a shared base model and scalable adapter modules.

[0038] In the step of allocating small time-slice resources: Step S210: Construct a second state vector based on the small-scale fading channel gain, the coordinates of the UAV in the small time slice, the computing resources required for the processing task of each UAV, and the service migration node vector of the previous large time slice. Input the second state vector into the second reinforcement learning model. The second reinforcement learning model outputs a second action vector, which includes the resource allocation in the server cluster.

[0039] The above-mentioned scheme simplifies the migration object from a "full model" to "lightweight adapter parameters" by pre-setting a shared base model on the edge side. Only differentiated data during agent operation is transmitted. If there is no missing base model migration data, the amount of base model migration data in the data migration vector is 0. To address the system instability caused by existing single-scale decision-making, this scheme further constructs a dual-time-scale optimization framework. By decoupling low-frequency migration decisions from high-frequency resource allocation decisions in time, and using a super-gradient-based reinforcement learning algorithm to handle hierarchical dependencies, this method aims to ensure millisecond-level real-time service response of UAVs through small time-slice resource allocation, while avoiding unnecessary frequent migrations over long periods through large time-slice service migration planning, thus achieving the optimal balance of long-term system benefits.

[0040] In some embodiments of the present invention, in the step of calculating the popularity ratio for each adapter in the server cluster, the popularity ratio corresponding to the adapter is calculated using the following formula: in, Indicates adapter k The popularity ratio Indicates adapter k The number of times drones were used in the previous large time slice, U This indicates the total number of drones.

[0041] In practical implementation, within the LoRA-based decoupled architecture, intelligent services consist of a "shared base model" and a "task-specific adapter." Therefore, It represents the distribution of request popularity for various specific task adapters in the current network.

[0042] It is a probability distribution vector, representing a large time slice. Initially, the frequency percentage of various adapters in the network requested by drones. Assume the system supports... Different task types are denoted as adapter sets. .but , Indicates adapter k The popularity ratio.

[0043] probability distribution vector It provides a global task distribution view for upper-layer agents. Its core function is to guide the caching strategy of the base model. If the aggregation of certain adapters is extremely hot, the agent will tend to pre-cache the shared base model that is compatible with it on more edge nodes, thereby ensuring that a large number of drones carrying these adapters can hit the cache during migration, and only need to transmit lightweight adapter data, maximizing the benefits of "trading storage for bandwidth".

[0044] In some embodiments of the present invention, in the step of constructing a service deployment vector based on the deployment location of each base model in the entire server cluster, a binary deployment matrix is ​​constructed based on the server location where each base model is deployed, and the service deployment vector is obtained by expanding the binary deployment matrix.

[0045] In some embodiments of the present invention, both the first reinforcement learning model and the second reinforcement learning model employ a DQN policy network.

[0046] In some embodiments of the present invention, the steps of the large time-slice service migration planning further include training a first reinforcement learning model. In the first reinforcement learning model training step, migration delay is calculated based on a global data migration vector; data freshness is calculated based on the updated data sent by the UAV to the connected server; and a first reward function is calculated based on the migration delay and data freshness.

[0047] In some embodiments of the present invention, in the step of calculating the first reward function based on the migration delay and data freshness, the first reward function is calculated using the following formula: in, Indicates a large time slice The first reward function, Indicates a large time slice The sum of the migration delays of each drone. The weight parameters represent the migration delay.U This indicates the total number of drones. Indicates drone In a short time slice Data freshness, The weighting parameter represents the freshness of the data.

[0048] In some embodiments of the present invention, in the step of calculating migration delay based on global data migration vector, it is determined whether data migration exists for each UAV based on the service migration node vector. If data migration exists for a UAV, the migration delay corresponding to each UAV is calculated using the following formula: in, Indicates a large time slice drones The corresponding migration delay, Indicates the drone migration start server To the migration destination server Number of jumps between; drones The server to be migrated to Total amount of data; This indicates the data transmission speed of the wired channel between servers.

[0049] The service migration node vector is defined as a binary matrix of dimension U×N. If an element of the matrix is ​​1, it indicates that the node is in the current time slice. Inside, drones The intelligent services are deployed to the base station A value of 0 indicates that the drone is not deployed; furthermore, if the drone... The intelligent services were not deployed in the base stations in the previous large time period. Then data migration exists.

[0050] In the actual implementation process, if data migration is required, the adapter data needs to be migrated.

[0051] In practical implementation, the data migration vector is defined as a vector with a dimension of 1. The binary matrix, denoted as .

[0052] Element meaning: Matrix element A value of 1 indicates that the base station has pre-cached the corresponding shared base station model; a value of 0 indicates that it has not been cached; if it is not cached, it needs to be cached again. Add the base model data; if cached, then =1.

[0053] In the specific implementation process middle, The overlay of data from the adapters to be migrated, including data from at least one adapter. The data representing the base model is determined by the deployment decision. Decide, Indicates whether to deploy the base model on the edge server. .

[0054] Specifically, such as Figure 4 As shown, it will serve drones. The intelligent service mathematical decomposition is divided into the following two parts: Shared base model: Contains the backbone parameters of the model, and its parameter count is... This part constitutes the vast majority of the total model size. It is applicable to the same basic tasks, such as vision and navigation tasks.

[0055] Task-specific adapters: contain incremental parameters trained for specific downstream tasks (such as status monitoring, logistics delivery, etc.). Its parameter quantity The parameters are very small, and the adapter parameters differ for different task types.

[0056] Based on the above decoupled architecture, when the upper-layer intelligent agent decides to deploy the drone... Service migration from the current base station to the target base station At that time, the system first checks the target base station. Has a base model compatible with the task been cached in the storage space? . For caching indicator variables, This indicates that the data has been cached. Then the amount of data transmitted via the backhaul link is... It was calculated that if the target base station has a pre-configured base station model, i.e., the cache is hit, then only a small amount of specific task adapter data needs to be transmitted. The base station model only needs to be transmitted when the cache is not hit. After the data arrives at the target base station, the edge server will receive the base station model. With adapter parameters Perform loading and merging, i.e. This allows for the rapid reconstruction of a complete intelligent service instance at the target node. .

[0057] This plan has the following constraints: ; This constraint limits the server's storage capacity, indicating that edge servers have limited storage space and cannot deploy all base models on a default full edge server configuration. If the system supports multiple types of base models, such as a visual base module for "security monitoring tasks" and "building maintenance" and a navigation base module for "logistics delivery," the system must selectively deploy on the edge servers if the edge server's storage space is extremely limited.

[0058] In some embodiments of the present invention, in the step of calculating data freshness based on the updated data sent by the drone to the connected server, the data freshness is calculated using the following formula: in, Indicates drone In a short time slice Data freshness; Indicates drone In a short time slice Total time consumed for updating data; Indicates drone In a short time slice Data freshness; The length of a small time slice.

[0059] In specific drone edge networks, such as Figure 6 As shown, and This indicates the moment the update task is generated by the drone. The update task is sent from the drone to the intelligent service deployed on the server and parsed, which involves the following process: ① First, the drone sends the update task to the edge server within its current coverage area, which will cause wireless transmission latency. ② When the intelligent service is not on the currently covered edge server, it will be transmitted to the edge server where the intelligent service is deployed via a wired link, causing forwarding latency. ③ After the update task reaches the edge server where the intelligent service resides, it must undergo computation before it can be parsed by the intelligent service. The latency incurred by the computation is... ,like Figure 5 As shown. Let Therefore, drones in The update task sent is in Only when the moment arrives can it truly be used by intelligent services, by which time the time since sending has already passed. Therefore, the information age of intelligent services has been reduced to [a certain level]. Let the time when the next update task is generated be... Then in Before the update task goes through the above process, the information age of the intelligent service will increase linearly until... Time drops to .

[0060] In practical implementation, this solution incorporates data freshness into the calculation of the first reward function. The drone needs to periodically send status update tasks to the intelligent service to ensure that the intelligent service on the edge server can continuously obtain the latest environmental status of the drone, thereby making correct inference decisions. The timeliness of the update tasks plays a decisive role in the accurate application of the intelligent service. Therefore, the concept of Age of Information (AoI) is introduced, representing data freshness. Unlike traditional "transmission latency," which only focuses on the speed of data transmission in the network, AoI characterizes how far back in environmental conditions the control decisions made by the intelligent service are based on. The physical meaning of AoI lies in revealing the "lag" of the intelligent service: the smaller the AoI value, the more "fresh" and safe the system is, based on newly generated environmental data; the larger the AoI value, the more outdated the inference decisions are, meaning the intelligent service can only use outdated inference decisions calculated based on stale data, which may result in significant deviations from the current actual environment.

[0061] In some embodiments of the present invention, in the step of calculating the total time consumed for updating data for each time slice of each UAV, the sum of the wireless transmission delay, wired transmission delay and calculation delay of the UAV time slice is calculated as the total time consumed for updating data. The wireless transmission delay is calculated using the following formula: The wired transmission delay is calculated using the following formula: The computation delay is calculated using the following formula: in, Indicates that the drone is in a small time slice wireless transmission latency, Indicates that the drone is in a small time slice Wired transmission latency, Indicates that the drone is in a small time slice computational delay, Indicates drone Update data size, Indicates drone The wireless transmission rate with the connected migration start server, Indicates the drone migration start server To the migration destination server Number of jumps between, This indicates the data transmission speed of the wired channel between servers. Indicates the server currently connected to the drone. The calculation proportion allocated in the previous large time slice, Indicates the server currently connected to the drone. Total computational load.

[0062] Specifically, regarding wireless transmission latency: using a frequency division multiple access protocol, drones... base station Uplink transmission rate : Wherein, is the bandwidth allocation ratio. Total bandwidth For transmission power, For channel gain, This represents the noise power. Based on this, the wireless transmission delay is calculated as follows: .

[0063] In some embodiments of the present invention, in the step of constructing a second state vector based on small-scale fading channel gain, the coordinates of the UAV in a small time slice, the computing resources required for the processing task of each UAV, and the service migration node vector of the previous large time slice, a gain vector is constructed based on the small-scale fading gain of each UAV and the server connected in the previous small time slice, a server connection vector is constructed based on the server connected to the UAV in the previous small time slice, and a resource demand vector is constructed based on the computing resources required for the processing task of each UAV. The gain vector, the server connection vector, the resource demand vector, and the service migration node vector of the previous large time slice are combined to construct the second state vector.

[0064] In the specific implementation process, in the second state vector middle, Representing the real-time channel gain matrix, it is used for the current time slot. A digital description of the physical quality of all "drone-to-base station" wireless communication links in the network. It consists of the gain coefficients of each sub-channel. The composition visually reflects the degree of attenuation of wireless signals as they propagate through the air.

[0065] It is a key basis for lower-level intelligent agents to allocate communication resources. According to Shannon's formula, the transmission rate... Directly depends on channel gain ,Right now Neural networks require observation. This is used to determine the current link quality. If a link has a high gain value, indicating a good channel, the agent may allocate sufficient bandwidth to achieve high-speed transmission. If a link has a low gain value, the agent may strategically allocate more bandwidth to compensate for losses, or choose to temporarily avoid the link to save resources.

[0066] In the specific implementation process The matrix constructed for the coordinates of the drone in a small time slice is defined as a matrix with dimension 1. A real matrix, denoted as .

[0067] Element meaning: Matrix number row vector , respectively representing drones The three-dimensional spatial coordinates at the current moment.

[0068] Function: Provides spatial distribution characteristics of UAVs to assist neural networks in determining distance constraints.

[0069] In the specific implementation process A matrix constructed from the computational resources required for each drone's processing task is defined as a matrix with dimension 1. A real matrix, denoted as .

[0070] Element meaning: Matrix number row vector .in Indicates drone Size of the generated update task data (in bits); This indicates the computational intensity required for the task (unit: cycles / bit).

[0071] In some embodiments of the present invention, the second action vector is composed of a communication bandwidth allocation matrix and a computing resource allocation matrix. The communication bandwidth allocation matrix is ​​configured with a bandwidth allocation ratio value for each channel in the server cluster, and the computing resource allocation matrix is ​​configured with a computing ratio value for each server in the server cluster.

[0072] In the specific implementation process, the resource allocation matrix is ​​calculated. Defined as a dimension A real matrix (with values ​​in the range [0,1]) is denoted as ; matrix elements Indicates base station What percentage of its total computing power (CPU frequency) is allocated to the drone? .

[0073] Communication bandwidth allocation matrix Defined as a dimension A real matrix (with values ​​in the range [0,1]) is denoted as ; matrix elements Indicates base station What percentage of its total communication bandwidth is allocated to drones? .

[0074] Furthermore, this invention aims to minimize the long-term weighted sum of migration latency and average AoI. This optimization problem is coupled with a first action vector. Second action vector The joint optimization objective function is defined as follows: The constraints are: Constraint 1: Constraint 2: Constraint 3: Constraint 4: Constraint 5: Among them, constraint 1 restricts each drone to only one intelligent service; constraints 2 and 3 are edge computing and bandwidth resource capacity constraints; and constraint 4 is based on storage limit. The base model cache constraints; constraint 5 defines the feasible domain of the variables.

[0075] To address the coupling problem between long-term migration decisions and short-term resource allocation in UAV edge networks, this embodiment models the optimization problem as a master-slave game model and constructs a centralized two-layer TD3 neural network architecture based on supergradients for solving the problem.

[0076] like Figure 7 As shown, this scheme constructs a hierarchically dependent two-layer Markov decision process, with the upper layer of the model having a macroscopic time period. The decision-making unit is the macroscopic state input vector, the first initial coordinate vector, and the service deployment vector at the beginning of each cycle. A first state vector is constructed from these vectors, and a first action vector is output based on the first reinforcement learning model. These actions determine the target node for service migration and the caching strategy of the base model, respectively. The upper-level optimization objective is to maximize the long-term cumulative return. The reward function implicitly includes the expectation of the optimal response of the lower-level followers; The lower layer, under the given macroscopic policy constraints of the upper layer, uses microscopic time slots. Decisions are made on a unit-by-unit basis. A second state vector is constructed based on the small-scale fading channel gain, the UAV's coordinates in a small time slice, the computational resources required for each UAV's processing task, and the service migration node vector of the previous large time slice. This second state vector is input into a second reinforcement learning model, which outputs a second action vector. The second reward function minimizes the instantaneous AoI cost. .

[0077] To achieve the synergistic optimization of the aforementioned two-layer strategy, a training process based on implicit hypergradients is adopted. The training process first updates the parameters of the lower layer, using the standard TD3 algorithm to minimize the Critic network loss, making it approximate the optimal response to the current upper-layer action. Subsequently, to update the upper layer, the system needs to calculate the upper-layer objective function with respect to the upper-layer parameters. The full gradient. Since the optimal policy of the lower layer is the implicit function of the action of the upper layer, the system calculates this gradient according to the chain rule: in the formula The term is the implicit Jacobian matrix, which represents the sensitivity of the upper-level decision change to the disturbance of the lower-level optimal response.

[0078] Since directly calculating the aforementioned implicit Jacobian matrix involves inverting a high-dimensional Hessian matrix, it is necessary to solve... The computational complexity is extremely high and the numerical values ​​are unstable. This embodiment further introduces a truncated Neumann series to iteratively approximate the solution for the inverse matrix terms. The specific approximation calculation logic is as follows: in Step size factor The truncation order is used. After synthesizing the final hypergradient using this approximation, the system performs gradient ascent updates on the upper-level policy network, thereby achieving end-to-end collaborative optimization of long-timescale migration strategies and short-timescale resource allocation strategies without explicitly solving the closed-form solution of the two-layer optimization problem.

[0079] The beneficial effects of this plan include: 1. This solution addresses the high bandwidth consumption issue of full migration by achieving low-latency benefits through a modular decoupling and differentiated migration mechanism based on LoRA, trading storage for bandwidth. In existing technologies, service migration typically involves the transfer of the entire container image or the migration of all parameters of a deep learning model, resulting in extremely high backhaul bandwidth consumption and migration latency. This invention utilizes LoRA technology to decouple intelligent services into a base model and a lightweight adapter. During migration, only adapter data needs to be transferred, significantly reducing the network resource consumption of service migration and greatly improving migration efficiency.

[0080] 2. This solution addresses the optimization challenge caused by the coupling of long and short timescale decisions. Through a two-layer spatiotemporal decoupling architecture, it resolves the oscillation problem caused by decision frequency mismatch, improving system stability and real-time performance. Existing technologies often handle service placement and resource allocation in a mixed manner at a single timescale, easily leading to frequent "ping-pong effects" or inability to respond promptly to instantaneous channel changes. This invention optimizes service topology at a macroscopic timescale to adapt to large-scale mobility, and optimizes resource allocation at a microscopic timescale to adapt to fast-fading channels. This layered processing mechanism significantly enhances the real-time response capability of the underlying resource allocation while ensuring service topology stability.

[0081] 3. This solution addresses the problem of convergence to suboptimal solutions caused by the disconnect between upper and lower layer decision-making. Based on the Stackelberg game algorithm, this solution enhances the synergy between upper and lower layer strategies. Traditional algorithms often neglect the hierarchical constraints of transfer decisions on resource allocation, leading to each layer acting independently. This invention introduces implicit hypergradients and computational mechanisms, accurately quantifying the gradient impact of upper-layer actions on the optimal response of lower layers through the chain rule. This allows the upper-layer agent to predict the reactions of lower layers and adjust its strategy, thereby guiding the system to converge to the globally optimal Nash equilibrium point and effectively reducing the long-term average information age.

[0082] 4. This solution addresses the backhaul link congestion and high latency issues caused by full model transmission in existing service migration technologies. This invention utilizes a LoRA-based modular decoupling and differentiated migration architecture to split edge intelligent services into a "shared base model" and a "hot-swappable task adapter." Combined with an edge-side base caching mechanism, it achieves low-latency migration benefits by trading storage for bandwidth. Compared to traditional virtual machine or full model migration, this solution only requires the transmission of lightweight adapter data, significantly reducing network communication overhead and meeting the service continuity requirements of high-speed UAV movement.

[0083] 5. This solution addresses the challenge of co-optimization between long-timescale migration decisions and short-timescale resource allocation in traditional edge computing. The invention proposes a two-layer spatiotemporal decoupling optimization framework, decomposing the optimization problem into macro-timescale service migration decisions and micro-timescale resource allocation decisions. This framework resolves the "ping-pong effect" and wasted computing resources caused by single-timescale decisions, significantly improving the system's decision stability and real-time response capabilities. While ensuring robust service topology, it achieves fine-grained scheduling of bandwidth and computing resources.

[0084] 6. To address the challenges of dimensionality explosion in traditional joint optimization algorithms and policy mismatch in independent divide-and-conquer algorithms, this invention employs a centralized two-layer optimization algorithm based on Stackelberg game theory. This scheme constructs a rigorous "upper-level leader-lower-level follower" hierarchical architecture and introduces implicit supergradient techniques, establishing derivative correlations between upper-level policies and the optimal responses of lower-level policies through a chain rule. This mechanism achieves deep global policy collaboration across time scales, guiding the system to converge to the globally optimal Stackelberg equilibrium point.

[0085] This invention also provides a device for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs). The device includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method described above.

[0086] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned UAV edge intelligent service migration and resource allocation method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0087] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0088] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0089] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for migrating and allocating resources for edge intelligent services of unmanned aerial vehicles (UAVs), characterized in that, The method includes steps such as large time-slice service migration planning and small time-slice resource allocation, wherein the large time-slice comprises multiple small time-slices: In the steps of the large time-slice service migration planning: For each adapter in the server cluster, calculate the popularity ratio and construct a macro-state input vector based on the popularity ratio of each adapter. The starting geographic coordinates of each drone in the drone cluster are constructed based on the server connected to at the start time of the current time slice, and a first starting coordinate vector is constructed based on the starting geographic coordinates of each drone. Based on the deployment location of each base model throughout the server cluster, a service deployment vector is constructed; The macroscopic state input vector, the first starting coordinate vector, and the service deployment vector are used to construct a first state vector. The first state vector is input into a first reinforcement learning model. The first reinforcement learning model outputs a first action vector, which includes a service migration node vector and a data migration vector. The data migration vector includes adapter migration data and base model migration data. In the step of allocating small time-slice resources: A second state vector is constructed based on the small-scale fading channel gain, the coordinates of the UAV in a small time slice, the computing resources required for each UAV's processing task, and the service migration node vector of the previous large time slice. The second state vector is input into the second reinforcement learning model, and the second reinforcement learning model outputs a second action vector, which includes the resource allocation in the server cluster.

2. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, In the step of constructing the service deployment vector based on the deployment location of each base model in the entire server cluster, a binary deployment matrix is ​​constructed based on the server location where each base model is deployed, and the service deployment vector is obtained by expanding the binary deployment matrix.

3. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, The steps of the large time-slice service migration planning also include training a first reinforcement learning model. In the first reinforcement learning model training step, migration latency is calculated based on the global data migration vector; data freshness is calculated based on the updated data sent by the UAV to the connected server; and a first reward function is calculated based on the migration latency and data freshness, using the following formula: in, Indicates a large time slice The first reward function, Indicates a large time slice The sum of the migration delays of each drone. The weight parameters represent the migration delay. U This indicates the total number of drones. Indicates drone In a short time slice Data freshness, The weighting parameter represents the freshness of the data.

4. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 1, characterized in that, In the step of calculating the popularity ratio for each adapter in the server cluster, the popularity ratio corresponding to the adapter is calculated using the following formula: in, Indicates adapter k The popularity ratio Indicates adapter k The number of times drones were used in the previous large time slice, U This indicates the total number of drones.

5. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, In the step of calculating migration latency based on global data migration vector, it is determined whether data migration exists for each UAV based on the service migration node vector. If data migration exists for a UAV, the migration latency corresponding to each UAV is calculated using the following formula: in, Indicates a large time slice drones The corresponding migration delay, Indicates the drone migration start server To the migration destination server Number of jumps between; drones The server to be migrated to Total amount of data; This indicates the data transmission speed of the wired channel between servers.

6. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 3, characterized in that, In the step of calculating data freshness based on the updated data sent by the drone to the connected server, the following formula is used to calculate data freshness: in, Indicates drone In a short time slice Data freshness; Indicates drone In a short time slice Total time consumed for updating data; Indicates drone In a short time slice Data freshness; The length of a small time slice.

7. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 6, characterized in that, The sum of the wireless transmission delay, wired transmission delay, and computation delay of a small time slice of the drone is calculated as the total time consumed for updating data; The wireless transmission delay is calculated using the following formula: The wired transmission delay is calculated using the following formula: The computation delay is calculated using the following formula: in, Indicates that the drone is in a small time slice wireless transmission latency, Indicates that the drone is in a small time slice Wired transmission latency, Indicates that the drone is in a small time slice computational delay, Indicates drone Update data size, Indicates drone The wireless transmission rate with the connected migration start server, Indicates the drone migration start server To the migration destination server Number of jumps between, This indicates the data transmission speed of the wired channel between servers. Indicates the server currently connected to the drone. The calculation proportion allocated in the previous large time slice, Indicates the server currently connected to the drone. Total computational load.

8. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to any one of claims 1 to 7, characterized in that, In the step of constructing the second state vector based on the small-scale fading channel gain, the coordinates of the UAV in the small time slice, the computing resources required for the processing task of each UAV, and the service migration node vector of the previous large time slice, a gain vector is constructed based on the small-scale fading gain of each UAV and the server connected in the previous small time slice, a server connection vector is constructed based on the server connected to the UAV in the previous small time slice, and a resource demand vector is constructed based on the computing resources required for the processing task of each UAV. The gain vector, server connection vector, resource demand vector, and service migration node vector of the previous large time slice are combined to construct the second state vector.

9. The method for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs) according to claim 8, characterized in that, The second action vector consists of a communication bandwidth allocation matrix and a computing resource allocation matrix. The communication bandwidth allocation matrix is ​​configured with a bandwidth allocation ratio for each channel in the server cluster, and the computing resource allocation matrix is ​​configured with a computing ratio for each server in the server cluster.

10. A device for migrating and allocating edge intelligent services for unmanned aerial vehicles (UAVs), characterized in that, The device includes a computer device, which includes a processor and a memory, wherein computer instructions are stored in the memory, and the processor is configured to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the method as described in any one of claims 1 to 9.

Images