Unmanned aerial vehicle task intelligent unloading strategy determination method, terminal and storage medium

Abstract

The invention provides an unmanned aerial vehicle task intelligent unloading strategy determination method, a terminal and a storage medium, and relates to the field of unmanned aerial vehicle task calculation. The method comprises the steps that an unmanned aerial vehicle task intelligent unloading strategy model of a target network system is constructed, and the optimization target is to maximize the long-term average weighted calculation rate of all unmanned aerial vehicles in the target network system; performing primary optimization on the unmanned aerial vehicle task intelligent unloading strategy model by adopting a Lyapunov function added with a penalty term to obtain an unmanned aerial vehicle task intelligent unloading strategy model after primary optimization; re-optimizing the unmanned aerial vehicle task intelligent unloading strategy model after the primary optimization by adopting a deep reinforcement learning model to obtain an optimized unmanned aerial vehicle task intelligent unloading strategy model; and obtaining an optimal unloading and resource allocation strategy of each unmanned aerial vehicle of the target network system in each time frame by using the optimized unmanned aerial vehicle task intelligent unloading strategy model. According to the invention, the calculation rate of the unmanned aerial vehicle task can be improved, and the resource allocation is optimal.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) mission computing, and in particular to a method for determining an intelligent unloading strategy for UAV missions, a terminal, and a storage medium. Background Technology

[0002] In recent years, communication systems for unmanned aerial vehicles (UAVs) have developed rapidly. These advancements have greatly propelled the development of the UAV industry and expanded its applications to numerous fields. In the commercial sector, UAV communication supports applications such as logistics delivery, precision agriculture, power line inspection, and urban mapping. With the growth of the consumer UAV market, users are increasingly demanding stable image transmission, low latency, and wide-area signal coverage. Furthermore, emerging technologies such as 5G, the Internet of Things (IoT), and edge computing are propelling UAV communication systems to a new level. 5G reduces latency in real-time control, while edge computing enables faster local processing. Their convergence lays a solid foundation for intelligent collaboration in complex environments and autonomous UAV operation.

[0003] On the other hand, with the rapid development of information technology, the number of terminal devices has increased dramatically. The explosive expansion of mobile internet services has led to the emergence of a large number of computationally intensive applications, such as Virtual Reality (VR), Augmented Reality (AR), autonomous driving, Artificial Intelligence (AI), and data analysis. These applications have increasingly stringent latency requirements, placing a heavy computational burden on drone terminals with limited processing power, and highlighting the growing contradiction between limited computing resources and low-latency requirements. However, traditional cloud computing, due to excessive transmission latency in its core network, is unable to handle the rapidly growing computing demands of drone terminals in real time.

[0004] To overcome the aforementioned limitations, the architecture using drones as relay nodes has attracted widespread attention. User equipment uploads data to the drone, which then offloads the computing tasks to base stations (BS) equipped with edge servers, thereby achieving efficient processing of large-scale workloads. This drone-assisted edge computing framework effectively reduces the computational burden on user equipment in air-to-ground integrated networks, significantly reduces task transmission latency, improves overall system efficiency and service quality, and provides a promising solution for future intelligent low-latency wireless networks.

[0005] Currently, numerous studies have explored UAV-assisted mobile edge computing (MEC). Among these methods, Lyapunov Coordinate Descent (LyCD) leverages its inherent stability to efficiently optimize resource allocation, achieving an effective balance between task latency and system stability. The Deep Deterministic Policy Gradient (DDPG) algorithm demonstrates strong performance in multi-UAV collaborative scenarios, and its meta-learning variant, Multi-Agent Meta-Learning (MAML)-DDPG, further enhances adaptability to time-varying optimization objectives and significantly improves offloading decision efficiency. The Soft Actor-Critic (SAC) algorithm, renowned for its superior exploration capabilities and robustness, has become the preferred solution for multi-objective optimization in complex environments, minimizing UAV energy consumption while ensuring service quality. Recent research often combines these algorithms with trajectory planning and load balancing through a joint optimization framework. Simulation results confirm their effectiveness in achieving low latency and energy efficiency, providing a solid theoretical foundation for practical applications.

[0006] However, existing technologies still have many shortcomings: traditional optimization methods often lack flexibility and are difficult to adapt to the random fluctuations in UAV mission arrival rates and channel states in dynamic wireless environments; pure deep reinforcement learning methods lack strict theoretical stability guarantees and are prone to problems such as queue backlog and uncontrolled energy consumption under complex constraints; at the same time, most schemes fail to achieve deep joint optimization of computation offloading and resource allocation, and in multi-UAV collaborative scenarios, it is difficult to balance high computational throughput, low energy consumption and stable queue behavior, resulting in uneven resource allocation and low computational speed, which cannot meet the needs of high-speed computing and efficient resource allocation in scenarios such as natural disasters and emergencies. Summary of the Invention

[0007] This application provides a method, terminal, and storage medium for determining intelligent offloading strategies for unmanned aerial vehicle (UAV) missions, in order to solve the problems of low computation speed and uneven resource allocation in existing UAV missions.

[0008] Firstly, this application provides a method for determining an intelligent unloading strategy for a drone mission, including: A drone mission intelligent offloading strategy model is constructed for a target network system. The drone mission intelligent offloading strategy model aims to maximize the long-term average weighted computing rate of all drones in the target network system. The target network system is a network system consisting of a base station equipped with a mobile edge computing server and multiple rotary-wing drone devices. The UAV mission intelligent offloading strategy model was initially optimized using a Lyapunov function with added penalty terms, resulting in the initially optimized UAV mission intelligent offloading strategy model. The initially optimized UAV mission intelligent offloading strategy model is further optimized using a deep reinforcement learning model to obtain the optimized UAV mission intelligent offloading strategy model. Using the optimized UAV mission intelligent offloading strategy model, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame are obtained.

[0009] In a second aspect, this application provides a terminal including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation thereof.

[0010] Thirdly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation thereof.

[0011] This application provides a method, terminal, and storage medium for determining intelligent offloading strategies for unmanned aerial vehicle (UAV) tasks. It constructs an intelligent offloading strategy model for a target network system, with the optimization objective of maximizing the long-term average weighted computation rate of all UAVs in the target network system. The target network system consists of a base station equipped with a mobile edge computing server and multiple rotary-wing UAV devices. The model is initially optimized using a Lyapunov function with added penalties. A deep reinforcement learning model is then used to further optimize the model, resulting in a final optimized model. Using this final optimized model, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame are obtained. By combining Lyapunov optimization theory with deep reinforcement learning, this application enables joint optimization of UAV task offloading and resource allocation in dynamic wireless environments, maintaining high throughput, low energy consumption, and stable queue behavior, thereby improving the computation rate of UAV tasks and achieving optimal resource allocation. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart illustrating the method for determining the intelligent unloading strategy for unmanned aerial vehicle (UAV) missions provided in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the application scenario of the target network system provided in the embodiments of this application; Figure 3 This is a schematic diagram of the channel model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the deep reinforcement learning model provided in the embodiments of this application; Figure 5 This is a schematic diagram illustrating the change of data queue length over time according to an embodiment of this application; Figure 6 This is a schematic diagram illustrating the change of energy consumption over time, provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating the change of computing speed over time, provided in an embodiment of this application. Figure 8This is a schematic diagram illustrating the change in energy consumption as the task arrives at its destination, provided in an embodiment of this application. Figure 9 This is a schematic diagram illustrating the variation of stable computing speed with the number of drones provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of the intelligent unloading strategy determination device for unmanned aerial vehicle (UAV) missions provided in the embodiments of this application; Figure 11 This is a schematic diagram of the terminal provided in the embodiments of this application. Detailed Implementation

[0014] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0016] Figure 1 The implementation flowchart of the intelligent unloading strategy determination method for UAV missions provided in the embodiments of this application is described in detail below: In step 101, a UAV mission intelligent offloading strategy model is constructed for the target network system. The UAV mission intelligent offloading strategy model aims to maximize the long-term average weighted computing rate of all UAVs in the target network system. The target network system is a network system consisting of a base station equipped with a mobile edge computing server and multiple rotary-wing UAV devices.

[0017] In this embodiment, based on the target network system, an intelligent unloading strategy model for drone tasks is constructed with the optimization objective of maximizing the long-term average weighted computation rate of all drones in the target network system. Maximizing the long-term average weighted computation rate of drones is crucial because drone task arrival and channel states fluctuate randomly, and short-term rates are meaningless; therefore, a long-term average is needed to ensure stable results.

[0018] For the target network system, please refer to Figure 2 As shown, this is a system consisting of a base station and A network system consisting of rotor-based unmanned aerial vehicles (UAVs) is established, with the base station equipped with a Mobile Edge Computing (MEC) server. All UAVs form a cluster. In addition, the embodiments of this application also employ... Figure 3 The Time Division Multiple Access (TDMA) channel model shown depicts multiple UAVs sharing a channel within the same time slot by allocating different time intervals. For ease of analysis, the entire target network system is divided into... A time interval of equal length, represented as The duration of each time slot is ,in, For the total time. Within each time slot, computational tasks from the UAV are assumed to be independent and identically distributed; the number of arriving tasks is represented by... It means that the conditions are met. (for ).in, In order to be with the first The constants related to drones.

[0019] Furthermore, all incoming computational tasks are temporarily stored in the drone's task cache queue for later processing. Within each time period, each drone device makes a binary offload decision, allowing the task to execute locally or be offloaded to the rotorcraft drone's MEC server. Let... Indicates the first Within the time interval, the first The unloading decision for each drone. For the MEC server, the total number of drone tasks received in each time interval equals the sum of the unloading tasks for all drone devices in the target network system. Similar to the drones, the MEC server also temporarily stores the received tasks in its task cache queue before processing them.

[0020] 1) Local computing (i.e.) ):set up Indicates unmanned aerial vehicle (UAV) equipment The number of CPU cycles required to process one bit of data. Indicates unmanned aerial vehicle (UAV) equipment CPI frequency, Indicates unmanned aerial vehicle (UAV) equipment In time period The time required for local computation within the drone device; therefore, In time period Local computational workload It can be represented as:

[0021] For the analysis of local computing energy consumption, an energy consumption model is adopted. In this model, This represents the energy coefficient related to the chip architecture, which is set to [value] in this embodiment of the application. Therefore, drone equipment In the Local computing power consumption over time intervals It can be represented as:

[0022] 2) Calculate unloading (i.e. In a remote unloading scenario, the drone uploads its computational tasks to the MEC server via a base station for processing. The MEC server then returns the computation results to the corresponding drone. Since the size of the computation results is typically much smaller than the size of the input data, downlink transmission time can be ignored in this embodiment. According to Shannon's capacity formula, the drone equipment... Uplink speed It can be represented in the following form:

[0023] in, For uplink bandwidth, To offload the required transmission power to the MEC server, For noise power, This refers to channel gain. Because a time-division multiple access channel is used, the computational unloading tasks of multiple UAVs share the same time. ,set up Indicates the first The time slots allocated to drone equipment Uninstallation time, This represents a collection of drones, subject to the following constraints:

[0024] in, For the first The time frame allocated to the first The unloading time of each drone is part of the total time. The percentage.

[0025] Therefore, drone equipment In the time slot Data volume unloaded during the period It can be represented as:

[0026] Energy consumption related to remote unloading for:

[0027] In summary, the total amount of data processing tasks for:

[0028] Total energy consumption yes:

[0029] Computation speed Defined as:

[0030] in, For drone equipment In the Calculate the power consumption of unloading within each time interval.

[0031] The formula for calculating power consumption is as follows:

[0032] but The embodiments of this application can... Set it to 1.

[0033] 3) Unmanned Aerial Vehicle (UAV) Task Queue Model: Let Indicates the first The drone equipment in the first The queue length at the start of the current time slot. The queue length for the next time slot can be modeled as:

[0034] in, For the first The drone equipment in the first The queue length at the start of each time slot To the number of tasks completed, The amount of work that the device can process in the current time slot is subject to conditions. The limitations. In subsequent derivations, causal constraints are imposed. Ensure for any All This is true. Therefore, the dynamic evolution of the queue can be simplified as follows:

[0035] And it is required that the backlog of all queues in the system meets the following requirements:

[0036] in, For the first The average data queue length of a drone device.

[0037] Assume the unmanned aerial vehicle (UAV) equipment is in the first The set of unloading decisions within each time frame is Unmanned aerial vehicle (UAV) equipment in the first The set of CPU frequencies within each time frame is Unmanned aerial vehicle (UAV) equipment in the first The set of time percentages used for unloading within each time frame is as follows: Unmanned aerial vehicle (UAV) equipment in the first The set of power consumption unloaded within each time frame is And make the uninstallation decision CPU frequency Percentage of uninstallation time Unloading power consumption The objective function of the intelligent unloading strategy model for UAV missions is denoted as P1:

[0038] in, To maximize the function, For the first Fixed weights for each drone For the first The drone in the first The computation rate per time frame The total number of time frames. This represents the total number of drones.

[0039] Accordingly, the constraint condition for constructing the UAV mission intelligent offloading strategy model of the target network system is the first constraint condition, which is:

[0040] in, For the first The maximum CPU frequency of a drone For mathematical expectation, For the first Power threshold for each drone For the first Maximum transmission power of each drone.

[0041] in, Corresponding to the time percentage constraint, it means the first The total shared task time occupied by each drone device during task unloading. The proportion; This represents the average power constraint, where the power threshold is... ; This represents the stability constraint of the data queue; and These correspond to upper and lower bound constraints and binary constraints, respectively. It's important to note that if the condition... If it is established, then It must be true; similarly, if the condition is true... If it is established, then Established.

[0042] In step 102, the UAV mission intelligent offloading strategy model is initially optimized using a Lyapunov function with added penalty terms, resulting in the initially optimized UAV mission intelligent offloading strategy model.

[0043] Since the UAV mission intelligent offloading strategy model constructed in step 101 has random fluctuations in the calculation of mission arrival rate and wireless channel state, this embodiment of the application uses a Lyapunov function with added penalty term to perform initial optimization on the UAV mission intelligent offloading strategy model, and obtains the initially optimized UAV mission intelligent offloading strategy model.

[0044] Wherein, P1 is a variable that depends on time 0 to... The problem involves dynamic optimization of historical information. This application's embodiment employs a Lyapunov function with added penalties, transforming P1 into an optimization problem that depends solely on the current time slot system information.

[0045] To satisfy the average power constraint in the first constraint condition, the following is introduced: A virtual energy queue Each drone corresponds to a virtual energy queue, with an initial energy queue. The update rules are as follows:

[0046] in, For the first The drone in the first Virtual energy queues within a time period For the first The drone in the first Virtual energy queues within a time period It is a positive scaling factor. For the first The drone in the first Energy consumption over a specific time period. Among them, It can be viewed as a queue, where the energy arrival rate is a random variable. The service rate is a fixed value. Intuitively speaking, when the virtual energy queue is stable, its average power consumption (i.e., the arrival rate of the virtual energy queue) is... Not exceeding the threshold This ensures that the average power requirement in the first constraint is met.

[0047] To control both the data queue and the virtual energy queue simultaneously, the total queue backlog is defined as follows: ,in Based on this, the Lyapunov function is introduced. To quantify the overall state of the system queue, that is:

[0048] Because, when When the queue size is small, the backlog is low for both queue types; conversely, when the backlog is large... When the queue length is large, at least one queue exhibits severe backlog. Longer queue lengths correspond to longer task waiting times, increasing the likelihood of instability in the target network system. To ensure the stability of the target network system, Lyapunov drift analysis is employed, namely:

[0049] in, The goal is to ensure stability by monitoring the trend of queue backlog changes. Decreasing the value of reduces the difference in queue length between adjacent time intervals, thus maintaining system stability. Therefore, in each time interval... In the following scenario, the upper bound of the drift plus penalty function is minimized:

[0050] in, For the drift-penalty function, This represents the weighting coefficient of the penalty term.

[0051] In addition, weighting coefficients It balances the trade-off between queue backlog and optimization objectives. Then, the derivation... The upper boundary.

[0052] First, according to the queue update rules, we have:

[0053] By analyzing the target network system Summing both sides of the queues, we get:

[0054] Further derivation yields:

[0055] Then, the overall queue situation is broken down into the data queue and the virtual energy queue for separate discussion.

[0056] First, the Lyapunov function of the data queue and its Lyapunov drift are defined as follows:

[0057] Taking the conditional expectation of both sides of equation (20), we get

[0058] Wherein, the first constant The derivation is as follows:

[0059] in, In order to be with the first A constant term related to drones, For the first The maximum average transmission rate achievable by a drone. Similarly, the Lyapunov function and drift of the virtual energy queue are also defined as:

[0060] Taking the expectation of both sides of equation (21), we get:

[0061] Among them, the second constant The derivation is as follows:

[0062] Adding the corresponding inequalities in formula (24) and formula (28), we get:

[0063] therefore, The upper bound of the drift penalty term in formula (17) is:

[0064] Based on the currently observed queue backlog status The task unloading and resource allocation are optimized using the opportunity expectation minimization method to minimize the upper limit specified in formula (17). This is because within this upper limit, only the second and third terms depend on the control decisions at each time point. Therefore, at the beginning of each time point, the constant term is ignored and multiplied by... Then, the optimal control action is determined by maximizing it:

[0065] To establish the constraint structure for each frame, an auxiliary variable for the unloading task calculation rate is introduced for each UAV device. The conditions are met. By integrating all relevant constraints into the same time frame, the first... The optimization task in the frame, namely the objective function of the initially optimized UAV mission intelligent offloading strategy model, is denoted as P2:

[0066] Accordingly, the second constraint is:

[0067] in, For the first The drone in the first The calculation rate of the unloading task auxiliary variable is calculated within each time frame.

[0068] It is worth noting that the constraints and It is essentially equivalent to the average power constraint in the first constraint condition, that is, under the optimal solution, only one term on the left side of the average power constraint in the first constraint condition remains non-zero.

[0069] In step 103, a deep reinforcement learning model is used to further optimize the initially optimized UAV mission intelligent offloading strategy model, resulting in an optimized UAV mission intelligent offloading strategy model.

[0070] In this embodiment, a deep reinforcement learning model is used to further optimize the initially optimized UAV mission intelligent offloading strategy model, aiming to achieve near-optimal performance with significantly reduced computational complexity. To achieve this, in the... To solve the initially optimized UAV mission intelligent offloading strategy model within a given time period, it is first necessary to obtain the state function of the target network system. This state function includes the channel gain of all UAV equipment. and system queue status Based on this state function, the joint unloading and resource allocation strategy vectors are then determined. .in, This indicates the binary unloading of decision variables, while This represents a continuous resource allocation scheme.

[0071] It can be seen that the objective function of the initially optimized UAV mission intelligent offloading strategy model is essentially a non-convex optimization problem, but with subproblems related to resource allocation. In uninstallation decision Under fixed conditions, this becomes a convex problem. To further solve for the optimal value of the initially optimized UAV mission intelligent unloading strategy model, we define... To achieve optimal resource allocation The optimal objective value of the subproblem is achieved, given the unloading decision. and system parameters Therefore, the objective function of the optimized UAV mission intelligent unloading strategy model is denoted as P3, that is:

[0072] in, This is the optimal unloading decision.

[0073] This application's embodiments introduce a deep reinforcement learning model, by constructing a mapping function. It can efficiently learn a low-complexity mapping from the system state vector to the optimal decision action, that is:

[0074] For example, when At that time, it can observe the state in just tens of milliseconds. Generate optimal control action output This significantly improves computational efficiency. (Refer to...) Figure 4 As shown, the deep reinforcement learning model consists of four main modules: the Actor module, the Critic module, the Policy Update module, and the Queue Management module. The Actor module receives parameters as input and generates a set of candidate offloading policies; the Critic module evaluates the performance of the candidate policies and selects the optimal offloading policy; the Policy Update module is responsible for maintaining and updating the experience replay buffer of the deep neural network; and the Queue Management module updates the system queue state based on task arrival and completion offloading, ensuring that the system state is accurately refreshed in subsequent time frames for scheduling and decision-making.

[0075] 1) Actor module The Actor module contains a deep neural network and an action quantizer. (Timeframe) Initially, the parameters of the deep neural network are represented as follows: Deep neural networks use system state vectors As input, a continuous unloading decision is generated, which is then converted by a quantizer into a feasible set of binary actions. The input-output relationship is as follows:

[0076] exist At that time, the above parameters are initialized according to a standard normal distribution. In this embodiment, the output layer of the deep neural network uses the Sigmoid activation function to ensure that the output is smooth and bounded. Subsequently, the unloading strategy generated by the deep neural network... Discretized using the defined quantization function One feasible candidate action, namely:

[0077] in, For the first A set of candidate actions within a time frame For the first The first time frame A candidate unloading decision vector To generate A quantization function for binary actions. An effective quantization function must find a balance between exploration and exploitation to guarantee convergence. On the one hand, candidate actions... It should be kept consistent with the output of the deep neural network By measuring sufficient proximity using Euclidean distance, the learned network information can be effectively utilized. On the other hand, candidate actions must maintain sufficient diversity to avoid premature convergence to a suboptimal strategy. To achieve this balance, embodiments of this application also employ a noisy ordered order-preserving quantization method, allowing the generation of any number of... The candidate actions are selected while maintaining structural consistency between the outputs.

[0078] 2) Critic module Following the Actor module, the Critic module is responsible for evaluating candidate unloading operations. And select the best action Specifically, the optimal action is determined as follows:

[0079] in, In order to under given conditions The optimal objective value is obtained by solving P2. It is worth noting that obtaining the final optimal action... Need to implement This evaluation is conducted in order to strike a balance between performance and computational efficiency. This application introduces an adaptive candidate action update strategy, which dynamically adjusts over time. The basic principle is that as the deep neural network of the Actor module gradually converges to the optimal policy, a smaller... It can effectively identify The nearest near-optimal action. Let Indicates the optimal action The index, and define The best candidate action exist The ranking position within a candidate set (with or without noise). In practical implementation, the maximum number of candidate actions. Initialized, and every Updated in 10 time frames, where... Number of candidate actions The adaptive update interval parameter. When the time frame... Meet the conditions Time (i.e.) Can be (Divisible), the adaptive update rule is:

[0080] in, make sure It increases gradually over time. If conditions... Then the current Keep it unchanged. If the update frequency is set too high (i.e....), Too small a refresh rate may cause oscillations and instability during training, while too low a refresh rate (i.e., ...) may cause oscillations and instability during training. If the value is too large, it may lead to unnecessary computational overhead and reduced efficiency.

[0081] 3) Policy Update module To facilitate efficient learning, an experience replay buffer is used to store the most recent training samples, ensuring adaptation to the latest system dynamics. Initially, the buffer is empty; as the number of stored samples exceeds [a certain threshold], [the buffer is activated]. At this point, deep neural network training begins. To mitigate the risk of overfitting, the network parameters are updated periodically. Specifically, every [time period missing]... Each time slot updates once. When the condition is met. At that time, a batch of samples is randomly selected from the playback buffer. The corresponding time index set is denoted as Then, the Adam optimizer is applied to minimize the average cross-entropy loss function on the selected batches. This updates the parameters of the deep neural network as follows:

[0082] in, For batch size, For transpose operation, This is a logarithmic operation. The updated network parameters after training. It is transferred to the Actor module for the next time slot, ensuring the temporal consistency of policy learning and improving adaptability to dynamic environments.

[0083] 4) Queue Management module After the Critic module completes its evaluation, it makes the optimal unloading decision. Corresponding optimal resource allocation scheme It is also obtained simultaneously. Subsequently, joint computation unloading and resource allocation operations are performed. Process task data according to formula (7) And consume the corresponding energy according to formula (10). Based on time slots Observed system state And the amount of newly arrived data The queue management module is in the time slot Initially, the data queue and energy queue are updated using formulas (12) and (14). Subsequently, the latest observed wireless channel gain With system parameters The inputs are combined and fed into the deep neural network, initiating a new round of iteration for the Actor module. Through this iterative learning mechanism, the deep neural network continuously learns from the latest state-action pairs, gradually optimizing its decision-making strategy. And it asymptotically converges to the optimal mapping of P3.

[0084] Using the same data, experiments were conducted using the embodiments of this application, the SAC algorithm, and the DDPG algorithm. It can be concluded that the embodiments of this application exhibit performance improvements compared to both the SAC and DDPG algorithms, specifically: Reference Figure 5 The relationship between data queue length and time frames was analyzed to evaluate the stability of the target network system. The average data queue length in the embodiment of this application remained stable at approximately 1 Mbps with minimal fluctuation, indicating good queue stability. This performance is mainly attributed to the combination of virtual energy queues and the Lyapunov optimization framework, which significantly limits task backlog growth and effectively prevents queue overflow. In contrast, the DDPG algorithm exhibited greater queue fluctuations, while the SAC algorithm showed the most severe fluctuations, with an average queue length significantly higher than that of the embodiment of this application. This indicates that both lack systematic queue constraint modeling and struggle to ensure long-term stability in dynamic environments.

[0085] Figure 6 The data shows energy consumption variations over different time periods. This embodiment maintains energy consumption stably within the range of 0.09 J / frame, achieving efficient resource utilization while maintaining high computational performance. While the DDPG algorithm has slightly lower average energy consumption, it exhibits significant instability. The SAC algorithm has the lowest energy consumption, but its computational throughput is reduced. This embodiment effectively combines energy control with computational performance through multi-objective joint optimization, thereby avoiding low energy consumption and low throughput. These results highlight the superior energy efficiency and resource utilization of this embodiment.

[0086] Figure 7The throughput represents the computational output at different time intervals, reflecting the system-level computing performance. The embodiments of this application achieve the highest and most stable throughput, consistently operating within the 17-20 Mbps range, significantly outperforming the DDPG and SAC algorithms. This improvement primarily stems from the use of a deep neural network in these embodiments to perform real-time learning and adaptive updates of the optimal offloading strategy. Therefore, even in highly dynamic and three-dimensional wireless environments, this algorithm can achieve precise matching between computational resources and task requirements. In contrast, the limited policy learning capabilities and generalization performance of the DDPG and SAC algorithms make it difficult to optimize computational scheduling under complex constraints.

[0087] like Figure 8 As shown, as the task arrival rate increases from 1.5 Mbps to 3.9 Mbps, the energy consumption of this embodiment remains stable within the range of 0.040-0.045 J, exhibiting a gradual increasing trend. Even at an arrival rate of 3.9 Mbps, the energy consumption remains below 0.045 J, far below the upper limit of 0.070 J. In contrast, while the SAC algorithm has slightly lower energy consumption, it fluctuates significantly, while the DDPG algorithm exhibits a sharp increase in energy consumption and the worst stability. The stable and efficient characteristics of this embodiment stem from its virtual energy queue and Lyapunov optimization mechanism. This mechanism, through dynamically balancing task offloading and energy constraints, avoids the "low energy consumption - low throughput" trade-off dilemma of the SAC algorithm and solves the energy instability problem of the DDPG algorithm.

[0088] Figure 9 The impact of varying drone numbers on computational stability is demonstrated. When the number of drones increases from 3 to 25, the embodiment in this application maintains a stable throughput of 1.25 to 1.75 Mbps, approaching the 2.00 Mbps upper limit with a linear growth trend. In contrast, the SAC and DDPG algorithms show slow rate increases, remaining below 1.5 Mbps even with 25 drones. This performance difference can be attributed to the real-time learning capability of the deep neural network in this embodiment, which efficiently handles high-dimensional channels and queue states, thereby achieving accurate resource allocation in multi-drone scenarios. The SAC and DDPG algorithms, limited by their policy generalization capabilities, struggle to adapt to the increased complexity brought about by a larger number of drones.

[0089] Based on the above experiments, the embodiments of this application have superior computing performance (high throughput), resource efficiency (low and stable energy consumption), and system stability (minimal queue fluctuations), providing an efficient, robust, and scalable solution for online computation offloading of rotorcraft UAV MEC networks.

[0090] In step 104, the optimized UAV mission intelligent offloading strategy model is used to obtain the optimal offloading strategy and resource allocation strategy for each UAV in the target network system in each time frame.

[0091] In this embodiment, real-time data of the target network system is acquired, including the number of drones, total time frames, parameters, update interval, energy consumption threshold, and learning rate. This real-time data is then input into an optimized drone task intelligent offloading strategy model, which outputs the optimal offloading strategy and resource allocation strategy for each drone in the target network system within each time frame.

[0092] Before inputting data, the optimized UAV mission intelligent unloading strategy model needs to be processed. This involves using a first-in-first-out (FIFO) mechanism to remove historical data from the model, ensuring that the output optimal unloading strategy and resource allocation strategy meet actual needs.

[0093] This application provides a method for determining intelligent offloading strategies for unmanned aerial vehicle (UAV) missions. It constructs an intelligent offloading strategy model for a target network system, with the optimization objective of maximizing the long-term average weighted computation rate of all UAVs in the target network system. The target network system consists of a base station equipped with a mobile edge computing server and multiple rotary-wing UAVs. The model is initially optimized using a Lyapunov function with added penalties. A deep reinforcement learning model is then used to further optimize the model, resulting in a final optimized model. Finally, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system are determined for each time frame using this optimized model. By combining Lyapunov optimization theory with deep reinforcement learning, this application enables joint optimization of UAV mission offloading and resource allocation in dynamic wireless environments, maintaining high throughput, low energy consumption, and stable queue behavior, thereby improving the computation rate of UAV missions and achieving optimal resource allocation.

[0094] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0096] Figure 10A schematic diagram of the structure of the intelligent unloading strategy determination device for UAV missions provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown, and are described in detail below: like Figure 10 As shown, the intelligent unloading strategy determination device 10 for UAV missions includes: Model building module 1001 is used to build a UAV mission intelligent offloading strategy model for the target network system. The UAV mission intelligent offloading strategy model aims to maximize the long-term average weighted computing rate of all UAVs in the target network system. The target network system is a network system consisting of a base station equipped with a mobile edge computing server and multiple rotary-wing UAV devices. The initial optimization module 1002 is used to perform initial optimization of the UAV mission intelligent offloading strategy model using a Lyapunov function with added penalty terms, so as to obtain the initially optimized UAV mission intelligent offloading strategy model. The model re-optimization module 1003 is used to further optimize the initially optimized UAV mission intelligent offloading strategy model using a deep reinforcement learning model, so as to obtain the optimized UAV mission intelligent offloading strategy model. The strategy determination module 1004 is used to obtain the optimal unloading strategy and resource allocation strategy for each UAV in the target network system in each time frame by utilizing the optimized UAV mission intelligent unloading strategy model.

[0097] This application provides a device for determining intelligent unmanned aerial vehicle (UAV) task offloading strategies. It constructs an intelligent unloading strategy model for a target network system, with the optimization objective of maximizing the long-term average weighted computation rate of all UAVs in the target network system. The target network system consists of a base station equipped with a mobile edge computing server and multiple rotary-wing UAVs. The model is initially optimized using a Lyapunov function with added penalties. A deep reinforcement learning model is then used to further optimize the model, resulting in a final optimized model. Using this final optimized model, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system are determined for each time frame. By combining Lyapunov optimization theory with deep reinforcement learning, this application enables joint optimization of UAV task offloading and resource allocation in dynamic wireless environments, maintaining high throughput, low energy consumption, and stable queue behavior, thereby improving the computation rate of UAV tasks and achieving optimal resource allocation.

[0098] In one possible implementation, the objective function of the intelligent unloading strategy model for UAV missions is:

[0099] in, To maximize the function, For the first Fixed weights for each drone For the first The drone in the first The computation rate per time frame The total number of time frames. The total number of drones, For the decision of unloading, CPU frequency, This represents the percentage of time spent uninstalling. To unload power consumption.

[0100] In one possible implementation, the constraint condition for constructing the UAV mission intelligent offloading strategy model of the target network system is the first constraint condition, which is:

[0101] in, For the first The time frame allocated to the first The percentage of unloading time for each drone in the total time. For the first The drone in the first Unloading decision for each time frame For the first CPU frequency of a drone For the first The maximum CPU frequency of a drone For the first The number of CPU cycles required for a drone to process one bit of data. For uplink bandwidth, For the first The drone in the first The power consumption of offloading is calculated for each time frame. For channel gain, For noise power, For the first The drone in the first The queue length starting from each time frame. For mathematical expectation, The energy coefficient, For the first Power threshold for each drone For the first Maximum transmission power of each drone.

[0102] In one possible implementation, the objective function of the initially optimized UAV mission intelligent offloading strategy model is:

[0103] in, To maximize the function, For the first The drone in the first The queue length starting from each time frame. The weighting coefficient for the penalty term. For the first Fixed weights for each drone For the first The drone in the first The computation rate per time frame For the first A virtual energy queue, For the first The drone in the first Energy consumption within a time frame The total number of drones, For drones in the The set of unloading decisions within a time frame. For drones in the The set of time percentages used for unloading within each time frame. For drones in the A set of CPU frequencies within a time frame. For drones in the The set of power consumption unloaded within each time frame. For the first The set of auxiliary variables for the calculation rate of the unloading task within each time frame.

[0104] In one possible implementation, the constraint condition of the initially optimized UAV mission intelligent offloading strategy model is the second constraint condition, which is:

[0105] in, For the first The time frame allocated to the first The unloading time of each drone is part of the total time. The proportion, For the first CPU frequency of a drone For the first The number of CPU cycles required for a drone to process one bit of data. For the first The drone in the first The offloading task calculation rate auxiliary variable within each time frame For uplink bandwidth, For the first The drone in the first The power consumption of offloading is calculated for each time frame. For channel gain, For noise power, For the first The maximum CPU frequency of a drone For the first Maximum transmission power of each drone For the first The drone in the first Unloading decisions for each time frame.

[0106] In one possible implementation, the objective function of the optimized UAV mission intelligent offloading strategy model is:

[0107] in, To maximize the function, For the optimal unloading decision, This represents the optimal target value achieved through optimal resource allocation. For drones in the The set of unloading decisions within a time frame. For drones in the The set of system parameters within a time frame This represents the total number of time frames.

[0108] In one possible implementation, the strategy determination module can specifically be used for: Acquire real-time data of the target network system, including the number of drones, total time frames, parameters, update interval, energy consumption threshold, and learning rate of the target network system; The real-time data of the target network system is input into the optimized UAV mission intelligent offloading strategy model, which outputs the optimal offloading strategy and resource allocation strategy for each UAV in the target network system in each time frame.

[0109] In one possible implementation, the device may further include a preprocessing module, which can be used for: A first-in-first-out (FIFO) mechanism is adopted to remove historical data from the optimized UAV mission intelligent offloading strategy model. Historical data includes the number of UAVs, total time frames, parameters, update interval, energy consumption threshold, and learning rate in the target network system. Accordingly, the strategy determination module can be used for: By utilizing the optimized UAV mission intelligent offloading strategy model after removing historical data, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame are obtained.

[0110] Figure 11 This is a schematic diagram of the terminal provided in an embodiment of this application. For example... Figure 11 As shown, the terminal 11 in this embodiment includes: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and executable on the processor 110. When the processor 110 executes the computer program 112, it implements the steps in the various embodiments of the UAV mission intelligent offloading strategy determination method described above, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when the processor 110 executes the computer program 112, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 10 The functions of each module are shown.

[0111] For example, the computer program 112 can be divided into one or more modules / units, which are stored in the memory 111 and executed by the processor 110 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 112 in the terminal 11. For example, the computer program 112 can be divided into... Figure 10 The modules shown.

[0112] The terminal 11 can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The terminal 11 may include, but is not limited to, a processor 110 and a memory 111. Those skilled in the art will understand that... Figure 11 This is merely an example of terminal 11 and does not constitute a limitation on terminal 11. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal may also include input / output devices, network access devices, buses, etc.

[0113] The processor 110 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0114] The memory 111 can be an internal storage unit of the terminal 11, such as a hard disk or memory of the terminal 11. The memory 111 can also be an external storage device of the terminal 11, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal 11. Furthermore, the memory 111 can include both internal storage units and external storage devices of the terminal 11. The memory 111 is used to store the computer program and other programs and data required by the terminal. The memory 111 can also be used to temporarily store data that has been output or will be output.

[0115] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0116] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0117] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0118] In the embodiments provided in this application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above-described embodiments of the intelligent unloading strategy determination method for UAV missions. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0122] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for determining an intelligent unloading strategy for a drone mission, characterized in that, include: A drone mission intelligent offloading strategy model is constructed for a target network system. The drone mission intelligent offloading strategy model aims to maximize the long-term average weighted computing rate of all drones in the target network system. The target network system is a network system consisting of a base station equipped with a mobile edge computing server and multiple rotary-wing drone devices. The UAV mission intelligent offloading strategy model was initially optimized using a Lyapunov function with added penalty terms, resulting in the initially optimized UAV mission intelligent offloading strategy model. The initially optimized UAV mission intelligent offloading strategy model is further optimized using a deep reinforcement learning model to obtain the optimized UAV mission intelligent offloading strategy model. Using the optimized UAV mission intelligent offloading strategy model, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame are obtained.

2. The method for determining the intelligent unloading strategy for UAV missions according to claim 1, characterized in that, The objective function of the intelligent unloading strategy model for UAV missions is: in, To maximize the function, For the first Fixed weights for each drone For the first The drone in the first The computation rate per time frame The total number of time frames. The total number of drones, For the decision of unloading, CPU frequency, This represents the percentage of time spent uninstalling. To unload power consumption.

3. The method for determining the intelligent unloading strategy for UAV missions according to claim 2, characterized in that, The constraint condition for constructing the UAV mission intelligent offloading strategy model of the target network system is the first constraint condition, which is: in, For the first The time frame allocated to the first The percentage of unloading time for each drone in the total time. For the first The drone in the first Unloading decision for each time frame For the first CPU frequency of a drone For the first The maximum CPU frequency of a drone For the first The number of CPU cycles required for a drone to process one bit of data. For uplink bandwidth, For the first The drone in the first The power consumption of offloading is calculated for each time frame. For channel gain, For noise power, For the first The drone in the first The queue length starting from each time frame. For mathematical expectation, The energy coefficient, For the first Power threshold for each drone For the first Maximum transmission power of each drone.

4. The method for determining the intelligent unloading strategy for UAV missions according to claim 1, characterized in that, The objective function of the initially optimized UAV mission intelligent offloading strategy model is: in, To maximize the function, For the first The drone in the first The queue length starting from each time frame. The weighting coefficient for the penalty term. For the first Fixed weights for each drone For the first The drone in the first The computation rate per time frame For the first A virtual energy queue, For the first The drone in the first Energy consumption within a time frame The total number of drones, For drones in the The set of unloading decisions within a time frame. For drones in the The set of time percentages used for unloading within each time frame. For drones in the A set of CPU frequencies within a time frame. For drones in the The set of power consumption unloaded within each time frame. For the first The set of auxiliary variables for the calculation rate of the unloading task within each time frame.

5. The method for determining the intelligent unloading strategy for UAV missions according to claim 4, characterized in that, The constraint condition of the initially optimized UAV mission intelligent offloading strategy model is the second constraint condition, which is: in, For the first The time frame allocated to the first The unloading time of each drone is part of the total time. The proportion, For the first CPU frequency of a drone For the first The number of CPU cycles required for a drone to process one bit of data. For the first The drone in the first The offloading task calculation rate auxiliary variable within each time frame For uplink bandwidth, For the first The drone in the first The power consumption of offloading is calculated for each time frame. For channel gain, For noise power, For the first The maximum CPU frequency of a drone For the first Maximum transmission power of each drone For the first The drone in the first Unloading decisions for each time frame.

6. The method for determining the intelligent unloading strategy for UAV missions according to claim 1, characterized in that, The objective function of the optimized UAV mission intelligent offloading strategy model is: in, To maximize the function, For the optimal unloading decision, This represents the optimal target value achieved through optimal resource allocation. For drones in the The set of unloading decisions within a time frame. For drones in the The set of system parameters within a time frame.

7. The method for determining the intelligent unloading strategy for UAV missions according to claim 1, characterized in that, The optimized UAV mission intelligent offloading strategy model is used to obtain the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame, including: Acquire real-time data of the target network system, including the number of drones, total time frames, parameters, update interval, energy consumption threshold, and learning rate of the target network system; The real-time data of the target network system is input into the optimized UAV mission intelligent offloading strategy model, and the optimal offloading strategy and resource allocation strategy for each UAV in the target network system in each time frame are output.

8. The method for determining the intelligent unloading strategy for a UAV mission according to claim 7, characterized in that, Before obtaining the optimal unloading strategy and resource allocation strategy for each UAV in the target network system within each time frame using the optimized UAV mission intelligent unloading strategy model, the method further includes: A first-in-first-out (FIFO) mechanism is used to remove historical data from the optimized UAV mission intelligent offloading strategy model. The historical data includes the number of UAVs, total time frames, parameters, update interval, energy consumption threshold, and learning rate in the target network system. Accordingly, the process of using the optimized UAV mission intelligent offloading strategy model to obtain the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame includes: By utilizing the optimized UAV mission intelligent offloading strategy model after removing historical data, the optimal offloading strategy and resource allocation strategy for each UAV in the target network system within each time frame are obtained.

9. A terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for determining the intelligent unloading strategy for unmanned aerial vehicle missions as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for determining the intelligent unloading strategy for unmanned aerial vehicle missions as described in any one of claims 1 to 8.

Images