Unmanned aerial vehicle deployment, calculation unloading and resource allocation joint optimization method based on space-air-ground integrated network

Abstract

The invention discloses an unmanned aerial vehicle deployment, calculation unloading and resource allocation joint optimization method based on a space-air-ground integrated network, and the method comprises the steps: constructing an SAGIN-IoRT edge calculation architecture assisted by an unmanned aerial vehicle (UAV), and building an unmanned aerial vehicle mobile model, a communication model, an energy consumption model and a calculation model; establishing an unmanned aerial vehicle deployment, calculation unloading and resource allocation joint optimization model; according to the unmanned aerial vehicle deployment problem, a distributed deployment algorithm based on MASD3 is adopted, part of observation information obtained by the unmanned aerial vehicle is utilized, and the unmanned aerial vehicle can adjust the flight path in a self-adaptive mode through autonomous decision making of an intelligent agent; for the strong coupling problem of calculation unloading and resource allocation, the calculation unloading sub-problem and the resource allocation sub-problem are decoupled. According to the method, the task processing time delay of the ground equipment can be remarkably reduced, the balance between the energy efficiency of the unmanned aerial vehicle and the number of coverage equipment is achieved, and a high-reliability and low-delay joint optimization scheme is provided for emergency communication, wide-area Internet of Things monitoring and other scenes.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing and resource optimization, and particularly relates to a joint optimization method for UAV deployment, computing offloading, and resource allocation based on a space-air-ground integrated network. Background Art

[0002] In the current era of rapid digital development, the application scenarios of the Internet of Remote Things (IoRT) are constantly expanding. Especially in remote areas, the demand for computing and communication services from a large number of Internet of Things devices has increased explosively. However, traditional terrestrial cellular networks have problems such as limited coverage and vulnerable signal blockage by terrain in remote areas, making it difficult to meet the real-time communication and efficient computing needs of ground devices. The space-air-ground integrated network (SAGIN) integrates the communication resources of space-based networks, air-based networks, and ground-based networks, providing stable and efficient communication and computing support for IoRT devices, and becoming a key technical means to meet the application requirements of the remote Internet of Things.

[0003] In SAGIN, as a dynamic relay and edge computing node, UAVs can provide low-latency communication and computing services for IoRT devices lacking terrestrial cellular network coverage, becoming a research hotspot in recent years. However, the existing technologies still face multiple challenges: First, most previous studies assume that UAVs hover statically or follow a preset linear trajectory, resulting in the lack of adaptability of trajectory optimization to dynamic network environments (such as the randomness of device distribution and channel state fluctuations), limiting the task throughput and resource utilization efficiency. Second, traditional non-learning optimization methods (such as convex optimization and particle swarm algorithms) rely on static environment assumptions, making it difficult to meet the real-time decision-making requirements in high-dimensional continuous action spaces, and having problems such as low convergence efficiency and insufficient non-convex constraint processing capabilities. In addition, although deep reinforcement learning (DRL) has shown advantages in dynamic optimization, mainstream algorithms such as the Deep Deterministic Policy Gradient (DDPG) and Twin Delayed Deep Deterministic Policy Gradient (TD3) have problems with value function estimation bias, and rely on global information (such as real-time user locations and full network channel states) in partially observable scenarios (such as disaster emergency communication), limiting the practical deployment feasibility.

[0004] In addition, existing computing offloading strategies mostly use binary decisions, forcing tasks to be completely executed locally or offloaded to a single node, ignoring the potential for heterogeneous resource collaboration between satellites, UAVs, and ground devices, resulting in increased risk of mission interruption and insufficient support for latency-sensitive applications. Therefore, how to achieve efficient and low-latency services for IoRT devices in SAGIN by jointly optimizing UAV dynamic deployment, fine-grained task splitting, and resource allocation is still a technical bottleneck that needs to be broken through. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to propose a joint optimization method for UAV deployment, computation offloading and resource allocation based on an integrated air-ground-space network, so as to improve the service quality and system performance of IoRT devices and reduce task processing delays.

[0006] Technical solution: The invention discloses a joint optimization method for unmanned aerial vehicle deployment, computation offloading and resource allocation based on an air-ground integrated network, comprising the following steps:

[0007] Build the UAV-assisted SAGIN-IoRT edge computing architecture, and establish the UAV mobility model, communication model, energy consumption model, and computing model;

[0008] With the goal of minimizing the total task processing delay of all ground IoRT devices, a joint optimization model of UAV deployment, computation offloading and resource allocation is established with the UAV flight speed limit, standardized bandwidth allocation ratio and maximum tolerable task delay as constraints.

[0009] Aiming at the problem of UAV deployment in the joint optimization model, a distributed deployment algorithm based on MASD3 is adopted. The partial observation information obtained by the UAV is used to realize the adaptive adjustment of the flight trajectory of the UAV through the autonomous decision-making of the intelligent agent. The distributed deployment algorithm based on MASD3 adopts a dual value network and dual action network structure, and configures two action networks and two value networks for each UAV, as well as two corresponding target action networks and two target value networks. By sampling multiple random Gaussian noise points and adding them to the target action, the model is prevented from falling into the local optimal solution. The training data is stored in the experience playback buffer, and the target network parameters are soft-updated to stabilize the training process.

[0010] Aiming at the strong coupling problem between computation offloading and resource allocation in the joint optimization model, a decoupled optimization framework based on the block coordinate descent method is designed. The numerical calculation method is used to solve the distributed computing offloading sub-problem, and the optimal task offloading ratio is determined according to the device status and network environment. The convex optimization technology is used to allocate bandwidth resources to improve bandwidth utilization.

[0011] Furthermore, the UAV deployment problem is formulated as a decentralized partially observable Markov decision process, defining the local observation information, continuous action space, and reward function of the UAV;

[0012] The observation information o of the k-th UAV at time t k,t , consists of the positions of the nearest M IoRT devices within its coverage and all UAV positions , that is

[0013] The action a k,t includes the speed v of the UAV k,t and the transmission power P from the j-th IoRT device to the k-th UAV j,k,t , that is a k,t = [v k,t , P j,k,t ;

[0014] The reward r k,t is the number of ground IoRT devices within the coverage that meet the minimum data transmission rate requirement minus the corresponding normalized energy consumption, plus the environmental penalty factor , that is where is the set of IoRT devices within the coverage area of the k-th UAV, SN j,k,t is the indicator function, is v k,t and P j,k,t is the total energy consumption when reaching the maximum value.

[0015]

[0016] Among them, ρ1 and ρ2 are the weights of the UAV and satellite data transmission rates respectively, R th is the minimum uplink transmission rate requirement. In addition, is v k,t and P j,k,t is the total energy consumption when reaching the maximum value, is the penalty value when the k-th UAV flies out of the specified area (when flying out ), is the penalty value when the k-th UAV collides with other UAVs (when colliding ).

[0017] Furthermore, the distributed deployment algorithm based on MASD3 obtains the next action by sampling G random Gaussian noise points and adding them to the target action , as follows:

[0018]

[0019] Among them, is the parameter of the target action network, is the parameter of the target value network.

[0020] Furthermore, the distributed deployment algorithm based on MASD3 introduces the softmax operator to optimize the value function. The value network takes the state s' and action a' as inputs and outputs a dimension of G×1. The action value function estimation is optimized through the softmax operator:

[0021]

[0022] where ζ≥0 is the softmax operator parameter, is the estimated value of the value function of the k-th UAV based on the g-th random Gaussian noise sample when taking action a' in state s'. g is the index value from 1 to G, and G is the number of random Gaussian noise sampling points.

[0023] Furthermore, the distributed deployment algorithm based on MASD3 uses an experience replay buffer to store training data. During training, D B samples are randomly sampled from the experience replay buffer for updating network parameters. For the k-th UAV, the value network parameters are updated by minimizing the loss function of the value network

[0024]

[0025] where is the loss function of the value network, is the i-th estimated value of the value network when taking action a in state s under the parameter , is the target value, and γ is the discount factor. At the same time, the action network parameters are updated using the sampled policy gradient

[0026]

[0027] where J is the gradient of the objective function of the action network with respect to the parameter , is the gradient operator with respect to the action network parameter , is the i-th action based on the observation o under the parameter k , is the gradient operator with respect to the action a k .

[0028] Finally, the weights of the four target networks are softly updated at the update rate ω (ω << 1).

[0029] Furthermore, a decoupled optimization framework based on the block coordinate descent method is designed to split the complex coupling problem of computing offloading and resource allocation into a computing offloading sub-problem and a resource allocation sub-problem. By alternately iteratively optimizing the computing offloading sub-problem and the resource allocation sub-problem, the global optimal solution is gradually approximated.

[0030] Furthermore, the numerical calculation method is used to solve the distributed computing offloading sub-problem, the computing delays of local, UAV, and satellite nodes are balanced based on the bucket effect, and the optimal solution vector is obtained by using the Sherman-Morrison formula.

[0031] The present invention also provides a joint optimization system for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network, including:

[0032] An edge computing architecture construction unit for constructing a SAGIN-IoRT edge computing architecture assisted by UAVs, and establishing a UAV movement model, a communication model, an energy consumption model, and a computing model;

[0033] A joint optimization model construction unit for establishing a joint optimization model of UAV deployment, computing offloading, and resource allocation with the goal of minimizing the total task processing delay of all ground IoRT devices, subject to the constraints of UAV flight speed limits, specified bandwidth allocation ratios, and maximum tolerable task delays;

[0034] A UAV deployment problem solving unit for solving the UAV deployment problem in the joint optimization model, using a distributed deployment algorithm based on MASD3, and using partial observation information obtained by the UAVs to adaptively adjust the flight trajectories of the UAVs through the autonomous decision-making of agents; the distributed deployment algorithm based on MASD3 adopts a dual-value network and a dual-action network structure, configures two action networks and two value networks for each UAV, as well as corresponding two target action networks and two target value networks; by sampling multiple random Gaussian noise points and adding them to the target actions, the model is prevented from falling into a local optimal solution, and the training data is stored in an experience replay buffer, and the target network parameters are softly updated to stabilize the training process;

[0035] A computing offloading and resource allocation problem solving unit for solving the strongly coupled problem of computing offloading and resource allocation in the joint optimization model, designing a decoupled optimization framework based on the block coordinate descent method, using a numerical calculation method to solve the distributed computing offloading sub-problem, and determining the optimal task offloading ratio according to the device status and network environment; using convex optimization techniques for bandwidth resource allocation to improve bandwidth utilization.

[0036] The present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor, wherein the computer program / instructions, when executed by the processor, implement the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network.

[0037] The present invention also provides a computer program product, including a computer program / instructions, wherein the computer program / instructions, when executed by the processor, implement the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network.

[0038] Beneficial effects: Compared with the prior art, the remarkable technical effects of the present invention are as follows: Aiming at the IoRT scenario with missing ground cellular network coverage in SAGIN, a low-latency task processing method based on joint optimization of UAV dynamic deployment, computing offloading, and resource allocation is proposed; this method realizes the comprehensive improvement of system performance through a hierarchical optimization framework; First, construct an edge computing architecture for UAV-assisted SAGIN-IoRT; Secondly, based on the architecture, establish a series of accurate mathematical models, including UAV movement models, communication models, energy consumption models, and computing models, and set optimization problems to clarify the problem constraints; Then, aiming at the optimization problem of UAV deployment in a dynamic environment, a distributed deployment algorithm based on the MASD3 algorithm is proposed, which maximizes the uplink transmission rate of IoRT devices and minimizes the energy consumption of UAVs, and the algorithm has advantages such as high average reward, stable convergence, high robustness, and high efficiency in different-scale IoRT device scenarios; Further, aiming at the strong coupling of the computing offloading and resource allocation problems, the block coordinate descent BCD method is used to decouple the problem, and it is efficiently solved through distributed numerical calculation and convex optimization algorithms respectively, overcoming the defects of low resource utilization and poor fault tolerance of traditional binary offloading strategies; Finally, through simulation experiments, it is verified that by jointly optimizing UAV deployment, computing offloading, and resource allocation, the task processing delay is significantly reduced; Through this method, the system achieves a balance between UAV energy consumption and communication efficiency, while supporting the real-time task requirements of high-density IoRT devices, and improving network robustness and service coverage.

[0039] In summary, in the space-air-ground integrated edge computing scenario assisted by UAVs, the present invention can improve resource utilization and enhance adaptability to dynamic environments through a joint method based on the MASD3 algorithm and BCD optimization, thereby effectively reducing task processing delay and providing low-latency and high-reliability technical support for practical application scenarios such as emergency communication and wide-area Internet of Things monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flowchart of the method of the embodiment of the present invention;

[0041] Figure 2 This is the edge computing architecture diagram of UAV-assisted SAGIN-IoRT for the embodiments of the present invention;

[0042] Figure 3 This is the architecture diagram of the UAV deployment algorithm based on MASD3 for the embodiments of the present invention. Detailed implementation manners

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0044] The core idea of the present invention is as follows: For the application scenario where a large number of IoRT devices in remote areas require computing resources but the ground cellular network coverage is missing, an edge computing architecture based on SAGIN is proposed, and the UAV deployment, computing offloading, and resource allocation are jointly optimized. First, an edge computing architecture of UAV-assisted SAGIN-IoRT is constructed; secondly, a series of accurate mathematical models are established based on the architecture, including UAV movement models, communication models, energy consumption models, and computing models, and the optimization problems are set, and the problem constraints are clarified; then, aiming at the optimization problem of UAV deployment in a dynamic environment, a distributed deployment algorithm based on the MASD3 algorithm is proposed; further, aiming at the strong coupling of the computing offloading and resource allocation problems, the BCD method is used to decouple the problems, and they are efficiently solved respectively through distributed numerical calculation and convex optimization algorithms, overcoming the defects such as low resource utilization rate and poor fault tolerance of traditional binary offloading strategies; finally, through simulation experiments, it is verified that by jointly optimizing UAV deployment, task parallel splitting, and multi-node resource collaborative scheduling, the task processing delay is significantly reduced. The present invention will be further described in detail below.

[0045] As Figure 1 shown, a method for jointly optimizing UAV deployment, computing offloading, and resource allocation based on an integrated space-air-ground network disclosed in the embodiments of the present invention mainly includes the following steps:

[0046] (1) Construct an edge computing architecture of UAV-assisted SAGIN-IoRT, which consists of satellites, UAVs, and ground Internet of Things (IoRT) devices. The satellite is responsible for providing access to cloud computing resources, achieving comprehensive coverage of the target area through the satellite backbone network, and providing low-latency cloud computing services for critical tasks; the UAV is equipped with a Mobile Edge Computing (MEC) server, which is flexibly deployed in the air, receives and processes tasks sent by ground IoRT devices, effectively reducing data transmission latency; ground IoRT devices generate computationally intensive tasks with time sensitivity, but their own computing capabilities are limited, and they achieve task offloading and processing through interaction with satellites and UAVs.

[0047] Specifically, the constructed UAV-assisted SAGIN-IoRT edge computing architecture includes a satellite responsible for providing cloud computing access services, and K UAVs carrying MEC servers, and their set is denoted as while the J ground IoRT devices generate latency-sensitive tasks, and their set is denoted as As shown in the appendix Figure 2 . This architecture is mainly targeted at remote areas. In the case of the absence of ground cellular network coverage, it provides services for a large number of IoRT devices that require computing resources. The UAVs fly in the air at a height of H and can receive and process the tasks sent by the ground IoRT devices within their coverage. The ground IoRT devices are randomly distributed in the specified area. Although they have limited computing power, they generate latency-sensitive computing tasks that have strict requirements for processing time. Each Internet of Robots and Things (IoRT) device carries a divisible computing task. Taking the jth ground IoRT device as an example, its task can be expressed as where S j represents the size of the input data of the computing task carried by this device, C j represents the total number of CPU cycles required to complete this task, represents the maximum tolerable delay for completing this task.

[0048] (2) Based on the edge computing architecture, a series of accurate mathematical models are established, including the UAV movement model, communication model, energy consumption model, and computing model. The UAV movement model describes the flight trajectory changes of the UAV at different times, comprehensively considering the UAV flight speed limit and environmental interference factors; the communication model is for the communication links between the ground IoRT devices and the UAVs and satellites, and adopts the probabilistic path loss model, fully considering the influence of the Line-of-Sight (LoS) and Non-Line-of-Sight (NLoS) links to accurately calculate the data transmission rate; the energy consumption model analyzes the energy losses during the flight and communication of the UAVs respectively to provide a basis for optimizing energy consumption; the computing model combines the task allocation strategy to evaluate the computing latency of tasks on local, UAV, and satellite nodes.

[0049] UAV movement model: The continuous time domain is discretized into T equal-length time slots. Within each time slot, it is assumed that the relevant parameters remain unchanged, but they will vary between different time slots. Let the horizontal coordinate of the kth UAV at time t be where, x k,t and y k,t represent the horizontal and vertical coordinates of the kth UAV on the horizontal plane at time t respectively, and satisfy X min ≤x k,t ≤X max , Y min ≤y k,t ≤Ymax , X min is the minimum limit of the abscissa of the k-th UAV on the horizontal plane at time t, X max is the maximum limit of the abscissa of the k-th UAV on the horizontal plane at time t, Y min is the minimum limit of the ordinate of the k-th UAV on the horizontal plane at time t, Y max is the maximum limit of the abscissa of the k-th UAV on the horizontal plane at time t, and its flight speed at time t is v k,t =(v x,k,t , v y,k,t ), where v x,k,t and v x,k,t respectively represent the horizontal and vertical speeds of the k-th UAV at time t, and the speed limit is 0 ≤ ‖v k,t ‖ ≤ V max , V max is the maximum limit of the flight speed of the k-th UAV. Then the horizontal position change formula of the k-th UAV is:

[0050] x k,t+1 = x k,t + v x,k,t τ + θ k,t

[0051] y k,t+1 = y k,t + v y,k,t τ + θ k,t

[0052] where, x k,t+1 and y k,t+1 respectively represent the abscissa and ordinate of the k-th UAV on the horizontal plane at time t + 1, θ k,t represents the influence factor introduced by uncertain factors such as the environment, and τ is the duration of each time slot.

[0053] Communication model: It includes the communication model from ground IoRT devices to UAVs and the communication model from ground IoRT devices to satellites.

[0054] Communication from ground IoRT devices to UAVs: Since the computational offloading time scale is much longer than the traditional resource scheduling time, large-scale channel fading is mainly considered, and a probabilistic path loss model is adopted. The path loss PL j,k,t from the j-th ground IoRT device to the k-th UAV at time t is:

[0055]

[0056] where, f c is the carrier frequency, c is the speed of light, is the horizontal distance between the j-th IoRT device and the k-th UAV, H is the UAV flight altitude, x j,t and y j,t respectively represent the horizontal coordinates of the j-th IoRT device on the horizontal plane at time t, is the LoS link probability, is the NLoS link probability, η LoS and η NLoS are the additional losses of the LoS and NLoS links above the free-space path loss respectively.

[0057] In addition, the LoS and NLoS link probabilities are respectively:

[0058]

[0059] where α and β are constants determined by the environment;

[0060] Then, the data transmission rate R from the j-th ground IoRT device to the k-th UAV is j,k,t as follows:

[0061]

[0062] where, is the bandwidth ratio allocated by the k-th UAV to the j-th ground IoRT device, B UAV is the available bandwidth of each UAV, P j,k,t is the transmission power from the ground IoRT device to the k-th UAV, σ 2 is the random Gaussian noise power.

[0063] Ground IoRT device to satellite communication: The data transmission rate R from the j-th ground IoRT device to satellite s is j,s,t as follows:

[0064]

[0065] where, B SAT is the available bandwidth of the satellite, P j,s,t is the transmission power from the ground IoRT device to satellite s, PL j,s,t is the path loss from the j-th ground IoRT device to satellite s at time t.

[0066] Energy consumption model: Focus on the flight energy consumption and communication energy consumption of the UAV itself.

[0067] UAV flight energy consumption: Mainly focus on the propulsion energy consumption during the UAV movement. The propulsion energy consumption of the k-th UAV at time t is:

[0068]

[0069] Among them, is the propulsion energy consumption of the k-th UAV at time t, and P prop is the UAV blade power, and U tip is the UAV rotor tip speed.

[0070] UAV communication energy consumption: If the task of the j-th ground IoRT device is transmitted to the k-th UAV, the communication energy consumption per unit time is:

[0071]

[0072] Among them, is the communication energy consumption per unit time when the task of the j-th ground IoRT device is transmitted to the k-th UAV at time t.

[0073] Calculation model: On the premise that the task can be split, the subtasks can be preferentially allocated to the ground IoRT devices for local processing, or dynamically offloaded to remote nodes (such as UAVs or satellite platforms) for execution according to the calculation resource requirements, so as to achieve the collaborative optimization of task processing efficiency and resource utilization rate.

[0074] Local calculation: When the task of the j-th ground IoRT device is selected for local processing, the delay is:

[0075]

[0076] Among them, is the delay of the task of the j-th ground IoRT device in local processing, is the proportion of the calculation task of the j-th ground IoRT device in local processing, and f IoRT is the computing power of the ground IoRT device, that is, the number of cycles that can be executed per second.

[0077] Remote calculation: When the task is offloaded to the UAV or satellite node for processing, the delay includes the time for the task to queue and wait for time slot resources, the task data upload time, and the time for the UAV or satellite node to execute the computing service. Assuming that the UAV or satellite node starts the computing service after receiving all the subtask transmissions, the delays of the j-th ground IoRT device offloaded to the k-th UAV node and the satellite node are respectively:

[0078]

[0079] Among them, is the delay of the j-th ground IoRT device offloaded to the k-th UAV node, is the delay of the j-th ground IoRT device offloaded to the satellite s node, is the proportion of the computing tasks offloaded from the j-th ground IoRT device to the k-th UAV node, is the proportion of the j-th ground IoRT device offloaded to the satellite. Since the remote calculation formula is independent of time slots, R here j,k,t and R j,s,t Delete the time slot t in the subscript, but the definition is the same as before, so R j,k is the data transmission rate from the j-th ground IoRT device to the k-th UAV, and R j,s is the data transmission rate from the j-th ground IoRT device to the satellite s. f UAV and f SAT are the computing capabilities of the UAV and the satellite respectively, and D j,k and D j,K+1 are variables introduced to simplify the delay equation. Since multiple subtasks can be processed in parallel, the delay T j for the j-th ground IoRT device to complete all computing tasks is:

[0080]

[0081] (3) Set the goal of the optimization problem to minimize the total task processing delay of all ground IoRT devices. To this end, set constraint conditions, including restrictions on the flight speed of UAVs to ensure flight safety and reasonable energy consumption; standardize the bandwidth allocation ratio to ensure the fairness and efficiency of resource allocation; set the maximum tolerable delay of tasks to meet the timeliness requirements of tasks.

[0082] Establish an optimization problem, and set the optimization goal to minimize the total task processing delay of all ground IoRT devices. It is necessary to obtain the optimal positions c UAV of all UAV deployments, the offloading decision matrix of each ground IoRT device, whose elements are the bandwidth allocation strategy matrix of each UAV, whose elements are In addition, set constraint conditions, including UAV flight speed limits, bandwidth allocation ratios, maximum tolerable task delays, etc. The optimization goal is as follows:

[0083]

[0084] Among them, P max is the maximum value of the transmission power from the ground IoRT device to the UAV. C1 ensures that the UAV flight speed is within the safe and device capabilities range. C2 - C3 limit the flight position range of the UAV. C4 stipulates the upper limit of the transmission power from the ground IoRT device to the UAV. C5 ensures that tasks can be fully allocated and processed. C6 clarifies the reasonable range of task offloading ratios. C7 - C8 standardize the UAV bandwidth allocation ratio. C9 ensures that tasks are completed within the maximum tolerable delay.

[0085] (4) To address the problem of optimizing UAV deployment in a dynamic environment, a distributed deployment algorithm based on Multi-Agent Softmax Deep Double Deterministic Policy Gradient (MASD3) is proposed. By using the partial observation information obtained by the UAV and through the autonomous decision-making of the agents, the UAV can adaptively adjust its flight trajectory. Compared with traditional offline policies, it effectively overcomes the problem of poor adaptability to dynamic scenarios and improves the flexibility of UAV deployment and the overall system performance.

[0086] Step (4) is to first optimize the Unmanned Aerial Vehicle (UAV) deployment to establish a communication coverage area, ensure task schedulability, and lay the foundation for step (5). Due to the time difference between UAV deployment adjustment (in seconds) and computational offloading along resource allocation (in milliseconds), by adopting a hierarchical design and following the idea of first ensuring connectivity and then optimizing performance, the global latency is minimized while ensuring the robustness of the system. Therefore, in step (4), the UAV deployment algorithm aims to maximize the uplink transmission rate of IoRT devices while minimizing the energy consumption of the UAV. The optimization objective is designed as follows:

[0087]

[0088] where λ is the weight of the energy consumption term.

[0089] The UAV deployment problem is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), defining the local observation information of the UAV (proximity device locations, other UAV locations), the continuous action space (speed and transmission power), and the reward function (number of covered devices minus normalized energy consumption).

[0090] In this model, each UAV acts as an agent and makes autonomous decisions in a dynamic network environment. The observation information o of the k-th UAV at time t k,t consists of the positions of the M nearest ground IoRT devices within its coverage area and all UAV positions , that is

[0091] The action a k,t includes the speed v of the UAV k,t and the transmission power P from the j-th ground IoRT device to the k-th UAV j,k,t , that is a k,t = [vk,t , P j,k,t .

[0092] Reward r k,t is the number of ground IoRT devices that meet the minimum data transmission rate requirement within the coverage range minus the corresponding normalized energy consumption, plus the environmental penalty factor That is where is the set of IoRT devices within the coverage area of the k-th UAV, SN j,k,t is the indicator function, is v k,t and P j,k,t is the total energy consumption when reaching the maximum value.

[0093]

[0094] where ρ1 and ρ2 are the weights of the UAV and satellite data transmission rates respectively, R th is the minimum uplink transmission rate requirement. In addition, is v k,t and P j,k,t is the total energy consumption when reaching the maximum value, is the penalty value for the k-th UAV flying out of the specified area (when flying out ), is the penalty value for the k-th UAV colliding with other UAVs (when colliding ).

[0095] For generality in the subsequent description, the current observation value, action, and the reward of the k-th UAV are defined as o k , a k and r k , o k ' and a k ' respectively represent the observation value and action of the k-th UAV at the next moment. In addition, define s = {o1, o2,..., o K} and s' = {o1', o2',..., o K '}, where s and s' represent the states of all UAVs at the current moment and the next moment respectively. Similarly, define a = {a1, a2,..., a K} and a' = {a1', a2',..., a K '}, where a and a' represent the action sets of all UAVs at the current moment and the next moment respectively. Finally, define r = {r1, r2,..., r K} as the reward set of all UAVs.

[0096] Next, as shown in the appendix Figure 3As shown, the MASD3 algorithm adopts a dual-value network and a dual-action network structure, introduces a softmax operator to optimize the value function estimation, and alleviates the underestimation bias of traditional algorithms. Two action networks are configured for each UAV and two value networks as well as two corresponding target action networks and two target value networks Through this structure, the stability and decision-making accuracy of the algorithm are enhanced, and the bias problem in the value function estimation of traditional algorithms is effectively alleviated.

[0097] The MASD3 algorithm adds multiple randomly sampled Gaussian noise points to the target action to increase the exploration of the training process and prevent the model from falling into local optimal solutions. This enables the UAV to more comprehensively explore different flight trajectories and transmission power combinations in complex dynamic environments, improving the quality of the deployment strategy. Specifically:

[0098] By sampling G randomly sampled Gaussian noise points and adding them to the target action the next action is obtained as:

[0099]

[0100] where are the parameters of the target action network, are the parameters of the target value network.

[0101] The value network takes the state s' and action a' as inputs and outputs a dimension of G×1. The action value function estimation is optimized through the softmax operator:

[0102]

[0103] where ζ≥0 is the softmax operator parameter, is the value function estimation value based on the g-th randomly sampled Gaussian noise sample when the k-th UAV is in state s′ and takes action a′. g is the index value from 1 to G, and G is the number of randomly sampled Gaussian noise points.

[0104] Adding Gaussian noise enhances exploration and prevents local optimality. By the above method of adding random Gaussian noise, the exploration of the training process is increased, and the model is prevented from falling into local optimal solutions.

[0105] During training, the training data is stored in an experience replay buffer, and the target network parameters are softly updated to stabilize the training process. During training, D B samples are randomly sampled from the experience replay buffer for updating the network parameters. For the k-th UAV, the value network parameters are updated by minimizing the loss function of the value network

[0106]

[0107] Among them, is the loss function of the value network, is the i-th value network estimation value when the state s takes the action a under the parameter , is the target value, and γ is the discount factor. At the same time, the action network parameters are updated using the sampled policy gradient

[0108]

[0109] Among them, is the objective function gradient of the action network with respect to the parameter , is the gradient operator with respect to the action network parameter , is the i-th action based on the observation o under the parameter k , is the gradient operator with respect to the action a k .

[0110] Finally, the weights of the four target networks are softly updated at the update rate ω (ω << 1). These two mechanisms stabilize the training process, accelerate the algorithm convergence speed, and improve the algorithm performance.

[0111] (5) Aiming at the strong coupling problem of computing offloading and resource allocation, a decoupling optimization framework based on the Block Coordinate Descent (BCD) method is proposed. The block coordinate descent method is used to split it into a computing offloading sub-problem and a resource allocation sub-problem, gradually approaching the global optimal solution, and effectively reducing the problem-solving complexity. On the one hand, a numerical calculation method is adopted to solve the distributed computing offloading sub-problem, and the optimal task offloading strategy is determined according to the device status and network environment; on the other hand, convex optimization technology is used for bandwidth resource allocation to improve the bandwidth utilization rate.

[0112] After the UAV obtains the best deployment position c UAV using step (4), it hovers. At this time, to solve the complex coupling problem of computing offloading and resource allocation, it is considered to be split into a computing offloading sub-problem and a resource allocation sub-problem, and the BCD method is used for alternating optimization. The computing offloading sub-problem aims to determine the task offloading ratio of each ground IoRT device, and the resource allocation sub-problem focuses on optimizing the ratio of UAV bandwidth allocated to each ground IoRT device. By alternately iterating these two sub-problems, the global optimal solution is gradually approached.

[0113] Regarding the computing offloading strategy, a distributed computing offloading algorithm based on numerical calculation methods is adopted. Assuming that each IoRT device is rational and self-interested, the total latency of all computing tasks in the original optimization objective can be transformed into the latency of the computing tasks carried by each IoRT device. Given the position c of the unmanned aerial vehicle (UAV) UAV and the resource allocation decision B, the original optimization problem in step (3) can be reformulated as:

[0114]

[0115] where Q goal (A) is the objective function of computing offloading, and the variable is A.

[0116] Based on the cask effect to balance the computing latencies of the local, UAV, and satellite nodes, a system of linear equations is established to solve for the optimal task offloading ratio. According to the latency relationship among local computing, UAV computing, and satellite computing in the computing model, to minimize the total latency for the j-th ground IoRT device to complete the computing task, it should be such that:

[0117]

[0118] where is the optimal latency for the j-th ground IoRT device to complete the computing task.

[0119] Further derivation gives:

[0120]

[0121] After a series of transformations, we get:

[0122]

[0123] where is the set of UAV and satellite nodes that the j-th ground IoRT device can choose.

[0124] Next, using the Sherman-Morrison formula to analyze the offloading ratio, the optimal solution vector can be obtained. A distributed request-response mechanism is designed to dynamically adjust the strategy, breaking through the limitations of traditional binary offloading strategies. The above equation is transformed into matrix form (E + uw T )A j = f, where A j is the offloading ratio matrix of the j-th IoRT device, E = diag(D j,1 ,…,D j,K+1 ), u = (1,1,…,1) T , w T = (t loc ,t loc ,…,tloc )

[0125] According to the Sherman-Morrison formula, the optimal solution vector can be obtained as follows:

[0126]

[0127] where is the optimal solution of A j .

[0128] Its component equations are as follows:

[0129]

[0130] where is the optimal proportion of the computing tasks offloaded from the j-th ground IoRT device to the k-th UAV node or satellite node, is the optimal proportion of the computing tasks processed locally by the j-th ground IoRT device.

[0131] During the actual iterative process, the ground IoRT device preferentially selects the available UAV or satellite node with a smaller current subtask load and dynamically adjusts the offloading strategy by sending a Request for Update (RFU) frame, and the UAV or satellite node randomly responds to the RFU signal.

[0132] Regarding the optimization of communication resource allocation, the bandwidth allocation is modeled as a convex optimization problem. It is verified that the second derivative of the objective function is positive definite, and the CVX toolbox is used to solve the optimal bandwidth allocation. Given the UAV position c UAV and the offloading decision A, the objective function of the resource allocation sub-problem at this time is:

[0133]

[0134] where Q goal (B) is the objective function of the communication resource allocation optimization, and the variable is B.

[0135] Taking the second derivative of the objective function with respect to B, we can obtain:

[0136]

[0137] In addition, the constraint conditions are linear, indicating that this problem is a convex optimization problem. The optimal bandwidth allocation strategy B can be obtained by using an optimization algorithm tool (such as the CVX toolbox).

[0138] Finally, based on the BCD method, the offloading and resource allocation are alternately iteratively calculated until the objective function converges (the relative error is less than the threshold 10 -4 ). By setting the iteration variable The initial value is 0, and the convergence threshold ε = 10 is set. -4 . In each iteration, first use the above-mentioned distributed computing offloading algorithm to solve the computing offloading sub-problem, obtain the updated computing offloading decision A, and then use the CVX toolbox to solve the resource allocation sub-problem to obtain the updated resource allocation decision B. Calculate the objective function value of the current iteration and the objective function value of the next iteration When , it is considered that the objective function converges, the iteration ends, and the final joint optimization strategy for computing offloading and resource allocation is obtained.

[0139] According to the description of this embodiment, those skilled in the art should not find it difficult to see that the present invention proposes a method for jointly optimizing UAV deployment, computing offloading, and resource allocation in SAGIN. By dynamically optimizing the UAV deployment and heterogeneous resource collaborative allocation strategy, the end-to-end task processing delay can be significantly reduced and the network energy efficiency can be improved.

[0140] The embodiment of the present invention also provides a joint optimization system for UAV deployment, computing offloading, and resource allocation based on a space-air-ground integrated network, including:

[0141] An edge computing architecture construction unit, used to construct a SAGIN-IoRT edge computing architecture assisted by unmanned aerial vehicles (UAVs), and establish a UAV movement model, communication model, energy consumption model, and computing model;

[0142] A joint optimization model construction unit, used to establish a joint optimization model for UAV deployment, computing offloading, and resource allocation with the goal of minimizing the total task processing delay of all ground IoRT devices, subject to UAV flight speed limits, specified bandwidth allocation ratios, and maximum tolerable task delays;

[0143] A UAV deployment problem solving unit, used to solve the UAV deployment problem in the joint optimization model, adopt a distributed deployment algorithm based on MASD3, utilize partial observation information obtained by the UAV, and through the autonomous decision-making of agents, realize the adaptive adjustment of the UAV flight trajectory; the distributed deployment algorithm based on MASD3 adopts a dual-value network and a dual-action network structure, configures two action networks and two value networks for each UAV, as well as corresponding two target action networks and two target value networks; by sampling multiple random Gaussian noise points and adding them to the target action, avoid the model falling into a local optimal solution, store training data through an experience replay buffer, and softly update the target network parameters to stabilize the training process;

[0144] The computing offloading and resource allocation problem solving unit is used to design a decoupled optimization framework based on the block coordinate descent method for the strongly coupled problem of computing offloading and resource allocation in the joint optimization model, solve the distributed computing offloading sub-problem using numerical calculation methods, and determine the optimal task offloading ratio according to the device status and network environment; utilize convex optimization techniques for bandwidth resource allocation to improve bandwidth utilization.

[0145] An electronic device disclosed in an embodiment of the present invention includes a memory, a processor, and a computer program / instructions stored in the memory and executable on the processor. When the computer program / instructions are executed by the processor, the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network in the foregoing embodiment are implemented.

[0146] An embodiment of the present invention also discloses a computer program product, including a computer program / instructions. When the computer program / instructions are executed by the processor, the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network in the foregoing embodiment are implemented.

[0147] The content not detailedly described in the present invention belongs to the prior art well-known to those skilled in the art.

Claims

1. A joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network, characterized in that, It includes the following steps: Construct a SAGIN-IoRT edge computing architecture assisted by an unmanned aerial vehicle (UAV), and establish a UAV movement model, communication model, energy consumption model, and computing model; With the goal of minimizing the total task processing delay of all ground IoRT devices, and subject to the UAV flight speed limit, standardized bandwidth allocation ratio, and maximum tolerable task delay, establish a joint optimization model for UAV deployment, computing offloading, and resource allocation; For the UAV deployment problem in the joint optimization model, adopt a distributed deployment algorithm based on MASD3. Utilize partial observation information obtained by the UAV, and through the autonomous decision-making of agents, realize the adaptive adjustment of the UAV flight trajectory. The distributed deployment algorithm based on MASD3 adopts a dual-value network and dual-action network structure, and configures two action networks and two value networks for each UAV, as well as the corresponding two target action networks and two target value networks. Add multiple randomly sampled Gaussian noise points to the target action to avoid the model falling into a local optimal solution, store training data through an experience replay buffer, and softly update the target network parameters to stabilize the training process; For the strongly coupled problem of computing offloading and resource allocation in the joint optimization model, design a decoupled optimization framework based on the block coordinate descent method. Adopt a numerical calculation method to solve the distributed computing offloading sub-problem, and determine the optimal task offloading ratio according to the device status and network environment. Use convex optimization technology for bandwidth resource allocation to improve bandwidth utilization.

2. The joint optimization method for UAV deployment, computing offloading and resource allocation based on the space-air-ground integrated network according to claim 1, wherein Construct the UAV deployment problem as a decentralized partially observable Markov decision process, and define the local observation information, continuous action space, and reward function of the UAV; Observation information \(o\) of the \(k\)-th UAV at time \(t\) k,t , which consists of the positions of the nearest \(M\) IoRT devices within its coverage and all UAV positions , that is Action a k,t includes the speed v of the UAV k,t and the transmission power P from the j-th IoRT device to the k-th UAV j,k,t , that is, a k,t = [v k,t , P j,k,t ; Reward r k,t Subtract the corresponding normalized energy consumption from the number of ground IoRT devices that meet the minimum data transmission rate requirement within the coverage area, and add the environmental penalty factor That is Where is the set of IoRT devices in the coverage area of the kth UAV, SN j,k,t is the indicator function, is ||v k,t || and P j,k,t The total energy consumption when reaching the maximum value where ρ1 and ρ2 are the weights of the data transmission rates of the UAV and the satellite, respectively, and R th is the minimum uplink transmission rate requirement. In addition, is ||v k,t || and P j,k,t when reaching the maximum value, the total energy consumption is the penalty value for the k-th UAV flying out of the specified area, is the penalty value for the k-th UAV colliding with other UAVs.

3. The joint optimization method for UAV deployment, computing offloading and resource allocation based on the space-air-ground integrated network according to claim 1, characterized in that The distributed deployment algorithm based on MASD3 samples G random Gaussian noise points and adds them to the target action to obtain the next action as follows: Among them, are the parameters of the target action network, are the parameters of the target value network.

4. The joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network according to claim 1, characterized in that The distributed deployment algorithm based on MASD3 introduces a softmax operator to optimize the value function. The value network takes the state s' and action a' as inputs, and the output dimension is G×1. Optimize the action value function estimation through the softmax operator: where ζ≥0 is the softmax operator parameter, is the estimated value of the value function for the k-th UAV at state s′ and taking action a′, based on the g-th sample of random Gaussian noise. g is the index value from 1 to G, and G is the number of random Gaussian noise sampling points.

5. The joint optimization method for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network according to claim 1, wherein The distributed deployment algorithm based on MASD3 uses an experience replay buffer to store training data, and randomly samples D B samples from the experience replay buffer during training for updating network parameters. For the k-th UAV, the value network parameters are updated by minimizing the loss function of the value network where, is the loss function of the value network, is the i-th value network estimated value when the state s takes the action a under the parameter , and is the target value, and γ is the discount factor. Meanwhile, the action network parameters are updated using the sampled policy gradient Among them, is the objective function gradient of the action network with respect to the parameter , is the gradient operator with respect to the action network parameter , is, under the parameter , the i-th action based on the observation o k , is the gradient operator with respect to the action a k . Finally, softly update the weights of the four target networks at the update rate ω.

6. The joint optimization method for UAV deployment, computing offloading and resource allocation based on the space-air-ground integrated network according to claim 1, wherein, Design a decoupled optimization framework based on the block coordinate descent method, split the complex coupling problem of computing offloading and resource allocation into a computing offloading sub-problem and a resource allocation sub-problem, and gradually approximate the global optimal solution by alternately iteratively optimizing the computing offloading sub-problem and the resource allocation sub-problem.

7. The joint optimization method for UAV deployment, computing offloading and resource allocation based on the space-air-ground integrated network according to claim 1, wherein The numerical calculation method is used to solve the distributed computing offloading sub-problem, balance the computing delays of local, UAV, and satellite nodes based on the cask effect, and obtain the optimal solution vector using the Sherman-Morrison formula.

8. A joint optimization system for UAV deployment, computing offloading, and resource allocation based on the space-air-ground integrated network, characterized in that It includes: An edge computing architecture construction unit for constructing a SAGIN-IoRT edge computing architecture assisted by an unmanned aerial vehicle (UAV), and establishing a UAV movement model, communication model, energy consumption model, and computing model; A joint optimization model construction unit for establishing a joint optimization model for UAV deployment, computing offloading, and resource allocation with the goal of minimizing the total task processing delay of all ground IoRT devices and subject to the UAV flight speed limit, standardized bandwidth allocation ratio, and maximum tolerable task delay; The UAV deployment problem solving unit is used to solve the UAV deployment problem in the joint optimization model. It adopts a distributed deployment algorithm based on MASD3, utilizes partial observation information obtained by the UAVs, and through the autonomous decision-making of agents, realizes the adaptive adjustment of the flight trajectories of the UAVs. The distributed deployment algorithm based on MASD3 adopts a dual value network and a dual action network structure, configures two action networks and two value networks for each UAV, as well as the corresponding two target action networks and two target value networks. By sampling multiple random Gaussian noise points and adding them to the target actions, it avoids the model falling into local optimal solutions, stores training data through an experience replay buffer, and softly updates the target network parameters to stabilize the training process. The computing offloading and resource allocation problem solving unit is used to solve the strongly coupled problem of computing offloading and resource allocation in the joint optimization model. It designs a decoupled optimization framework based on the block coordinate descent method, adopts a numerical calculation method to solve the distributed computing offloading sub-problem, and determines the optimal task offloading ratio according to the device status and network environment. It uses convex optimization technology for bandwidth resource allocation to improve bandwidth utilization.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program / instructions stored on the memory and executable on the processor. It is characterized in that when the computer program / instructions are executed by the processor, the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the air-ground-space integrated network according to any one of claims 1-7 are realized.

10. A computer program product comprising computer programs / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the joint optimization method for UAV deployment, computing offloading, and resource allocation based on the air-ground-space integrated network according to any one of claims 1-7 are realized.

Images