Unmanned aerial vehicle computing resource optimization system and method based on low-altitude intelligent networking

Abstract

This application provides a UAV general computing resource optimization system and method based on low-altitude intelligent network. Based on a centralized coarse scheduling mechanism, it identifies access nodes that provide communication and computing services to the UAV and obtains the resource constraint information of these access nodes. Based on a distributed fine-grained optimization mechanism, a reinforcement learning decision model deployed locally on the UAV generates a resource optimization strategy that satisfies latency and energy consumption optimization objectives based on the resource constraint information of the access nodes and the parameter information of the current task. This resource optimization strategy achieves an effective balance between latency and energy consumption while ensuring efficient execution of UAV tasks. This solves the problem of sparse task arrival and hybrid action space optimization under non-Markovian processes, alleviates the energy and computing power limitations of the UAV itself, and effectively improves the scalability of low-altitude intelligent network and the flexibility of UAV terminal adaptation.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) communication technology, and in particular to a UAV computing resource optimization system and method based on low-altitude intelligent network. Background Technology

[0002] With the rapid development of the low-altitude economy, low-altitude drones are gradually showing a trend of routine and high-density operation. During flight missions, drones typically carry artificial intelligence algorithms to continuously perform tasks such as environmental perception, data collection, target recognition, and intelligent analysis. The widespread application of artificial intelligence algorithms in drones has further generated massive, real-time computing and communication demands, placing higher requirements on the communication transmission and computing scheduling capabilities of low-altitude networks.

[0003] Currently, most research on low-altitude network resource optimization focuses on ground base station communication scenarios. For example, it involves determining the relationship between low-altitude flight terminals and access nodes, and allocating corresponding communication resources such as channels, bandwidth, or transmit power; or, based on the uplink channel estimation results of low-altitude flight terminals, selecting target access nodes from multiple candidate access nodes that meet preset receive power conditions, and activating the target access nodes to provide communication services to the low-altitude flight terminals. To overcome the coverage limitations of ground networks, existing technologies are gradually introducing low-orbit satellites into low-altitude networks, forming a Low-Altitude Intelligent Network (LAIN) that connects communication nodes such as low-altitude flight terminals, ground base stations, and low-orbit satellites. Existing technologies typically utilize reinforcement learning, convex optimization, or game theory to optimize task allocation, offloading strategies, and resource configuration in the LAIN, thereby improving its service capabilities to some extent.

[0004] However, existing research on low-altitude intelligent network resource scheduling mostly focuses on application scenarios where drones are used as temporary base stations, communication relays, or edge servers. In the practical application of the low-altitude economy, drones are more often used as low-altitude flight terminals, with ground base stations or low-orbit satellites providing communication and computing services. However, there is a lack of corresponding resource coordination and scheduling mechanisms for this application scenario. To build an effective low-altitude intelligent network, it is necessary to overcome a series of shortcomings and technical challenges in the existing communication technology system and network architecture. These challenges are mainly reflected in the following three aspects:

[0005] (1) Lack of ubiquitous reliable connectivity. Due to the large differences in flight altitude and long flight distance of UAVs, it is difficult to achieve complete coverage by ground-based networks alone, and low-orbit satellites are needed to make up for the coverage blind spots of ground-based networks. However, existing technologies usually design resource optimization algorithms separately for ground-based networks and space-based networks built by low-orbit satellites, and lack an optimization mechanism that can uniformly and collaboratively schedule the communication and computing resources provided by the two, thus making it difficult to fully guarantee the reliable connectivity of UAVs.

[0006] (2) The centralized resource scheduling mechanism has performance bottlenecks. Most existing low-altitude intelligent network resource optimization adopts a centralized scheduling mechanism. With the continuous increase in the number of UAVs and the continuous expansion of the low-altitude intelligent network business, the signaling overhead and processing burden of the centralized scheduling mechanism will increase accordingly, and its scalability is poor, making it difficult to meet the real-time and flexible resource scheduling requirements of high-density low-altitude intelligent networks.

[0007] (3) The energy and computing power of UAVs are limited. UAVs consume limited onboard energy when performing communication transmission and computing tasks. Due to limitations in onboard energy, size and weight, the communication and computing capabilities of UAVs are constrained to a certain extent, making it difficult for them to independently carry out computing-intensive services. This seriously restricts the service capabilities of UAVs as intelligent mobile terminals.

[0008] Therefore, existing low-altitude intelligent network resource optimization schemes suffer from problems such as the lack of a unified joint scheduling framework for air-ground networks, insufficient scalability of centralized scheduling, and difficulty in meeting the computing requirements of UAV terminals. Summary of the Invention

[0009] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a UAV computing resource optimization system and method based on low-altitude intelligent network, which solves the technical problems of existing low-altitude intelligent network resource optimization schemes, such as the lack of a unified joint scheduling framework for air-ground networks, insufficient scalability of centralized scheduling, and difficulty for UAV terminals to meet computing requirements.

[0010] To achieve the above and other related objectives, a first aspect of this application provides a UAV computing resource optimization system based on a low-altitude intelligent network. The UAV computing resource optimization system includes: a UAV; a low-altitude intelligent network, including: a network control center and multiple ground base stations and multiple low-orbit satellites respectively communicatively connected to the network control center; a distributed computing architecture, including: a local computing unit located on the UAV, base station computing units located on each ground base station, satellite computing units located on each low-orbit satellite, and a cloud server; the cloud server is communicatively connected to each ground base station and each low-orbit satellite of the low-altitude intelligent network; wherein, the system optimizes UAV computing resources by: the UAV responding to the current task and initiating a session establishment request for the network control center to confirm... A ground base station or a low-Earth orbit satellite is designated as the access node; the access node establishes a communication connection and session with the UAV and sends resource constraint information to the UAV; the UAV constructs an observation vector based on the resource constraint information and the parameter information of the current task, and generates a resource optimization strategy for the current task based on a pre-trained reinforcement learning decision model; the UAV determines the uplink transmission power and the target execution nodes for multiple sub-tasks of the current task based on the resource optimization strategy, and assigns each sub-task to the corresponding target execution node for execution based on the uplink transmission power; wherein, the target execution node includes at least one of the UAV, the access node, and the cloud server; each target execution node executes its respective sub-task and feeds back the execution results to the UAV.

[0011] In some embodiments of the first aspect of this application, the reinforcement learning decision model is trained using a KL divergence-controlled proximal policy optimization algorithm driven by immediate rewards; the reinforcement learning decision model includes a policy network, a value network, and a KL divergence monitoring module.

[0012] In some embodiments of the first aspect of this application, the training method of the reinforcement learning decision model includes: acquiring UAV general computing resource optimization data generated during the execution of multiple historical UAV resource optimization tasks, and constructing multiple training samples to build a training dataset; wherein each training sample includes: historical observation vector, historical action vector, and historical instant reward; inputting the historical observation vector in the training sample into the policy network and outputting the corresponding predicted action vector to generate a predicted resource optimization policy for the historical UAV resource optimization task; inputting the historical observation vector in the training sample into the value network and outputting the predicted expected reward for the historical UAV resource optimization task, and performing a difference operation with the historical instant reward to calculate the advantage estimate of the predicted resource optimization policy; defining loss functions for the policy network and the value network respectively, and using the loss functions to measure the loss of the policy network and the value network respectively based on the advantage estimate of the predicted resource optimization policy, so as to update the network parameters of the policy network and the value network through backpropagation; sequentially inputting each training sample into the reinforcement learning decision model, and repeating the above steps to iteratively update the policy network and the value network until a converged reinforcement learning decision model is obtained.

[0013] In some embodiments of the first aspect of this application, the method for iteratively updating the value network includes: calculating the average KL divergence among multiple predictive resource optimization strategies generated during continuous iteration using the KL divergence monitoring module to monitor whether the value network stops early; and stopping the updating of the value network when the average KL divergence exceeds a preset KL divergence threshold.

[0014] In some embodiments of the first aspect of this application, the resource constraint information includes: the received signal-to-interference-plus-noise ratio (SINR) of the access node, used to characterize the ratio of the target signal power to the sum of noise power and interference power when the access node receives the uplink signal from the UAV; the maximum allowed transmit power of the access node, used to characterize the upper limit of transmit power used by the access node when allowing the UAV to transmit task data through the uplink channel; the allocatable wireless communication bandwidth of the access node, used to characterize the uplink transmission bandwidth allocated by the access node for the UAV from the currently available spectrum resources; and the allocatable computing resource quantity of the distributed computing architecture, used to characterize the upper limit of computing power that each target execution node can provide for the current task of the UAV.

[0015] In some embodiments of the first aspect of this application, when the access node feeds back the execution results of each subtask to the drone, it simultaneously feeds back the updated resource constraint information to the drone.

[0016] In some embodiments of the first aspect of this application, after the UAV receives the resource constraint information sent by the access node, it further includes: determining the validity period of the resource constraint information, and maintaining a communication connection and session with the access node during the validity period.

[0017] In some embodiments of the first aspect of this application, when the access node is a ground base station, the resource optimization strategy includes at least one: a local resource configuration strategy, a base station resource configuration strategy, and a cloud resource configuration strategy; when the access node is a low-Earth orbit satellite, the resource optimization strategy includes at least one: a local resource configuration strategy, a satellite resource configuration strategy, and a cloud resource configuration strategy; wherein, the local resource configuration strategy includes at least: the amount of task data and the required computing resources of the local computing subtask offloaded to the UAV; the base station resource configuration strategy includes at least: the uplink transmission power of the UAV transmitting the task to the ground base station acting as the access node, and the amount of task data and the required computing resources of the base station computing subtask offloaded to the ground base station; the satellite resource configuration strategy includes at least: the uplink transmission power of the UAV transmitting the task to the low-Earth orbit satellite acting as the access node, and the amount of task data and the required computing resources of the satellite computing subtask offloaded to the low-Earth orbit satellite; the cloud resource configuration strategy includes at least: the amount of task data and the required computing resources of the cloud computing subtask offloaded to the cloud server.

[0018] In some embodiments of the first aspect of this application, the method of assigning multiple subtasks of the current task to corresponding target execution nodes includes: when the access node is a ground base station, the UAV, according to the resource optimization strategy, calls its local computing unit to execute a local computing subtask, and based on the uplink transmission power of the ground base station, sends relevant data of the base station computing subtask and the cloud computing subtask to the ground base station for the base station computing unit to execute the base station computing subtask. At the same time, the ground base station forwards relevant data of the cloud computing subtask to the cloud server for the cloud server to execute the cloud computing subtask; when the access node is a low-Earth orbit satellite, the UAV, according to the resource optimization strategy, calls its local computing unit to execute a local computing subtask, and based on the uplink transmission power of the low-Earth orbit satellite, sends relevant data of the satellite computing subtask and the cloud computing subtask to the low-Earth orbit satellite for the satellite computing unit to execute the satellite computing subtask. At the same time, the low-Earth orbit satellite forwards relevant data of the cloud computing subtask to the cloud server for the cloud server to execute the cloud computing subtask.

[0019] To achieve the above and other related objectives, a second aspect of this application provides a method for optimizing the computing resources of unmanned aerial vehicles (UAVs) based on a low-altitude intelligent network (LAH). This method, applied to a UAV, includes: responding to a current task and initiating a session establishment request so that a network control center in the LHA can determine a ground base station or a low-orbit satellite as an access node; establishing a communication connection and session with the access node and obtaining its resource constraint information; constructing an observation vector based on the resource constraint information and the parameter information of the current task, and generating a resource optimization strategy for the current task based on a pre-trained reinforcement learning decision model; determining the uplink transmission power and target execution nodes for multiple sub-tasks of the current task based on the resource optimization strategy, and assigning each sub-task to its corresponding target execution node for execution based on the uplink transmission power, and receiving the execution results of each sub-task; wherein the target execution node includes at least one of the UAV, the access node, and the cloud server.

[0020] As described above, this application has the following beneficial effects: This application provides a UAV general computing resource optimization system and method based on low-altitude intelligent network. Based on a centralized coarse scheduling mechanism, it determines the access node that provides communication and computing services to the UAV and obtains the resource constraint information of the access node. Based on a distributed fine-grained optimization mechanism, through a reinforcement learning decision model deployed locally on the UAV, it generates a resource optimization strategy that meets the time and energy consumption optimization objectives according to the resource constraint information of the access node and the parameter information of the current task. This resource optimization strategy achieves an effective balance between time and energy consumption while ensuring the efficient execution of UAV tasks. This solves the problem of mixed action space optimization under sparse task arrival and non-Markov process, alleviates the problem of limited energy and computing power of the UAV itself, and effectively improves the scalability of low-altitude intelligent network and the flexibility of UAV terminal adaptation. Attached Figure Description

[0021] Figure 1 The diagram shown is a structural schematic of a UAV general computing resource optimization system based on low-altitude intelligent network in one embodiment of this application.

[0022] Figure 2 The diagram shown is a flowchart illustrating the optimization of general computing resources for unmanned aerial vehicles (UAVs) in one embodiment of this application.

[0023] Figure 3 The diagram shown is a schematic representation of UAV general computing resource optimization based on centralized coarse scheduling and distributed fine optimization in one embodiment of this application.

[0024] Figure 4 The diagram shown is a schematic representation of the training of a reinforcement learning decision model in one embodiment of this application.

[0025] Figure 5The diagram shown is a flowchart illustrating a method for optimizing the computing resources of unmanned aerial vehicles (UAVs) based on low-altitude intelligent networks, according to one embodiment of this application. Detailed Implementation

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0027] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with essentially the same function and effect, without limiting their order. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that the terms "first" and "second" do not necessarily imply that they are different.

[0028] To address the problems mentioned above, this application provides a UAV computing resource optimization system and method based on low-altitude intelligent network, aiming to solve the technical problems of existing low-altitude intelligent network resource optimization schemes, such as the lack of a unified joint scheduling framework for air-ground networks, insufficient scalability of centralized scheduling, and the inability of UAV terminals to meet computing requirements.

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application are further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0030] like Figure 1 The diagram illustrates a structural schematic of a UAV computing resource optimization system based on a low-altitude intelligent network, as described in this embodiment of the application. The UAV computing resource optimization system based on a low-altitude intelligent network in this embodiment mainly includes a UAV, a low-altitude intelligent network, and a distributed computing architecture.

[0031] The low-altitude intelligent network is used to provide air-ground coordinated wireless communication services for drones, including: a network control center, multiple ground base stations, and multiple low-orbit satellites. Among them, such as... Figure 1As shown, each ground base station forms a corresponding ground-based network to establish wireless communication connections with drones located within its coverage area. After the drone establishes communication with the ground base station, an air-to-ground link is formed. Each low-orbit satellite forms a space-based network to establish wireless communication connections with drones located within its coverage area. After the drone establishes communication with the low-orbit satellite, an air-to-space link is formed. The network control center communicates with each ground base station and each low-orbit satellite to form a control link.

[0032] It should be understood that while ground-based network solutions are simple to implement, their coverage is poor, and communication links may be interrupted in coverage blind spots. Space-based network solutions offer better coverage, but their propagation delay and transmission power are high, and their bandwidth is small. This application establishes a space-ground integrated low-altitude intelligent network, using both ground-based and space-based networks to provide wireless communication services for UAVs. This leverages ground base stations and low-orbit satellites to achieve complementary network coverage for UAVs, balancing the continuity of communication coverage with propagation delay performance.

[0033] The distributed computing architecture includes: a local computing unit on the UAV, base station computing units located at various ground base stations, satellite computing units located on various low-Earth orbit satellites, and a cloud server, thus forming a multi-level computing resource system on the UAV's local side, base station side, satellite side, and cloud side, providing dynamically deployable computing resource support for the UAV. The local computing unit may consist of a processor, memory, and corresponding task processing programs onboard the UAV, used to process communication tasks and / or computational tasks such as environmental perception, data acquisition, target recognition, and image processing using the UAV's own computing resources; its computing power is limited. The base station computing unit is an edge computing device deployed at or connected to the corresponding ground base station, possessing more computing resources and greater computing power than the local computing unit. It is used to receive communication or computational sub-tasks offloaded by the UAV to the ground base station and execute the corresponding sub-tasks using the base station's computing resources. The satellite computing unit is an edge computing device deployed at or connected to the corresponding low-Earth orbit satellite, possessing more computing resources and greater computing power than the local computing unit. It is used to receive communication or computational sub-tasks offloaded by the UAV to the low-Earth orbit satellite and execute the corresponding sub-tasks using the base station's computing resources. The cloud server has more computing resources and greater computing power than local computing units, base station computing units and satellite computing units, and is used to process communication or computing subtasks with large computing loads or high requirements for local computing resources.

[0034] like Figure 1As shown, the cloud server is communicatively connected to various ground base stations and low-Earth orbit satellites in the low-altitude intelligent network. When a ground base station establishes a communication connection and session with the UAV as an access node, the UAV can send communication or computing sub-tasks to be processed by the cloud server to the cloud server through the ground base station of the access node. The cloud server then executes the corresponding communication or computing sub-tasks and returns the execution results to the UAV through the ground base station of the access node. When a low-Earth orbit satellite establishes a communication connection and session with the UAV as an access node, the UAV can send communication or computing sub-tasks to be processed by the cloud server to the cloud server through the low-Earth orbit satellite of the access node. The cloud server then executes the corresponding communication or computing tasks and returns the execution results to the UAV through the low-Earth orbit satellite of the access node.

[0035] During mission execution, the UAV can determine the target execution node for the current mission based on the mission data volume, communication and computing requirements, latency threshold, and available computing resources of the local computing unit, base station computing unit, satellite computing unit, and cloud server. For example, missions with low computing requirements or high latency requirements can be executed by the local computing unit; missions within the coverage area of ​​a ground base station and suitable for near-end offloading can be executed by the base station computing unit; missions not within the coverage area of ​​a ground base station can be executed by the satellite computing unit; and missions with high computing requirements can be offloaded to the cloud server via a ground base station or low-Earth orbit satellite for execution. The same mission can also be divided into multiple mission parts, which are executed collaboratively by at least two of the local computing unit, base station computing unit, satellite computing unit, and cloud server.

[0036] By using local computing units, base station computing units, satellite computing units, and the cloud server, UAV missions can be processed in a distributed manner across the local side, base station side, satellite side, and cloud side. This can alleviate the problem of limited energy and computing power of the UAV itself, and provide a foundation for subsequent joint optimization of communication resource allocation strategies, mission offloading strategies, and computing resource allocation strategies.

[0037] The UAV computing resource optimization system based on low-altitude intelligent network is used to perform UAV computing resource optimization, such as... Figure 2 As shown, the specific method mainly includes the following steps.

[0038] Step S1: The UAV responds to the current task and initiates a session establishment request so that the network control center can determine a ground base station or a low-orbit satellite as an access node.

[0039] Specifically, when one or more tasks to be executed by the UAV arrive sparsely, in response to each task, the UAV initiates a session establishment request to establish a communication connection and conduct a session with a ground base station or a low-orbit satellite of the low-altitude intelligent network. It should be understood that each sparse task is independent of the others, and there is no need for tasks to queue. The session establishment request includes the UAV's movement status and parameter information of the current task; the movement status includes at least: the UAV's current position, the number of clock cycles required for the current task, and the predicted position of the UAV within multiple clock cycles; the parameter information includes at least: the task data volume of the current task, the task communication and computing requirements, the task latency threshold, and the available computing resources of the UAV's local computing unit.

[0040] In one embodiment, the network control center can coordinate with various ground base stations and low-orbit satellites to determine the access node for the UAV based on a priority access strategy for ground base stations, using a centralized coarse-grained scheduling method. Figure 3 The aforementioned step, "task-triggered access," includes the following specific steps.

[0041] ① The network control center performs identity authentication on the drone.

[0042] Specifically, the drone sends a registration request via broadcast. A ground base station or low-orbit satellite in the vicinity of the drone acts as a relay to send the registration request to the network control center, which then authenticates the drone's identity.

[0043] Subsequently, when one or more tasks to be performed by the UAV arrive sparsely, the UAV sends a session establishment request via broadcast. Ground base stations or low-orbit satellites in the vicinity of the UAV act as relays to send the session establishment request to the network control center. The network control center then coordinates with each ground base station and each low-orbit satellite to determine one or more candidate access nodes. The specific implementation is described in the following steps.

[0044] ② In response to the session establishment request sent by the UAV after the identity authentication verification is passed, the network control center collects the real-time channel load, airspace coverage and received signal-to-interference-plus-noise ratio of multiple ground base stations in the vicinity of the UAV, and selects one or more ground base stations that meet the communication conditions of the UAV as candidate access nodes.

[0045] If the airspace coverage of the ground base station covers the UAV and the received signal-to-interference-plus-noise ratio (SINR) of the ground base station exceeds a preset SINR threshold, then the ground base station is determined to meet the communication conditions of the UAV and can be identified as a candidate access node for the UAV.

[0046] ③ If none of the ground base stations meet the communication conditions of the UAV, then the real-time channel load, airspace coverage and received signal-to-interference-plus-noise ratio of multiple low-orbit satellites in the vicinity of the UAV are collected, and one or more low-orbit satellites that meet the communication conditions of the UAV are selected as candidate access nodes.

[0047] If the airspace coverage of a low-orbit satellite covers the UAV, and the received signal-to-interference-plus-noise ratio (SINR) of the low-orbit satellite exceeds a preset SINR threshold, then the low-orbit satellite is determined to meet the communication conditions of the UAV, and can be identified as a candidate access node for the UAV.

[0048] ④ The network control center selects the candidate access node with the highest channel quality as the access node of the UAV based on the real-time channel load and received signal-to-interference-plus-noise ratio of each candidate access node.

[0049] ⑤ The network control center, based on the access node and combined with the retained historical service license information, authorizes the drone with resources. If the resource authorization verification passes, the network control center controls the access node to establish a two-way communication connection with the drone and allows the session; if the resource authorization verification fails, the network control center rejects the session establishment request.

[0050] Step S2: The access node establishes a communication connection and session with the UAV and sends resource constraint information to the UAV.

[0051] The resource constraint information refers to the information fed back by the Access Node (AN) to the Unmanned Aerial Vehicle (UAV), which limits the task offloading path, wireless transmission resources, and computing resources that can be used for the current task, such as... Figure 3 The step shown is "AN Feedback Constraint". The resource constraint information includes: the received signal-to-interference-plus-noise ratio of the access node, the maximum allowable transmit power of the access node, the allocatable wireless communication bandwidth of the access node, and the allocatable computing resources of the distributed computing architecture. Specifically, when the access node is a terrestrial base station, the allocatable computing resources of the distributed computing architecture include the allocatable computing resources of the base station computing unit of the terrestrial base station and the allocatable computing resources of the cloud server; when the access node is a low-Earth orbit satellite, the allocatable computing resources of the distributed computing architecture include the allocatable computing resources of the satellite computing unit of the low-Earth orbit satellite and the allocatable computing resources of the cloud server.

[0052] The Received Signal-to-Interference-plus-Noise Ratio (SINR) characterizes the ratio of the target signal power to the sum of noise power and interference power when the access node receives the uplink signal from the UAV. Therefore, the SINR reflects the uplink channel quality between the UAV and the access node and can be used as a basis for determining the uplink transmission rate and transmission delay. Since the UAV cannot directly obtain the uplink channel response of the access node, preferably, the access node can estimate the uplink channel quality based on the received uplink signal, obtain the SINR, and feed it back to the UAV via downlink control signaling through the access node.

[0053] The maximum allowed transmit power is used to characterize the upper limit of transmit power that the access node allows the UAV to use when transmitting mission data through the uplink channel. Transmit power refers to the power output by the UAV when sending uplink signals to the access node via its antenna. Its magnitude affects the reception quality of the uplink signal at the access node, the uplink transmission rate of mission data, and the communication power consumption of the UAV. In one embodiment, the access node can determine the maximum allowed transmit power based on uplink channel quality, interference level, and network resource configuration through an uplink closed-loop power control mechanism, and feed it back to the UAV. This allows the UAV to use the maximum allowed transmit power as a power constraint when optimizing mission resources to determine the actual uplink transmission power, thereby balancing the uplink transmission rate and the UAV's communication power consumption while meeting mission transmission requirements.

[0054] The allocatable wireless communication bandwidth is used to characterize the uplink transmission bandwidth allocated by the access node to the UAV from the currently available spectrum resources. It should be understood that wireless communication bandwidth refers to the channel frequency range that can be occupied when transmitting data. The allocatable wireless communication bandwidth reflects the amount of spectrum resources available for the UAV to transmit task data for the current task. It can be determined by the access node based on the currently available spectrum resources, the number of connected UAVs, the current task load, and interference conditions, and fed back to the UAV. The UAV can then use this as a wireless resource constraint, combined with the received signal-to-interference-plus-noise ratio (SINR) and actual uplink transmission power, to determine the uplink transmission rate, transmission delay, and communication power consumption, thereby providing a basis for the joint optimization of subsequent communication resource allocation strategies, task offloading strategies, and computing resource allocation strategies.

[0055] The allocatable computing resources are used to characterize the upper limit of computing power that the ground base station, the low-Earth orbit satellite, or the cloud server can provide for the current task of the UAV. The computing resources refer to the processor's computational power available when executing a computing task, and can be characterized by processor clock frequency, the number of computation cycles that can be executed per unit time, or the amount of data that can be processed per unit time. The ground base station, the low-Earth orbit satellite, or the cloud server can determine the corresponding allocatable computing resources based on their total computing resources, occupied computing resources, reserved computing resources, and current task load, and feed this information back to the UAV. The UAV uses this as a computing resource constraint to estimate the computational latency of the base station computing unit, the low-Earth orbit satellite computing unit, and the cloud server to offload the current task, and accordingly determine the task offloading ratio and computing resource allocation ratio, thereby avoiding the task offloading amount exceeding the actual processing capacity of the computing units.

[0056] In one embodiment, after receiving the resource constraint information sent by the access node, the UAV further includes: determining the validity period of the resource constraint information, and maintaining a communication connection and session with the access node within the validity period. It should be noted that this application does not limit the specific value of the resource constraint validity period.

[0057] Specifically, during the validity period of the resource constraint information, the access node maintains a communication connection with the UAV. The UAV does not need to repeatedly perform the session establishment process for each newly arrived task, and can determine the actual uplink transmission power, task offloading strategy, and computing resource allocation strategy for the current task based on the resource constraint information. When the validity period of the resource constraint information expires, the original resource constraint information becomes invalid, the access node releases the communication and computing resources maintained for the UAV, and disconnects the session and communication connection with the UAV to avoid long-term occupation of network resources.

[0058] Upon arrival of each task, the UAV determines whether the validity period of the resource constraint information has expired. If it has not expired, the UAV performs general computing resource optimization using the current access node and the corresponding resource constraint information. If it has expired, the UAV re-initiates a session establishment request to the low-altitude intelligent network to determine a new access node and re-establish a communication connection and session with the UAV, and feeds back the updated resource constraint information to the UAV. The UAV then re-determines the validity period of the resource constraint information.

[0059] In this embodiment, by setting the validity period of the resource constraint information, it is possible to reduce the number of repeated session accesses of the UAV to the low-altitude intelligent network and the related signaling overhead in sparse task scenarios, and also to prevent the UAV from continuously using resource constraint information that can no longer reflect the current channel state and resource state, thus ensuring the real-time performance and accuracy of UAV computing resource optimization.

[0060] Step S3: The UAV constructs an observation vector based on the resource constraint information and the parameter information of the current task, and generates a resource optimization strategy for the current task based on a pre-trained reinforcement learning decision model.

[0061] In one embodiment, the current task is a task Its observation vector can be expressed as:

[0062] ;Formula (1)

[0063] in, For the current task The observation vector; This refers to the connection status between the drone and the access node. This indicates that the access node is a ground base station. This indicates that the access node is a low-orbit satellite; The received signal-to-interference-plus-noise ratio (SINR) of the access node; This refers to the maximum allowed transmit power of the access node; The allocatable wireless communication bandwidth for the access node; The amount of available computing resources in the local computing unit of the UAV; The amount of allocatable computing resources for the access node; The amount of allocable computing resources for the cloud server; The amount of task data for the current task; In one embodiment, the computational load of the current task can be represented by the number of Central Processing Unit (CPU) cycles required to complete the task. This is the task latency threshold for the current task.

[0064] In one embodiment, to improve the training stability of the reinforcement learning decision model, the observation vector can be normalized to an interval. The normalized observation vector is:

[0065] . Formula (2)

[0066] The reinforcement learning decision model is used to determine the current task. Normalized observation vector Generate resource optimization strategies that meet the objectives of latency and energy consumption optimization.

[0067] The resource optimization strategy is used to characterize how the current task is offloaded among the local computing unit, the base station computing unit, and the cloud server, and the computing resources to be allocated to the offloaded computing task and the uplink transmission power to send the computing task, including at least one: resource configuration strategy, base station resource configuration strategy, satellite resource configuration strategy, and cloud resource configuration strategy.

[0068] The resource allocation strategy includes at least: the amount of task data and required computing resources for the local computing subtasks offloaded to the UAV. The base station resource allocation strategy includes at least: the uplink transmission power of the UAV transmitting tasks to the ground base station acting as an access node, and the amount of task data and required computing resources for the base station computing subtasks offloaded to the ground base station. The satellite resource allocation strategy includes at least: the uplink transmission power of the UAV transmitting tasks to the low-Earth orbit satellite acting as an access node, and the amount of task data and required computing resources for the satellite computing subtasks offloaded to the low-Earth orbit satellite. The cloud resource allocation strategy includes at least: the amount of task data and required computing resources for the cloud computing subtasks offloaded to the cloud server.

[0069] It should be noted that when the access node is a terrestrial base station, the resource optimization strategy includes at least one of a local resource configuration strategy, a base station resource configuration strategy, and a cloud resource configuration strategy; when the access node is a low-orbit satellite, the resource optimization strategy includes at least one of a local resource configuration strategy, a satellite resource configuration strategy, and a cloud resource configuration strategy.

[0070] The latency and energy consumption optimization objectives include: maximizing the task latency satisfaction rate, reducing the average weighted energy consumption of tasks, and reducing the average task completion latency of infeasible tasks. Specifically, the resource optimization strategy should enable as many tasks as possible to meet the latency threshold, and while maintaining the same or essentially unchanged task latency satisfaction rate, reduce the average weighted energy consumption of tasks; for tasks that cannot meet the latency threshold (i.e., infeasible tasks), shorten their average task completion latency.

[0071] In one embodiment, the formula for calculating the task latency satisfaction rate is:

[0072] ;Formula (3)

[0073] in, For task latency satisfaction rate; The total number of tasks within the observation period; This represents an indicator function used to "mark" whether a condition is true or false. When the condition within the parentheses is true, ... ,otherwise ; For the current task The actual completion delay; For the current task The latency threshold is the allowable latency from transmitting task data to completing the task.

[0074] The formula for calculating the average task completion delay for infeasible tasks is as follows:

[0075] ;Formula (4)

[0076] in, The average task completion delay for infeasible tasks; When executing the current task If the drone is able to transmit at a power not exceeding the maximum allowable transmission power of the access node... Under the premise of meeting the transmission delay threshold If the task is feasible, then the task is considered feasible. Otherwise, the task is infeasible, and it is called an infeasible task. It should be noted that, To perform the current task At that time, the allowable latency during the transmission of task data, for communication tasks that only need to transmit data, For computational tasks, .

[0077] The formula for calculating the average weighted energy consumption of the task is as follows:

[0078] ;Formula (5)

[0079] in, , , These are the energy consumption weighting coefficients for the local computing unit of the UAV, the computing unit of the access node (such as the base station computing unit of a ground base station or the satellite computing unit of a low-orbit satellite), and the cloud server, respectively, satisfying... ; The first task energy consumption is generated by the local computing unit of the UAV for transmitting data and executing local computing subtasks; The energy consumption of the computing unit of the access node (such as the base station computing unit of a ground base station or the satellite computing unit of a low-orbit satellite) for transmitting data and executing base station computing sub-tasks. This refers to the energy consumption of the cloud server due to data transmission and execution of cloud computing subtasks. To ensure consistency in measurement, the average weighted energy consumption index of the task is normalized when constructing the latency and energy consumption optimization target, i.e.:

[0080] ;Formula (6)

[0081] in, To perform the current task This refers to the maximum theoretical energy consumption achievable under time delay constraints. In a preferred embodiment, for communication tasks, For computational tasks, ; This represents the upper limit of energy consumption during the transmission of task data. , , Execute the current task respectively The upper limit of task energy consumption generated by unloading local computing subtasks, base station computing subtasks / satellite computing subtasks, and cloud computing subtasks.

[0082] Therefore, the optimization objectives for latency and energy consumption can be expressed as:

[0083] ;Formula (7)

[0084] in, , The connection status between the drone and the cloud server; This refers to the set of uplink transmission power of the drone. , For the current task Uplink transmission power; For task unloading strategy, , , , These represent the proportions of computing subtasks offloaded to the local computing unit of the UAV, the computing unit offloaded to the access node (such as the base station computing unit of a ground base station or the satellite computing unit of a low-orbit satellite), and the computing subtasks offloaded to the cloud server, respectively, with a value range of [missing information]. And the sum of the three is 1; For computing resource allocation strategies, , , , These are the proportions of computing resources required by the local computing unit of the UAV, the computing unit of the access node (such as the base station computing unit of a ground base station or the satellite computing unit of a low-orbit satellite), and the cloud server. , These are the time delay weighting coefficient and the energy consumption weighting coefficient, respectively, both of which are non-negative and satisfy the condition that... .

[0085] In one embodiment, the reinforcement learning decision model is trained using a KL divergence-controlled proximal policy optimization algorithm driven by immediate rewards.

[0086] It should be understood that Proximal Policy Optimization (PPO) is currently the most mainstream, stable, and easily implemented policy gradient algorithm in reinforcement learning. It ensures that the new policy does not deviate too far from the old policy by pruning the ratio of the new to the old policy (PPO-Clip), thus maintaining sample efficiency while ensuring training stability. However, the PPO algorithm mainly uses cumulative reward-driven methods and is designed for Markov processes. The UAV general-purpose resource optimization involved in this application is a non-Markov process and is not suitable for the PPO algorithm.

[0087] This application introduces an immediate reward mechanism and a KL divergence early stopping mechanism into the existing PPO algorithm, forming a KL divergence-controlled proximal policy optimization algorithm driven by immediate rewards. By directly learning the immediate reward mapping and canceling generalized advantage estimation, a KL divergence early stopping mechanism is introduced for the value network to improve the model training stability. This solves the problem of hybrid action space optimization under sparse task arrival and non-Markov process conditions. It can achieve an effective balance between low latency and low energy consumption under the conditions of satellite-ground fusion network, overcome the limitations of existing context slot machine algorithms that only support a single low-dimensional action space and have high algorithm complexity, and at the same time ensure the stability of model training.

[0088] Furthermore, the reinforcement learning model is small and requires low computing power, allowing it to be deployed locally on the drone, such as within the drone's control unit, and supporting calls from the local computing unit to achieve... Figure 3 The distributed fine-grained optimization is shown.

[0089] In this embodiment, as Figure 4 As shown, the reinforcement learning decision model includes a policy network, a value network, and a KL divergence monitoring module, and the training method includes the following steps.

[0090] ① Obtain UAV general computing resource optimization data generated during the execution of multiple historical UAV resource optimization tasks, and construct multiple training samples to build a training dataset.

[0091] Each training sample includes: historical observation vectors Historical action vectors and historical real-time rewards .

[0092] Among them, historical observation vector Consistent with the observation vector expression shown in formula (1), in one embodiment, the historical observation vector can be... Normalization processing. Specifically, , , , , , , These are the connection status between the UAV and the access node, the received signal-to-interference-plus-noise ratio, the maximum allowed transmit power, the allocable wireless communication bandwidth, the available computing resources of the local computing unit, the allocable computing resources of the access node, and the allocable computing resources of the cloud server when performing historical UAV resource optimization tasks. , , These are the task data volume, task computation volume, and task latency threshold for historical drone resource optimization tasks.

[0093] Historical Action Vector ,in, The connection status between the drone and the cloud server when performing historical drone resource optimization tasks; The drone power control coefficient when performing historical drone resource optimization tasks; The task offloading ratio when performing historical drone resource optimization tasks. , , , These are the offloading ratios for local computing subtasks, access node computing subtasks (such as base station computing subtasks or satellite computing subtasks), and cloud computing subtasks, respectively. The proportion of computational resources allocated when performing historical UAV resource optimization tasks. , , , These are the computing resource allocation ratios for local computing subtasks, access node computing subtasks (such as base station computing subtasks or satellite computing subtasks), and cloud computing subtasks. For discrete action variables, the historical action vector The remaining components are all continuous motion components.

[0094] Historical Instant Rewards The time delay and energy consumption optimization values ​​for performing historical UAV resource optimization tasks can be calculated using formulas (3)-(7).

[0095] ②The historical observation vectors in the training samples Input the policy network and output the corresponding predicted action vector. Generate a predictive resource optimization strategy for the historical UAV resource optimization task.

[0096] In one embodiment, such as Figure 4 As shown, the policy network includes a feature extraction module, a discrete branch, a continuous branch, and a masking module. The discrete branch and the continuous branch are connected to the feature extraction module and the masking module, respectively.

[0097] The feature extraction module includes multiple fully connected layers (FC layers) connected in sequence, used to extract state features representing the relationship between communication state, computing resource state, and task requirements from observation information. In a specific embodiment, such as... Figure 4 As shown, the feature extraction module includes a first fully connected layer ( Figure 4 As shown in FC1), the second fully connected layer ( Figure 4 The FC2 shown) and the third fully connected layer ( Figure 4 (FC3 shown).

[0098] After obtaining the state characteristics of the historical UAV resource optimization task, the policy network generates corresponding discrete and continuous actions through the discrete and continuous branches respectively, thereby adapting to the mixed action space of UAV resource optimization actions that simultaneously include discrete and continuous decision variables.

[0099] The discrete branch, based on the Bernoulli distribution, generates discrete actions for the historical execution of the UAV resource optimization task. Specifically, it outputs probability parameters conforming to the Bernoulli distribution and samples discrete actions based on these probability parameters. These discrete actions can be used to indicate whether to offload at least a portion of the historical UAV resource optimization task to the cloud server or whether to enable cloud computing paths; that is, the discrete actions represent the predicted connection state between the UAV and the cloud server during the historical execution of the UAV resource optimization task. .

[0100] The continuous branch generates continuous actions for the historical UAV resource optimization task based on a normal distribution constrained by a hyperbolic tangent function. Specifically, it outputs the mean and standard deviation of a normal distribution, samples initial continuous actions based on these mean and standard deviation, and then maps the initial continuous actions to a preset value range using a hyperbolic tangent function to obtain continuous actions. The continuous actions may include UAV power control coefficients. Task uninstallation ratio and computing resource allocation ratio This allows the UAV to determine its actual uplink transmission power based on its power control coefficient and the maximum allowable transmission power fed back by the access node. Based on the task offloading ratio, the historical UAV resource optimization task is divided into local computing subtasks, base station computing subtasks, satellite computing subtasks, and cloud computing subtasks, thus determining the task offloading strategy. Based on the computing resource allocation ratio, the local computing resources, ground base station computing resources, low-orbit satellite computing resources, and cloud server computing resources called by each subtask are determined, thus determining the computing resource allocation strategy. This generates a predictive resource optimization strategy for the historical UAV resource optimization task.

[0101] The masking module constrains the continuous actions based on the discrete actions of the historical UAV resource optimization task, and outputs the predicted action vector of the historical UAV resource optimization task. Specifically, when the access node is a terrestrial base station, local computing subtasks, base station computing subtasks, and cloud computing subtasks are allowed to be generated; when the access node is a low-Earth orbit satellite, local computing subtasks, satellite computing subtasks, and cloud computing subtasks are allowed to be generated; when the cloud server is unreachable or the corresponding resource constraint information has expired, the masking module blocks the cloud server computing actions to avoid outputting resource optimization decisions that cannot be executed under the current network condition. The predicted action vector... Discrete actions including the historical UAV resource optimization task and feasible continuous actions ,in, , , , , These are the predicted offloading ratios for local computing subtasks, access node computing subtasks (such as base station computing subtasks or satellite computing subtasks), and cloud computing subtasks, respectively. , , These are the predicted computing resource allocation ratios for local computing subtasks, access node computing subtasks (such as base station computing subtasks or satellite computing subtasks), and cloud computing subtasks, respectively.

[0102] In one embodiment, such as Figure 4 As shown, the predicted action vector is obtained. Then, it also includes the predicted action vector. Normalization is performed to convert each effective motion component after motion masking into a non-negative value, and the sum of each effective motion component is used as the normalization benchmark to obtain the normalized motion.

[0103] ③ The historical observation vectors in the training samples Input the value network and output the predicted expected reward of the historical UAV resource optimization task. In order to match historical instant rewards Perform interpolation to calculate the advantage estimate of the predicted resource optimization strategy.

[0104] In one embodiment, the value network includes a plurality of fully connected layers connected in sequence; in a specific embodiment, such as... Figure 4 As shown, the value network includes a fourth fully connected layer ( Figure 4 The FC4 shown), the fifth fully connected layer ( Figure 4 The FC5 shown) and the sixth fully connected layer ( Figure 4 (FC6 shown).

[0105] The value network is used to calculate the transmission latency, computation latency, communication energy consumption, and computation energy consumption during the execution of the historical UAV resource optimization task, and based on this, to generate the predicted expected reward for the historical UAV resource optimization task. In this embodiment, the value network abandons the cumulative discount reward and directly learns the instant reward mapping, which is suitable for hybrid action space optimization under sparse task arrival and non-Markovian processes.

[0106] In one embodiment, taking into account both task latency constraint satisfaction and task energy consumption, a preset reward function is used to generate the predicted expected reward. Specifically, when the completion latency of a historical UAV resource optimization task does not exceed the task latency threshold, a positive reward is given, and the corresponding reward is deducted based on communication and computing energy consumption. When the completion latency of a historical UAV resource optimization task exceeds the task latency threshold, a negative reward is given, and the penalty is increased according to the degree to which the latency threshold is exceeded. Thus, the value network can guide the policy network to prioritize improving the task latency constraint satisfaction rate and reduce the average weighted energy consumption of the task while meeting the latency requirements.

[0107] In one embodiment, the reward function can be defined as:

[0108] ;Formula (8)

[0109] ;Formula (9)

[0110] ;Formula (10)

[0111] in, To predict expected rewards; For delay-related rewards; This is an energy consumption incentive item; For the first The actual completion delay of a historical drone resource optimization task; For the first The latency threshold for a historical drone resource optimization task; When executing the first During a historical drone resource optimization task, if the drone can transmit without exceeding the maximum allowable transmission power of the access node... Under the premise of meeting the transmission delay threshold , ;otherwise, ; , These are the time delay weighting coefficient and the energy consumption weighting coefficient, respectively.

[0112] The reward function incorporates three metrics: task latency satisfaction rate, average task completion latency of infeasible tasks, and average weighted energy consumption of tasks. This enables joint optimization of the UAV terminal's uplink transmission power, task offloading strategy, and computing resource allocation strategy. Consequently, it maximizes the task latency satisfaction rate while reducing the average weighted energy consumption of tasks and decreasing the average task completion latency of infeasible tasks, achieving an effective balance between latency and energy consumption while ensuring efficient UAV task execution.

[0113] ④ Define loss functions for the policy network and the value network respectively, and use these loss functions to measure the losses of the policy network and the value network according to the advantage estimate of the predicted resource optimization strategy, so as to update the network parameters of the policy network and the value network through backpropagation.

[0114] In one embodiment, such as Figure 4 As shown, the loss function of the value network can be a defined mean squared error (MSE) loss function. Using the MSE function, the loss of the value network is measured based on the advantage estimate of the predicted resource optimization strategy, so as to update the network parameters of the value network through backpropagation.

[0115] In one embodiment, such as Figure 4 As shown, the loss function of the policy network includes a PPO-Clip truncated objective function. The PPO-Clip truncated objective function constrains each advantage estimate, reducing the risk of training instability caused by excessively large single parameter updates, and limiting the magnitude of change of the new policy relative to the old policy, thus avoiding complete bias towards a single resource optimization strategy.

[0116] like Figure 4As shown, the loss function of the policy network may further include an entropy regularization term. This entropy regularization term is determined based on the entropy of the discrete and continuous action distributions of each historical UAV resource optimization task, and is used to compare the predicted action vector output by the policy network. Compared with the historical action vectors in the training samples The loss of the policy network is calculated to maintain its ability to explore different resource optimization actions and prevent the resource optimization strategy from converging to a single action too early.

[0117] Specifically, the loss function of the policy network can be defined as:

[0118] ;Formula (11)

[0119] in, This is the estimated advantage value of the predicted resource optimization strategy; This represents the cropping probability ratio. ; , This is the clipping threshold; This is the entropy regularization term.

[0120] ⑤ Input each training sample into the reinforcement learning decision model in sequence, and repeat steps ②-④ to iteratively update the policy network and the value network until a converged reinforcement learning decision model is obtained.

[0121] To avoid training oscillations caused by the magnitude of the value network update, in one embodiment, such as Figure 4 As shown, the KL divergence monitoring module can calculate the average KL divergence among multiple predicted resource optimization strategies generated during continuous iterations to monitor whether the value network stops prematurely. Specifically, when the average KL divergence exceeds a preset KL divergence threshold, the value network is stopped from being updated, and only the policy network is continuously updated until convergence. In a preferred embodiment, the KL divergence threshold is set to 5, but it should be noted that this application is not specifically limited to this.

[0122] Based on the convergent reinforcement learning decision model, according to the current task Normalized observation vector This allows for the generation of resource optimization strategies that satisfy both latency and energy consumption optimization objectives, achieving an effective balance between these two factors. Specifically, the normalized observation vector... The policy network is input, and the corresponding predicted action vector is output, which is used to generate the resource optimization policy. It should be noted that the normalized observation vector... The computation process of the input policy network is the same as that of the policy network on historical observation vectors during training. The calculations will not be elaborated here for the sake of simplicity.

[0123] like Figure 3 As shown, this application utilizes a centralized upper-layer mechanism jointly implemented by the network control center and the access nodes (ground base stations or low-Earth orbit satellites). This mechanism only performs macro-level resource constraints and coarse-grained scheduling on UAV session establishment requests, determining the access node. Simultaneously, based on the reinforcement learning decision model deployed locally on the UAV, a distributed lower-layer mechanism is implemented, autonomously completing fine-grained uplink transmission power control, task offloading, and computing resource allocation. This achieves a resource scheduling mechanism combining centralized coarse scheduling and distributed fine optimization. Even if the number of UAVs connected to the low-altitude intelligent network increases, there is no need to increase significant signaling overhead and processing burden, effectively improving the scalability of the low-altitude intelligent network and the flexibility of UAV terminal adaptation.

[0124] Step S4: The UAV determines the uplink transmission power and the target execution nodes of multiple sub-tasks of the current task according to the resource optimization strategy, and assigns each sub-task to the corresponding target execution node for execution based on the uplink transmission power.

[0125] The target execution node describes the communication or computing device actually performing the current task, including at least one of the drone, the access node, and the cloud server. The access node is a ground base station or a low-Earth orbit satellite.

[0126] Specifically, when the access node is a ground base station, the resource optimization strategy includes at least one: a local resource configuration strategy, a base station resource configuration strategy, and a cloud resource configuration strategy. In this case, the UAV, according to the resource optimization strategy, invokes its local computing unit to execute a local computing subtask, and based on the uplink transmission power of the ground base station, sends relevant data of the base station computing subtask and the cloud computing subtask to the ground base station for its base station computing unit to execute the base station computing subtask. Simultaneously, the ground base station forwards relevant data of the cloud computing subtask to the cloud server, awaiting execution of the cloud computing subtask by the cloud server.

[0127] When the access node is a low-Earth orbit (LEO) satellite, the resource optimization strategy includes at least one: a local resource configuration strategy, a satellite resource configuration strategy, and a cloud resource configuration strategy. In this case, the UAV, according to the resource optimization strategy, invokes its local computing unit to execute local computing subtasks. Based on the uplink transmission power of the LEO satellite, it sends relevant data of the satellite computing subtasks and cloud computing subtasks to the LEO satellite for its satellite computing unit to execute the satellite computing subtasks. Simultaneously, the LEO satellite forwards relevant data of the cloud computing subtasks to the cloud server, whereby the cloud server executes the cloud computing subtasks.

[0128] Step S5: Each target execution node executes its respective sub-task and feeds back the execution results to the UAV.

[0129] The drone, the ground base station, the low-orbit satellite, and / or the cloud server each execute their respective sub-tasks. After the tasks are completed, the execution results are fed back to the drone via the original route. Figure 3 The step shown is "Forming and Executing a Strategy". That is, when the access node is a ground base station, the execution results of the base station computing subtask and the cloud computing subtask are fed back to the UAV via the connected ground base station; when the access node is a low-Earth orbit satellite, the execution results of the satellite computing subtask and the cloud computing subtask are fed back to the UAV via the connected low-Earth orbit satellite.

[0130] In one embodiment, such as Figure 3 As shown, when the access node reports the execution results of each subtask to the drone, it simultaneously sends updated resource constraint information back to the drone. The drone then re-determines the validity period of the updated resource constraint information to await the arrival of the next task. If the validity period of the updated resource constraint information expires and no task arrives, the drone disconnects the session and communication connection with the access node.

[0131] It should be understood that the module division in the system embodiments of this application is illustrative and only represents a logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor or functional module, exist as separate physical entities, or be divided into more functional modules. The integrated modules or units described above can be implemented in hardware or as software functional modules.

[0132] like Figure 5 The diagram illustrates a flowchart of a UAV computing resource optimization method based on a low-altitude intelligent network, as described in this embodiment of the application. The UAV computing resource optimization method based on a low-altitude intelligent network described in this embodiment is applied to UAVs and mainly includes the following steps.

[0133] Step S501: In response to the current task, initiate a session establishment request so that the network control center in the low-altitude intelligent network can determine a ground base station or a low-orbit satellite as an access node.

[0134] Step S502: Establish a communication connection and session with the access node, and obtain the resource constraint information of the access node.

[0135] Step S503: Based on the resource constraint information and the parameter information of the current task, construct the observation vector, and generate the resource optimization strategy for the current task based on the pre-trained reinforcement learning decision model.

[0136] Step S504: Based on the resource optimization strategy, determine the uplink transmission power and the target execution nodes of multiple subtasks of the current task, and based on the uplink transmission power, assign each subtask to the corresponding target execution node for execution, and receive the execution results of each subtask.

[0137] The target execution node includes at least one of the drone, the access node, and the cloud server.

[0138] It should be understood that the UAV computing resource optimization method based on low-altitude intelligent network described in this application and the various UAV computing resource optimization systems based on low-altitude intelligent network provided in the above system embodiments belong to the same inventive concept, and will not be described again here for the sake of brevity.

[0139] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0140] In summary, this application provides a UAV general computing resource optimization system and method based on low-altitude intelligent network. Based on a centralized coarse scheduling mechanism, it identifies access nodes that provide communication and computing services to the UAV and obtains the resource constraint information of these access nodes. Based on a distributed fine-grained optimization mechanism, a reinforcement learning decision model deployed locally on the UAV generates a resource optimization strategy that satisfies latency and energy consumption optimization objectives based on the resource constraint information of the access nodes and the parameter information of the current task. This resource optimization strategy achieves an effective balance between latency and energy consumption while ensuring efficient execution of UAV tasks. This solves the problem of sparse task arrival and hybrid action space optimization under non-Markovian processes, alleviates the limitations of UAV's own energy and computing power, and effectively improves the scalability of low-altitude intelligent network and the flexibility of UAV terminal adaptation.

[0141] Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0142] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A UAV general computing resource optimization system based on low-altitude intelligent network, characterized in that, include: Drones; The low-altitude intelligent network includes: a network control center and multiple ground base stations and multiple low-orbit satellites that are respectively connected to the network control center; The distributed computing architecture includes: a local computing unit located on the UAV, a base station computing unit located at various ground base stations, a satellite computing unit located at various low-orbit satellites, and a cloud server; the cloud server is communicatively connected to the various ground base stations and various low-orbit satellites of the low-altitude intelligent network. The system optimizes UAV computing resources in the following ways: The UAV responds to the current task and initiates a session establishment request so that the network control center can determine a ground base station or a low-orbit satellite as an access node; The access node establishes a communication connection and session with the UAV and sends resource constraint information to the UAV; The UAV constructs an observation vector based on the resource constraint information and the parameter information of the current task, and generates a resource optimization strategy for the current task based on a pre-trained reinforcement learning decision model. The UAV determines the uplink transmission power and the target execution nodes for multiple subtasks of the current task according to the resource optimization strategy, and assigns each subtask to the corresponding target execution node for execution based on the uplink transmission power; wherein, the target execution node includes at least one of the UAV, the access node, and the cloud server; Each target execution node executes its respective sub-task and feeds back the execution results to the UAV.

2. The UAV computing resource optimization system based on low-altitude intelligent network according to claim 1, characterized in that, The reinforcement learning decision model is trained using a KL divergence-controlled proximal policy optimization algorithm driven by immediate rewards; the reinforcement learning decision model includes a policy network, a value network, and a KL divergence monitoring module.

3. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 2, characterized in that, The training methods for the reinforcement learning decision-making model include: Acquire UAV general computing resource optimization data generated during the execution of multiple historical UAV resource optimization tasks, and construct multiple training samples to build a training dataset; wherein, each training sample includes: historical observation vector, historical action vector, and historical real-time reward; The historical observation vectors in the training samples are input into the policy network, and the corresponding predicted action vectors are output to generate the predicted resource optimization policy for the historical UAV resource optimization task. The historical observation vectors in the training samples are input into the value network, and the predicted expected reward of the historical UAV resource optimization task is output. The difference between the expected reward and the historical real-time reward is calculated to calculate the advantage estimate of the predicted resource optimization strategy. Loss functions are defined for the policy network and the value network, respectively. Using these loss functions, the losses of the policy network and the value network are measured based on the advantage estimate of the predicted resource optimization strategy, so as to update the network parameters of the policy network and the value network through backpropagation. Each training sample is sequentially input into the reinforcement learning decision model, and the above steps are repeated to iteratively update the policy network and the value network until a converged reinforcement learning decision model is obtained.

4. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 3, characterized in that, The methods for iteratively updating the value network include: The KL divergence monitoring module calculates the average KL divergence among multiple predicted resource optimization strategies generated during continuous iterations to monitor whether the value network stops prematurely. When the average KL divergence exceeds a preset KL divergence threshold, the value network is stopped from being updated.

5. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 1, characterized in that, The resource constraint information includes: The received signal-to-interference-plus-noise ratio of the access node is used to characterize the ratio of the target signal power to the sum of noise power and interference power when the access node receives the uplink signal from the UAV. The maximum allowed transmit power of the access node is used to characterize the upper limit of the transmit power used by the access node when allowing the UAV to transmit mission data through the uplink channel; The allocatable wireless communication bandwidth of the access node is used to characterize the uplink transmission bandwidth allocated by the access node to the UAV from the currently available spectrum resources. The allocatable computing resources of the distributed computing architecture are used to characterize the upper limit of computing power that each target execution node can provide for the current task of the UAV.

6. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 5, characterized in that, When the access node feeds back the execution results of each subtask to the drone, it simultaneously feeds back the updated resource constraint information to the drone.

7. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 1 or 6, characterized in that, After receiving the resource constraint information sent by the access node, the UAV further includes: determining the validity period of the resource constraint information, and maintaining a communication connection and session with the access node within the validity period.

8. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 1, characterized in that, When the access node is a ground base station, the resource optimization strategy includes at least one: local resource configuration strategy, base station resource configuration strategy, and cloud resource configuration strategy; When the access node is a low-Earth orbit satellite, the resource optimization strategy includes at least one: a local resource configuration strategy, a satellite resource configuration strategy, and a cloud resource configuration strategy. The local resource configuration strategy includes at least the amount of task data and the required computing resources of the local computing subtasks offloaded to the UAV. The base station resource configuration strategy includes at least: the uplink transmission power of the UAV transmitting tasks to the ground base station as an access node, and the task data volume and required computing resources of the base station computing subtasks offloaded to the ground base station; The satellite resource allocation strategy includes at least: the uplink transmission power of the UAV to the low-orbit satellite as the access node, and the task data volume and required computing resources of the satellite computing sub-tasks offloaded to the low-orbit satellite. The cloud resource configuration strategy includes at least the task data volume and required computing resources of the cloud computing subtasks offloaded to the cloud server.

9. The UAV general computing resource optimization system based on low-altitude intelligent network according to claim 8, characterized in that, The methods for assigning multiple subtasks of the current task to their respective target execution nodes include: When the access node is a ground base station, the UAV calls the local computing unit to execute local computing subtasks according to the resource optimization strategy, and sends the relevant data of the base station computing subtasks and cloud computing subtasks to the ground base station based on the uplink transmission power of the ground base station, so that the base station computing unit of the ground base station can execute the base station computing subtasks. At the same time, the ground base station forwards the relevant data of the cloud computing subtasks to the cloud server, waiting for the cloud server to execute the cloud computing subtasks. When the access node is a low-Earth orbit satellite, the UAV, according to the resource optimization strategy, calls the local computing unit to execute local computing sub-tasks, and based on the uplink transmission power of the low-Earth orbit satellite, sends the relevant data of the satellite computing sub-tasks and cloud computing sub-tasks to the low-Earth orbit satellite so that the satellite computing unit of the low-Earth orbit satellite can execute the satellite computing sub-tasks. At the same time, the low-Earth orbit satellite forwards the relevant data of the cloud computing sub-tasks to the cloud server, waiting for the cloud server to execute the cloud computing sub-tasks.

10. A method for optimizing the general computing resources of unmanned aerial vehicles (UAVs) based on low-altitude intelligent networks, applied to UAVs, characterized in that: include: In response to the current task, a session establishment request is initiated so that the network control center in the low-altitude intelligent network can determine a ground base station or a low-orbit satellite as the access node; Establish a communication connection and session with the access node, and obtain the resource constraint information of the access node; Based on the resource constraint information and the parameter information of the current task, an observation vector is constructed, and a resource optimization strategy for the current task is generated based on a pre-trained reinforcement learning decision model. According to the resource optimization strategy, the uplink transmission power and the target execution nodes of multiple subtasks of the current task are determined, and each subtask is assigned to the corresponding target execution node for execution based on the uplink transmission power, and the execution results of each subtask are received; wherein, the target execution node includes at least one of the UAV, the access node and the cloud server.

Images