Inspection method, unmanned aerial vehicle, device and storage medium

Abstract

The application provides a method for inspection, a UAV, a device and a storage medium, comprising: using a reinforcement learning model to maximize a reward function constructed according to a current task queue length, an uplink transmission rate, a total processing delay and a total time slot energy consumption as an objective, processing a current state, and determining a current action from a plurality of candidate actions; and performing the current action to perform inspection. The application constructs a state space including a current task queue length, a spatial distance, a channel gain state and a current battery remaining energy, so that the resource allocation process can simultaneously perceive energy constraints, link changes and business load changes; by including communication transmission power, sensing transmission power and calculation offloading ratio into a unified continuous action space for joint optimization control, the resource configuration can be dynamically adjusted under different channel conditions and power levels, and the stability and operation efficiency of the inspection process are improved.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to an inspection method, a drone, equipment, and a storage medium. Background Technology

[0002] During the inspection process, related technologies typically optimize communication resource configuration and computational offloading strategies independently, or use fixed-ratio offloading or resource allocation based on instantaneous information. This makes it difficult to adapt to dynamic changes in channel status and task load, resulting in poor stability. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an inspection method, a drone, equipment and storage medium.

[0004] To achieve the above objectives, this application provides an inspection method for unmanned aerial vehicles (UAVs), comprising: processing the current state of the UAV using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption; determining the current action from multiple candidate actions; and executing the current action to perform inspection. The current state includes the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state obtained based on the spatial distance, and the current remaining battery energy. The candidate actions include communication transmission power, sensing transmission power, and computational offload ratio. The uplink transmission rate is obtained based on the channel gain state and the communication transmission power. The total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computational resources. The total time slot energy consumption is obtained based on the communication energy consumption obtained based on the communication transmission power, the sensing energy consumption obtained based on the sensing transmission power, and the local computation energy consumption.

[0005] This application provides a drone, including: a processing module, used to process the current state of the drone using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption, and to determine the current action from multiple candidate actions; wherein the current state includes the current task queue length, the spatial distance between the drone and the target edge node, the channel gain state obtained based on the spatial distance, and the current remaining battery energy; the candidate actions include communication transmission power, sensing transmission power, and computational offload ratio; the uplink transmission rate is obtained based on the channel gain state and the communication transmission power; the total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computing resources; the total time slot energy consumption is obtained based on the communication energy consumption obtained based on the communication transmission power, the sensing energy consumption obtained based on the sensing transmission power, and the local computation energy consumption; and an execution module, used to execute the current action for inspection.

[0006] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described inspection method.

[0007] This application provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to execute the above-described inspection method.

[0008] This application provides an inspection method, drone, equipment, and storage medium that achieves coordinated optimization scheduling of communication resources, sensing resources, and computing resources. Specifically, by constructing a state space including the current task queue length, spatial distance, channel gain status, and current battery remaining energy, the resource allocation process can simultaneously sense energy constraints, link changes, and service load changes. By incorporating communication transmission power, sensing transmission power, and computing offload ratio into a unified continuous action space for joint optimization control, resource configuration can be dynamically adjusted under different channel conditions and power levels, improving the stability and operational efficiency of the inspection process. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1This is a schematic diagram illustrating an application scenario of the inspection method according to an embodiment of this application;

[0011] Figure 2 This is a flowchart illustrating the inspection method according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the drone according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0013] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0014] With the continuous expansion of power systems and the increasing complexity of transmission line operating environments, transmission line inspection plays an increasingly important role in ensuring the safe and stable operation of the power grid. Traditional manual inspection methods suffer from low efficiency, high labor intensity, and high safety risks, and are also difficult to implement in complex terrain conditions such as mountainous and forested areas. In recent years, drones, due to their high mobility, flexible deployment, and wide coverage, have been gradually applied to the field of transmission line inspection to complete tasks such as image acquisition, equipment status detection, and defect identification.

[0015] In actual inspection processes, drones typically need to perform environmental perception and data processing tasks simultaneously. On the one hand, they need to continuously perceive the positions of surrounding obstacles and towers through airborne radar or visual sensors to ensure flight safety; on the other hand, they need to process the collected high-definition image data in real time or upload it to edge nodes for defect identification and analysis. These tasks involve multiple types of resources, including communication transmission, perception and detection, and computing processing. These three types of resources are coupled in terms of transmission power allocation, processing latency, and energy consumption.

[0016] Meanwhile, power transmission lines are mostly located in remote areas with limited terrestrial cellular communication coverage. Drones typically rely on air-to-ground links to interact with edge computing nodes deployed along these lines. Due to battery capacity limitations, drones must complete sensing and computing tasks under limited energy conditions when performing inspection missions. An inadequate resource scheduling strategy can lead to communication interruptions, decreased detection accuracy, or mission termination.

[0017] Based on this, the relevant technologies provide two technical solutions, specifically: Technical Solution 1 provides a method and system for multi-UAV collaborative task allocation in power transmission and distribution line inspection. The method first establishes a multi-UAV collaborative inspection task model based on the requirements of power transmission and distribution line inspection tasks, and defines decision variables such as UAV task execution, task access, and charging node selection to describe the task execution status of the UAVs during the inspection process. Based on this, considering factors such as UAV flight distance, flight time, and task execution time, an optimization objective function for multi-UAV collaborative inspection task allocation is constructed, and a task allocation optimization model is established by combining task uniqueness constraints, power constraints, and time constraints. Furthermore, the UAV task allocation scheme is encoded using multi-chromosome encoding, and a dual-population genetic algorithm is used to search and optimize the task allocation scheme. Through chromosome crossover, mutation, and selection operations, the task allocation scheme is continuously updated to obtain the optimal inspection path and task allocation result that meets the constraints. Finally, a multi-UAV collaborative inspection task allocation scheme is generated based on the optimization results, achieving efficient scheduling of power transmission and distribution line inspection tasks, thereby improving the operational efficiency of multi-UAV collaborative inspection and reducing inspection time costs. This invention achieves reasonable allocation and path optimization of multi-UAV inspection tasks by constructing a multi-UAV collaborative task optimization model and solving it using a genetic algorithm.

[0018] Technical Solution 2 proposes an integrated method and system for UAV-assisted edge perception computing based on deep reinforcement learning. This method first constructs a system architecture consisting of UAVs, edge servers, and user devices, and divides tasks according to the UAV service cycle. Simultaneously, it establishes system models for communication rate, task transmission latency, computation latency, and energy consumption. Based on this, considering computational power, task latency constraints, and energy constraints, a joint optimization model is constructed with the objectives of maximizing system service success rate and minimizing energy consumption. Subsequently, by constructing an Actor-Critic structured deep reinforcement learning model, the UAV service cycle and computational resource allocation strategy are trained and optimized. Ultimately, the optimal task scheduling and resource allocation strategy is obtained, realizing dynamic resource scheduling of the UAV-assisted edge perception computing system, thereby improving system service performance and reducing energy consumption.

[0019] Technical Solution 1 primarily addresses the multi-UAV task allocation problem in power transmission and distribution line inspection scenarios. It optimizes UAV inspection paths and task allocation schemes by establishing an inspection task model and utilizing a dual-population genetic algorithm. This method is essentially an offline optimization strategy based on heuristic evolutionary algorithms, relying on preset encoding methods, fitness functions, and crossover and mutation operators. When the scale of the inspection task or environmental conditions change, the algorithm's performance becomes highly sensitive to parameter settings, often requiring readjustment or recalculation. Furthermore, this solution primarily focuses on flight distance and task time as optimization objectives, insufficiently considering factors such as changes in communication status during the inspection process, task data processing requirements, and computational resource utilization. It fails to establish a unified decision-making framework for the collaborative optimization of communication, sensing, and computational resources. Simultaneously, this method lacks an adaptive decision-making mechanism for dynamic environmental changes. In complex situations such as fluctuating link quality or random task arrival, it struggles to achieve real-time online optimization and dynamic adjustment, thus exhibiting limitations in system real-time performance and resource coordination efficiency.

[0020] Technical Solution 2 constructs a UAV-assisted sensing computing system model and utilizes deep reinforcement learning methods to optimize service cycles and computing resource allocation strategies, aiming to improve system service success rate and reduce energy consumption. However, this method primarily models from the perspective of UAV service cycle scheduling and computing resource allocation, failing to adequately consider the business characteristics of specific application scenarios, particularly neglecting system modeling for key constraints such as flight safety, real-time sensing requirements, and task processing latency in power transmission line inspection tasks. Furthermore, this solution focuses on service success rate and energy consumption, lacking a unified optimization framework for the coupling relationship between communication, sensing, and computing resources. Simultaneously, this method relies on reinforcement learning models for policy training, which may lead to slow convergence or insufficient policy stability under rapidly changing environmental conditions or improper parameter settings. Therefore, its real-time performance and resource collaborative optimization capabilities in complex dynamic environments remain somewhat limited.

[0021] In summary, existing technologies independently optimize communication resources or computational offloading strategies, failing to integrate sensing and detection constraints with battery energy budgets into a unified decision-making framework. Some methods employ fixed-ratio offloading or resource allocation based on instantaneous information, making it difficult to adapt to dynamic changes in channel conditions and task load. Furthermore, shortcomings remain in continuous power control, detection probability constraint modeling, and long-term energy management; a collaborative optimization mechanism for communication, sensing, and computation for transmission line inspection scenarios is still lacking.

[0022] Therefore, how to achieve joint scheduling of communication, sensing and computing resources while ensuring the safety constraints of detection, and also taking into account the battery energy budget of drones, has become an urgent technical problem to be solved in the field of drone inspection of power transmission lines.

[0023] This application provides an inspection method to address the problems in related technologies where communication, sensing, and computing resources are independently scheduled, making it difficult to uniformly consider flight safety constraints and battery energy budgets. Specifically, during power line inspection, UAVs need to simultaneously perform environmental perception and image data processing tasks. There is a coupling relationship between communication transmission power, sensing transmission power, and computing offload ratio. Furthermore, the air-to-ground channel status and task load have dynamic changing characteristics, making it difficult for related technologies based on fixed strategies or instantaneous information to achieve long-term stable optimization. Simultaneously, detection probability constraints and battery capacity limitations impose rigid requirements on system operation; if not incorporated into a unified decision-making framework, this could lead to flight safety risks or mission failure. To address these issues, this application constructs a unified communication-sensing-computing collaborative control framework. Under the premise of ensuring detection safety constraints, it achieves joint and continuous optimization of communication, sensing, and computing resources, thereby improving the overall system effectiveness and enhancing the stability and reliability of UAV inspection tasks.

[0024] refer to Figure 1 This is a schematic diagram illustrating an application scenario of the inspection method provided in this application embodiment. The application scenario includes a drone 101, a target edge node 102, and a server 103. The drone 101, target edge node 102, and server 103 can be connected via wired and / or wireless communication networks. The drone 101 is a flight terminal equipped with a sensor-computer interface module, capable of performing tasks such as power transmission line inspection, safety sensing, inspection data collection, data transmission, and some computational processing. The target edge node 102 can be deployed at the towers along the 2059 power transmission line to receive task data unloaded by the drone 101 and perform edge processing on the corresponding computational tasks. In this application embodiment, the number of target edge nodes is not limited. The server 103 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. In this embodiment, server 103 can be used to collaborate with target edge node 102 to achieve inspection data aggregation, centralized management, global scheduling, and remote support.

[0025] In this embodiment, a discrete time-slot approach can be used for joint decision-making and resource scheduling. The time slots are numbered t=1,2,…,T, and each time slot has a length of Δt. At the beginning of each time slot, the UAV can collect its current state, including the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state determined based on the spatial distance, and the current remaining battery energy, which serves as input to the reinforcement learning model. The reinforcement learning model processes the current state with the goal of maximizing the reward function and determines the current action from multiple candidate actions. The candidate actions include at least communication transmit power, sensing transmit power, and calculation offload ratio. The reward function can be constructed by combining the current task queue length, uplink transmission rate, total processing delay, and total time slot energy consumption. The uplink transmission rate can be determined based on the current channel gain state and communication transmit power; the total processing delay can be determined based on the uplink transmission rate, task data volume, computation offloading ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge computing resources; the total energy consumption of the time slot can be determined based on the communication energy consumption corresponding to the communication transmit power, the sensing energy consumption corresponding to the sensing transmit power, and the local computing energy consumption. Through these methods, joint adaptive control of sensing, communication, and computing resources can be achieved in power transmission line inspection scenarios, improving inspection efficiency and system performance.

[0026] The drone 101 can perform corresponding inspection, sensing, communication, and computing operations according to its current actions. Specifically, based on the computing offload ratio, a portion of the task data is processed by the drone 101 using its local computing resources. The remaining task data is transmitted to the target edge node 102 via an air-to-ground (A2G) communication link between the drone 101 and the target edge node 102, where it is processed using edge-side computing resources. After completing the corresponding computing processing, the target edge node 102 can upload the processing results and / or operational status information to the server 103 via an Optical Fiber Composite Overhead Ground Wire (OPGW). The server 103 then performs data aggregation, storage analysis, global scheduling, and remote support. This creates a collaborative inspection application scenario combining local drone processing, near-end edge node processing, and remote server support.

[0027] In one optional embodiment, the reward function includes a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensed transmit power to ensure safety throughout the inspection flight process; the energy constraint is constructed based on the current remaining battery energy to ensure endurance safety.

[0028] The following is combined with Figure 1 The application scenarios described above illustrate the inspection method according to exemplary embodiments of this application. It should be noted that the above application scenarios are merely shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. Rather, the embodiments of this application can be applied to any applicable scenario. For example, the embodiments of this application can be extended to multi-UAV collaborative inspection or cloud-edge-device layered collaborative architecture. Furthermore, this invention can be applied to technical fields such as power distribution network inspection, pipeline inspection, emergency rescue, and mobile edge computing, where there are requirements for communication, sensing, and computational coupling and scheduling.

[0029] like Figure 2 As shown in the figure, this application provides an inspection method applied to a drone, which may include: S1. Using a reinforcement learning model, the current state of the UAV is processed to maximize the reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The current action is determined from multiple candidate actions. The current state includes the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state obtained based on the spatial distance, and the current battery energy remaining. The candidate actions include communication transmission power, sensing transmission power, and calculation offload ratio.

[0030] S2, Perform the current action to conduct an inspection.

[0031] To characterize the dynamic decision-making problem during UAV inspection, the system can be modeled as a Markov decision process in a continuous action space. In time slot t, the current state can be defined as... E t This can represent the current remaining battery energy of the drone in time slot t; d t h represents the spatial distance between the drone and the target edge node. t Q represents the channel gain state, which is a state parameter characterizing the strength of the wireless channel transmission quality between the UAV and the target edge node. It reflects the gain or attenuation of the signal transmission in the current communication link. t This indicates the current task queue length. The current task queue length can refer to the length of the UAV's onboard task queue at the start of the current time slot t, and can be represented as... .

[0032] The aforementioned state variables collectively reflect the real-time operational status of the system in terms of energy, communication environment, and task load. Given these states, a reinforcement learning model can output actions, and the action space can be represented as follows: ,in, Indicates the communication transmission power of the drone; This indicates the drone's sensing transmission power; This indicates the calculation of the unloading ratio. ∈[0,1]. By jointly controlling the above continuous action variables, collaborative optimization scheduling of communication resources, sensing resources, and computing resources can be achieved.

[0033] In this embodiment, the communication transmission power refers to the transmission power used by the UAV communication module to send offload data to the target edge node in the current time slot; the sensing transmission power refers to the transmission power used by the UAV sensing module to perform line environment perception or target detection in the current time slot; and the calculated offload ratio refers to the allocation ratio of task data in the current time slot between local computation and edge-side computation, where some data is processed locally by the UAV and the remainder is transmitted to the target edge node for processing via the communication link. The locally processed data volume and the offloaded data volume can be obtained based on the calculated offload ratio and the task data volume. The task data volume refers to the amount of data processed locally and the amount of offloaded data in the current time slot. The amount of data collected from the inspection image / video task can be expressed as: (bits), which can be characterized by a random arrival process and satisfy 0 ≤ ≤ max , max This can represent the maximum amount of data for a task. The amount of data processed locally can be represented as (1 ) The amount of unloaded data can be expressed as .

[0034] The drone can utilize local computing resources to process locally processed data and then transmit the unloaded data to the target edge node for further processing via a ground-to-air communication link. After completing the corresponding calculations, the target edge node can further transmit the processing results, status information, or alarm information to the backend server for centralized storage, comprehensive analysis, and inspection management.

[0035] The uplink transmission rate can be obtained from the channel gain state and the communication transmit power. This uplink transmission rate refers to the uplink transmission rate at which the UAV sends data to the target edge node, and is used to characterize the UAV's ability to transmit to the target edge node.

[0036] The total processing latency can be obtained based on the uplink transmission rate, task data volume, computation offloading ratio, number of CPU cycles required per bit of task, local computing frequency, and available frequency of edge computing resources. This total processing latency can refer to the total latency corresponding to the completion of processing of task data in the current time slot. The local computing frequency can refer to the frequency of the UAV in the time slot. t The local CPU computing frequency can be expressed as loc And satisfy loc ≤ max , loc This can represent the maximum local computing frequency. The CPU cycles required per bit of task can be expressed as... The available frequency of edge computing resources refers to the computing resources that the target edge node can allocate to the UAV unloading task in the current time slot, and is used to characterize the edge node's ability to process unloading data.

[0037] The total energy consumption of a time slot can be obtained by considering the communication energy consumption based on communication transmission power, the sensing energy consumption based on sensing transmission power, and the local computing energy consumption. Total time slot energy consumption refers to the total energy consumed by the UAV during the current time slot while performing an inspection task. Communication energy consumption refers to the energy consumed by the UAV during data offloading based on communication transmission power. Sensing energy consumption refers to the energy consumed by the UAV during sensing operations based on sensing transmission power. Local computing energy consumption refers to the energy consumed by the UAV when performing calculations on locally processed data.

[0038] A reward function can be constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The reward function indicates the merit of executing the current candidate action in the current state, comprehensively considering factors such as communication performance, sensing effect, latency, and energy consumption. A candidate action refers to the set of all potential behaviors that can be executed based on the current state.

[0039] A reinforcement learning model is used to process the current state with the goal of maximizing the reward function, determining the current action from multiple candidate actions. The current action can refer to the actual action to be performed, as determined from the set of candidate actions.

[0040] In this embodiment, an inspection method is provided that achieves coordinated optimization scheduling of communication resources, sensing resources, and computing resources. Specifically, by constructing a state space including the current task queue length, spatial distance, channel gain state, and current battery remaining energy, the resource allocation process can simultaneously sense energy constraints, link changes, and service load changes. By incorporating communication transmission power, sensing transmission power, and computing offload ratio into a unified continuous action space for joint optimization control, resource configuration can be dynamically adjusted under different channel conditions and power levels, thereby improving the stability and operational efficiency of the inspection process.

[0041] In an optional embodiment, to ensure that the output action can be implemented, a joint feasible region projection can be applied to the action: if the policy network outputs... Then through the projection operator Obtain the actual action. In this application embodiment, the feasible domain can refer to the set of actions that satisfy system physical constraints and security constraints, such as transmit power range limits and mission ratio constraints. Projection can refer to the process of adjusting the actions output by the strategy to the range of the feasible domain to ensure that the actions meet the actual execution conditions.

[0042] In one implementation, the projection process of the projection operator can satisfy... , , , , It can represent the total transmit power threshold, so that the strategy output meets the physical and safety constraints at the execution level, avoiding flight risks caused by inaction during the exploration phase.

[0043] In an optional embodiment, after performing the current action, it may include obtaining the next state. The next task queue length and the next remaining battery energy in the next state need to be calculated by the current action. Specifically: In some embodiments, the process of obtaining the next task queue length in the next state may include: Step 1: Calculate the local processing rate based on the local computing frequency, time slot length, and the number of CPU cycles required per bit of task.

[0044] In this embodiment of the application, the time slot length can refer to the time slot change of the next state relative to the current state, and the local processing rate can refer to the amount of task data that the UAV completes locally within time slot t.

[0045] The local processing rate can be determined by the UAV's local computing power. In one implementation, the local processing rate can be expressed as:

[0046] in It can represent the local processing speed. It can be expressed as, The number of CPU cycles required per bit of task. It can represent the length of a time slot.

[0047] Step 2: Obtain the first data based on the allocated calculation frequency, time slot length, and number of CPU cycles required per bit task; obtain the second data based on the target uplink transmission rate and time slot length; obtain the offloading processing rate based on the smaller value between the first and second data.

[0048] In this embodiment of the application, the allocated computing frequency can refer to the computing frequency that the target edge node can allocate to the target task, the target uplink transmission rate can refer to the uplink transmission rate corresponding to the current action, and the offloading processing rate can refer to the amount of offloading task data processed by the target edge node within time slot t.

[0049] The offloading processing rate can be determined by both the communication transmission rate and the computing power of the target edge node. In one implementation, the offloading processing rate can be expressed as:

[0050] in, It can represent the unloading processing rate. R t For the target uplink transmission rate, It can represent the frequency of allocation calculation.

[0051] Step 3: Based on the data obtained by subtracting the local processing rate and the unloading processing rate from the current task queue length and the queue constraints, obtain the current remaining task quantity; the queue constraints are that the task queue length is not less than 0 and not greater than the maximum task queue length; based on the sum of the current remaining task quantity and the task data quantity and the queue constraints, obtain the next task queue length.

[0052] The expression for this step can be:

[0053] in, This can represent the length of the UAV's onboard mission queue at the start of the next time slot t+1. The positive part function can be pointed to, where This can be used to ensure that the length of the task queue is non-negative. It can represent the amount of task data remaining after subtracting the local processing rate and the unloading processing rate from the current task queue length, but it cannot be negative. This can represent the maximum task queue length.

[0054] The queue evolution model described above can be used to depict the dynamic changes in the inspection task between local processing by the UAV and unloading processing at the target edge.

[0055] In some embodiments, the process of obtaining the next remaining battery energy in the next state may include: taking the constraint that the battery energy consumed during the entire inspection time does not exceed the initial energy, and obtaining the next remaining battery energy based on the difference between the current remaining battery energy and the total energy consumption of the target time slot.

[0056] In this embodiment, the total energy consumption of the target time slot can refer to the total energy consumption of the time slot calculated based on the communication transmission power and sensing transmission power corresponding to the current action. The calculation method of the total energy consumption of the time slot can be referred to in the following description. The total inspection time can refer to the continuous operation time that the UAV can maintain from takeoff to perform the mission, return to base, and safe landing.

[0057] The expression for the remaining battery power can be: ,in, It can indicate the remaining power of the next battery. It can indicate the current remaining battery power. It can represent the total energy consumption of the target time slot and satisfy the overall energy budget. Where T can represent the total inspection time. It can be the initial energy.

[0058] In this embodiment of the application, the next state of the UAV can be obtained based on the next task queue length, the remaining battery energy in the next time slot, the spatial distance between the UAV and the target edge node in the next time slot, and the channel gain state in the next time slot, so as to optimize the reinforcement learning model.

[0059] In one alternative embodiment, the process of obtaining the uplink transmission rate may include: Step 1: Obtain the spatial distance and elevation angle based on the UAV's flight altitude and horizontal distance relative to the target edge node.

[0060] In this embodiment, the flight altitude can be represented as H, and the horizontal distance can refer to the planar distance between the UAV's projected position on the ground and the target edge node, which can be represented as... Based on this, the actual spatial propagation distance (i.e., spatial distance) between the UAV and the target edge node can be expressed as: This spatial distance can be used to characterize the length of the propagation path of a wireless signal in space.

[0061] The elevation angle refers to the angle between the drone and the edge node of the target, and can be expressed as... .

[0062] Step 2: Based on the elevation angle, obtain the line-of-sight propagation loss and non-line-of-sight propagation loss.

[0063] Considering the complex terrain of the transmission line corridor leading to significant obstruction, a probabilistic Loss-of-Stake (LoS) model can be used to characterize the communication link.

[0064] Line-of-sight propagation loss refers to the path loss of a wireless signal during propagation when there are no obstructions. Its expression can be:

[0065] in, This can refer to line-of-sight propagation loss. , It can refer to a constant related to the type of environment.

[0066] Non-line-of-sight propagation refers to the path loss of wireless signals during propagation due to obstructions or reflections. Its expression can be:

[0067] in, This can refer to non-line-of-sight propagation loss.

[0068] Step 3: Obtain the free space path loss based on the spatial distance and carrier wavelength.

[0069] The carrier wavelength refers to the spatial distance occupied by a complete oscillation cycle of a high-frequency carrier signal used to carry and transmit information in its propagation direction.

[0070] The free-space path loss term refers to the attenuation of a wireless signal with increasing propagation distance under unobstructed conditions, and its expression can be:

[0071] in, It can represent the free space path loss term. It can represent the carrier wavelength.

[0072] Step 4: Obtain the average path loss based on line-of-sight propagation loss, non-line-of-sight propagation loss, and free space path loss.

[0073] The expression for this step can be:

[0074] in, This can represent the average path loss. This can represent the additional loss during line-of-sight propagation. This can represent the additional loss during non-line-of-sight propagation.

[0075] Step 5: Obtain the channel gain state based on the average path loss.

[0076] Based on the path loss model described above, the channel gain state of the link can be further obtained. h t It has an inverse relationship with the average path loss, that is... .

[0077] Step 6: Obtain the uplink transmission rate based on the link channel gain status, system bandwidth, communication transmit power, and noise power spectral density.

[0078] The expression for this step can be:

[0079] in, It can represent the uplink transmission rate. It can represent the communication transmission power. N can represent the system bandwidth, and N0 can represent the noise power spectral density.

[0080] In an alternative embodiment, the process of obtaining the total processing delay may include: Step 1: Calculate the offloading transmission delay based on the uplink transmission rate, the offloading ratio, and the amount of task data.

[0081] In this embodiment, the unloading transmission delay can represent the time required for the UAV to unload the task to the target edge node. The expression can be:

[0082] Step 2: Calculate the local computation latency based on the number of CPU cycles required per bit of task, the offload ratio, the amount of task data, and the local computation frequency.

[0083] In this embodiment of the application, local computation latency can represent the time required for the UAV to process the task locally. The expression can be: .

[0084] Step 3: Based on the number of CPU cycles required per bit of task, the offload ratio, the amount of task data, and the available frequency of edge computing resources, obtain the edge processing latency.

[0085] In this embodiment of the application, edge processing latency can represent the time required for a task to process at the target edge node. The expression can be: .

[0086] Step 4: Calculate the total processing delay based on the offload transmission delay, local calculation delay, and edge processing delay; where the total processing delay is the maximum value between the sum of the offload transmission delay and the edge processing delay and the local calculation delay.

[0087] The expression for this step can be:

[0088] in, It can represent the total processing delay.

[0089] In one alternative embodiment, the process of obtaining the total energy consumption of a time slot may include: Step 1: Calculate the communication power consumption based on the communication transmission power and time slot length.

[0090] The expression for communication energy consumption can be: ,in, It can represent communication energy consumption. It can represent the communication transmission power.

[0091] Step 2: Obtain the sensing power consumption based on the sensing transmission power and time slot length.

[0092] The expression for perceived energy consumption can be: ,in, It can represent perceived energy consumption. It can represent the perceived transmission power.

[0093] Step 3: Calculate the local calculation energy consumption based on the local calculation frequency and time slot length.

[0094] The expression for calculating energy consumption locally can be: ,in, It can represent local calculation energy consumption. It can represent the hardware correlation coefficient.

[0095] Step 4: Calculate the total time slot energy consumption based on communication energy consumption, sensing energy consumption, and local computing energy consumption.

[0096] The expression for the total energy consumption of a time slot can be: ,in, It can represent the total energy consumption of a time slot.

[0097] In one optional embodiment, the constraints of the reward function may include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensed transmit power to ensure safety throughout the inspection flight process; the energy constraint is constructed based on the current remaining battery energy to ensure range safety.

[0098] In this application embodiment, the implementation method of safety constraints and energy constraints is not limited. Safety constraints can be constructed by means of hard constraint projection, Lagrange multiplier method, risk-sensitive optimization method or detection probability gating function, etc., and energy constraints can be constructed by means of piecewise penalty function, continuous decay function, remaining flight time prediction model, power safety threshold, etc.

[0099] The following example illustrates the construction of safety constraints using a detection probability gating function and energy constraints using a power safety threshold.

[0100] In one optional embodiment, a reinforcement learning model is used to process the current state with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The current action is determined from multiple candidate actions, and may include: Step 1: Based on the sensing transmit power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target, and receiver noise power corresponding to the current candidate action, obtain the target echo signal-to-noise ratio corresponding to the current candidate action.

[0101] Unmanned aerial vehicles (UAVs) utilize airborne radar for continuous detection of poles / obstacles. Radar antenna gain represents the antenna's ability to concentrate energy radiated in a specific direction; the target radar cross section represents the power density of the radar energy reflected back to the radar receiver from the target object; and receiver noise power refers to the inherent interference signal energy generated within the receiver due to factors such as electron thermal motion. In a single-pulse system, the target echo signal-to-noise ratio... It can be approximated by the radar equation as follows:

[0102] Among them, G r It can represent the radar antenna gain. It can represent the carrier wavelength. It can represent the radar cross section of the target, d tar,t N can represent the actual distance between the drone and the target object. r It can represent the receiver noise power.

[0103] Step 2: Use the Markum Q function to process the target echo signal-to-noise ratio and false alarm probability to obtain the detection probability corresponding to the current candidate action.

[0104] Based on the Neyman–Pearson criterion, given a false alarm probability P faUnder the given conditions, the detection probability can be maximized by setting a detection threshold. The false alarm probability represents the probability of incorrectly identifying a target object when it is not present; the detection probability represents the probability of correctly detecting a target object when it is present. The detection probability can be obtained using the Markum Q function, and its expression is:

[0105] in, It can represent the detection probability. It can represent the Markum Q function.

[0106] Step 3: By comparing the detection probability corresponding to the current candidate action with the minimum detection probability, determine whether the current candidate action can be used as the current action.

[0107] In this embodiment of the application, a minimum detection probability P is introduced to ensure flight safety. th As a constraint, its expression can be: Based on this constraint, a feasibility check can be performed to determine whether it can be used as the current action. In an optional embodiment, the projection operator must also satisfy the following during the projection process: .

[0108] In an optional embodiment, after determining whether a candidate action can be used as the current action by comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the process may include: determining a safety constraint based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability; determining an energy constraint based on the relationship between the current remaining battery energy and the minimum battery energy; obtaining a reward value based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the safety constraint, and the energy constraint; and determining the candidate action corresponding to the maximum reward value as the current action.

[0109] In this embodiment, the objective is to maximize the long-term weighted total utility, and the instantaneous utility function is constructed as a normalized weighted combination of multiple indicators. It can be represented as:

[0110] in, This can represent the weighting coefficient corresponding to the uplink transmission rate. It can represent the maximum uplink transmission rate. This can represent the weighting coefficients corresponding to the safety constraint terms, and the function... ( ) can represent safety constraints, when When the condition is not met, the value is 1; when the condition is not met, the value is -α (α≥1) to impose an explicit strong penalty, thus ensuring safety priority from a mechanism perspective. This can represent the weighting coefficient corresponding to the total processing delay. This can represent the maximum total processing latency. This can represent the weighting coefficient corresponding to the total energy consumption of a time slot. This can represent the total energy consumption of the maximum time slot. This can represent the weight coefficient corresponding to the current task queue length. This can represent the maximum task queue length. Where w i ≥0.

[0111] To transform long-term energy budget risk into a single-step learnable signal, an energy constraint term can be introduced. The expression can be: ,in, It is the penalty coefficient, and >0,1{ } is an indicator function. If the current remaining battery energy is less than the minimum battery energy, the value is 1; otherwise, the value is 0.

[0112] Based on this, the reward function The expression can be: .

[0113] In this embodiment, by directly embedding the detection probability safety threshold into the resource scheduling decision-making process, a "safety-first" resource allocation strategy is implemented through gating or penalty mapping mechanisms to ensure the safety of inspection flights, rather than relying on post-event verification. By constructing a power safety threshold and a risk penalty function, the full-range battery capacity constraint is transformed into a single-step decision signal, realizing the mapping from long-term energy budget to real-time control strategy and improving mission completion stability. The aforementioned reward construction of "perception safety gating + energy threshold mapping" strongly constrains safety and endurance risks during strategy training, and the reward function is used to comprehensively evaluate communication rate, perception safety, processing latency, energy consumption, and queue stability.

[0114] In the embodiments of this application, the implementation method of the reinforcement learning model is not limited. Continuous control methods such as Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), approximate dynamic programming or model predictive control, and improved Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm can be used. The improved TD3 algorithm will be used as an example for illustration below.

[0115] In one optional embodiment, the reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The process of training the reinforcement learning model includes: inputting the current state and current action into the first and second evaluation networks to obtain a first evaluation value and a second evaluation value; obtaining the target next action based on the output of the next state input into the target policy network and truncated noise; inputting the next state and the target next action into the target first evaluation network and the target second evaluation network to obtain a target first evaluation value and a target second evaluation value; and determining the target reward value based on the smaller of the target first evaluation value and the target second evaluation value. Obtain the target evaluation value; update the first evaluation network by minimizing the mean square error between the first evaluation value and the target evaluation value; update the second evaluation network by minimizing the mean square error between the second evaluation value and the target evaluation value; after the first and second evaluation networks have been updated a preset number of times, update the policy network with the goal of maximizing the updated first and second evaluation values; using a soft update mechanism, transfer the parameters of the updated policy network, the first evaluation network, and the second evaluation network to the corresponding target policy network, target first evaluation network, and target second evaluation network respectively with smoothing coefficients; repeat the above process until a preset termination condition is reached to obtain the trained reinforcement learning model.

[0116] In a preferred embodiment, an improved TD3 can be used for continuous action policy learning. A policy network is then constructed. Output deterministic action prototypes to construct the first evaluation network. 1(s,a) and the second evaluation network 2(s,a) is used to reduce the overestimation bias of the evaluation value and to construct the corresponding target policy network. Target First Evaluation Network Objective 2 Evaluation Network During the training phase, interactive samples (s) will be used. t ,a t ,r t ,s t+1 Store the data in the experience replay pool and update the network parameters by batch sampling.

[0117] To achieve a smooth target policy, truncated noise is added to the target actions. Based on this, the next target action is... The expression can be: ,in, It can represent the next state. It can represent truncated noise.

[0118] Target evaluation value The expression can be: Where γ∈(0,1) is the discount factor. It can represent the target reward value, that is, the reward value corresponding to the current action.

[0119] The first and second evaluation networks are updated by minimizing the mean square error, and its expression can be: .

[0120] The policy network can employ a delayed update mechanism. Every preset number of iterations, the policy network is updated once, with the goal of maximizing the updated first and second evaluation values. The optimization objective can be... The updated first evaluation value is the result of inputting the current state and the action output by the current policy network based on the current state into the updated first evaluation network; the updated second evaluation value is the result of inputting the current state and the action output by the current policy network based on the current state into the updated second evaluation network.

[0121] Target Policy Network Target First Evaluation Network Objective Second Evaluation Network A soft update method can be used, and its update expression can be:

[0122] Where ρ∈(0,1) are the soft update coefficients. This can represent the parameters of the target policy network. Parameters that can represent policy networks This represents the parameters of the target-first evaluation network. This represents the parameters of the second evaluation network for the objective. Indicates the parameters of the first evaluation network. This represents the parameters of the second evaluation network.

[0123] By using the improved dual-delay deep deterministic policy gradient training mechanism described above, which includes "dual evaluation network to suppress overestimation, delayed policy update, and target policy smoothing," the policy achieves more stable convergence and higher policy quality in environments with strong time-varying channels and random task arrival.

[0124] Once training converges, during the online deployment phase, exploration noise is disabled, and the state s is collected in real time. t The policy network outputs an action prototype, which is then processed by the projection operator and a safety feasibility check to obtain an executable action a. t This allows for joint control of communication transmission power, sensing transmission power, and computational offloading ratio, enabling real-time collaborative optimization of communication, sensing, and computing resources during the inspection process.

[0125] In summary, based on the highly coupled characteristics of UAV communication, sensing, and computing resources in the power transmission line inspection scenario, this application embodiment achieves coordinated optimization scheduling of communication resources, sensing resources, and computing resources through unified system modeling, continuous joint control mechanism, and security and energy constraint embedding strategy. The specific beneficial effects are as follows.

[0126] First, by constructing a unified state space, key variables such as the remaining battery power of the UAV, the air-to-ground channel status, relative distance, and task queue length are incorporated into a unified decision-making framework. This enables the resource allocation process to simultaneously perceive energy constraints, link changes, and service load changes. Compared to related technologies that only optimize single communication or computing resources, the embodiments of this application can achieve multi-resource collaborative scheduling in a dynamic inspection environment, improving the global consistency and stability of system decisions, thereby avoiding resource imbalance problems caused by local optima.

[0127] Secondly, this embodiment establishes a continuous action space joint control mechanism, using communication transmission power, sensing transmission power, and computational offloading ratio as unified decision variables for real-time adjustment, thereby achieving refined control of resource allocation. Compared with traditional methods using discrete power levels or fixed offloading ratios, this embodiment can dynamically adjust resource configuration under different channel conditions and power levels, reducing unnecessary power waste, improving spectrum utilization efficiency and computational resource utilization efficiency, and thus enhancing the overall system performance.

[0128] Furthermore, this application's embodiments directly embed radar detection probability constraints into the resource scheduling mechanism. By constructing a safety gating function, the strategy is automatically penalized and corrected when the detection probability falls below a safety threshold, prioritizing perception safety over communication and computational gains. This mechanism avoids the passive approach of related technologies that only performs safety verification after the mission is completed, ensuring flight safety constraints are guaranteed at the decision-making stage, effectively improving the safety and reliability of the inspection process.

[0129] Furthermore, this application embodiment constructs an energy risk mapping mechanism to transform long-term battery capacity constraints into a single-step learnable penalty signal, and introduces a power safety threshold to achieve risk warning and adjustment. When the remaining power approaches the threshold, the system automatically adjusts the power allocation and offloading ratio, prioritizing core sensing functions and basic communication needs. This mechanism effectively prevents mission termination due to power depletion, improves the efficiency of UAV endurance utilization and mission completion rate, and enhances the sustainability of system operation.

[0130] Furthermore, this application employs a dual-value network structure and a delayed update mechanism to construct an improved continuous control policy generation method. By suppressing value overestimation and enhancing training stability, the resource scheduling policy can still converge stably under conditions of highly time-varying channels and random task arrivals. Compared to related technologies based on static optimization or single-value network reinforcement learning methods, this invention exhibits stronger adaptability and robustness in dynamic inspection environments.

[0131] In summary, the embodiments of this application, through key technical means such as unified multi-resource modeling, continuous joint control, security constraint embedding, and energy risk mapping, have achieved efficient collaborative optimization of synergistic computing resources in the scenario of UAV inspection of power transmission lines. Under the premise of ensuring flight safety and power constraints, the overall system utility has been improved, business processing latency and energy consumption have been reduced, and the system operation stability and task completion reliability have been enhanced, which has significant engineering application value.

[0132] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0133] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] Based on the same inventive concept, and corresponding to any of the above embodiments, this application also provides an unmanned aerial vehicle (UAV).

[0135] refer to Figure 3 The drone 300 may include: The processing module 301 can be used to process the current state of the UAV using a reinforcement learning model with the objective of maximizing the reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption, and determine the current action from multiple candidate actions. The current state includes the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state obtained based on the spatial distance, and the current remaining battery energy. Candidate actions include communication transmission power, sensing transmission power, and computational offload ratio. The uplink transmission rate is obtained based on the channel gain state and communication transmission power. The total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computing resources. The total time slot energy consumption is obtained based on the communication energy consumption based on the communication transmission power, the sensing energy consumption based on the sensing transmission power, and the local computation energy consumption.

[0136] The execution module 302 can be used to perform the current action for inspection.

[0137] In one optional embodiment, the constraints of the reward function include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensed transmit power to ensure safety throughout the inspection flight process; the energy constraint is constructed based on the current remaining battery energy to ensure endurance safety.

[0138] In an optional embodiment, the detection device may further include a state construction module 303, which is configured to: obtain a local processing rate based on the local calculation frequency, time slot length, and the number of CPU cycles required per bit of task; obtain first data based on the allocated calculation frequency, time slot length, and the number of CPU cycles required per bit of task; obtain second data based on the target uplink transmission rate and time slot length; obtain an unloading processing rate based on the smaller value between the first data and the second data; and obtain the current remaining task quantity based on the data obtained by subtracting the local processing rate and the unloading processing rate from the current task queue length and the queue constraints; and queue... The constraints are: the task queue length is not less than 0 and not greater than the maximum task queue length; the next task queue length is obtained based on the sum of the current remaining task quantity and the task data quantity, as well as the queue constraints; the battery energy consumed during the entire inspection time does not exceed the initial energy, and the next remaining battery energy is obtained based on the difference between the current remaining battery energy and the total energy consumption of the target time slot; the total energy consumption of the target time slot is the total energy consumption of the time slot corresponding to the current action; the next state of the UAV is obtained based on the next task queue length, the next remaining battery energy, the spatial distance of the next time slot, and the channel gain state of the next time slot to optimize the reinforcement learning model.

[0139] In one optional embodiment, the process of obtaining the uplink transmission rate may include: obtaining the spatial distance and elevation angle based on the flight altitude and horizontal distance of the UAV relative to the target edge node; obtaining the line-of-sight propagation loss and non-line-of-sight propagation loss based on the elevation angle; obtaining the free space path loss based on the spatial distance and carrier wavelength; obtaining the average path loss based on the line-of-sight propagation loss, non-line-of-sight propagation loss and free space path loss; obtaining the channel gain state based on the average path loss; and obtaining the uplink transmission rate based on the channel gain state, system bandwidth, communication transmit power and noise power spectral density.

[0140] In one optional embodiment, the process of obtaining the total processing latency may include: obtaining the offloading transmission latency based on the uplink transmission rate, the calculated offloading ratio, and the task data volume; obtaining the local computation latency based on the number of CPU cycles required per bit of task, the calculated offloading ratio, the task data volume, and the local computation frequency; obtaining the edge processing latency based on the number of CPU cycles required per bit of task, the calculated offloading ratio, the task data volume, and the available frequency of edge-side computation resources; and obtaining the total processing latency based on the offloading transmission latency, the local computation latency, and the edge processing latency; wherein the total processing latency is the maximum value between the sum of the offloading transmission latency and the edge processing latency and the local computation latency.

[0141] In one optional embodiment, the process of obtaining the total energy consumption of the time slot may include: obtaining the communication energy consumption based on the communication transmission power and the time slot length; obtaining the sensing energy consumption based on the sensing transmission power and the time slot length; obtaining the local computing energy consumption based on the local computing frequency and the time slot length; and obtaining the total energy consumption of the time slot based on the communication energy consumption, the sensing energy consumption, and the local computing energy consumption.

[0142] In one optional embodiment, a reinforcement learning model is used to process the current state of the UAV with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The current action is determined from multiple candidate actions, and may include: For any candidate action, the following operations are performed: based on the sensing transmit power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target object, and receiver noise power corresponding to the current candidate action, the target echo signal-to-noise ratio is obtained; the target echo signal-to-noise ratio and false alarm probability are processed using the Markum Q function to obtain the detection probability corresponding to the current candidate action; the detection probability corresponding to the current candidate action is compared with the minimum detection probability to determine whether the current candidate action can be used as the current action.

[0143] In an optional embodiment, after comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the method further includes: determining a safety constraint term based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability; determining an energy constraint term based on the relationship between the current remaining battery energy and the minimum battery energy; obtaining a reward value for the candidate action based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the safety constraint term, and the energy constraint term; and determining the candidate action corresponding to the maximum reward value as the current action.

[0144] In an optional embodiment, the reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The device further includes a training module 304, which can be used to: input the current state and the current action into the first evaluation network and the second evaluation network to obtain a first evaluation value and a second evaluation value; obtain the target next action based on the output result of the next state input into the target policy network and truncation of noise; input the next state and the target next action into the target first evaluation network and the target second evaluation network to obtain the target first evaluation value and the target second evaluation value; and obtain the target next action based on the smaller value of the target first evaluation value and the target second evaluation value. The reward value is used to obtain the target evaluation value. The first evaluation network is updated by minimizing the mean square error between the first evaluation value and the target evaluation value. The second evaluation network is updated by minimizing the mean square error between the second evaluation value and the target evaluation value. After the first and second evaluation networks have been updated a preset number of times, the policy network is updated with the goal of maximizing the updated first and second evaluation values. A soft update mechanism is used to transfer the parameters of the updated policy network, the first evaluation network, and the second evaluation network to the corresponding target policy network, target first evaluation network, and target second evaluation network, respectively, with smoothing coefficients. The above process is repeated until a preset termination condition is reached to obtain the trained reinforcement learning model.

[0145] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0146] The apparatus described above is used to implement the corresponding inspection method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0147] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the inspection method described in any of the above embodiments.

[0148] Figure 4 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0149] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0150] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0151] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0152] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0153] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0154] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0155] The electronic devices described above are used to implement the corresponding inspection methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0156] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the inspection method as described in any of the above embodiments.

[0157] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0158] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the inspection method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0159] It should be noted that the embodiments of this application can also be further described in the following ways: An inspection method applied to a drone includes: processing the current state of the drone using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption; determining the current action from multiple candidate actions; and executing the current action to perform inspection. The current state includes the current task queue length, the spatial distance between the drone and a target edge node, a channel gain state obtained based on the spatial distance, and the current remaining battery energy. The candidate actions include communication transmission power, sensing transmission power, and computational offload ratio. The uplink transmission rate is obtained based on the channel gain state and the communication transmission power. The total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computing resources. The total time slot energy consumption is obtained based on the communication energy consumption based on the communication transmission power, the sensing energy consumption based on the sensing transmission power, and the local computation energy consumption.

[0160] Optionally, the constraints of the reward function include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensed transmission power to ensure safety throughout the inspection flight process; the energy constraint is constructed based on the current remaining battery energy to ensure endurance safety.

[0161] Optionally, after performing the current action, the method further includes: obtaining the local processing rate based on the local computing frequency, the time slot length, and the number of CPU cycles required per bit of task; obtaining first data based on the allocated computing frequency, the time slot length, and the number of CPU cycles required per bit of task; obtaining second data based on the target uplink transmission rate and the time slot length; obtaining the offloading processing rate based on the smaller value between the first data and the second data; and obtaining the current remaining task quantity based on the data obtained by subtracting the local processing rate and the offloading processing rate from the current task queue length and the queue constraint; wherein the queue constraint is the task queue. The length is not less than 0 and not greater than the maximum task queue length; the next task queue length is obtained based on the sum of the current remaining task quantity and the task data quantity, as well as the queue constraints; the remaining battery energy is obtained based on the difference between the current remaining battery energy and the total energy consumption of the target time slot, with the constraint that the battery energy consumed during the entire inspection time does not exceed the initial energy; the total energy consumption of the target time slot is the total energy consumption of the time slot corresponding to the current action; the next state of the UAV is obtained based on the next task queue length, the next remaining battery energy, the spatial distance of the next time slot, and the channel gain state of the next time slot to optimize the reinforcement learning model.

[0162] Optionally, the process of obtaining the uplink transmission rate includes: obtaining the spatial distance and elevation angle based on the flight altitude and horizontal distance of the UAV relative to the target edge node; obtaining the line-of-sight propagation loss and non-line-of-sight propagation loss based on the elevation angle; obtaining the free-space path loss based on the spatial distance and carrier wavelength; obtaining the average path loss based on the line-of-sight propagation loss, the non-line-of-sight propagation loss, and the free-space path loss; obtaining the channel gain state based on the average path loss; and obtaining the uplink transmission rate based on the channel gain state, the system bandwidth, the communication transmit power, and the noise power spectral density.

[0163] Optionally, the process of obtaining the total processing latency includes: obtaining the offloading transmission latency based on the uplink transmission rate, the computation offloading ratio, and the task data volume; obtaining the local computation latency based on the number of CPU cycles required per bit of task, the computation offloading ratio, the task data volume, and the local computation frequency; obtaining the edge processing latency based on the number of CPU cycles required per bit of task, the computation offloading ratio, the task data volume, and the available frequency of edge-side computing resources; and obtaining the total processing latency based on the offloading transmission latency, the local computation latency, and the edge processing latency; wherein the total processing latency is the maximum value between the sum of the offloading transmission latency and the edge processing latency and the local computation latency.

[0164] Optionally, the process of obtaining the total energy consumption of the time slot includes: obtaining the communication energy consumption based on the communication transmission power and the time slot length; obtaining the sensing energy consumption based on the sensing transmission power and the time slot length; obtaining the local computing energy consumption based on the local computing frequency and the time slot length; and obtaining the total energy consumption of the time slot based on the communication energy consumption, the sensing energy consumption, and the local computing energy consumption.

[0165] Optionally, a reinforcement learning model is used to process the current state of the UAV with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. This process determines the current action from multiple candidate actions, including: for any candidate action, performing operations such as: obtaining the target echo signal-to-noise ratio (SNR) corresponding to the current candidate action based on the sensing transmit power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target object, and receiver noise power; processing the target echo SNR and false alarm probability using a Markum Q-function to obtain the detection probability corresponding to the current candidate action; and comparing the detection probability corresponding to the current candidate action with the minimum detection probability to determine whether the current candidate action can be used as the current action.

[0166] Optionally, after comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the method further includes: determining a safety constraint term based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability; determining an energy constraint term based on the relationship between the current remaining battery energy and the minimum battery energy; obtaining a reward value for the candidate action based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the safety constraint term, and the energy constraint term; and determining the candidate action corresponding to the maximum reward value as the current action.

[0167] Optionally, the reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The process of training the reinforcement learning model includes: inputting the current state and the current action into the first evaluation network and the second evaluation network to obtain a first evaluation value and a second evaluation value; obtaining the target next action based on the output of the next state input into the target policy network and truncated noise; inputting the next state and the target next action into the target first evaluation network and the target second evaluation network to obtain a target first evaluation value and a target second evaluation value; and determining the target reward based on the smaller of the target first evaluation value and the target second evaluation value. The first evaluation network is updated by minimizing the mean square error between the first evaluation value and the target evaluation value; the second evaluation network is updated by minimizing the mean square error between the second evaluation value and the target evaluation value; after the first evaluation network and the second evaluation network have been updated a preset number of times, the policy network is updated with the goal of maximizing the updated first evaluation value and the second evaluation value; a soft update mechanism is used to transfer the parameters of the updated policy network, the first evaluation network and the second evaluation network to the corresponding target policy network, the target first evaluation network and the target second evaluation network with smoothing coefficients, respectively; the above process is repeated until a preset termination condition is reached to obtain the trained reinforcement learning model.

[0168] A drone includes: a processing module for processing the current state of the drone using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption, and determining a current action from multiple candidate actions; wherein the current state includes the current task queue length, the spatial distance between the drone and a target edge node, a channel gain state obtained based on the spatial distance, and the current remaining battery energy; the candidate actions include communication transmission power, sensing transmission power, and computational offload ratio; the uplink transmission rate is obtained based on the channel gain state and the communication transmission power; the total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computing resources; the total time slot energy consumption is obtained based on the communication energy consumption obtained based on the communication transmission power, the sensing energy consumption obtained based on the sensing transmission power, and the local computation energy consumption; and an execution module for executing the current action to perform inspection.

[0169] Optionally, the constraints of the reward function include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensed transmission power to ensure safety throughout the inspection flight process; the energy constraint is constructed based on the current remaining battery energy to ensure endurance safety.

[0170] Optionally, the apparatus further includes a state construction module, which is configured to: obtain the local processing rate based on the local computing frequency, the time slot length, and the number of CPU cycles required per bit of task; obtain first data based on the allocated computing frequency, the time slot length, and the number of CPU cycles required per bit of task; obtain second data based on the target uplink transmission rate and the time slot length; obtain the offloading processing rate based on the smaller value between the first data and the second data; and obtain the current remaining task quantity based on the data obtained by subtracting the local processing rate and the offloading processing rate from the current task queue length and the queue constraint; wherein the queue constraint is... The task queue length is not less than 0 and not greater than the maximum task queue length; the next task queue length is obtained based on the sum of the current remaining task quantity and the task data quantity, as well as the queue constraints; with the constraint that the battery energy consumed during the entire inspection time does not exceed the initial energy, the next remaining battery energy is obtained based on the difference between the current remaining battery energy and the total energy consumption of the target time slot; the total energy consumption of the target time slot is the total energy consumption of the time slot corresponding to the current action; the next state of the UAV is obtained based on the next task queue length, the next remaining battery energy, the spatial distance of the next time slot, and the channel gain state of the next time slot to optimize the reinforcement learning model.

[0171] Optionally, the process of obtaining the uplink transmission rate includes: obtaining the spatial distance and elevation angle based on the flight altitude and horizontal distance of the UAV relative to the target edge node; obtaining the line-of-sight propagation loss and non-line-of-sight propagation loss based on the elevation angle; obtaining the free-space path loss based on the spatial distance and carrier wavelength; obtaining the average path loss based on the line-of-sight propagation loss, the non-line-of-sight propagation loss, and the free-space path loss; obtaining the channel gain state based on the average path loss; and obtaining the uplink transmission rate based on the channel gain state, the system bandwidth, the communication transmit power, and the noise power spectral density.

[0172] Optionally, the process of obtaining the total processing latency includes: obtaining the offloading transmission latency based on the uplink transmission rate, the computation offloading ratio, and the task data volume; obtaining the local computation latency based on the number of CPU cycles required per bit of task, the computation offloading ratio, the task data volume, and the local computation frequency; obtaining the edge processing latency based on the number of CPU cycles required per bit of task, the computation offloading ratio, the task data volume, and the available frequency of edge-side computing resources; and obtaining the total processing latency based on the offloading transmission latency, the local computation latency, and the edge processing latency; wherein the total processing latency is the maximum value between the sum of the offloading transmission latency and the edge processing latency and the local computation latency.

[0173] Optionally, the process of obtaining the total energy consumption of the time slot includes: obtaining the communication energy consumption based on the communication transmission power and the time slot length; obtaining the sensing energy consumption based on the sensing transmission power and the time slot length; obtaining the local computing energy consumption based on the local computing frequency and the time slot length; and obtaining the total energy consumption of the time slot based on the communication energy consumption, the sensing energy consumption, and the local computing energy consumption.

[0174] Optionally, a reinforcement learning model is used to process the current state of the UAV with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. This process determines the current action from multiple candidate actions, including: for any candidate action, performing operations such as: obtaining the target echo signal-to-noise ratio (SNR) corresponding to the current candidate action based on the sensing transmit power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target object, and receiver noise power; processing the target echo SNR and false alarm probability using a Markum Q-function to obtain the detection probability corresponding to the current candidate action; and comparing the detection probability corresponding to the current candidate action with the minimum detection probability to determine whether the current candidate action can be used as the current action.

[0175] Optionally, after comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the method further includes: determining a safety constraint term based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability; determining an energy constraint term based on the relationship between the current remaining battery energy and the minimum battery energy; obtaining a reward value for the candidate action based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the safety constraint term, and the energy constraint term; and determining the candidate action corresponding to the maximum reward value as the current action.

[0176] Optionally, the reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The device further includes a training module, which is configured to: input the current state and the current action into the first evaluation network and the second evaluation network to obtain a first evaluation value and a second evaluation value; obtain the target next action based on the output of the next state input into the target policy network and truncated noise; input the next state and the target next action into the target first evaluation network and the target second evaluation network to obtain a target first evaluation value and a target second evaluation value; and determine the target next action based on the smaller of the target first evaluation value and the target second evaluation value. The reward value is used to obtain the target evaluation value; the first evaluation network is updated by minimizing the mean square error between the first evaluation value and the target evaluation value; the second evaluation network is updated by minimizing the mean square error between the second evaluation value and the target evaluation value; after the first evaluation network and the second evaluation network have been updated a preset number of times, the policy network is updated with the goal of maximizing the updated first evaluation value and the second evaluation value; a soft update mechanism is used to transfer the parameters of the updated policy network, the first evaluation network and the second evaluation network to the corresponding target policy network, the target first evaluation network and the target second evaluation network with smoothing coefficients, respectively; the above process is repeated until a preset termination condition is reached to obtain the trained reinforcement learning model.

[0177] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the inspection method.

[0178] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the inspection method.

[0179] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0180] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0181] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0182] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. An inspection method applied to unmanned aerial vehicles (UAVs), comprising: The current state of the UAV is processed using a reinforcement learning model with the objective of maximizing the reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption, and the current action is determined from multiple candidate actions. Perform the current action to conduct an inspection; The current state includes the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state obtained based on the spatial distance, and the current remaining battery energy; the candidate actions include communication transmission power, sensing transmission power, and calculation offload ratio. The uplink transmission rate is obtained based on the channel gain state and the communication transmit power; the total processing latency is obtained based on the uplink transmission rate, task data volume, computation offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge computing resources; the total time slot energy consumption is obtained based on the communication energy consumption based on the communication transmit power, the sensing energy consumption based on the sensing transmit power, and the local computation energy consumption.

2. The method according to claim 1, characterized in that, The constraints of the reward function include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensing transmission power to ensure safety throughout the inspection flight process. The energy constraint is constructed based on the current remaining battery energy to ensure range safety.

3. The method according to claim 1, characterized in that, After performing the current action, the following is also included: The local processing rate is obtained based on the local computing frequency, time slot length, and the number of CPU cycles required per bit of task; The first data is obtained based on the allocated calculation frequency, the time slot length, and the number of CPU cycles required per bit of task; The second data is obtained based on the target uplink transmission rate and the time slot length; The unloading processing rate is obtained based on the smaller value between the first data and the second data; The remaining number of tasks is obtained by subtracting the local processing rate and the unloading processing rate from the current task queue length and by the queue constraints; the queue constraints are that the task queue length is not less than 0 and not greater than the maximum task queue length. The length of the next task queue is obtained based on the sum of the current remaining task quantity and the task data quantity, as well as the queue constraints. With the constraint that the battery energy consumed during the entire inspection process does not exceed the initial energy, the remaining battery energy for the next time slot is obtained based on the difference between the current remaining battery energy and the target total energy consumption of the time slot; the target total energy consumption of the time slot is the total energy consumption of the time slot corresponding to the current action. The next state of the UAV is obtained based on the next task queue length, the remaining battery energy, the spatial distance of the next time slot, and the channel gain state of the next time slot, so as to optimize the reinforcement learning model.

4. The method according to claim 1, characterized in that, The process of obtaining the uplink transmission rate includes: The spatial distance and elevation angle are obtained based on the UAV's flight altitude and horizontal distance relative to the target edge node; Based on the elevation angle, the line-of-sight propagation loss and non-line-of-sight propagation loss are obtained; The free space path loss is obtained based on the spatial distance and carrier wavelength. The average path loss is obtained based on the line-of-sight propagation loss, the non-line-of-sight propagation loss, and the free space path loss; The channel gain state is obtained based on the average path loss; The uplink transmission rate is obtained based on the channel gain state, the system bandwidth, the communication transmit power, and the noise power spectral density.

5. The method according to claim 1, characterized in that, The process of obtaining the total processing delay includes: The offloading transmission delay is obtained based on the uplink transmission rate, the calculated offloading ratio, and the task data volume. The local computation latency is obtained based on the number of CPU cycles required per bit of task, the computation offload ratio, the task data volume, and the local computation frequency. The edge processing latency is obtained based on the number of CPU cycles required per bit task, the computation offload ratio, the task data volume, and the available frequency of edge-side computing resources. The total processing delay is obtained based on the offloading transmission delay, the local calculation delay, and the edge processing delay; The total processing delay is the maximum value of the sum of the offloading transmission delay and the edge processing delay and the local calculation delay.

6. The method according to claim 1, characterized in that, The process of obtaining the total energy consumption of the time slot includes: The communication power consumption is obtained based on the communication transmission power and the time slot length; The sensing power consumption is obtained based on the sensing transmission power and the time slot length; The local computing energy consumption is obtained based on the local computing frequency and the time slot length; The total energy consumption of the time slot is obtained based on the communication energy consumption, the sensing energy consumption, and the local computing energy consumption.

7. The method according to claim 1, characterized in that, The current state of the UAV is processed using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The current action is determined from multiple candidate actions, including: For any candidate action, the operations performed include: The target echo signal-to-noise ratio corresponding to the current candidate action is obtained based on the sensing transmission power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target object, and receiver noise power. The target echo signal-to-noise ratio and false alarm probability are processed using the Markum Q function to obtain the detection probability corresponding to the current candidate action; The detection probability corresponding to the current candidate action is compared with the minimum detection probability to determine whether the current candidate action can be used as the current action.

8. The method according to claim 7, characterized in that, After comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the method further includes: Based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability, the safety constraint term is determined; Based on the relationship between the current remaining battery energy and the minimum battery energy, determine the energy constraint term; The reward value for the candidate action is obtained based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the security constraint, and the energy constraint. The candidate action corresponding to the maximum reward value is determined and used as the current action.

9. The method according to any one of claims 1-8, characterized in that, The reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The process of training the reinforcement learning model includes: The current state and the current action are input into the first evaluation network and the second evaluation network to obtain a first evaluation value and a second evaluation value; Based on the output of the next state input to the target policy network and the truncated noise, the next action of the target is obtained; The next state and the next action of the target are input into the first evaluation network and the second evaluation network of the target to obtain the first evaluation value and the second evaluation value of the target; The target evaluation value is obtained by combining the smaller of the first target evaluation value and the second target evaluation value with the target reward value. The first evaluation network is updated by minimizing the mean square error between the first evaluation value and the target evaluation value; The second evaluation network is updated by minimizing the mean square error between the second evaluation value and the target evaluation value; After the first evaluation network and the second evaluation network have been updated a preset number of times, the strategy network is updated with the goal of maximizing the updated first evaluation value and the second evaluation value. A soft update mechanism is adopted to migrate the parameters of the updated policy network, the first evaluation network and the second evaluation network to the corresponding target policy network, the target first evaluation network and the target second evaluation network with smoothing coefficients, respectively. Repeat the above process until the preset termination condition is met to obtain the trained reinforcement learning model.

10. An unmanned aerial vehicle (UAV), comprising: The processing module is used to process the current state of the UAV using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption, and to determine the current action from multiple candidate actions. The current state includes the current task queue length, the spatial distance between the UAV and the target edge node, the channel gain state obtained based on the spatial distance, and the current remaining battery energy. The candidate actions include communication transmission power, sensing transmission power, and computational offload ratio. The uplink transmission rate is obtained based on the channel gain state and the communication transmission power. The total processing latency is obtained based on the uplink transmission rate, task data volume, computational offload ratio, number of CPU cycles required per bit of task, local computation frequency, and available frequency of edge-side computing resources. The total time slot energy consumption is obtained based on the communication energy consumption based on the communication transmission power, the sensing energy consumption based on the sensing transmission power, and the local computation energy consumption. The execution module is used to perform the current action for inspection.

11. The apparatus according to claim 10, characterized in that, The constraints of the reward function include a safety constraint and / or an energy constraint. The safety constraint is constructed based on the sensing transmission power to ensure safety throughout the inspection flight process. The energy constraint is constructed based on the current remaining battery energy to ensure range safety.

12. The apparatus according to claim 10, characterized in that, The device further includes a state construction module, which is used for: The local processing rate is obtained based on the local computing frequency, the time slot length, and the number of CPU cycles required per bit task; the first data is obtained based on the allocated computing frequency, the time slot length, and the number of CPU cycles required per bit task. The second data is obtained based on the target uplink transmission rate and the time slot length; The unloading processing rate is obtained based on the smaller value between the first data and the second data; The remaining number of tasks is obtained by subtracting the local processing rate and the unloading processing rate from the current task queue length and by the queue constraints; the queue constraints are that the task queue length is not less than 0 and not greater than the maximum task queue length. The length of the next task queue is obtained based on the sum of the current remaining task quantity and the task data quantity, as well as the queue constraints. With the constraint that the battery energy consumed during the entire inspection process does not exceed the initial energy, the remaining battery energy for the next time slot is obtained based on the difference between the current remaining battery energy and the target total energy consumption of the time slot; the target total energy consumption of the time slot is the total energy consumption of the time slot corresponding to the current action. The next state of the UAV is obtained based on the next task queue length, the remaining battery energy, the spatial distance of the next time slot, and the channel gain state of the next time slot, so as to optimize the reinforcement learning model.

13. The apparatus according to claim 10, characterized in that, The process of obtaining the uplink transmission rate includes: The spatial distance and elevation angle are obtained based on the UAV's flight altitude and horizontal distance relative to the target edge node; Based on the elevation angle, the line-of-sight propagation loss and non-line-of-sight propagation loss are obtained; The free space path loss is obtained based on the spatial distance and carrier wavelength. The average path loss is obtained based on the line-of-sight propagation loss, the non-line-of-sight propagation loss, and the free space path loss; The channel gain state is obtained based on the average path loss; The uplink transmission rate is obtained based on the channel gain state, the system bandwidth, the communication transmit power, and the noise power spectral density.

14. The apparatus according to claim 10, characterized in that, The process of obtaining the total processing delay includes: The offloading transmission delay is obtained based on the uplink transmission rate, the calculated offloading ratio, and the task data volume. The local computation latency is obtained based on the number of CPU cycles required per bit of task, the computation offload ratio, the task data volume, and the local computation frequency. The edge processing latency is obtained based on the number of CPU cycles required per bit task, the computation offload ratio, the task data volume, and the available frequency of edge-side computing resources. The total processing delay is obtained based on the offloading transmission delay, the local calculation delay, and the edge processing delay; The total processing delay is the maximum value of the sum of the offloading transmission delay and the edge processing delay and the local calculation delay.

15. The apparatus according to claim 10, characterized in that, The process of obtaining the total energy consumption of the time slot includes: The communication power consumption is obtained based on the communication transmission power and the time slot length; The sensing power consumption is obtained based on the sensing transmission power and the time slot length; The local computing energy consumption is obtained based on the local computing frequency and the time slot length; The total energy consumption of the time slot is obtained based on the communication energy consumption, the sensing energy consumption, and the local computing energy consumption.

16. The apparatus according to claim 10, characterized in that, The current state of the UAV is processed using a reinforcement learning model with the objective of maximizing a reward function constructed based on the current task queue length, uplink transmission rate, total processing latency, and total time slot energy consumption. The current action is determined from multiple candidate actions, including: For any candidate action, the operations performed include: The target echo signal-to-noise ratio corresponding to the current candidate action is obtained based on the sensing transmission power, radar antenna gain, carrier wavelength, target radar cross section, actual distance between the UAV and the target object, and receiver noise power. The target echo signal-to-noise ratio and false alarm probability are processed using the Markum Q function to obtain the detection probability corresponding to the current candidate action; The detection probability corresponding to the current candidate action is compared with the minimum detection probability to determine whether the current candidate action can be used as the current action.

17. The apparatus according to claim 16, characterized in that, After comparing the detection probability corresponding to the current candidate action with the minimum detection probability, the method further includes: Based on the relationship between the detection probability corresponding to the current candidate action and the minimum detection probability, the safety constraint term is determined; Based on the relationship between the current remaining battery energy and the minimum battery energy, determine the energy constraint term; The reward value for the candidate action is obtained based on the ratio of the current task queue length to the maximum task queue length, the ratio of the uplink transmission rate to the maximum uplink transmission rate, the ratio of the total processing latency to the maximum total processing latency, the ratio of the total time slot energy consumption to the maximum total time slot energy consumption, the security constraint, and the energy constraint. The candidate action corresponding to the maximum reward value is determined and used as the current action.

18. The apparatus according to any one of claims 10-17, characterized in that, The reinforcement learning model includes a policy network, a first evaluation network, a second evaluation network, a target policy network, a target first evaluation network, and a target second evaluation network. The device further includes a training module, which is used for: The current state and the current action are input into the first evaluation network and the second evaluation network to obtain a first evaluation value and a second evaluation value; Based on the output of the next state input to the target policy network and the truncated noise, the next action of the target is obtained; The next state and the next action of the target are input into the first evaluation network and the second evaluation network of the target to obtain the first evaluation value and the second evaluation value of the target; The target evaluation value is obtained by combining the smaller of the first target evaluation value and the second target evaluation value with the target reward value. The first evaluation network is updated by minimizing the mean square error between the first evaluation value and the target evaluation value; The second evaluation network is updated by minimizing the mean square error between the second evaluation value and the target evaluation value; After the first evaluation network and the second evaluation network have been updated a preset number of times, the strategy network is updated with the goal of maximizing the updated first evaluation value and the second evaluation value. A soft update mechanism is adopted to migrate the parameters of the updated policy network, the first evaluation network and the second evaluation network to the corresponding target policy network, the target first evaluation network and the target second evaluation network with smoothing coefficients, respectively. Repeat the above process until the preset termination condition is met to obtain the trained reinforcement learning model.

19. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of any one of claims 1 to 9.

Images