A drone inspection method, equipment and medium based on deep reinforcement learning

Abstract

The present invention relates to a drone inspection method, device and medium based on deep reinforcement learning, and belongs to the field of drone communication technology. The method steps include obtaining the location information of the base station and the drone, calculating the distance between them, and determining the probability of the existence of a line-of-sight link and the large-scale path loss based on this distance, and then constructing a channel model. Then, based on the channel model, the data transmission time between the drone and the base station is calculated, while considering the data transmission time between the base station and the data terminal through multi-hop transmission. Finally, an objective function is established to minimize the total inspection data transmission time, and solved by a deep learning algorithm to obtain the optimal flight path and transmission power of the drone. The present invention simplifies the comprehensive consideration of three-dimensional space and time and reduces the computational complexity.

Description

Technical Field

[0001] The present invention relates to a drone inspection method, equipment and medium based on deep reinforcement learning, and belongs to the field of drone communication technology. Background Art

[0002] Drones, with their flexible deployment capabilities and ease of control, have shown great potential in applications such as real-time environmental monitoring in remote areas, power base station inspections, and aerial logistics. In particular, proper drone trajectory planning can significantly improve the overall performance of drone wireless networks. However, drone trajectory planning typically involves complex computations in high-dimensional space, including considerations of both three-dimensional space and time, which undoubtedly increases the difficulty and computational cost of the problem.

[0003] In the early days of drone path planning, algorithms primarily relied on basic graph search methods, such as Dijkstra's algorithm. These methods effectively found the shortest path or optimal solution, which was particularly important for applications with strict optimality requirements. However, when dealing with large graphs or state spaces, the complexity of these graph search algorithms increased dramatically, leading to a significant increase in computational cost.

[0004] Prior art, such as the Chinese patent application with publication number "CN119001800B," discloses a beacon system and navigation method for drone navigation, including the following steps: collecting performance parameters generated by the beacon node closest to the drone based on its initial position; calculating the beacon node spacing within the drone's target flight area based on the performance parameters, and arranging a number of beacon nodes on the ground in an equidistant manner; monitoring the signal strength received by the beacon node based on the performance parameters and signal loss, comparing the received signal strength with a signal strength threshold, and when the received signal strength is less than the signal strength threshold, recording the current beacon node position, detecting the location of obstacles, and adjusting the beacon node to the target position based on the obstacle position; collecting drone position information through the beacon node at the target position, and selecting a path based on the current drone position information using a particle swarm optimization algorithm, so that the drone reaches its destination according to the selected optimal path. However, the above patent mainly focuses on the navigation problem of the drone, achieving path planning through beacon nodes and a particle swarm optimization algorithm, and ignores communication efficiency, which is also an important consideration during drone inspections. In practical applications, drones may need to frequently transmit data to base stations. Inefficient communication can affect the time and effectiveness of inspection tasks. While this method uses a particle swarm optimization algorithm for path planning, it is relatively traditional and lacks the intelligence and adaptability offered by deep reinforcement learning. Summary of the Invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes a drone inspection method, equipment and medium based on deep reinforcement learning.

[0006] The technical solutions of the present invention are as follows:

[0007] In one aspect, the present invention provides a drone inspection method based on deep reinforcement learning, comprising the following steps:

[0008] Obtain the location information of the base station and the drone in the inspection area, and calculate the distance between the base station and the drone based on the location information;

[0009] The data transmission during the UAV inspection process is divided into air-ground communication and multi-hop transmission communication;

[0010] For air-to-ground communications, the probability of a line-of-sight link between the drone and the base station is calculated based on the distance between the base station and the drone; as well as the large-scale path loss between the drone and the base station;

[0011] A channel model between the UAV and the base station is constructed based on the large-scale path loss and the probability, the UAV's transmit power is used as input, a signal-to-noise ratio is output, and the data transmission time between the UAV and the base station is calculated based on the signal-to-noise ratio;

[0012] For multi-hop transmission communication, the data transmission time between the base station and the data terminal is calculated based on the multi-hop transmission power of the base station;

[0013] Based on the data transmission time between the UAV and the base station and the data transmission time between the base station and the data terminal, an objective function is constructed with the goal of minimizing the inspection data transmission time, and corresponding constraints are formulated;

[0014] The objective function is solved using a deep learning algorithm to obtain the optimal flight path and transmission power of the UAV while minimizing the inspection data transmission time. The UAV performs inspections based on the optimal flight path and transmission power.

[0015] As a preferred embodiment, the position of the base station is quantified by a two-dimensional coordinate system. The location information of each base station is expressed as:

[0016] ;

[0017] Where, Indicates the The location information of base stations, Indicates the Base stations Axis coordinates, Indicates the Base stations axis coordinates;

[0018] The inspection time Split into Time periods, expressed as:

[0019] ;

[0020] ;

[0021] Where, Indicates the time periods, The index representing the time period;

[0022] The position of the UAV is quantified through a two-dimensional coordinate system. The position information of the UAV in each time period is expressed as:

[0023] ;

[0024] ;

[0025] Where, Indicates the The location information of the drone in each time period, Indicates the time period of drones Axis coordinates, Indicates the time period of drones Axis coordinates, Indicates the altitude of the drone, Indicates the The flight speed of the drone in each time period, Indicates the preset maximum flight speed.

[0026] As a preferred embodiment, the probability of a line-of-sight link between the UAV and the base station is calculated based on the distance between the base station and the UAV, which can be expressed as follows:

[0027] ;

[0028] Where, 、 Represents environmental parameters, represents the exponential function, Indicates in Time period The elevation angle from the base station to the drone, Indicates in The UAV and the The probability that there is a line-of-sight link between the base stations;

[0029] The calculation method of elevation angle is:

[0030] ;

[0031] Where, represents the inverse tangent function, Indicates the The drone and the The distance to the base station.

[0032] As a preferred embodiment, the large-scale path loss between the UAV and the base station is expressed as follows:

[0033] ;

[0034] Where, Indicates that the drone and The large-scale path loss of each base station, represents the path loss base value, represents the excess path loss coefficient, Indicates that the drone and The distance between base stations, represents the path loss exponent of the line-of-sight link, Represents the path loss exponent of the non-line-of-sight link;

[0035] The channel model between the UAV and the base station is constructed based on the large-scale path loss and the probability, and is expressed as follows:

[0036] ;

[0037] Where, Indicates that the drone and The channels of the base stations, represents the small-scale fading coefficient;

[0038] Taking the UAV's transmit power as input, the output signal-to-noise ratio is expressed as:

[0039] ;

[0040] Where, Indicates that the drone and The signal-to-noise ratio of each base station, Indicates the transmission power of the UAV, Indicates that the drone and The additive white Gaussian noise power of each base station;

[0041] The data transmission time between the UAV and the base station is calculated based on the signal-to-noise ratio and expressed as follows:

[0042] ;

[0043] Where, Indicates that the drone and The data transmission time of each base station, Indicates that the drone is heading to The amount of data transmitted by each base station, Indicates bandwidth, Represents the base-2 logarithm function.

[0044] As a preferred embodiment, the data transmission time between the base station and the data terminal is calculated by the multi-hop transmission power of the base station, which is expressed as follows:

[0045] ;

[0046] ;

[0047] Where, Indicates the The data transmission time between the base station and the data terminal, represents the multi-hop transmission channel gain, It follows the gamma distribution. represents the gamma function, represents the shape parameter, represents the scale parameter, Indicates the The multi-hop transmission power of each base station, Indicates the data terminal and the The additive white Gaussian noise power of each base station.

[0048] As a preferred embodiment, the objective function is to minimize the inspection data transmission time. , expressed as:

[0049] ;

[0050] The corresponding constraints include reliable transmission constraints, maximum flight distance constraints, flight area constraints, and transmit power constraints, where:

[0051] The reliable transmission constraint is expressed as follows:

[0052] ;

[0053] The maximum flight distance constraint is expressed as follows:

[0054] ;

[0055] The flight area constraint is expressed as follows:

[0056] ;

[0057] The transmit power constraint is expressed as:

[0058] ;

[0059] Where, Indicates the number of base stations, represents the minimum function, represents the constraints, represents the signal-to-noise ratio threshold, Indicates the The location information of the drone in each time period, represents the trajectory of the drone, Indicates the lower bound of the drone's flight area, Indicates the upper bound of the drone’s flight area, Indicates the maximum transmit power of the drone, Indicates the inspection data transmission time.

[0060] As a preferred embodiment, a deep learning algorithm is used to solve the objective function to obtain the optimal flight path and transmission power of the drone while minimizing the inspection data transmission time, specifically:

[0061] Using a 4-tuple Describe the objective function;

[0062] in, represents the state space, represents the action space, represents the state transition probability, represents the reward function;

[0063] The UAV is considered as an intelligent agent that can perform flight trajectory planning and transmit power selection, where the state space, action space and reward function of the agent are expressed as:

[0064] In the Time-segment state space Expressed as:

[0065] ;

[0066] In the Time period action space Expressed as:

[0067] ;

[0068] ;

[0069] Where, Indicates the The flight direction of the drone in each time period, Indicates the The transmission power of the UAV in each time period;

[0070] In the Time period reward function Expressed as:

[0071] .

[0072] As a preferred implementation, the deep enhancement algorithm is a double-delay deep deterministic policy gradient algorithm TD3.

[0073] As a preferred implementation, the deep reinforcement learning method is the TD3 deep reinforcement learning method.

[0074] On the other hand, the present invention also provides an electronic device having a computer program stored thereon, which, when executed by a processor, implements the drone inspection method based on deep reinforcement learning as described in any embodiment of the present invention.

[0075] On the other hand, the present invention also provides a computer-readable storage medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the drone inspection method based on deep reinforcement learning as described in any embodiment of the present invention.

[0076] The present invention has the following beneficial effects:

[0077] 1. The present invention divides the UAV inspection process into air-to-ground communication between the UAV and the base station and multi-hop transmission communication from the base station to the data terminal, calculates the data transmission time of the two separately, and then adds these times as the total inspection data transmission time. This method can more accurately evaluate the communication efficiency during the inspection process.

[0078] 2. This invention uses deep reinforcement learning methods to plan the flight path and transmission power of the drone, with the goal of minimizing the inspection data transmission time, thereby significantly improving the overall efficiency of drone inspections.

[0079] 3. Compared with traditional graph search algorithms (such as Dijkstra algorithm), which have the problem of sharply increased computational complexity when processing large-scale graphs or state spaces, the present invention converts complex three-dimensional space and time comprehensive consideration problems into optimizable objective functions through steps such as constructing a channel model and calculating the signal-to-noise ratio and data transmission time, thereby reducing the computational complexity.

[0080] 4. This invention considers factors such as environmental parameters and the elevation angle from the base station to the drone when calculating the probability of a line-of-sight link between the drone and the base station, thereby more accurately assessing the reliability of the communication link. By constructing a channel model and calculating the signal-to-noise ratio, it is possible to assess the communication quality between the drone and the base station in real time. Based on the assessment results, the flight path and transmit power are adjusted to ensure stable and reliable communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1 The present invention is a flowchart for implementing the method.

[0082] Figure 2 Schematic diagram of the inspection scene of the present invention.

[0083] Figure 3 This is a network structure diagram of the deep reinforcement learning algorithm of the present invention.

[0084] Figure 4 This is a comparison chart of the data transmission completion time between the present invention and the ant colony algorithm. DETAILED DESCRIPTION

[0085] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0086] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are to be executed.

[0087] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0088] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0089] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0090] Example 1:

[0091] See also Figure 1The present invention provides a UAV inspection method based on deep reinforcement learning, comprising the following steps:

[0092] Obtain the location information of the base station and the drone in the inspection area, and calculate the distance between the base station and the drone based on the location information;

[0093] The data transmission during the UAV inspection process is divided into air-ground communication and multi-hop transmission communication;

[0094] The communication between the drone and the base station during the drone inspection process is divided into air-ground communication, and the communication between the base station and the data terminal is divided into multi-hop transmission communication;

[0095] For air-to-ground communications, the probability of a line-of-sight link between the drone and the base station is calculated based on the distance between the base station and the drone; as well as the large-scale path loss between the drone and the base station;

[0096] A channel model between the UAV and the base station is constructed based on the large-scale path loss and the probability, the UAV's transmit power is used as input, a signal-to-noise ratio is output, and the data transmission time between the UAV and the base station is calculated based on the signal-to-noise ratio;

[0097] For multi-hop transmission communication, the data transmission time between the base station and the data terminal is calculated based on the multi-hop transmission power of the base station;

[0098] Based on the data transmission time between the UAV and the base station and the data transmission time between the base station and the data terminal, an objective function is constructed with the goal of minimizing the inspection data transmission time, and corresponding constraints are formulated;

[0099] The objective function is solved using a deep learning algorithm to obtain the optimal flight path and transmission power of the UAV while minimizing the inspection data transmission time. The UAV performs inspections based on the optimal flight path and transmission power.

[0100] See also Figure 2 As a preferred embodiment, the position of the base station is quantified by a two-dimensional coordinate system. The location information of each base station is expressed as:

[0101] ;

[0102] Where, Indicates the The location information of base stations, Indicates the Base stations Axis coordinates, Indicates the Base stations axis coordinates;

[0103] The inspection time Split into Time periods, expressed as:

[0104] ;

[0105] ;

[0106] Where, Indicates the time periods, The index representing the time period;

[0107] The position of the UAV is quantified through a two-dimensional coordinate system. The position information of the UAV in each time period is expressed as:

[0108] ;

[0109] ;

[0110] Where, Indicates the The location information of the drone in each time period, Indicates the time period of drones Axis coordinates, Indicates the time period of drones Axis coordinates, Indicates the altitude of the drone, Indicates the The flight speed of the drone in each time period, Indicates the preset maximum flight speed.

[0111] As a preferred embodiment, the probability of a line-of-sight link between the UAV and the base station is calculated based on the distance between the base station and the UAV, which can be expressed as follows:

[0112] ;

[0113] Where, 、 Represents environmental parameters, represents the exponential function, Indicates in Time period The elevation angle from the base station to the drone, Indicates in The UAV and the The probability that there is a line-of-sight link between the base stations;

[0114] The calculation method of elevation angle is:

[0115] ;

[0116] Where, represents the inverse tangent function, Indicates the The drone and the The distance between base stations;

[0117] The distance is calculated using the formula:

[0118] .

[0119] As a preferred embodiment, the large-scale path loss between the UAV and the base station includes two cases: line-of-sight link and non-line-of-sight link, which can be expressed as follows:

[0120] ;

[0121] Where, Indicates that the drone and The large-scale path loss of each base station, represents the path loss base value, represents the excess path loss coefficient, which shows the additional attenuation due to the larger path loss of non-line-of-sight connections. Indicates that the drone and The distance between base stations, represents the path loss exponent of the line-of-sight link, Represents the path loss exponent of the non-line-of-sight link;

[0122] The channel model between the UAV and the base station is constructed based on the large-scale path loss and the probability, and is expressed as follows:

[0123] ;

[0124] Where, Indicates that the drone and The channels of the base stations, represents the small-scale fading coefficient;

[0125] In this embodiment, the small-scale fading coefficient is Rayleigh fading in the case of a non-line-of-sight link and is Ricean fading with a 15 dB Ricean fading factor in the case of a line-of-sight link;

[0126] Taking the UAV's transmit power as input, the output signal-to-noise ratio is expressed as:

[0127] ;

[0128] Where, Indicates that the drone and The signal-to-noise ratio of each base station, Indicates the transmission power of the UAV, Indicates that the drone and The additive white Gaussian noise power of each base station is a preset value;

[0129] The data transmission time between the UAV and the base station is calculated based on the signal-to-noise ratio and expressed as follows:

[0130] ;

[0131] Where, Indicates that the drone and The data transmission time of each base station, Indicates that the drone is heading to The amount of data transmitted by each base station, Indicates bandwidth, Represents the base-2 logarithm function.

[0132] As a preferred embodiment, the data transmission time between the base station and the data terminal is calculated by the multi-hop transmission power of the base station, which is expressed as follows:

[0133] ;

[0134] The distribution of the multi-hop transmission channel gain has the shape parameter and scale parameters The gamma random variable is expressed as:

[0135] ;

[0136] Where, Indicates the The data transmission time between the base station and the data terminal, represents the multi-hop transmission channel gain, It follows the gamma distribution. represents the gamma function, Represents the shape parameter, which is the preset value. Represents the scale parameter, which is the preset value. Indicates the The multi-hop transmission power of each base station, Indicates the data terminal and the The additive white Gaussian noise power of each base station.

[0137] As a preferred embodiment, the objective function is to minimize the inspection data transmission time. , expressed as:

[0138] ;

[0139] The corresponding constraints include reliable transmission constraints, maximum flight distance constraints, flight area constraints, and transmit power constraints, where:

[0140] The reliable transmission constraint is expressed as follows:

[0141] ;

[0142] The maximum flight distance constraint is expressed as follows:

[0143] ;

[0144] The flight area constraint is expressed as follows:

[0145] ;

[0146] The transmit power constraint is expressed as:

[0147] ;

[0148] Where, Indicates the number of base stations, Represents the trajectory of the UAV to minimize and UAV transmission power is the minimum function of the target, represents the constraints, represents the signal-to-noise ratio threshold, Indicates the The location information of the drone in each time period, represents the trajectory of the drone, Indicates the lower bound of the drone's flight area, Indicates the upper bound of the drone’s flight area, Indicates the maximum transmit power of the drone, Indicates the inspection data transmission time. Indicates taking the absolute value.

[0149] See also Figure 3 TD3's deep reinforcement learning approach is based on Deep Deterministic Policy Gradient (DDPG) and is designed to address the overestimation problem of the DDPG algorithm. TD3 employs two Q-networks, an actor network and a critic network, to address the overestimation problem by selecting the minimum target Q. In each update, the Q-target network with the smallest Q value is selected as the Q-target. TD3 updates the actor and target networks less frequently than the current critic network. This less frequent update method results in lower variance in value estimates, making the current critic network more stable. Target policy smoothing regularization is also employed.

[0150] As a preferred embodiment, a deep learning algorithm is used to solve the objective function to obtain the optimal flight path and transmission power of the drone while minimizing the inspection data transmission time, specifically:

[0151] Using a 4-tuple Describe the objective function;

[0152] in, represents the state space, represents the action space, represents the state transition probability, represents the reward function;

[0153] The UAV is considered as an intelligent agent that can perform flight trajectory planning and transmit power selection, where the state space, action space and reward function of the agent are expressed as:

[0154] In the Time-segment state space Expressed as:

[0155] ;

[0156] In the Time period action space Expressed as:

[0157] ;

[0158] ;

[0159] Where, Indicates the The flight direction of the drone in each time period, Indicates the The transmission power of the UAV in each time period;

[0160] In the Time period reward function Expressed as:

[0161] .

[0162] As a preferred implementation, the deep enhancement algorithm is a double-delay deep deterministic policy gradient algorithm TD3.

[0163] In order to verify the effectiveness of the method of this embodiment, the steps described in this embodiment are used to calculate the transmission completion time at a flight altitude of 100m and 150m respectively. Figure 4As the number of base stations increases, the data transmission completion time of all algorithms will increase. When the drone is flying at an altitude of 150m, it takes more time to complete data transmission than when it is flying at 100m. The transmission completion time at flight altitudes of 150m and 100m is further calculated based on the Ant Colony Algorithm (ACO). In all scenarios, the data transmission completion time of the deep reinforcement learning algorithm proposed in this invention is always shorter than the data transmission completion time based on the Ant Colony Algorithm (ACO).

[0164] Example 2:

[0165] This embodiment provides an electronic device having a computer program stored thereon. When the computer program is executed by a processor, the drone inspection method based on deep reinforcement learning as described in any embodiment of the present invention is implemented.

[0166] Example 3:

[0167] This embodiment provides a computer-readable storage medium for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the drone inspection method based on deep reinforcement learning as described in any embodiment of the present invention.

[0168] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0169] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0171] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk, and other media that can store program code.

[0172] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A drone inspection method based on deep reinforcement learning, characterized in that: The following steps are involved: Obtain the location information of the base station and the drone in the inspection area, and calculate the distance between the base station and the drone based on the location information; The data transmission during the UAV inspection process is divided into air-ground communication and multi-hop transmission communication; For air-to-ground communication, the probability of a line-of-sight link between the UAV and the base station is calculated based on the distance between the base station and the UAV, which can be expressed as: ； Where, 、 Represents environmental parameters, represents the exponential function, Indicates in Time period The elevation angle from the base station to the drone, Indicates in The UAV and the The probability that there is a line-of-sight link between the base stations; and large-scale path loss between the drone and the base station; Based on the large-scale path loss and the above probability, the channel model between the UAV and the base station is constructed. The UAV's transmit power is used as input and the output signal-to-noise ratio is expressed as: ； Where, Indicates that the drone and The signal-to-noise ratio of each base station, Indicates the transmission power of the UAV, Indicates that the drone and The additive white Gaussian noise power of each base station, Indicates that the drone and channels of base stations; The data transmission time between the UAV and the base station is calculated based on the signal-to-noise ratio and expressed as follows: ； Where, Indicates that the drone and The data transmission time of each base station, Indicates that the drone is heading to The amount of data transmitted by each base station, Indicates bandwidth, represents the logarithmic function with base 2; For multi-hop transmission communication, the data transmission time between the base station and the data terminal is calculated using the multi-hop transmission power of the base station, which can be expressed as follows: ； ； Where, Indicates the The data transmission time between the base station and the data terminal, represents the multi-hop transmission channel gain, It follows the gamma distribution. represents the gamma function, represents the shape parameter, represents the scale parameter, Indicates the The multi-hop transmission power of each base station, Indicates the data terminal and the The additive white Gaussian noise power of each base station; Based on the data transmission time between the UAV and the base station and the data transmission time between the base station and the data terminal, an objective function is constructed with the goal of minimizing the inspection data transmission time. , expressed as: ； And formulate corresponding constraints, including: ； ； ； ； Where, Indicates the number of base stations, represents the minimum function, represents the constraints, represents the signal-to-noise ratio threshold, Indicates the The location information of the drone in each time period, represents the trajectory of the drone, Indicates the lower bound of the drone's flight area, Indicates the upper bound of the drone’s flight area, Indicates the maximum transmit power of the drone, Indicates the inspection data transmission time. Indicates the time periods, Indicates the preset maximum flight speed. Indicates the The location information of the drone in each time period, The index representing the time period; The objective function is solved using a deep learning algorithm to obtain the optimal flight path and transmission power of the UAV while minimizing the inspection data transmission time. The UAV performs inspections based on the optimal flight path and transmission power.

2. The UAV inspection method based on deep reinforcement learning according to claim 1 is characterized in that: The position of the base station is quantified by a two-dimensional coordinate system. The location information of each base station is expressed as: ； Where, Indicates the The location information of base stations, Indicates the Base stations Axis coordinates, Indicates the Base stations axis coordinates; The inspection time Split into Time periods, expressed as: ； ； The position of the UAV is quantified through a two-dimensional coordinate system. The position information of the UAV in each time period is expressed as: ； ； Where, Indicates the time period of drones Axis coordinates, Indicates the time period of drones Axis coordinates, Indicates the altitude of the drone, Indicates the The flight speed of the drone in a certain time period.

3. The UAV inspection method based on deep reinforcement learning according to claim 2 is characterized in that: The calculation method of elevation angle is: ； Where, represents the inverse tangent function, Indicates the The drone and the The distance to the base station.

4. The UAV inspection method based on deep reinforcement learning according to claim 3 is characterized in that: The large-scale path loss between the drone and the base station is expressed as: ； Where, Indicates that the drone and The large-scale path loss of each base station, represents the path loss base value, represents the excess path loss coefficient, Indicates that the drone and The distance between base stations, represents the path loss exponent of the line-of-sight link, Represents the path loss exponent of the non-line-of-sight link; The channel model between the UAV and the base station is constructed based on the large-scale path loss and the probability, and is expressed as follows: ； Where, represents the small-scale fading coefficient.

5. The UAV inspection method based on deep reinforcement learning according to claim 4 is characterized in that: The objective function is solved using a deep learning algorithm to obtain the optimal flight path and transmission power of the drone while minimizing the inspection data transmission time. Specifically, Using a 4-tuple Describe the objective function; in, represents the state space, represents the action space, represents the state transition probability, represents the reward function; The UAV is considered as an intelligent agent that can perform flight trajectory planning and transmit power selection, where the state space, action space and reward function of the agent are expressed as: In the Time-segment state space Expressed as: ； In the Time period action space Expressed as: ； ； Where, Indicates the The flight direction of the drone in each time period, Indicates the The transmission power of the UAV in each time period; In the Time period reward function Expressed as: 。 6. The UAV inspection method based on deep reinforcement learning according to claim 5 is characterized in that: The deep reinforcement algorithm is the double-delay deep deterministic policy gradient algorithm TD3.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the drone inspection method based on deep reinforcement learning as described in any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the drone inspection method based on deep reinforcement learning as described in any one of claims 1 to 6 is implemented.

Images