Distribution network unmanned aerial vehicle autonomous route generation method and system based on deep reinforcement learning

Abstract

The invention discloses a distribution network unmanned aerial vehicle autonomous route generation method and system based on deep reinforcement learning. Comprising the following steps: carrying a multispectral camera through an unmanned aerial vehicle to collect a distribution network line image in real time, identifying a tower, a wire and an insulator by adopting an improved target detection algorithm fused with a visual Transform module, and fusing RTK positioning and visual SLAM to construct a three-dimensional environment map; an intelligent body based on PPO is constructed, the state space of the intelligent body comprises the relative position of the unmanned aerial vehicle, the line angle and the obstacle distance, the action space comprises the flight speed, the pan-tilt angle and zoom control, and the decision is optimized through a composite reward function; an epsilon-greedy strategy is adopted to explore an optimal path during first tour inspection, a route file containing a position and a shooting parameter is generated based on an LSTM network, and dynamic recombination is supported to adapt to a special tour inspection requirement; and continuously optimizing the inspection strategy by utilizing transfer learning and online strategy gradient updating. According to the invention, high-precision autonomous decision-making and dynamic path planning of unmanned aerial vehicle inspection are realized, and the inspection efficiency and safety of power equipment are improved.

Description

Technical Field

[0001] This invention belongs to the field of autonomous inspection technology for unmanned aerial vehicles (UAVs), specifically involving a method for generating autonomous flight paths for power distribution network UAVs based on deep reinforcement learning, which is applicable to the intelligent inspection of power distribution network lines. Background Technology

[0002] With the rapid development of smart grids, the demand for distribution network inspection is increasing. Traditional manual inspection methods suffer from low efficiency, high cost, and poor security. Drone inspection is gradually becoming mainstream due to its flexibility and efficiency, but existing drone inspection technologies still have the following problems:

[0003] Insufficient environmental perception: Traditional drones rely on preset routes or remote control operation, making it difficult to adapt to complex and ever-changing power distribution network environments. Their accuracy in identifying key targets such as poles, conductors, and insulators is limited, leading to incomplete inspection coverage or missed inspections. Weak autonomous decision-making capability: Existing drone path planning is mostly based on fixed waypoints or simple obstacle avoidance algorithms, unable to dynamically adjust flight strategies to adapt to sudden obstacles or special inspection needs, affecting inspection efficiency and safety. Low level of intelligence: Traditional methods rely on manual experience to adjust parameters, lacking adaptive learning capabilities, making it difficult to optimize key parameters such as shooting angle and flight speed, affecting data acquisition quality. Poor transfer adaptability: Different line environments vary greatly; existing algorithms need to be retrained in new scenarios, resulting in high computational costs and difficulty in rapid deployment.

[0004] In recent years, deep reinforcement learning (DRL) has demonstrated advantages in the field of autonomous decision-making, but its application in distribution network drone inspection still faces challenges, such as state space modeling, reward function design, and real-time policy optimization. Therefore, there is an urgent need for a high-precision, adaptive, and intelligent method for generating autonomous drone routes to improve the automation level and reliability of distribution network inspection. Summary of the Invention

[0005] This invention proposes an autonomous flight path generation method for power distribution network drones based on deep reinforcement learning. This invention aims to solve the technical problems of poor environmental adaptability, unintelligent obstacle avoidance decision-making, and low inspection efficiency in the existing technology of autonomous flight path generation for power distribution network drones.

[0006] In a first aspect, embodiments of this application provide a method for generating autonomous flight paths for distribution network unmanned aerial vehicles (UAVs) based on deep reinforcement learning, the method comprising:

[0007] S1. Real-time images of power distribution lines are collected using a multispectral camera mounted on a drone. The improved target detection algorithm is used to identify key targets such as towers, conductors and insulators. RTK positioning data and visual SLAM technology are then integrated to construct a three-dimensional environment map.

[0008] S2. Construct an intelligent agent based on the PPO algorithm. Its state space includes the drone's position relative to the pole, the angle of the route, and the distance to obstacles. Its action space includes flight speed adjustment, gimbal angle adjustment, and zoom control. Decisions are optimized through a composite reward function, which includes a coverage reward that is positively correlated with the completeness of the shooting target and a safety reward that is negatively correlated with the distance to obstacles.

[0009] S3. During the first inspection, the optimal flight path is explored through the ε-greedy strategy. Based on the LSTM network, the route file containing latitude, longitude, altitude and shooting angle is generated by memorizing the route features. It also supports dynamic reorganization of waypoints to adapt to special inspection needs.

[0010] S4. Adapt the inspection strategy to the new environment using transfer learning techniques and continuously optimize the inspection strategy through online policy gradient updates.

[0011] Optionally, in one implementation of the first aspect of the present invention, step S1 involves acquiring real-time images of the power distribution network lines using a multispectral camera mounted on a drone, and identifying key targets such as towers, conductors, and insulators using an improved target detection algorithm model, specifically including:

[0012] Real-time images of power distribution lines are collected using a multispectral camera mounted on a drone.

[0013] An improved target detection algorithm model is used to process the acquired images and identify key targets such as towers, conductors and insulators. The improved target detection algorithm model integrates a visual Transformer module to enhance the ability to extract local texture features and uses an improved IoU loss function to optimize the bounding box regression accuracy.

[0014] By integrating RTK positioning data with visual SLAM technology, a three-dimensional environmental map of the power distribution network is constructed; the recognition results are then correlated with the three-dimensional environmental map to generate an inspection report.

[0015] Optionally, in one implementation of the first aspect of the present invention, the model structure of the improved target detection algorithm includes:

[0016] The backbone network adopts a visual Transformer architecture that integrates spatial local perception modules. Specifically, a spatial local perception block consisting of 3×3 dilated convolutions and residual connections is inserted before the Transformer block. The dilation rate of the dilated convolutions is set to 2 to enhance the ability to extract local texture features of towers, conductors and insulators.

[0017] The detection head employs a depthwise separable convolutional structure from a single-stage object detection algorithm and introduces an improved IoU loss function, the calculation formula of which is as follows:

[0018] Losstotal =Loss angle +Loss shape +IOU+1,

[0019] in,

[0020]

[0021] Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and C h σ is the vertical height difference between the center points of the predicted bounding box and the ground truth bounding box, and σ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. This serves as the angle normalization baseline, used to measure the directional deviation between the predicted bounding box and the ground truth bounding box; w h The ratio of the difference between the width w or height h of the predicted bounding box and the ground truth bounding box, where θ is the shape penalty coefficient that controls the sensitivity to shape differences;

[0022] A Dropout layer with a probability of 0.3 is added to the Multilayer Perceptron (MLP) module to prevent model overfitting and improve generalization ability.

[0023] The network training used a batch size of 16 and an initial learning rate of 0.001, and the input image size was uniformly adjusted to 224×224 pixels.

[0024] Optionally, in one implementation of the first aspect of the present invention, step S1, which involves fusing RTK positioning data with visual SLAM technology to construct a three-dimensional environment map, specifically includes:

[0025] (1) Multi-sensor synchronization module, which realizes μs-level time synchronization of RTK-GNSS, IMU and visual sensor through hardware timestamp;

[0026] (2) Enhanced visual SLAM subsystem, which integrates the ORB-SLAM3 framework and adopts the power equipment feature weighting strategy. When identifying towers or insulators, the density of regional feature points is increased by 300%, and the point cloud weight is doubled.

[0027] (3) Adaptive fusion positioning module: establishes RTK error ellipse model, prioritizes RTK data when the positioning accuracy is better than 2cm, smoothly switches to visual SLAM when the signal is blocked, and achieves sub-centimeter positioning through extended Kalman filter;

[0028] (4) Intelligent 3D reconstruction engine, using TSDF voxel mesh with a basic resolution of 5cm, performs 1cm high-precision local reconstruction of key parts of conductors and towers, and automatically performs global bundled adjustment and optimization every 100 meters of the route.

[0029] (5) Online quality assessment system, which monitors point cloud density and power equipment integrity in real time, automatically triggers the replenishment mechanism when the point cloud density in the tower area is less than 500 points / cubic meter.

[0030] Optionally, in one implementation of the first aspect of the present invention, the construction of the agent based on the PPO algorithm in step S2 includes the following steps:

[0031] S2.1 Constructing the state space: The state space of the intelligent agent includes the position coordinates of the UAV relative to the tower, the deviation value of the route direction angle, and the obstacle distance vector. The position of the UAV relative to the tower is represented by the three-dimensional coordinates (Δx, Δy, Δz) obtained by fusion of RTK positioning and visual SLAM. The route direction angle is calculated based on the direction of the conductor recognized by five consecutive frames of images, with a heading yaw angle θ ranging from [-π, π]. The obstacle distance is obtained by LiDAR and dynamically normalized to the interval [0, 1].

[0032] S2.2 Construct the motion space, which includes: flight speed adjustment, gimbal pitch / yaw angle adjustment, and camera zoom control parameters. Flight speed adjustment is a discrete output {-1, 0, +1}, corresponding to a 20% deceleration, a hold, and a 20% acceleration, respectively. Gimbal angle adjustment is a continuous output of pitch angle φ∈[-30°, +30°] and azimuth angle ψ∈[-45°, +45°]. Zoom control is a graded output of optical zoom {1×, 2×, 4×}.

[0033] S2.3 Construct a composite reward function and optimize the decision using the composite reward function. The reward function consists of the coverage reward R. cover Safety Rewards R safety and energy reward R energy The composition, specifically, is as follows:

[0034] R t =α·R cover +β·R safety +γ·R energy ,

[0035] Among them, the coverage reward R cover Positively correlated with the completeness of the target image, the safety reward R is calculated as the percentage of the visible area of ​​key tower components in the current frame. safety Negatively correlated with obstacle distance, short-range flight behavior is penalized according to an exponential decay function, with energy reward R. energy Negatively correlated with the change in velocity, α, β, and γ are adjustment coefficients;

[0036] S2.4 Network Architecture: A dual-Critic network structure is adopted, which includes a 3-layer MLP with 256 nodes. The input layer receives state features, and the output layer generates the action mean μ and variance σ respectively. The agent is updated collaboratively through the policy network and the value network. During the training phase, the experience replay mechanism is used to store the state-action-reward tuple, and the gradient is calculated based on the generalized advantage estimation (GAE) to optimize the policy.

[0037] Optionally, in one implementation of the first aspect of the present invention, in step S3, the path planning for the initial inspection employs an ε-greedy strategy in conjunction with the LSTM network for optimization, specifically including...

[0038] Next steps:

[0039] (1) An ε-greedy strategy is adopted for exploratory flight, and the combination of flight parameters is randomly adjusted within a safe threshold range. The flight parameters include latitude and longitude, flight altitude and gimbal angle.

[0040] (2) The LSTM network is used to extract features from the time-series data collected during the exploration flight, learn the spatial topology features of the route and the optimal shooting parameters, and generate a structured route file containing waypoint location information and shooting parameters.

[0041] (3) Based on the requirements of special patrol missions, the waypoint sequence in the route file is dynamically adjusted in combination with real-time environmental data to achieve online optimization of the patrol route;

[0042] The spatial topology features include the tower spacing and the route angle, and the optimal shooting parameters include the shooting height, gimbal tilt angle, and zoom ratio.

[0043] Optionally, in one implementation of the first aspect of the present invention, the training dataset of the LSTM network includes historical inspection paths, 3D models of towers, and manually annotated optimal shooting points; during dynamic waypoint reorganization, the reorganization strategy is evaluated online through Q-learning, prioritizing the retention of waypoints with coverage rewards higher than a threshold, and dynamically adjusting waypoint priorities based on real-time environmental factors; the generated route files can be exported to standard aerial survey formats, including KML and CSV formats, and interact with a geographic information system (GIS) platform to achieve multi-UAV collaborative path collision avoidance.

[0044] Optionally, in one implementation of the first aspect of the present invention, step S4, adapting to the new environment using transfer learning techniques and continuously optimizing the inspection strategy through online policy gradient updates, includes:

[0045] Transfer learning adaptation steps: Adapt the model to the environment based on the pre-trained deep reinforcement learning model, where: the parameters of the feature extraction layer of the model are kept unchanged; only the output layer parameters of the policy network are updated; and the domain adaptation loss function is used to reduce the difference in data distribution between the old and new environments.

[0046] Online strategy optimization steps: During the inspection process, perform the following operations: collect and store status-action-reward data in real time; periodically perform strategy gradient update calculations; and use importance sampling techniques to correct data biases.

[0047] Dynamic policy evaluation steps: Policy performance is evaluated in the following ways: Monte Carlo tree search is used to simulate policy effects; feasible policies are selected in combination with safety constraints; wherein, the pre-trained model adopts the PPO algorithm, the domain adaptation technique includes MMD loss function or adversarial training method, and the policy gradient algorithm adopts the Actor-Critic method.

[0048] The online update mechanism employs a policy gradient update formula:

[0049]

[0050] in, π represents the gradient of the policy objective function J(θ) with respect to the policy parameter angle θ, indicating the direction of adjustment of the parameter angle θ to maximize the long-term cumulative reward; θ (a|s) denotes the parameterized policy function, representing the probability of choosing action a in state s; Q represents the gradient of the logarithmic probability of the policy function with respect to the parameter angle θ; π (s,a) represents the state-action value function, indicating the expected cumulative reward Q after performing action a in state s, following policy π. π (s,a) is estimated using a Critic network and variance is reduced using GAE.

[0051] Secondly, embodiments of this application provide an autonomous flight path generation system for distribution network drones based on deep reinforcement learning, applied to the autonomous flight path generation method for distribution network drones based on deep reinforcement learning as described in the first aspect, the system comprising:

[0052] The environmental perception module is used to collect images of power distribution lines in real time using a multispectral camera mounted on a drone, identify key targets such as towers, conductors and insulators using an improved target detection algorithm, and build a three-dimensional environmental map by integrating RTK positioning data and visual SLAM technology.

[0053] The deep reinforcement learning decision module is used to construct an intelligent agent based on the PPO algorithm. Its state space includes the drone's position relative to the pole, the angle of the route and the distance to obstacles, and the action space includes flight speed adjustment, gimbal angle adjustment and zoom control. The decision is optimized through a composite reward function, which includes a coverage reward that is positively correlated with the completeness of the shooting target and a safety reward that is negatively correlated with the distance to obstacles.

[0054] The adaptive route generation module is used to explore the optimal flight path through the ε-greedy strategy during the first inspection. It generates route files containing latitude, longitude, altitude and shooting angle based on the memory of route features of the LSTM network, and supports dynamic reorganization of waypoints to adapt to special inspection needs.

[0055] The online optimization module is used to adapt to new environments using transfer learning techniques and continuously optimize inspection strategies through online policy gradient updates.

[0056] Thirdly, embodiments of this application provide an electronic device, including:

[0057] processor;

[0058] Memory used to store processor-executable instructions;

[0059] The processor is configured to implement the autonomous flight path generation method for distribution network UAVs based on deep reinforcement learning as described in the first aspect when executing the instructions.

[0060] Fourthly, embodiments of this application provide a computer-readable storage medium storing a program that instructs a device to execute the autonomous flight path generation method for distribution network unmanned aerial vehicles based on deep reinforcement learning as described in the first aspect.

[0061] This invention discloses a method and system for autonomous route generation by a UAV in power distribution networks based on deep reinforcement learning. The method includes: acquiring real-time images of power distribution lines using a multispectral camera mounted on the UAV; identifying towers, conductors, and insulators using an improved target detection algorithm incorporating a visual Transformer module; and constructing a 3D environment map by integrating RTK localization and visual SLAM technologies; constructing an agent based on the PPO algorithm, whose state space includes the UAV's relative position, line angle, and obstacle distance, and whose action space includes flight speed, gimbal angle, and zoom control; optimizing decisions through a composite reward function; exploring the optimal path using an ε-greedy strategy during the initial inspection; generating a route file containing position and shooting parameters based on an LSTM network, supporting dynamic reconfiguration to adapt to special inspection needs; and continuously optimizing the inspection strategy using transfer learning and online policy gradient updates. This invention achieves high-precision autonomous decision-making and dynamic path planning for UAV inspections, improving the efficiency and safety of power equipment inspections.

[0062] Beneficial effects:

[0063] 1. High-precision target detection and environment modeling. An improved target detection algorithm, combined with a visual Transformer module and an optimized IoU loss function, significantly improves the identification accuracy of towers, conductors, and insulators, reducing missed and false detections. By integrating RTK high-precision positioning and visual SLAM technology, a sub-centimeter-level 3D environment map is constructed, ensuring the reliability of UAV positioning and obstacle avoidance.

[0064] 2. Intelligent Dynamic Path Planning. Based on the PPO algorithm, the agent optimizes decisions through a composite reward function (coverage reward, safety reward, and energy consumption reward), achieving adaptive adjustments to flight speed, gimbal angle, and zoom magnification, thus improving inspection efficiency and data quality. Combining the ε-greedy strategy and LSTM network, the agent explores the optimal path and memorizes route characteristics during the initial inspection, supporting dynamic waypoint reorganization to adapt to different special inspection mission requirements.

[0065] 3. Strong environmental adaptability and transferability. Utilizing transfer learning techniques, it can quickly adapt to new environments, requiring only minor adjustments to the policy network output layer, reducing computational costs and improving deployment efficiency. An online policy gradient update mechanism continuously optimizes the inspection policy, and Monte Carlo tree search is used to evaluate policy performance, ensuring long-term stability and security.

[0066] 4. High Efficiency and Scalability. The generated flight path files support standard formats such as KML and CSV, seamlessly integrating with GIS platforms to enable multi-UAV collaborative operations and path collision avoidance. The system features real-time quality assessment and automatically triggers supplementary aircraft mechanisms to ensure the integrity and reliability of inspection data. This invention significantly improves the intelligence level and operational efficiency of power distribution network UAV inspections, reduces the need for manual intervention, and provides an efficient, safe, and adaptive solution for power equipment operation and maintenance. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of a method for generating autonomous flight paths for distribution network UAVs based on deep reinforcement learning, provided as an embodiment of this application.

[0068] Figure 2 This is a diagram of the target detection model architecture provided in an embodiment of this application.

[0069] Figure 3 This is a diagram of the backbone network structure provided in one embodiment of this application.

[0070] Figure 4 This is a diagram of the detection head architecture provided in one embodiment of this application.

[0071] Figure 5A system architecture diagram for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning is provided as an embodiment of this application.

[0072] Figure 6 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0074] It should be noted that, in the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application.

[0075] It should be noted that in the embodiments of this application, the terms "first," "second," etc., are used only for descriptive purposes and should not be construed as indicating or implying relative importance, nor as indicating or implying order. Features specified as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0076] Based on the embodiments described in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] Example 1

[0078] Figure 1 This is a schematic diagram of a method for generating autonomous flight paths for distribution network UAVs based on deep reinforcement learning, provided as an embodiment of this application.

[0079] like Figure 1 As shown, a method for autonomous flight path generation of distribution network drones based on deep reinforcement learning includes:

[0080] S1. Real-time images of power distribution lines are collected using a multispectral camera mounted on a drone. A model with an improved target detection algorithm is used to identify key targets such as towers, conductors, and insulators. RTK positioning data and visual SLAM technology are then integrated to construct a three-dimensional environmental map.

[0081] The drone, equipped with a multispectral camera, captures real-time images of power distribution lines, a prerequisite for intelligent inspection. Multispectral sensors can monitor indicators such as abnormal equipment temperature, gas leaks, and vegetation health, providing foundational data for subsequent image recognition and analysis. Simultaneously, the integration of RTK (Real-Time Kinematic) positioning data with visual SLAM (Simultaneous Localization and Mapping) technology to construct a 3D environmental map requires high-precision positioning and spatial modeling capabilities. RTK technology provides centimeter-level accuracy, while visual SLAM enables autonomous navigation and map building through visual information. This combined technology ensures stable drone flight in complex environments and accurate acquisition of 3D information about the target area.

[0082] Specifically, in this embodiment, in step S1, images of the power distribution network lines are acquired in real time using a multispectral camera mounted on a drone. This process obtains high-resolution image data, providing a foundation for subsequent identification and modeling. An improved target detection algorithm is used to process the acquired images, identifying key targets such as towers, conductors, and insulators. The improved target detection algorithm incorporates a visual Transformer module to enhance local texture feature extraction capabilities and employs an improved IoU loss function to optimize bounding box regression accuracy. The visual Transformer module effectively captures local texture features in the image, improving the accuracy of target detection. Improved IoU loss functions, such as CIoU (Complete-IoU) or DIoU (Distance-IoU), better measure the overlapping area, center point distance, and aspect ratio between two bounding boxes, thereby improving the accuracy of bounding box regression.

[0083] By integrating RTK (Real-Time Kinematic) positioning data with visual SLAM (Simultaneous Localization and Mapping) technology, a 3D environmental map of power distribution lines is constructed. RTK technology provides centimeter-level accuracy in positioning information, while visual SLAM technology enables autonomous navigation and map building through visual information. This combined technology ensures stable flight of the UAV in complex environments and accurate acquisition of 3D information of the target area. The identification results are then correlated with the 3D environmental map to generate inspection reports. By associating the identified key targets such as towers, conductors, and insulators with the 3D environmental map, detailed inspection reports can be generated, including information on the target's location, status, and potential defects, providing support for subsequent maintenance and management.

[0084] Figure 2This is a diagram of the target detection model architecture provided in an embodiment of this application. Figure 3 This is a diagram of the backbone network structure provided in one embodiment of this application. Figure 4 This is a diagram of a detection head architecture provided in one embodiment of this application. Figure 2 As shown, the improved target detection algorithm's model structure includes the following key parts:

[0085] Backbone network: such as Figure 3 As shown, the backbone network adopts a visual Transformer architecture that integrates spatial local perception modules. Specifically, a spatial local perception block consisting of 3×3 dilated convolutions and residual connections is inserted before the Transformer block, where the dilation rate of the dilated convolutions is set to 2, to enhance the extraction capability of local texture features of towers, conductors, and insulators.

[0086] like Figure 4 As shown, the detection head employs a depthwise separable convolutional structure based on a single-stage object detection algorithm, and introduces an improved IoU loss function. The formula for calculating this loss function is:

[0087] Loss total =Loss angle +Loss shape +IOU+1,

[0088] in,

[0089]

[0090] Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and C h σ is the vertical height difference between the center points of the predicted bounding box and the ground truth bounding box, and σ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. This serves as the angle normalization baseline, used to measure the directional deviation between the predicted bounding box and the ground truth bounding box; w h The ratio of the difference between the width w or height h of the predicted bounding box and the ground truth bounding box, where θ is the shape penalty coefficient that controls the sensitivity to shape differences.

[0091] Multilayer Perceptron (MLP) Module: A Dropout layer with a probability of 0.3 is added to the MLP module to prevent overfitting and improve generalization ability. Training Parameters: The network is trained with a batch size of 16 and an initial learning rate of 0.001. The input image size is uniformly adjusted to 224×224 pixels.

[0092] In step S1, a three-dimensional environment map is constructed by fusing RTK positioning data with visual SLAM technology, which specifically includes the following aspects:

[0093] Multi-sensor synchronization module: Achieves μs-level time synchronization between RTK-GNSS, IMU, and vision sensors through hardware timestamps. This synchronization mechanism ensures the temporal alignment of data from different sensors, thereby improving the accuracy of positioning and mapping.

[0094] Enhanced Visual SLAM Subsystem: Integrating the ORB-SLAM3 framework and employing a power equipment feature weighting strategy, it increases the density of regional feature point extraction by 300% when identifying towers or insulators, while assigning double the point cloud weight. ORB-SLAM3 is a highly efficient monocular SLAM system capable of providing real-time localization and mapping capabilities.

[0095] The adaptive fusion localization module establishes an RTK error ellipse model, prioritizing RTK data when the positioning accuracy is better than 2cm, smoothly switching to visual SLAM when the signal is obstructed, and achieving sub-centimeter-level positioning through extended Kalman filtering. This adaptive fusion strategy effectively addresses positioning requirements in different environments and improves the system's robustness.

[0096] Intelligent 3D Reconstruction Engine: Employing a TSDF voxel mesh with a base resolution of 5cm, it performs high-precision local reconstruction of key parts of conductors and towers at 1cm resolution, and automatically performs global bundled adjustment and optimization every 100 meters of the flight path. The TSDF (Tetrahedral Signed Distance Function) voxel mesh provides high-precision 3D reconstruction, suitable for detailed modeling of power equipment.

[0097] Specifically, a hierarchical reconstruction system can be constructed: first, a global model is performed using a TSDF voxel mesh with a base resolution of 5cm; then, high-precision reconstruction at 1cm is used for key areas (such as conductor connection points and tower support structures). By introducing adaptive resolution and semantic information, efficient and accurate local reconstruction can be achieved. This approach not only meets the requirements for high-precision reconstruction but also maintains good performance even with limited computational resources.

[0098] Specifically, the hierarchical architecture for multi-scale voxel management can adopt an octree index structure, with a base layer of a 5cm resolution global mesh, dynamically refined to 1cm high-precision sub-blocks for key component areas such as conductor joints and insulators. Dynamic switching conditions can be set based on target detection results (such as the improved YOLOv7 model) and point cloud density (>500 points / cm). 3 Automatic resolution enhancement is triggered. Parallel updates of multi-resolution voxels on the GPU are achieved through CUDA atomic operations. The core algorithms include: Basic TSDF update (5cm): global fast reconstruction, truncation distance τ = 10cm. High-precision TSDF enhancement (1cm): introducing curvature constraints, local geometric error ≤ 0.8mm.

[0099] A hierarchical architecture based on multi-scale voxel management is proposed, employing an octree index structure to manage voxel data at different resolutions. The base layer of this architecture is a 5cm resolution global mesh for rapid reconstruction of the overall scene, while critical component areas such as wire joints and insulators are dynamically refined into 1cm high-precision sub-blocks. This hybrid precision scheme effectively balances geometric accuracy with the use of computational resources.

[0100] Regarding dynamic switching conditions, the system can adjust based on target detection results (such as the improved YOLOv7 model) and point cloud density (>500 points / cm²). 3 Automatic resolution enhancement is triggered. This mechanism ensures higher geometric accuracy in critical areas while avoiding unnecessary computational overhead.

[0101] To achieve parallel updates on the GPU, this study employs CUDA atomic operations, enabling efficient updating and rendering of multi-resolution voxels. The core algorithms include basic TSDF (truncated signed distance function) updates (5cm) and high-precision TSDF enhancement (1cm). In the basic layer, a truncated distance τ = 10cm is used for rapid reconstruction of the overall scene; while in the high-precision layer, curvature constraints are introduced to ensure that local geometric errors do not exceed 0.8mm.

[0102] Compared with traditional fixed-resolution reconstruction methods (such as KinectFusion), the hybrid precision scheme proposed in this study significantly reduces geometric errors from 3.1 cm to 0.8 cm in key areas such as wire connectors, while reducing memory consumption by 42%. This improvement is due to the adaptive properties of the octree structure, which can store only non-empty voxels, thereby greatly reducing memory usage.

[0103] By combining octree indexing, dynamic resolution switching, and CUDA parallel computing, efficient and high-precision 3D reconstruction was achieved, providing a feasible technical solution for real-time applications.

[0104] This invention proposes a hybrid precision scheme combining a 5cm base resolution with 1cm local reconstruction of critical areas, achieving real-time resolution switching through octree indexing and CUDA atomic operations. Compared to traditional fixed-resolution reconstruction (such as KinectFusion), this reduces geometric errors in critical areas such as wire connectors from 3.1cm to 0.8cm, while simultaneously reducing memory consumption by 42%.

[0105] The online quality assessment system monitors point cloud density and power equipment integrity in real time. When the point cloud density in the tower area falls below 500 points / cubic meter, a supplementary monitoring mechanism is automatically triggered. This mechanism ensures the high quality and integrity of the 3D environmental map, improving the reliability of inspections.

[0106] The combination of these technologies makes the construction of 3D environment maps more efficient and accurate, providing a solid foundation for subsequent inspection tasks.

[0107] S2. Construct an intelligent agent based on the PPO algorithm. Its state space includes the drone's position relative to the pole, the angle of the route, and the distance to obstacles. Its action space includes flight speed adjustment, gimbal angle adjustment, and zoom control. Decisions are optimized through a composite reward function, which includes a coverage reward positively correlated with the completeness of the shooting target and a safety reward negatively correlated with the distance to obstacles.

[0108] An agent based on the PPO (Proximal Policy Optimization) algorithm was constructed. Its state space includes the drone's position relative to the pole, the flight path angle, and obstacle distances. Its action space includes flight speed adjustment, gimbal angle adjustment, and zoom control. PPO is a policy gradient algorithm that, by limiting the magnitude of policy updates, allows for smooth optimization across multiple actions, thus avoiding convergence to local optima. The design of the reward function is crucial; it includes a coverage reward positively correlated with the completeness of the captured target and a safety reward negatively correlated with obstacle distances. This composite reward function guides the agent to maximize target coverage while ensuring safety.

[0109] In step S2, the construction of an agent based on the PPO algorithm specifically includes the following aspects:

[0110] State Space Construction: The state space of the agent includes the UAV's position coordinates relative to the tower, the deviation value of the route angle, and the obstacle distance vector. The UAV's position relative to the tower is represented by three-dimensional coordinates (Δx, Δy, Δz) obtained by fusion of RTK positioning and visual SLAM; the route angle is calculated based on the yaw angle θ of the conductor recognized from 5 consecutive frames of images, ranging from [-π, π]; the obstacle distance is obtained by LiDAR and dynamically normalized to the interval [0, 1].

[0111] The motion space is constructed as follows: The motion space includes flight speed adjustment, gimbal pitch / yaw angle adjustment, and camera zoom control parameters. Flight speed adjustment is a discrete output {-1, 0, +1}, corresponding to a 20% deceleration, a hold, and a 20% acceleration, respectively; gimbal angle adjustment is a continuous output of pitch angle φ∈[-30°, +30°] and azimuth angle ψ∈[-45°, +45°]; zoom control is a graded output {1×, 2×, 4×} optical zoom.

[0112] Constructing a composite reward function: Decision-making is optimized through a composite reward function, which consists of coverage reward, safety reward, and energy consumption reward. Coverage reward is positively correlated with the completeness of target image capture and is calculated as the proportion of the visible area of ​​key components of the tower in the current frame; safety reward is negatively correlated with obstacle distance and penalizes close-range flight behavior according to an exponential decay function; energy consumption reward is negatively correlated with the change in speed.

[0113] Specifically, a composite reward function is constructed, and the decision is optimized through the composite reward function. The reward function consists of a coverage reward R. cover Safety Rewards R safety and energy reward R energy The composition, specifically, is as follows:

[0114] R t =α·R cover +β·R safety +γ·R energy ,

[0115] Among them, the coverage reward R cover Positively correlated with the completeness of the target image, the safety reward R is calculated as the percentage of the visible area of ​​key tower components in the current frame. safety Negatively correlated with obstacle distance, short-range flight behavior is penalized according to an exponential decay function, with energy reward R. energy It is negatively correlated with the change in velocity, and α, β, and γ are adjustment coefficients.

[0116] Network Architecture: A dual-Critic network structure is adopted, consisting of a 3-layer, 256-node MLP. The input layer receives state features, and the output layer generates the action mean μ and variance σ, respectively. The agent is updated collaboratively through the policy network and the value network. During the training phase, an experience replay mechanism is used to store state-action-reward tuples, and gradients are calculated based on generalized advantage estimation (GAE) to optimize the policy.

[0117] S3. During the first inspection, the optimal flight path is explored through the ε-greedy strategy. Based on the LSTM network, the route file containing latitude, longitude, altitude and shooting angle is generated by memorizing the route features. It also supports dynamic reorganization of waypoints to adapt to special inspection needs.

[0118] In S3, the optimal flight path is explored during the initial inspection using an ε-greedy strategy. A route file containing latitude, longitude, altitude, and shooting angle is generated based on the route features memorized by an LSTM network, and dynamic waypoint reconfiguration is supported to adapt to special inspection needs. LSTM (Long Short-Term Memory) is a deep learning model suitable for processing sequential data, capable of capturing the spatiotemporal characteristics of routes. The ε-greedy strategy balances exploration and utilization, ensuring the agent explores more possible paths initially, ultimately converging to the optimal strategy. The ability to dynamically reconfigure waypoints allows the system to adjust the flight path according to real-time conditions, improving the flexibility and adaptability of the inspection.

[0119] In step S3, the path planning for the initial inspection employs an ε-greedy strategy in conjunction with the LSTM network for optimization, specifically including the following steps:

[0120] An ε-greedy strategy is employed for exploratory flight: within a safe threshold range, combinations of flight parameters, including latitude and longitude, flight altitude, and gimbal angle, are randomly adjusted. The ε-greedy strategy balances exploration and exploitation, ensuring the agent explores more possible paths initially and ultimately converges to the optimal strategy.

[0121] The LSTM network is used to extract features from the time-series data collected during the exploration flight: the LSTM network can capture the spatiotemporal features of the route, learn the spatial topology features of the route (such as tower spacing and route direction angle) and the optimal shooting parameters (such as shooting height, gimbal pitch angle and zoom), and generate a structured route file containing waypoint location information and shooting parameters.

[0122] Based on the requirements of special patrol missions, the waypoint sequence in the route file is dynamically adjusted in conjunction with real-time environmental data: by adjusting the waypoint sequence online, the inspection path is optimized online to meet the needs of special patrol missions.

[0123] In step S3, the training dataset for the LSTM network includes historical inspection routes, 3D models of towers, and manually annotated optimal shooting locations. This data provides the LSTM network with rich spatiotemporal features, enabling it to learn the spatial topology features of the route and optimal shooting parameters, thereby generating a structured route file containing waypoint location information and shooting parameters.

[0124] During dynamic waypoint reorganization, Q-learning is used to evaluate the reorganization strategy online, prioritizing waypoints with coverage rewards exceeding a threshold and dynamically adjusting waypoint priorities based on real-time environmental factors. This mechanism ensures the flexibility and adaptability of inspection routes, enabling optimization according to actual needs.

[0125] The generated flight path files can be exported to standard aerial survey formats, including KML and CSV, and can interact with Geographic Information System (GIS) platforms to achieve multi-UAV collaborative path collision avoidance. This feature enables efficient management and execution of inspection tasks, especially in complex environments.

[0126] S4. Adapt the inspection strategy to the new environment using transfer learning techniques and continuously optimize the inspection strategy through online policy gradient updates.

[0127] In S4, transfer learning is used to adapt to new environments, and the inspection strategy is continuously optimized through online policy gradient updates. Transfer learning enables the rapid application of a model trained in one environment to a new one, reducing training time and computational costs. Online policy gradient updates allow the agent to continuously adjust its strategy during actual operation to adapt to environmental changes and task requirements. This mechanism improves the system's robustness and adaptability, ensuring high inspection efficiency and accuracy across different scenarios.

[0128] In S4, transfer learning techniques are used to adapt to the new environment, and the inspection strategy is continuously optimized through online policy gradient updates. Specifically, this includes the following aspects:

[0129] The transfer learning adaptation process involves adapting the pre-trained deep reinforcement learning model to the new environment, while preserving the parameters of the model's feature extraction layer; updating only the output layer parameters of the policy network; and employing a domain adaptation loss function to reduce the differences in data distribution between the old and new environments. This transfer learning method effectively utilizes existing knowledge and reduces the time and computational resources required for retraining in the new environment.

[0130] Online strategy optimization steps: During the inspection process, the following operations are performed: real-time collection and storage of status-action-reward data; periodic calculation of strategy gradient updates; and correction of data bias using importance sampling techniques. These steps ensure that the strategy can be adjusted according to the latest environmental information, thereby improving the adaptability and accuracy of the inspection.

[0131] Dynamic policy evaluation steps: Policy performance is evaluated through the following methods: Monte Carlo tree search is used to simulate policy effects; feasible policies are selected based on safety constraints. The pre-trained model employs the PPO algorithm, the domain adaptation techniques include the MMD loss function or adversarial training methods, and the policy gradient algorithm uses the Actor-Critic method. The combination of these techniques enables the policy to achieve efficient inspection tasks while ensuring safety.

[0132] Online update mechanism: Employs a policy gradient update formula:

[0133]

[0134] in, π represents the gradient of the policy objective function J(θ) with respect to the policy parameter angle θ, indicating the direction of adjustment of the parameter angle θ to maximize the long-term cumulative reward; θ (a|s) denotes the parameterized policy function, representing the probability of choosing action a in state s; Q represents the gradient of the logarithmic probability of the policy function with respect to the parameter angle θ; π (s,a) represents the state-action value function, indicating the expected cumulative reward Q after performing action a in state s, following policy π. π (s,a) is estimated using a Critic network, and variance is reduced using GAE. This formula ensures that the strategy can be optimized based on the latest environmental information, thereby improving the efficiency and accuracy of inspections.

[0135] The inspection processes from S1 to S4 combine multispectral sensing, 3D modeling, reinforcement learning algorithms, deep learning models, and online learning strategies to form a complete intelligent inspection system. This system not only efficiently completes inspection tasks but also continuously improves inspection quality and efficiency through continuous learning and optimization. The application of these technologies provides strong support for automated inspections in industries such as power grids and coal mines.

[0136] This invention discloses an autonomous flight path generation method and system for power distribution network drones based on deep reinforcement learning. It constructs a high-precision 3D map by fusing an improved target detection algorithm with RTK-SLAM, achieves dynamic flight path planning by combining the PPO algorithm with an LSTM network, and utilizes transfer learning to adapt to new environments. This invention significantly improves inspection efficiency, safety, and adaptability, and is suitable for complex power distribution network scenarios.

[0137] Example 2

[0138] like Figure 5 As shown, this application provides an architecture diagram of an autonomous flight path generation system for distribution network UAVs based on deep reinforcement learning, which is applied to the autonomous flight path generation system for distribution network UAVs based on deep reinforcement learning as described in Embodiment 1. It includes an environment perception module 11, a deep reinforcement learning decision module 12, an adaptive flight path generation module 13, and an online optimization module 14.

[0139] The environmental perception module 11 is used to collect images of power distribution lines in real time through a multispectral camera mounted on a drone, identify key targets such as towers, conductors and insulators using an improved target detection algorithm, and build a three-dimensional environmental map by integrating RTK positioning data and visual SLAM technology.

[0140] The deep reinforcement learning decision module 12 is used to construct an intelligent agent based on the PPO algorithm. Its state space includes the position of the drone relative to the tower, the angle of the route and the distance to the obstacle. Its action space includes flight speed adjustment, gimbal angle adjustment and zoom control. The decision is optimized through a composite reward function, which includes a coverage reward that is positively correlated with the completeness of the shooting target and a safety reward that is negatively correlated with the distance to the obstacle.

[0141] The adaptive route generation module 13 is used to explore the optimal flight path through the ε-greedy strategy during the first inspection. It generates route files containing latitude, longitude, altitude and shooting angle based on the memory route features of the LSTM network, and supports dynamic reorganization of waypoints to adapt to special inspection needs.

[0142] The online optimization module 14 is used to adapt to the new environment using transfer learning technology and continuously optimize the inspection strategy through online policy gradient updates.

[0143] Figure 6 This is an electronic device provided in one embodiment of this application. For example... Figure 6 As shown, the electronic device includes at least the following components: processor 101 and memory 100, communication interface 103, and bus 102.

[0144] In this embodiment of the application, memory 100 is used to store executable instructions of processor 101, which, when configured to execute instructions, implements the method as described in the first aspect.

[0145] In embodiments of this application, a computer-readable storage medium includes instructions that instruct a device to perform the method as described in the first aspect. For example, the instructions instruct the device to perform... Figure 1 The method is shown in the process steps.

[0146] In one embodiment of this application, the program operating in the electronic device may be a program that controls a central processing unit (CPU) or similar device to achieve the functions of the above-described embodiments of the present invention (a program that enables the computer to function). Information processed by these devices is then temporarily stored in random access memory (RAM) during processing, and subsequently stored in various ROMs such as read-only memory (FlashROM) and hard disk drives (HDDs), and read, corrected, and written by the CPU as needed.

[0147] It should be noted that a portion of the electronic device described above can also be implemented using a computer. In this case, the program for implementing the control function can be recorded on a computer-readable recording medium, and the program recorded on the recording medium can be read into the computer and executed.

[0148] It should be noted that the term "computer" as used here refers to a computer built into an electronic device, employing hardware including an operating system and peripheral devices. Furthermore, "computer-readable recording media" refers to removable media such as floppy disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard drives built into a computer.

[0149] Furthermore, a "computer-readable recording medium" can include: a medium that dynamically stores a program for a short period of time, such as a communication line used when transmitting a program via a network such as the Internet or a communication line such as a telephone line; or a medium that stores a program for a fixed period of time, such as volatile memory inside a computer that serves as a server or client in this case. In addition, the aforementioned program can be a program used to implement the above-mentioned functions, or it can be a program that can implement the above-mentioned functions by combining with programs already recorded in the computer.

[0150] Furthermore, the electronic device in the above embodiments can also be implemented as an assembly (device group) composed of multiple devices. Each device constituting the device group can possess some or all of the functions or functional blocks of the electronic device in the above embodiments. As a device group, it is sufficient to have all the functions or functional blocks of the electronic device.

[0151] Those skilled in the art should recognize that the above embodiments are only used to illustrate this application and are not intended to limit this application. Any appropriate changes and variations made to the above embodiments within the essential spirit and scope of this application fall within the scope of protection claimed in this application.

Claims

1. A method for autonomous flight path generation of distribution network unmanned aerial vehicles (UAVs) based on deep reinforcement learning, characterized in that, The method includes: S1. Real-time images of power distribution lines are collected using a multispectral camera mounted on a drone. The improved target detection algorithm is used to identify key targets such as towers, conductors and insulators. RTK positioning data and visual SLAM technology are then integrated to construct a three-dimensional environment map. S2. Construct an intelligent agent based on the PPO algorithm. Its state space includes the drone's position relative to the pole, the angle of the route, and the distance to obstacles. Its action space includes flight speed adjustment, gimbal angle adjustment, and zoom control. Decisions are optimized through a composite reward function, which includes a coverage reward that is positively correlated with the completeness of the shooting target and a safety reward that is negatively correlated with the distance to obstacles. S3. During the first inspection, the optimal flight path is explored through the ε-greedy strategy. Based on the LSTM network, the route file containing latitude, longitude, altitude and shooting angle is generated by memorizing the route features. It also supports dynamic reorganization of waypoints to adapt to special inspection needs. S4. Adapt the inspection strategy to the new environment using transfer learning techniques and continuously optimize the inspection strategy through online policy gradient updates.

2. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 1, characterized in that, In step S1, images of the power distribution network lines are acquired in real time using a multispectral camera mounted on a drone. An improved target detection algorithm is then used to identify key targets such as towers, conductors, and insulators. Specifically, this includes: Real-time images of power distribution lines are collected using a multispectral camera mounted on a drone. An improved target detection algorithm model is used to process the acquired images and identify key targets such as towers, conductors and insulators. The improved target detection algorithm model integrates a visual Transformer module to enhance the ability to extract local texture features and uses an improved IoU loss function to optimize the bounding box regression accuracy. By integrating RTK positioning data with visual SLAM technology, a three-dimensional environmental map of the power distribution network is constructed; the recognition results are then correlated with the three-dimensional environmental map to generate an inspection report.

3. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 2, characterized in that, The improved target detection algorithm's model structure includes: The backbone network adopts a visual Transformer architecture that integrates spatial local perception modules. Specifically, a spatial local perception block consisting of 3×3 dilated convolutions and residual connections is inserted before the Transformer block. The dilation rate of the dilated convolutions is set to 2 to enhance the ability to extract local texture features of towers, conductors and insulators. The detection head employs a depthwise separable convolutional structure from a single-stage object detection algorithm and introduces an improved IoU loss function, the calculation formula of which is as follows: Loss total =Loss angle +Loss shape +IOU+1, in, Where IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and C h σ is the vertical height difference between the center points of the predicted bounding box and the ground truth bounding box, and σ is the Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box. This serves as the angle normalization baseline, used to measure the directional deviation between the predicted bounding box and the ground truth bounding box; w h The ratio of the difference between the width w or height h of the predicted bounding box and the ground truth bounding box, where θ is the shape penalty coefficient that controls the sensitivity to shape differences; A Dropout layer with a probability of 0.3 is added to the Multilayer Perceptron (MLP) module to prevent model overfitting and improve generalization ability. The network training used a batch size of 16 and an initial learning rate of 0.001, and the input image size was uniformly adjusted to 224×224 pixels.

4. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 2, characterized in that, In step S1, the 3D environment map is constructed by fusing RTK positioning data with visual SLAM technology, specifically including: The multi-sensor synchronization module achieves μs-level time synchronization between RTK-GNSS, IMU, and visual sensors through hardware timestamps; The enhanced visual SLAM subsystem integrates the ORB-SLAM3 framework and adopts a power equipment feature weighting strategy. When identifying towers or insulators, it increases the density of regional feature point extraction by 300% and assigns 2 times the point cloud weight. The adaptive fusion positioning module establishes an RTK error ellipse model, prioritizes RTK data when the positioning accuracy is better than 2cm, smoothly switches to visual SLAM when the signal is blocked, and achieves sub-centimeter positioning through extended Kalman filtering. The intelligent 3D reconstruction engine uses a TSDF voxel mesh with a base resolution of 5cm to perform high-precision local reconstruction of key parts of conductors and towers with a resolution of 1cm. It automatically performs global bundled adjustment and optimization every 100 meters of the flight path. The online quality assessment system monitors point cloud density and power equipment integrity in real time. When the point cloud density in the tower area is less than 500 points / cubic meter, the system automatically triggers the replenishment mechanism.

5. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 2, characterized in that, The construction of the agent based on the PPO algorithm in step S2 includes the following steps: S2.1 Constructing the state space: The state space of the intelligent agent includes the position coordinates of the UAV relative to the tower, the deviation value of the route direction angle, and the obstacle distance vector. The position of the UAV relative to the tower is represented by the three-dimensional coordinates (Δx, Δy, Δz) obtained by fusion of RTK positioning and visual SLAM. The route direction angle is calculated based on the direction of the conductor recognized by five consecutive frames of images, with a heading yaw angle θ ranging from [-π, π]. The obstacle distance is obtained by LiDAR and dynamically normalized to the interval [0, 1]. S2.2 Construct the motion space, which includes: flight speed adjustment, gimbal pitch / yaw angle adjustment, and camera zoom control parameters. Flight speed adjustment is a discrete output {-1, 0, +1}, corresponding to a 20% deceleration, a hold, and a 20% acceleration, respectively. Gimbal angle adjustment is a continuous output of pitch angle φ∈[-30°, +30°] and azimuth angle ψ∈[-45°, +45°]. Zoom control is a graded output of optical zoom {1×, 2×, 4×}. S2.3 Construct a composite reward function and optimize the decision using the composite reward function. The reward function consists of the coverage reward R. cover Safety Rewards R safety and energy reward R energy The composition, specifically, is as follows: R t =α·R cover +β·R safety +γ·R energy , Among them, the coverage reward R cover Positively correlated with the completeness of the target image, the safety reward R is calculated as the percentage of the visible area of ​​key tower components in the current frame. safety Negatively correlated with obstacle distance, short-range flight behavior is penalized according to an exponential decay function, with energy reward R. energy Negatively correlated with the change in velocity, α, β, and γ are adjustment coefficients; S2.4 Network Architecture: A dual-Critic network structure is adopted, which includes a 3-layer MLP with 256 nodes. The input layer receives state features, and the output layer generates the action mean μ and variance σ respectively. The agent is updated collaboratively through the policy network and the value network. During the training phase, the experience replay mechanism is used to store the state-action-reward tuple, and the gradient is calculated based on the generalized advantage estimation (GAE) to optimize the policy.

6. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 5, characterized in that, In step S3, the path planning for the initial inspection adopts an ε-greedy strategy in conjunction with the LSTM network for optimization, specifically including the following steps: S3.

1. An ε-greedy strategy is adopted for exploratory flight, and the combination of flight parameters is randomly adjusted within a safe threshold range. The flight parameters include latitude and longitude, flight altitude and gimbal angle. S3.

2. Use an LSTM network to extract features from the time-series data collected during the exploration flight, learn the spatial topology features of the route and the optimal shooting parameters, and generate a structured route file containing waypoint location information and shooting parameters. S3.

3. Based on the requirements of special patrol missions, dynamically adjust the waypoint sequence in the route file in conjunction with real-time environmental data to achieve online optimization of the inspection path; The spatial topology features include the tower spacing and the route angle, and the optimal shooting parameters include the shooting height, gimbal tilt angle, and zoom ratio.

7. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 6, characterized in that, in, The training dataset of the LSTM network includes historical inspection paths, 3D models of towers, and manually annotated optimal shooting points. During dynamic waypoint reorganization, the reorganization strategy is evaluated online through Q-learning, prioritizing waypoints with coverage rewards higher than a threshold, and dynamically adjusting waypoint priorities based on real-time environmental factors. The generated route files can be exported to standard aerial survey formats, including KML and CSV formats, and can interact with the Geographic Information System (GIS) platform to achieve multi-UAV collaborative path collision avoidance.

8. The method for autonomous flight path generation of distribution network UAVs based on deep reinforcement learning according to claim 5, characterized in that, S4, which utilizes transfer learning techniques to adapt to the new environment and continuously optimizes the inspection strategy through online policy gradient updates, includes: Transfer learning adaptation steps: Adapt the model to the environment based on the pre-trained deep reinforcement learning model, where: the parameters of the feature extraction layer of the model are kept unchanged; only the output layer parameters of the policy network are updated; and the domain adaptation loss function is used to reduce the difference in data distribution between the old and new environments. Online strategy optimization steps: During the inspection process, perform the following operations: collect and store status-action-reward data in real time; periodically perform strategy gradient update calculations; and use importance sampling techniques to correct data biases. Dynamic policy evaluation steps: Policy performance is evaluated in the following ways: Monte Carlo tree search is used to simulate policy effects; feasible policies are selected in combination with safety constraints; wherein, the pre-trained model adopts the PPO algorithm, the domain adaptation technique includes MMD loss function or adversarial training method, and the policy gradient algorithm adopts the Actor-Critic method. The online update mechanism employs a policy gradient update formula: in, π represents the gradient of the policy objective function J(θ) with respect to the policy parameter angle θ, indicating the direction of adjustment of the parameter angle θ to maximize the long-term cumulative reward; θ (a|s) denotes the parameterized policy function, representing the probability of choosing action a in state s; Q represents the gradient of the logarithmic probability of the policy function with respect to the parameter angle θ; π (s,a) represents the state-action value function, indicating the expected cumulative reward Q after performing action a in state s, following policy π. π (s,a) is estimated using a Critic network and variance is reduced using GAE.

9. A deep reinforcement learning-based autonomous flight path generation system for power distribution network unmanned aerial vehicles (UAVs), applied to the deep reinforcement learning-based autonomous flight path generation method for power distribution network UAVs as described in any one of claims 1 to 8, characterized in that, The system includes: The environmental perception module is used to collect images of power distribution lines in real time using a multispectral camera mounted on a drone, identify key targets such as towers, conductors and insulators using an improved target detection algorithm, and build a three-dimensional environmental map by integrating RTK positioning data and visual SLAM technology. The deep reinforcement learning decision module is used to construct an intelligent agent based on the PPO algorithm. Its state space includes the drone's position relative to the pole, the angle of the route and the distance to obstacles, and the action space includes flight speed adjustment, gimbal angle adjustment and zoom control. The decision is optimized through a composite reward function, which includes a coverage reward that is positively correlated with the completeness of the shooting target and a safety reward that is negatively correlated with the distance to obstacles. The adaptive route generation module is used to explore the optimal flight path through the ε-greedy strategy during the first inspection. It generates route files containing latitude, longitude, altitude and shooting angle based on the memory of route features of the LSTM network, and supports dynamic reorganization of waypoints to adapt to special inspection needs. The online optimization module is used to adapt to new environments using transfer learning techniques and continuously optimize inspection strategies through online policy gradient updates.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that instructs the device to execute the method for generating autonomous flight paths of a power distribution UAV based on deep reinforcement learning as described in any one of claims 1 to 8.

Images