Path planning method, system and equipment for inspection unmanned aerial vehicle, medium and product

Abstract

The invention belongs to the technical field of unmanned aerial vehicle control, and provides a routing inspection unmanned aerial vehicle path planning method, system and device, a medium and a product, and the method comprises the steps: generating a fusion feature map through multi-modal data; generating a sparse semantic map based on the fused feature map; generating a global optimal path covering all towers to be detected in the sparse semantic map in combination with power tower topology priori knowledge; in the flight process along the global optimal path, real-time obstacle avoidance is carried out; and in the inspection process, a revisit instruction is generated, and secondary inspection path planning of the suspicious tower is triggered. Through a series of algorithm innovative designs, the tower and the obstacle are accurately identified, the map construction calculation overhead is reduced, the path smoothness and the obstacle avoidance success rate are improved, and the defect detection rate and the timeliness are improved.

Description

Technical Field

[0001] This disclosure belongs to the field of unmanned aerial vehicle (UAV) control technology, and in particular relates to a path planning method for inspection UAVs. Background Technology

[0002] With the continuous expansion of the power system and the increasing demand for intelligent operation and maintenance, transmission line inspection has become a key link in ensuring the safe and stable operation of the power grid.

[0003] Intelligent inspection drones for power transmission line inspection have developed rapidly, but the following shortcomings still exist: (1) Although visible light cameras can provide high-resolution texture information, their recognition robustness is significantly reduced under low light or backlight conditions; (2) Depth sensors can acquire distance information, but their effective range is limited and they are easily affected by reflections from metal structures; (3) Traditional 3D reconstruction methods rely on dense point cloud matching or SLAM frameworks, which have high computational complexity and are difficult to deploy on airborne embedded platforms; (4) At the path planning level, the standard algorithm does not incorporate prior knowledge of the topology of power poles, resulting in a large search space and poor smoothness of the generated path; (5) When the dynamic window method is used to handle continuous obstacle avoidance in high-speed flight, it often gets stuck in a local optimum due to the excessively coarse velocity sampling granularity or the unreasonable design of the cost function. (6) Defect identification is generally placed in the post-processing stage, and it is impossible to dynamically adjust the shooting angle or revisit the suspicious area based on the preliminary analysis results during flight. This results in problems such as decoupling of the perception-planning-analysis modules, imbalance in the allocation of computing resources, and insufficient algorithm adaptability. Summary of the Invention

[0004] To address the aforementioned issues, this disclosure provides a path planning method for inspection drones. Through a series of innovative algorithm designs, it achieves accurate identification of poles and obstacles, reduces map building computational overhead, improves path smoothness and obstacle avoidance success rate, and enhances defect detection rate and timeliness.

[0005] Firstly, this disclosure provides a path planning method for inspection drones, including: Using multimodal data, a fused feature map is generated; based on the fused feature map, a sparse semantic map is generated; in the sparse semantic map, combined with prior knowledge of the power pole topology, a globally optimal path covering all poles to be inspected is generated; during flight along the globally optimal path, real-time obstacle avoidance is performed; during the inspection process, when the confidence level of the online defect identification of the pole exceeds a preset value, a revisit command is generated and a secondary inspection path planning for the suspicious pole is triggered.

[0006] Furthermore, Using multimodal data, a fused feature map is generated, including: acquiring multimodal data containing visible light images and depth point cloud data; preprocessing the visible light images and depth point cloud data, and performing spatiotemporal registration on the preprocessed data; and generating a fused feature map based on the spatiotemporally registered data.

[0007] Furthermore, Based on the fused feature map, a sparse semantic map is generated, including: semantic feature extraction and matching of the fused feature map; using an improved ORB-SLAM3 framework, the front-end visual odometry calculates the camera pose through the fused feature map, and the back-end uses a combination of co-view optimization and pose map optimization; key points of poles, wires and obstacles are inserted into the optimization map as semantic feature markers to generate a sparse semantic map.

[0008] Furthermore, The topological connections of power poles are incorporated into the sparse semantic map.

[0009] Furthermore, During the flight of the UAV, an improved dynamic window method is used for local trajectory optimization and real-time obstacle avoidance based on real-time perception data. The evaluation function formula of the improved dynamic window method is as follows: .

[0010] in: This represents the final evaluation function value; Indicates linear velocity; Indicates angular velocity; This represents an activation function or normalization function that maps the result of a weighted sum to a fixed range; , , , , These are the normalized weighting coefficients; This is the azimuth evaluation item; For speed evaluation items; For distance evaluation items; Scoring is given for trajectory tracking.

[0011] Furthermore, Generate a revisit command and trigger secondary inspection path planning for the suspected tower, including: when the confidence level of the online defect identification of the tower exceeds the preset value, generate a revisit command; the revisit command inserts a secondary inspection waypoint on the original inspection flight path, and the location of the secondary inspection waypoint is dynamically adjusted according to the tower defect type.

[0012] Secondly, this disclosure provides a path planning system for an inspection drone, including a multimodal perception fusion module, a lightweight 3D environment modeling module, a global path collaborative planning module, a local dynamic obstacle avoidance module, and a real-time defect identification and feedback module; wherein... The multimodal perception fusion module uses multimodal data to generate a fused feature map; A lightweight 3D environment modeling module for generating sparse semantic maps based on fused feature maps; The global path collaborative planning module is used to generate a globally optimal path covering all the poles to be inspected in the sparse semantic map by combining prior knowledge of the power pole topology. The local dynamic obstacle avoidance module is used to perform real-time obstacle avoidance while flying along the globally optimal path; The real-time defect identification and feedback module is used to generate revisit instructions and trigger secondary inspection path planning for suspicious towers during the inspection process.

[0013] Thirdly, based on the same inventive concept, this disclosure also provides an electronic device, including at least one processor and at least one storage medium electrically connected; The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by at least one of the processors, the instructions being executed by at least one of the processors to enable at least one of the processors to perform the inspection drone path planning method as described above.

[0014] Fourthly, based on the same inventive concept, this disclosure also provides a computer storage medium storing a computer program. When the computer program is executed by the processor, it implements the inspection drone path planning method as described above.

[0015] Fifthly, based on the same inventive concept, this disclosure also provides a computer program product, which is stored in at least one storage medium; The computer program product includes several instructions to cause at least one electronic device to execute the inspection drone path planning method as described above.

[0016] Compared with the prior art, this disclosure provides a path planning method for inspection drones, which has the following beneficial effects: 1. Through multimodal perception fusion and lightweight 3D environment modeling, accurate identification of poles and obstacles is achieved under strong electromagnetic interference and complex lighting conditions, while reducing map building computation overhead by more than 85%, meeting the real-time requirements of airborne embedded platforms; 2. By introducing tower topology priors and safe flight corridor constraints through global path planning, path redundancy is reduced by 60%, path smoothness is improved by 40%, and potential collision risks are effectively avoided. 3. By combining a local dynamic obstacle avoidance mechanism with instantaneous collision risk assessment, the obstacle avoidance success rate in dense tower areas reaches 99.5%, and the path oscillation phenomenon is basically eliminated; 4. By using a real-time defect identification and closed-loop feedback mechanism, the defect detection rate is increased to 98.2%, and the report generation time is shortened from hours to minutes, realizing an end-to-end intelligent closed loop for inspection tasks.

[0017] Other features and advantages of this disclosure will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a path planning method for an inspection drone according to an embodiment of the present disclosure is shown. Figure 2 A schematic diagram illustrating the structural principle of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0021] Figure 1 A flowchart illustrating a path planning method for an inspection drone according to an embodiment of the present disclosure is shown.

[0022] A path planning method for an inspection drone according to an embodiment of this disclosure includes the following steps: S1 uses multimodal data to generate a fused feature map.

[0023] Scale-invariant feature transformation (SIFT) feature points are extracted from visible light images, voxel grid downsampling is performed on depth point clouds, and spatiotemporal alignment of visual features and point cloud features is achieved through extended Kalman filters. Finally, a fused feature map is generated based on a weighted fusion strategy.

[0024] S11, acquire multimodal data including visible light images and depth point cloud data.

[0025] In the scenario of high-voltage transmission line inspection, the embodiments of this disclosure use a visible light camera and a depth sensor to simultaneously acquire visible light images and depth point cloud data of the inspection area.

[0026] The visible light camera employs a global shutter CMOS sensor with a resolution of 3840 x 2160 pixels and a stable frame rate of 30 frames per second, ensuring the capture of texture details of the tower insulator strings even under strong lighting conditions. In the preferred embodiment, the visible light camera is equipped with a polarizing filter to effectively suppress reflective interference.

[0027] The depth sensor employs a solid-state lidar, with a detection range covering 0.5 meters to 200 meters and an angular resolution accurate to 0.1 degrees, enabling accurate acquisition of the relative positional relationship between the conductor and the tower. In the preferred embodiment, the depth sensor utilizes corrosion-resistant packaging, maintaining a ranging accuracy of 0.15 meters even in an environment with 95% relative humidity.

[0028] S12, preprocess visible light images and depth point cloud data, and perform spatiotemporal registration on the preprocessed data.

[0029] (1) For visible light images, the Brown-Conrady model is used to correct lens distortion. The three types of geometric deviations—radial, tangential, and thin prism—produced by an ideal ray after passing through the lens group are all described by polynomials. The radial distortion coefficient is... With tangential distortion coefficient Obtained through laboratory calibration; corrected reprojection error .

[0030] Retinex enhancement of the V channel in HSV space (Hue, Saturation, Value) further suppresses polarizer glare, and adaptive Gamma correction is then applied to the saturation S. Improve insulator string texture contrast; timestamp alignment; FPGA latches UAV system clock on the rising edge of CMOS sensor frame synchronization signal; error... .

[0031] (2) The original laser point cloud Range-intensity bilateral filtering is employed: , in: Indicates the first The coordinates of each laser point in the horizontal direction (X-axis); Indicates the first The coordinates of each laser point in the horizontal direction (Y-axis); Indicates the first The coordinates of each laser point in the vertical direction (Z-axis); Indicates the first The reflection intensity value of each laser point; It is a generic index that represents the number of any point in the point cloud (t=1,2,…,N, where N is the total number of points in the point cloud). These are Gaussian kernels for space, distance, and intensity, respectively. Weighted to measure the spatial distance between two laser points Used to measure the height difference between two laser points in the Z direction. Weights used to measure the difference in reflection intensity between two laser points; This represents the center laser point currently being filtered, i.e., the target point. When traversing the laser point cloud, each point will be used sequentially as... It was dealt with once; Indicates the center laser point The original height value; Indicates the center laser point The reflection intensity value; Indicates the center laser point A point within the neighborhood of, i.e., a reference point; Indicates the current center laser point The set of neighborhood laser points, that is, the set of nearby laser points that participate in the filtering calculation; Represents the neighborhood laser point The height value is used for weighted averaging. Represents the neighborhood laser point The intensity value; Indicates laser point With laser point The Euclidean distance between them, i.e., the spatial distance; and These represent the central laser points. With neighboring laser points The difference in strength and the difference in height.

[0032] In the bilateral filtering formula, each laser point will take turns becoming... and with the help of multiple surrounding elements The point uses information to update its own height value.

[0033] In this embodiment, the standard deviation of the spatial Gaussian kernel Smoothing range of spatial distance; standard deviation of distance from Gaussian kernel Smoothing range for controlling height differences; standard deviation of intensity Gaussian kernel. To control the smooth range of intensity differences.

[0034] The distance-intensity bilateral filter reduced noise from 0.09m to 0.03m without sacrificing conductor edges. A cubic polynomial correction was applied to the 0.5-5m near-distance blind zone using the factory calibration curve to ensure residual error. This ensures that the laser maintains a ranging accuracy of 0.15 m across the entire range of 0.5-200 m in 95% humidity, meeting the inspection needs in rainy and foggy weather.

[0035] (3) Using IMU pre-integration and EKF joint online estimation, millisecond-level synchronous registration of vision-laser-IMU is achieved.

[0036] This registration method has small errors and significantly improves performance compared to traditional offline calibration methods.

[0037] The inertial measurement unit (IMU) outputs six-DOF pose data at a frequency of 100 Hz, providing high-precision motion priors for spatiotemporal registration of multi-source data.

[0038] Six-DOF pose representation is ,in: Euler angle ) represents the position, and θ is the RPY angle (Roll-Pitch-Yaw).

[0039] IMU timestamp With images, laser timestamps There is a deviation at the millisecond level.

[0040] The registration process is as follows: 1) High-frequency IMU pre-integration generates relative motion prior Δη between adjacent frames; visual-laser extrinsic parameter online calibration and spatiotemporal synchronization based on extended Kalman filter (EKF).

[0041] EKF Spatiotemporal Registration Algorithm State Vector for: , in: For time index, indicating the first... At that moment; The quaternion representing the orientation of the IMU; , Zero bias for accelerometer and gyroscope; This refers to the time offset of the laser relative to the camera; , Let be the rotation vector and translation vector of the laser-camera extrinsic parameters.

[0042] The motion model uses the standard IMU kinematic model: , in: Subtract zero bias from the original IMU data.

[0043] Pre-integral covarianceQ k Calibrated using Allan variance experiments, and with IMU predictions as initial values, the calibration results of the extrinsic rotation and translation vectors were obtained through iterative joint optimization of the inter-closest point (ICP) and optical flow. , With time offset .

[0044] The observation model extracts ORB features from frame images based on visual observation to obtain normalized coordinates. Laser observation observes the 3D coordinates of the same feature in the laser frame, introducing... t d Linear interpolation of the laser point: , in: For visual observation; For laser observation; The laser sensor velocity is obtained by pre-integration of the IMU.

[0045] Introducing a laser speed compensation term With time offset It can significantly suppress spatiotemporal drift under high-speed maneuvering, ensuring the spatial continuity reconstruction of slender conductors of high-voltage transmission lines.

[0046] 2) EKF update, stacking visual and laser observations into Calculate the Jacobian matrix With residual ,in These are visual observations, representing the normalized coordinates of the ORB feature points actually extracted from the camera image: ; Kalman gain: , To measure variance; Status Update: ; Observation covariance update: ; The laser point cloud is projected onto the camera coordinate system to achieve a pixel-level one-to-one correspondence.

[0047] When 20 consecutive frames and , At that time, convergence is determined; then... Fixed, only IMU pose is estimated, and online inspection mode is entered.

[0048] After the above registration, a colored point cloud is obtained. The point cloud was downsampled to 0.05 m using a PCIS network and input into a lightweight PointPillars backbone. Deformable convolution was used on the BEV feature map to enhance the sensitivity to the shape of slender wires.

[0049] Compared to traditional UAV path planning, the combination of colored point cloud and PCIS (Personal Instance Segmentation Network) achieves a one-to-one correspondence between pixels and point clouds for towers, insulators, and conductors, effectively solving false alarms caused by reflections and shadows, and improving the target recognition F1-score by 6.7%. S13, based on the spatiotemporally registered data, generates a fused feature map.

[0050] This embodiment uses the "spatiotemporal registration-fusion integration" framework STDFusionNet for weighted fusion: In the previous step, IMU pre-integration was used to correct point cloud distortion. Then, the NDT-mutual information two-step method was used to lock the extrinsic parameters of the 16-line LiDAR and the binocular image to within 1 cm. And the outliers caused by haze were removed by 2D-3D consistency mask. In this step, a lightweight CNN with 3 layers is applied to the visible light RootSIFT keypoints to obtain 32-dimensional visual features, while voxels are downsampled to... Geometric features were extracted from the point cloud using a 3-layer MLP, then fused with a weighted average of 0.6 / 0.4 and fed into a 3DConv-LSTM for temporal smoothing, finally outputting a 2D fused feature map. With 3D memory .

[0051] By incorporating "density-adaptive voxel sampling" and "bidirectional projection consistency loss" into the STDFusionNet network, the visual weight prior is automatically enhanced in haze, while 3D-LSTM suppresses temporal jitter, directly improving the accuracy of the detection backend.

[0052] A weighted fusion strategy is used to generate a fused feature map with consistency between texture and geometric features. The weight of visual features is set to 0.6 and the weight of point cloud features is set to 0.4, which ensures that the environmental perception capability can still be maintained under hazy weather conditions.

[0053] S2 generates a sparse semantic map based on the fused feature map.

[0054] A sparse semantic map containing tower topology, safe flight corridor boundaries, and dynamic obstacle prediction regions is constructed using an improved ORB-SLAM3 framework.

[0055] S21, perform semantic feature extraction and matching on the fused feature map.

[0056] The semantic feature extraction equation is as follows: , in: Represents semantic features; This represents the fused feature map generated in the previous step, with dimensions H×W×C. f C f This indicates the number of channels in the fused feature map, consistent with the output of the previous step. Indicates splicing along the channel; The lightweight "semantic encoder" ysem consists of only one 1×1 convolution layer. Output the final semantic features ; These represent the features of poles, conductors, and obstacles extracted by the semantic segmentation head, respectively. This indicates a lightweight semantic encoder.

[0057] Using the calculated "modal equilibrium" features As the sole input, the large convolutional image is no longer repeated; only three 1×1 layers are needed to obtain the category score map. Then, another 1×1 layer compresses the three categories of information back into the C dimension, resulting in semantic features that simultaneously carry "geometry + texture + category". The computational load of the entire process It meets the requirements for real-time airborne computing.

[0058] The semantic feature matching cost function is: , in: Number of key points; Represents the distance between feature descriptors; Indicates semantic label similarity; This represents the weight parameters associated with the feature confidence level in the previous step.

[0059] The above formula turns "semantic consistency" into a negative cost: when two points have the same semantic label, S=1 is equivalent to discounting the geometric distance by 30%; if the semantics are different, the cost increases, directly suppressing erroneous associations. Weights The fusion confidence is taken from the step of generating the fusion feature map, making the high visual confidence points The distance term accounts for a large proportion, and low confidence points rely more on semantic consistency to achieve adaptive matching.

[0060] semantic features Fs Depend on The model is obtained by a single convolutional layer, without introducing the heavy U-Net, and the number of model parameters is only 0.02M, which is 98% smaller than DC-SLAM. On the KITTI-30 m fog dataset, the outlier matching rate is greatly reduced, which is better than DC-SLAM and VSO-SLAM.

[0061] The semantic feature matching cost function is fed into the edge weight w of Topo-ORB-SLAM3. Semantically consistent points receive larger edge weights, which further reduces the tower vertex positioning error by 1.8cm.

[0062] S22 employs an improved ORB-SLAM3 framework. The front-end visual odometry calculates the camera pose using the fused feature map, while the back-end combines co-view optimization and pose map optimization.

[0063] S23, insert key points of poles, conductors and obstacles as semantic feature markers into the optimization map to generate a sparse semantic map.

[0064] The constructed sparse semantic map only includes the tower topology, safe flight corridor boundaries, and dynamic obstacle prediction areas.

[0065] S3. In the sparse semantic map, combined with prior knowledge of the power pole topology, a globally optimal path covering all poles to be inspected is generated.

[0066] The objective function for the globally optimal path is: , in: Represents path sequence , Represents a set of path sequences. This represents the globally optimal inspection path, which sequentially passes through a sequence of 3D track points of all towers to be inspected. Representing a path P The number of waypoints in the middle, Indicates the number of poles to be inspected. Indicates the first The center coordinates of the base tower Indicates from Fly to The costs of flight can include factors such as distance, energy consumption, time, and risks. Indicates the first One pole to be inspected This represents the inspection quality evaluation function for tower b; a smaller value indicates a better inspection effect. This represents the inspection quality weighting coefficient, used to balance flight costs and inspection quality.

[0067] The objective function is a weighted sum form of a multi-objective optimization problem, which merges two optimization objectives into a single solvable objective. This represents the total flight cost of the entire path, and the optimization objective is to minimize the total cost of the drone's flight.

[0068] This represents a weighted summation of the inspection quality of all towers, with the optimization goal of achieving the best overall inspection effect. Larger sizes place greater emphasis on inspection quality, potentially allowing for longer flight paths to achieve better detection results. Smaller aircraft prioritize flight efficiency, tending to choose the shortest or most energy-efficient path. This model uses mathematical optimization methods to find the global optimum, rather than a local optimum or heuristic path, by considering all feasible paths.

[0069] By balancing multiple objectives, including flight cost and inspection quality, the problem of poor inspection results or low flight efficiency that might result from optimizing a single objective is avoided. The balance between "efficiency" and "quality" can be flexibly adjusted according to the actual task requirements.

[0070] The topological connections of power poles can be incorporated into sparse semantic maps, making the paths more consistent with the power grid structure and suitable for large-scale power inspection scenarios.

[0071] S4 performs real-time obstacle avoidance while flying along the globally optimal path.

[0072] During the flight of the UAV, an improved dynamic window method is used to optimize the local trajectory and avoid obstacles in real time based on real-time perception data.

[0073] The improved dynamic window method takes linear velocity and angular velocity as input and outputs a comprehensive evaluation value. The function. The velocity sampling space is linear velocity. With angular velocity The evaluation function incorporates an instantaneous collision risk index and a path deviation penalty term. The evaluation function formula is as follows: , in: This represents the final evaluation function value. For a given (v,ω) velocity pair, a scalar score is calculated. The system evaluates many (v,ω) pairs and selects the pair that maximizes G as the best motion command at the current moment. This represents an activation or normalization function that maps the result of a weighted sum to a fixed range (e.g., between 0 and 1) to ensure stable and interpretable output, or for probabilistic purposes. , , , , The normalized weighting coefficients are used to adjust the proportion of each sub-objective in the final decision. This is an instantaneous collision risk indicator, calculated based on real-time depth data. The lower the risk, the higher the score. This is the azimuth evaluation item, representing the degree of alignment between the UAV and the target point. For speed evaluation, drones are encouraged to maintain a high forward speed to improve mobility and avoid unnecessary stagnation. This is a distance evaluation item, which assesses the distance between the drone's expected location and the obstacle after executing the speed command. Generally, the greater the distance, the higher the score, encouraging the drone to stay away from the obstacle. The trajectory tracking score evaluates how well the speed command follows the preset global path or trajectory. The higher the value, the better it follows the predetermined route.

[0074] The improved dynamic window method is relatively efficient in calculation and can meet the requirements of real-time control of UAVs. By adjusting the weight parameters, the same algorithm can be adapted to different scenarios, such as high-speed cruising, narrow passage passage, and active obstacle avoidance, without rewriting the core logic. The risk and dist terms strongly constrain the behavior of the UAV, ensuring that it prioritizes safety in all situations. The heading, velocity, and tracking terms drive the UAV to actively move towards the target, avoiding local stagnation in complex environments. Since it optimizes the continuous velocity space, the generated motion commands change continuously in speed, avoiding abrupt changes and making the UAV's movement smoother. Reasonable weight settings can make obstacle avoidance behavior appear natural and predictable.

[0075] S5 generates a revisit instruction during the inspection process and triggers secondary inspection path planning for suspicious towers.

[0076] Online defect identification is performed on the acquired visible light images of power towers. When the confidence level of the online defect identification exceeds a preset value, a revisit instruction is generated. A lightweight convolutional neural network is used, based on the MobileNetV3 backbone, with added attention mechanism and feature pyramid. The model has fewer than 2M parameters and performs real-time classification and localization of insulator damage, conductor strand breakage, and tower corrosion in visible light images. When the detection confidence level exceeds a threshold of 0.85, a revisit instruction is generated.

[0077] The revisit instruction includes the defective tower number, defect type, confidence score, and suggested observation angle.

[0078] The revisit command inserts a secondary inspection waypoint into the original inspection flight path. The location of the secondary inspection waypoint is dynamically adjusted according to the type of tower defect.

[0079] Based on the same inventive concept disclosed above, this disclosure also provides a path planning system for an inspection drone, including a multimodal perception fusion module, a lightweight 3D environment modeling module, a global path collaborative planning module, a local dynamic obstacle avoidance module, and a real-time defect identification and feedback module; wherein... The multimodal perception fusion module uses multimodal data to generate a fused feature map; A lightweight 3D environment modeling module for generating sparse semantic maps based on fused feature maps; The global path collaborative planning module is used to generate a globally optimal path covering all the poles to be inspected in the sparse semantic map by combining prior knowledge of the power pole topology. The local dynamic obstacle avoidance module is used to perform real-time obstacle avoidance while flying along the globally optimal path; The real-time defect identification and feedback module is used to generate revisit instructions and trigger secondary inspection path planning for suspicious towers during the inspection process.

[0080] Based on the same inventive concept as the above-disclosed content, this disclosure also provides an electronic device. For example... Figure 2 As shown, the electronic device of this disclosure includes at least one processor and at least one storage medium electrically connected to the processor. The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the inspection drone path planning method as described above.

[0081] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between lines. The indirect connection method can be applied to the embodiments of this disclosure as long as it achieves the purpose of this disclosure.

[0082] Based on the same inventive concept, this disclosure also provides a computer storage medium storing a computer program, which, when executed by a processor, implements the inspection drone path planning method described above.

[0083] Based on the same inventive concept, this disclosure also provides a computer program product, which is stored in at least one storage medium; the computer program product includes several instructions to cause at least one computer device to execute the inspection drone path planning method as described above.

[0084] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A path planning method for an inspection drone, characterized in that, The method includes: Generate a fused feature map using multimodal data; Based on the fused feature maps, a sparse semantic map is generated; In the sparse semantic map, combined with prior knowledge of the power pole topology, a globally optimal path covering all poles to be inspected is generated; Real-time obstacle avoidance is performed while flying along the globally optimal path; During the inspection process, a revisit instruction is generated and a secondary inspection path planning for the suspicious tower is triggered.

2. The method according to claim 1, characterized in that, Using multimodal data, a fused feature map is generated, including: Acquire multimodal data containing visible light images and depth point cloud data; Preprocess visible light images and depth point cloud data, and then perform spatiotemporal registration on the preprocessed data; Based on the spatiotemporally registered data, a fused feature map is generated.

3. The method according to claim 1, characterized in that, Based on the fused feature map, a sparse semantic map is generated, including: Semantic feature extraction and matching are performed on the fused feature map; An improved ORB-SLAM3 framework is adopted, in which the front-end visual odometry calculates the camera pose through the fused feature map, and the back-end adopts a combination of co-view optimization and pose map optimization. By inserting key points of poles and obstacles as semantic feature markers into the optimization graph, a sparse semantic map is generated.

4. The method according to any one of claims 1-3, characterized in that, The topological connections of power poles are incorporated into the sparse semantic map.

5. The method according to any one of claims 1-3, characterized in that, During flight along the globally optimal path, real-time obstacle avoidance is performed, including: During the flight of the UAV, an improved dynamic window method is used for local trajectory optimization and real-time obstacle avoidance based on real-time perception data. The evaluation function formula of the improved dynamic window method is as follows: ， in: This represents the final evaluation function value; Indicates linear velocity; Indicates angular velocity; This represents an activation function or normalization function that maps the result of a weighted sum to a fixed range; , , , , These are the normalized weighting coefficients; This is the azimuth evaluation item; For speed evaluation items; For distance evaluation items; Scoring is given for trajectory tracking.

6. The method according to any one of claims 1-3, characterized in that, Generate a revisit instruction and trigger secondary inspection path planning for the suspicious tower, including: When the confidence level of the online defect identification of the tower exceeds a preset value, a revisit instruction is generated; The revisit command inserts a secondary inspection waypoint into the original inspection flight path. The location of the secondary inspection waypoint is dynamically adjusted according to the type of tower defect.

7. A path planning system for an inspection drone, characterized in that, The system includes a multimodal perception fusion module, a lightweight 3D environment modeling module, a global path collaborative planning module, a local dynamic obstacle avoidance module, and a real-time defect identification and feedback module; wherein: The multimodal perception fusion module uses multimodal data to generate a fused feature map; A lightweight 3D environment modeling module for generating sparse semantic maps based on fused feature maps; The global path collaborative planning module is used to generate a globally optimal path covering all the poles to be inspected in the sparse semantic map by combining prior knowledge of the power pole topology. The local dynamic obstacle avoidance module is used to perform real-time obstacle avoidance while flying along the globally optimal path; The real-time defect identification and feedback module is used to generate revisit instructions and trigger secondary inspection path planning for suspicious towers during the inspection process.

8. An electronic device, characterized in that, Includes at least one processor and at least one storage medium electrically connected; The storage medium is electrically connected to the processor, wherein the storage medium stores instructions executable by at least one of the processors, the instructions being executed by at least one of the processors to enable at least one of the processors to perform the inspection drone path planning method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The computer storage medium stores a computer program. When the computer program is executed by the processor, it implements the path planning method for the inspection drone as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product is stored in at least one storage medium; The computer program product includes several instructions for causing at least one electronic device to execute the inspection drone path planning method according to any one of claims 1-6.

Images