Unmanned aerial vehicle-based overhead line insulator inspection path planning method

Abstract

The invention discloses an unmanned aerial vehicle-based overhead line insulator inspection path planning method. The method comprises the following steps of providing an inspection mathematical model based on preprocessed overhead line scene point cloud and unmanned aerial vehicle carried hardware parameters; the model generates an optimal waypoint coordinate capable of covering the shooting requirement of each insulator by using the identified insulator point cloud information; then, an LKH algorithm and an improved A * algorithm are used for calculating an optimal path for connecting the waypoints, and camera shooting parameters of the waypoints are further optimized according to the shooting effect; and finally, performing experimental measurement on the field point cloud data of the seven overhead lines, and performing comparative analysis on the result. According to the model, the waypoint coverage rate and the path optimization efficiency are improved, the picture quality is improved through adjustment of shooting parameters, and the method is suitable for unmanned aerial vehicle inspection tasks in actual engineering and has wide application prospects.

Description

Technical Field

[0001] The invention relates to an overhead line insulator inspection path planning method based on an unmanned aerial vehicle, and belongs to the technical field of new energy. Background Art

[0002] In the acceptance project of overhead transmission lines, the inspection and acceptance of insulators is an important part. And the UAV inspection path planning is an important part of the insulator acceptance collection. In the path planning process, the first thing to consider is the flight safety of the UAV to ensure that the UAV does not collide with obstacles in the overhead line environment. Secondly, the total distance of the planned path is directly related to the inspection efficiency of the UAV. In addition, the selection of the camera viewpoint position in the path will also affect the insulator aerial coverage and the insulator image quality. The current power line UAV inspection path solution is not very applicable. The main problem is that the surrounding environment of the overhead power line is very complex. In most cases, it still depends on manual flight by the pilot, and it is difficult to plan a safe and reliable automatic acceptance route to ensure the safety of the UAV while collecting high-quality data that meets the acceptance requirements.

[0003] For safety and efficiency reasons, the overhead line UAV inspection path planning is considered as a three-dimensional coverage path planning (CPP) problem. Most of the existing UAV automatic flight path generation methods solve the CPP problem in two steps. First, the planning algorithm determines a set of viewpoints that meet the coverage requirements of the inspection application. The process of finding these viewpoints is called the viewpoint planning (VPP) problem. Then, the planning algorithm finds the shortest path between the selected viewpoints, and the UAV flies along this path to complete the automatic inspection task. According to the different focuses of the coverage path planning scheme in different scenarios, the planning schemes can be divided into three types: focusing on the inspection time, focusing on the inspection environment, and focusing on the inspection target. Path planning methods that focus on inspection time focus on minimizing the total working time. However, such methods only consider the inspection time allocation, do not consider the obstacle avoidance problem in the inspection environment, and do not evaluate the inspection quality of aerial photography targets; path planning methods that focus on the inspection environment focus on the impact of the inspection site environment on the inspection path. However, the implementation effect of such methods on a single drone remains to be studied; path planning algorithms that focus on inspection targets focus on the image acquisition effect of the inspection target. However, such methods have low operating efficiency in large-scale complex scenes and cannot meet the real-time requirements of inspection projects. Although there are many related studies on drone inspection path planning, many algorithms do not consider the operating efficiency in large scene environments. At the same time, for small-scale inspection targets such as insulators, it is also necessary to consider the image acquisition quality and inspection coverage during drone inspections. Summary of the invention

[0004] Technical issues:

[0005] The purpose of the present invention is to solve the problem that the existing path planning algorithm does not take into account the small-scale inspection targets such as insulators in a large scene environment, and at the same time, the problems of image acquisition quality and inspection coverage, and provide a path planning method based on the combination of a drone obstacle avoidance model and a photography model.

[0006] Technical solution:

[0007] To achieve the above object, the method adopted by the present invention is: a method for planning an overhead line insulator inspection path based on a drone, comprising the following steps in sequence:

[0008] S1: Use drones to collect point clouds of overhead line scenes to obtain point cloud data;

[0009] S2: Identify the insulator point cloud information in the point cloud data to obtain the three-dimensional coordinate position of the insulator.

[0010] S3: To prevent accidents caused by collisions between drones and poles, power lines or other ground obstacles during inspection, a drone obstacle avoidance model is established based on the point cloud of overhead line scenes;

[0011] S4: When taking images of insulators, in order to ensure that the insulators occupy a certain size in the image and the aerial images are clear, a drone photography model with a camera mounted on the drone needs to be established;

[0012] S5: Generate an efficient and reasonable drone inspection path using pre-acquired LiDAR point cloud data and information such as drone obstacle avoidance and photography models;

[0013] Furthermore, in the step S1, a drone equipped with a laser radar (LiDAR) is used to fly semi-automatically according to the inspection workflow, and the laser point cloud is scanned downward from a safe distance above the overhead line to obtain point cloud data of the complete scene.

[0014] Furthermore, the specific steps of step S2 are as follows:

[0015] S2-1: Preprocessing the original point cloud data of the overhead line scene, mainly including point cloud filtering and tower area pre-extraction;

[0016] S2-2: After completing the point cloud filtering and tower area pre-extraction, the point cloud magnitude of each tower area has been effectively reduced compared to the original point cloud. Then the types and characteristics of insulators are analyzed, and insulators are divided into two categories: suspension insulators and tension insulators. A feature set is constructed based on the three-dimensional features of the point clouds of various types of insulators, and the weight of each feature is determined based on the entropy weight method. The final point cloud feature matrix is ​​used to train the random forest classifier to complete insulator identification.

[0017] Furthermore, in the step S3, a drone obstacle avoidance model is established based on the filtered and simplified point cloud data, the three-dimensional point cloud space composed of the point cloud data is divided using cubic grids with a side length of l, and a danger level is set for each grid to reflect the collision risk that would exist if the drone is within the range of the grid, thereby establishing an obstacle avoidance model based on the danger level mechanism.

[0018] Furthermore, the specific steps of step S4 are as follows:

[0019] S4-1: Construct camera imaging model diagram;

[0020] S4-2: The camera focal length f is calculated as shown in formula (1):

[0021]

[0022] In the above formula, f min is the equivalent minimum focal length of the camera lens, and z is the camera zoom factor. The camera's diagonal field of view d fov The calculation formula is as follows:

[0023]

[0024] In the above formula, cmosw is the camera sensor width, cmosh is the camera sensor height, and f is the camera focal length.

[0025] S4-3: Using the camera's diagonal field of view d fov The actual shooting range calculation formula corresponding to the photo can be obtained as follows:

[0026]

[0027] In the above formula, h, w, and x represent the height, width, and diagonal length of the actual shooting range corresponding to the photo, respectively; d is the distance between the actual object being photographed and the camera (referred to as the object distance), and pixelw and pixelh represent the pixel width and pixel height of the photo, respectively.

[0028] Furthermore, the specific steps of step S5 are as follows:

[0029] S5-1: Generate a set of UAV inspection waypoints according to the acceptance project requirements and safety obstacle avoidance conditions. The inspection waypoints include the three-dimensional coordinate position of the UAV when inspecting the insulator and the camera shooting direction. The insulator point cloud position information obtained by identification is combined with the UAV inspection obstacle avoidance model, camera imaging model, gimbal angle parameters, the distance between the camera and the target insulator (object distance), the range of insulators covered by the photo and other constraints to determine the UAV inspection insulator waypoint set;

[0030] S5-2: After the drone inspection waypoints are generated, it is necessary to determine the routes connecting these waypoints to form a complete drone inspection path for collecting insulator images. First, the Euclidean distance between two coordinate points in three-dimensional space is used as the cost function to calculate the distance between any two points in the waypoint set. Then, the Lin-Kernighan-Helsgaun heuristic (LKH) algorithm is used as the TSP solver to obtain the waypoint connection sequence with the shortest total distance. Due to the presence of obstacles on the overhead line site, the waypoints cannot be directly connected by straight lines to form a path, and obstacle avoidance judgment is required. Therefore, the A* algorithm is added, and the connection path between the waypoints is generated in combination with the on-site point cloud obstacle avoidance model. Finally, consider optimizing the turning time of the drone inspection and smoothing the generated path;

[0031] S5-3: The complete waypoint also includes camera shooting parameters, so the camera shooting parameters need to be determined. The camera shooting parameters mainly include zoom factor and exposure parameters. The zoom factor can be determined by the photography model in step S4. The exposure parameters are obtained by calculating the image ratio of overexposed and dark areas in the image.

[0032] Beneficial effects:

[0033] The insulator inspection path planning method proposed in the present invention is based on the traditional transmission line acceptance only based on two-dimensional image processing. It uses the three-dimensional point cloud information collected by the automatic inspection of the drone to identify and locate the insulator point cloud position. It can not only classify the on-site point cloud and use it as a priori information for planning the insulator inspection path, but also serve as a three-dimensional data information source for insulator defect detection. In addition, the inspection mathematical model of the present invention combines the obstacle avoidance model and the photography model, so as to better meet the safety and efficiency requirements of on-site inspections, provide important safety guarantees for generating inspection waypoint trajectories, and provide important data support for the selection of drone inspection waypoint positions and shooting postures. In addition, the present invention adds the A* algorithm on the basis of the LKH algorithm, combines the obstacle avoidance model to generate the connection path between the waypoints, and smoothly optimizes the generated path. At the same time, the present invention also optimizes the corresponding camera shooting parameters for the backlit images and low-light images taken by the drone during the inspection process. The method of the present invention was experimented on 7 overhead transmission lines (including 23 pole towers and a total of 287 insulators). The average insulator coverage rate, the total length of the generated path, the total length of the smoothed path, and the running time of the path planning algorithm for each line were statistically analyzed. It can be obtained that the average insulator coverage rate corresponding to the generated route is 90.01%, which meets the acceptance project requirements; the average running time of the path generation algorithm is 88.74s, which meets the real-time requirements; the smoothed path further shortens the inspection distance and improves the inspection efficiency. At the same time, the camera parameter optimization algorithm was quantitatively experimentally analyzed, and the camera parameters were optimized and re-shot. The experiment proved that the optimized camera parameters can effectively improve the shooting image effect and can be used in the insulator inspection application scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the overall process of the method of the present invention;

[0035] Figure 2 A photographic model diagram in the method of the present invention;

[0036] Figure 3 A comparison diagram of images taken before and after the optimization of photographic parameters in the method of the present invention;

[0037] Figure 4 A visualization diagram of waypoints and inspection path results is generated for the method of the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is described in detail below, but the protection scope of the present invention is not limited to the embodiments. Figure 1 As shown, the overhead line insulator inspection path planning method based on a drone in this embodiment includes the following steps in sequence:

[0039] S1: Use drones to collect point clouds of overhead line scenes to obtain point cloud data;

[0040] S2: Identify the insulator point cloud information in the point cloud data to obtain the three-dimensional coordinate position of the insulator.

[0041] S3: To prevent accidents caused by collisions between drones and poles, power lines or other ground obstacles during inspection, a drone obstacle avoidance model is established based on the point cloud of overhead line scenes;

[0042] S4: When taking images of insulators, in order to ensure that the insulators occupy a certain size in the image and the aerial images are clear, a drone photography model with a camera mounted on the drone needs to be established;

[0043] S5: Generate an efficient and reasonable drone inspection path using pre-acquired LiDAR point cloud data and information such as drone obstacle avoidance and photography models;

[0044] In this embodiment, in step S1, a drone equipped with a laser radar (LiDAR) is used to fly semi-automatically according to the inspection workflow, and the laser point cloud is scanned downward from a safe distance above the overhead line to obtain point cloud data of the complete scene.

[0045] In this embodiment, the specific steps of step S2 are as follows:

[0046] S2-1: Preprocessing the original point cloud data of the overhead line scene, mainly including point cloud filtering and tower area pre-extraction;

[0047] S2-2: After completing the point cloud filtering and tower area pre-extraction, the point cloud magnitude of each tower area has been effectively reduced compared to the original point cloud. Then the types and characteristics of insulators are analyzed, and insulators are divided into two categories: suspension insulators and tension insulators. A feature set is constructed based on the three-dimensional features of the point clouds of various types of insulators, and the weight of each feature is determined based on the entropy weight method. The final point cloud feature matrix is ​​used to train the random forest classifier to complete insulator identification.

[0048] In this embodiment, in the step S3, a drone obstacle avoidance model based on the filtered and simplified point cloud data is established, the three-dimensional point cloud space composed of the point cloud data is divided using cubic grids with a side length of l, and a danger level is set for each grid to reflect the collision risk that would exist if the drone was within the range of the grid, thereby establishing an obstacle avoidance model based on the danger level mechanism.

[0049] In this embodiment, the specific steps of step S4 are as follows:

[0050] S4-1: Camera imaging model as shown in the figure Figure 2 As shown;

[0051] S4-2: The camera focal length f is calculated as shown in formula (6):

[0052]

[0053] In this embodiment, f min is the equivalent minimum focal length of the camera lens, and z is the camera zoom factor. The camera's diagonal field of view d fov The calculation formula is as follows:

[0054]

[0055] In this embodiment, cmosw is the camera sensor width, cmosh is the camera sensor height, and f is the camera focal length.

[0056] S4-3: Using the camera's diagonal field of view d fov The actual shooting range calculation formula corresponding to the photo can be obtained as follows:

[0057]

[0058] In this embodiment, h, w, and x respectively represent the height, width, and diagonal length of the actual shooting range corresponding to the photo; d is the distance between the actual object being photographed and the camera (referred to as the object distance), and pixelw and pixelh respectively represent the pixel width and pixel height of the photo.

[0059] In this embodiment, the specific steps of step S5 are as follows:

[0060] S5-1: Generate a set of UAV inspection waypoints according to the acceptance project requirements and safety obstacle avoidance conditions. The inspection waypoints include the three-dimensional coordinate position of the UAV when inspecting the insulator and the camera shooting direction. The insulator point cloud position information obtained by identification is combined with the UAV inspection obstacle avoidance model, camera imaging model, gimbal angle parameters, the distance between the camera and the target insulator (object distance), the range of insulators covered by the photo and other constraints to determine the UAV inspection insulator waypoint set;

[0061] S5-2: After the drone inspection waypoints are generated, it is necessary to determine the routes connecting these waypoints to form a complete drone inspection path for collecting insulator images. First, the Euclidean distance between two coordinate points in three-dimensional space is used as the cost function to calculate the distance between any two points in the waypoint set. Then, the Lin-Kernighan-Helsgaun heuristic (LKH) algorithm is used as the TSP solver to obtain the waypoint connection sequence with the shortest total distance. Due to the presence of obstacles on the overhead line site, the waypoints cannot be directly connected by straight lines to form a path, and obstacle avoidance judgment is required. Therefore, the A* algorithm is added, and the connection path between the waypoints is generated in combination with the on-site point cloud obstacle avoidance model. Finally, consider optimizing the turning time of the drone inspection and smoothing the generated path;

[0062] S5-3: The complete waypoint also includes camera shooting parameters, so the camera shooting parameters need to be determined. The camera shooting parameters mainly include zoom factor and exposure parameters. The zoom factor can be determined by the photography model in step S4. The exposure parameters are obtained by calculating the image ratio of overexposed and dark areas in the image.

[0063] Here, in the waypoint generation algorithm, the constraints of waypoint generation are mainly set based on the drone obstacle avoidance model and the photography model. Since the size of the DJI M300RTK is 0.81m×0.67m×0.43m. Considering a certain safety margin, the grid size of this invention is selected as 1m 3 , that is, set the side length l of the obstacle avoidance grid to 1m. The setting of the danger level M in the obstacle avoidance model is related to the wind speed at the acceptance site. If the wind speed is high, a larger M needs to be set to ensure flight safety; if the wind speed is low, a smaller M can be set to improve the efficiency of obstacle avoidance grid generation. M is usually taken as an integer value in the range of 4 to 8. Object distance range D min and D max Usually, they are set to 15m and 20m respectively. Because the insulator is usually some distance away from the edge of the tower, considering the flight safety distance, D min It cannot be set too small, and if the object distance is too far, the clarity of the insulator will decrease. max The gimbal pitch angle range is determined by the controllable rotation angle of the hardware itself. At the same time, in order to make the camera cover a wider range of insulators, the pitch range is further reduced. min and θ max Set to 12° and 30° respectively.

[0064] In the path generation algorithm, before using the A* algorithm and when smoothing the generated path, it is necessary to use linear interpolation to generate line points and perform obstacle avoidance judgment. The step size of the linear interpolation method is usually set to 0.1m, because if the step size is set too large, it may cause the obstacle avoidance judgment to be missed and thus generate a path passing through obstacles; if the step size is set too small, more path points will be generated, reducing the algorithm execution efficiency and the drone inspection efficiency. The A* algorithm path search grid is directly selected as the obstacle avoidance grid, so its path search step size is set to the grid side length of 1m.

[0065] In the camera parameter optimization algorithm, the length of the insulator point cloud bounding box needs to be expanded by a certain length Δl and set to the actual shooting range height corresponding to the photo. Δl is usually set to 0.5m to ensure that the photo can completely include the insulator. When the captured image is divided into rectangular windows and traversed, in order to accurately reflect the image information and meet the real-time processing requirements, the rectangular window is set to 16×16 and the pixel neighborhood side length δ is set to 11. The backlit image and low-light image taken with automatic exposure parameters are shown as follows: Figure 3 (a) and Figure 3 As shown in (b), the normally exposed insulator images obtained by retaking the photos after the camera parameters are optimized are as follows: Figure 3 (c) and Figure 3 (d) as shown.

[0066] The above parameter settings are summarized as shown in Table 1.

[0067] Table 1. Parameter selection

[0068]

[0069]

[0070] The average insulator coverage rate, total length of generated paths, total length of smoothed paths, and running time of path planning algorithm for each line are statistically analyzed. The experimental data of various indicators are shown in Table 2. Among them, the waypoints and paths generated for line 6 are visualized. The results are shown in Figure 4 Finally, the average insulator coverage rate corresponding to the generated route is 90.01%, which meets the requirements of the acceptance project; the average running time of the path generation algorithm is 88.74s, which meets the real-time requirements; the smoothed path further shortens the inspection distance and improves the inspection efficiency.

[0071] Table 2. Waypoint generation and path planning experimental data

[0072]

[0073] A quantitative experimental analysis of the camera parameter optimization algorithm was conducted. The experiment was conducted using images taken from the insulators of line 7. During the inspection of the line, 5 insulator images with backlight or low light were detected. The images were retaken after the camera parameters were optimized. The images of the insulators taken under normal lighting conditions were used as reference images, and the quality of the images taken before and after optimization was analyzed using three evaluation indicators: average brightness, contrast, and peak signal-to-noise ratio (PSNR). The evaluation indicators of the 5 images were calculated, and the evaluation indicator values ​​of the images taken before and after parameter optimization were expressed in array form: (image indicator value before optimization, image indicator value after optimization). The experimental results are shown in Table 3:

[0074] Table 3. Camera parameter optimization experimental data

[0075]

[0076]

[0077] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A method for planning the inspection path of overhead line insulators based on drones, characterized in that: The following steps are involved: S1: Use drones to collect point clouds of overhead line scenes to obtain point cloud data; S2: Identify the insulator point cloud information in the point cloud data, so as to obtain the three-dimensional coordinate position of the insulator; S3: To prevent accidents caused by collisions between drones and poles, power lines or other ground obstacles during inspection, a drone obstacle avoidance model is established based on the point cloud of overhead line scenes; S4: When taking images of insulators, in order to ensure that the insulators occupy a certain size in the image and the aerial images are clear, a drone photography model with a camera mounted on a drone is established; S5: Generate an efficient and reasonable drone inspection path using pre-acquired point cloud data and drone obstacle avoidance and photography model information.

2. The method for planning an overhead line insulator inspection path based on a drone according to claim 1, characterized in that: In the step S1, a drone equipped with a LiDAR is used to fly semi-automatically according to the inspection workflow, and the laser point cloud is scanned downward from a safe distance above the overhead line to obtain point cloud data of the complete scene.

3. The method for planning the inspection path of overhead line insulators based on a drone according to claim 1 is characterized in that the specific steps of step S2 are as follows: S2-1: Preprocessing the original point cloud data of the overhead line scene, including point cloud filtering and tower area pre-extraction; S2-2: After completing point cloud filtering and tower area pre-extraction, the order of magnitude of point clouds in each tower area has been effectively reduced compared to the original point cloud; then the types and characteristics of insulators are analyzed, and insulators are divided into two categories: suspension insulators and tension insulators. A feature set is constructed based on the three-dimensional features of the point clouds of each type of insulator, and the weight of each feature is determined based on the entropy weight method. The final point cloud feature matrix is ​​used to train a random forest classifier to complete insulator identification.

4. The method for planning an overhead line insulator inspection path based on a drone according to claim 1, characterized in that: In the step S3, a drone obstacle avoidance model is established, and the three-dimensional point cloud space composed of the point cloud data is divided using a cubic grid with a side length of l, and a danger level is set for each grid to reflect the collision risk that will exist if the drone is within the range of the grid, thereby establishing an obstacle avoidance model based on the danger level mechanism.

5. The method for planning an overhead line insulator inspection path based on a drone according to claim 1, characterized in that: The specific steps of step S4 are as follows: S4-1: Construct camera imaging model diagram; S4-2: The camera focal length f is expressed as the equivalent minimum focal length f of the camera lens min and the camera zoom factor z, the camera's diagonal field of view d fov It is expressed as a function of the camera sensor width cmosw and the camera sensor height cmosh and the camera focal length f; The camera focal length f is calculated as follows: Camera diagonal field of view d fov The calculation formula is as follows: S4-3: Using the camera's diagonal field of view d fov The actual shooting range corresponding to the photo can be obtained, and the calculation formula is as follows: Where h, w, and x represent the height, width, and diagonal length of the actual shooting range corresponding to the photo, respectively; d is the distance between the actual object being photographed and the camera, and pixelw and pixelh represent the pixel width and pixel height of the photo, respectively.

6. The method for planning an overhead line insulator inspection path based on a drone according to claim 5, characterized in that: The specific steps of step S5 are as follows: S5-1: Generate a set of UAV inspection waypoints according to the acceptance project requirements and safety obstacle avoidance conditions. The inspection waypoints include the three-dimensional coordinate position of the UAV when inspecting the insulator and the camera shooting direction. The insulator point cloud position information obtained by identification is combined with the UAV obstacle avoidance model, camera imaging model, gimbal angle parameters, the distance between the camera and the target insulator, and the constraints of the insulator range covered by the photo to determine the waypoint set of the UAV inspection insulator; S5-2: After the UAV inspection waypoints are generated, it is necessary to determine the route connecting these waypoints to form a complete inspection path for the UAV to collect insulator images; first, the Euclidean distance between two coordinate points in three-dimensional space is used as the cost function to calculate the distance between any two points in the waypoint set, and then the heuristic algorithm is used as the TSP solver to obtain the waypoint connection sequence with the shortest total distance; due to the presence of obstacles on the overhead line site, the waypoints cannot be directly connected by straight lines to form a path, and obstacle avoidance judgment is also required. Therefore, the A* algorithm is added to generate the connection path between the waypoints in combination with the on-site point cloud obstacle avoidance model; finally, the turning time of the UAV inspection is optimized and the generated path is smoothed; S5-3: The complete waypoint also includes camera shooting parameters, so the camera shooting parameters need to be determined; the camera shooting parameters include zoom factor and exposure parameters; the zoom factor is determined by the drone photography model in step S4, and the exposure parameters are obtained by calculating the image ratio of overexposed and dark areas in the image.

Images