A method and system for locking and tracking power lines using lidar

Abstract

The present invention discloses a method and system for using lidar to perform wire locking and tracking processing on power lines. Among them, the method includes using lidar to collect laser point clouds in real time to obtain point cloud data; using a deep learning model to segment the laser point clouds to obtain power line points and tower points; clustering the tower points and calculating the central point coordinates of the tower points; using a RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines in the current line channel to obtain the straight line equations of all straight lines; selecting a target tracking straight line; determining whether the deviation angle and distance of the target tracking straight line are less than or equal to a preset standard deviation threshold; if so, using the target tracking straight line to predict the flight prediction position of the aircraft. The technical solution of the present invention can solve the problems in the prior art that wire locking and tracking are unstable, the traditional Hough transform has low efficiency, large memory occupation, and high requirements for the computing power and computing resources of the computing unit.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and system for using lidar to perform line-locking tracking processing on power lines. Background Art

[0002] In related technologies of spatial ranging, it is often necessary to measure and extract straight lines in three-dimensional space. For example, map surveying and mapping, UAV reconnaissance, and transmission line measurement technologies. In existing transmission line measurements, for the measurement and extraction of straight lines, usually, a general image of the transmission towers is obtained through UAV inspection, and then the straight line data in three-dimensional space is extracted using relevant UAV line-following algorithms.

[0003] Most of the related technologies involved in current UAV line-following algorithms adopt the scheme of taking images with a camera. Specifically, by taking pictures of the line and then using the hough transform algorithm to extract the line equation. The disadvantages of the above scheme are that the camera is easily affected by light and needs to be taken at close range, otherwise the resolution is insufficient or the image is not clear and the line in the image cannot be recognized. In addition, there are occlusions in the image of the line taken, and the background straight lines in the scene are also easily extracted. These factors make the scheme of taking images with a camera unstable and unreliable. Summary of the Invention

[0004] The present invention provides a method for using lidar to perform line-locking tracking processing on power lines, and uses the RANSAC multi-line extraction algorithm with constraint conditions to extract the straight line equations of all straight lines on the current line channel, aiming to solve the problems of unstable line-locking tracking in the prior art, low efficiency of traditional hough transform and least squares algorithms, large memory occupancy, and high requirements for the computing power and computing resources of the computing unit.

[0005] To achieve the above object, according to the first aspect of the present invention, a method for using lidar to perform line-locking tracking processing on power lines is proposed, and the main steps include:

[0006] Use lidar to collect laser point clouds in real time to obtain point cloud data;

[0007] Use a deep learning model to segment the laser point clouds included in the point cloud data to obtain power line points and tower points corresponding to the current line channel;

[0008] Cluster the tower points in the current line channel, and use the clustered tower points to calculate and generate the center point coordinates of the tower points;

[0009] Use the RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines in the current line channel according to the center point coordinates of the tower points, and obtain the straight line equations of all straight lines in the current line channel;

[0010] Select a target tracking straight line according to the straight-line equations of all the straight lines in the current line channel;

[0011] Judge whether the deviation angle and distance of the target tracking straight line are less than or equal to a preset standard deviation threshold;

[0012] If it is determined that the deviation angle and distance are less than or equal to the preset standard deviation threshold, use the target tracking straight line to predict the flight prediction position of the aircraft.

[0013] Preferably, the step of clustering the tower pole points in the current line channel and calculating the central point coordinates of the tower pole points using the clustered tower pole points includes:

[0014] Extract the tower pole points within a predetermined distance range of the current line channel;

[0015] Use the DBSCAN algorithm to perform clustering operations on the tower pole points and select the category with the largest number of tower pole points;

[0016] Use the coordinates of all the tower pole points in the category with the largest number of tower pole points to calculate the average to obtain the central point coordinates of the tower pole points.

[0017] Preferably, in the above power line locking tracking processing method, the step of using the RANSAC multi-line extraction algorithm with constraint conditions to extract all the straight lines of the current line channel according to the central point coordinates of the tower pole points includes:

[0018] Use the RANSAC multi-line extraction algorithm to extract the circuit line points corresponding to the current line channel to obtain the straight lines of the current frame;

[0019] Obtain the straight lines of the previous frame of the current frame straight lines, and constrain the current frame straight lines according to the direction of the previous frame straight lines to judge whether the included angle between the two frames of straight lines exceeds a preset angle threshold;

[0020] If the included angle between the two frames of straight lines exceeds the preset angle threshold, discard the current frame straight lines and remove the power line points from the current line channel;

[0021] If the included angle between the two frames of straight lines does not exceed the preset angle threshold, determine that the straight line segment corresponding to the current frame straight lines is a valid line segment;

[0022] Obtain the central coordinates of the nearest tower pole point, and constrain the valid line segment according to the positional relationship between the central coordinates of the nearest tower pole point and the valid line segment;

[0023] Constrain the valid line segment according to the distance between the valid line segment and the power line points of the current line channel;

[0024] Extract the effective line segments after constraint and repeat the above steps until all the straight lines are extracted.

[0025] Preferably, in the above power line locking and tracking processing method, the steps of obtaining the power line points and tower points corresponding to the current line channel include:

[0026] Obtain the current position and flight direction of the aircraft;

[0027] Select the line channel in the three-dimensional space that is closest to the current position and flight direction of the aircraft;

[0028] Select the power line points and tower points corresponding to the current line channel from the point cloud data.

[0029] Preferably, in the above power line locking and tracking processing method, the step of using the straight line equations of all the straight lines in the current line channel and selecting the straight line with the highest voting score as the target tracking straight line according to the preset voting score mechanism includes:

[0030] Use the straight line equations of all the straight lines in the current line channel to calculate the distances from each straight line to the eccentric coordinate;

[0031] Vote and score the distances from each straight line to the eccentric coordinate according to the preset voting mechanism, and select the straight line with the highest voting score as the target tracking straight line;

[0032] Constrain the target tracking straight line to obtain the straight line equation of the target tracking straight line.

[0033] Preferably, in the above power line locking and tracking processing method, the step of calculating the distances from each straight line to the eccentric coordinate includes:

[0034] Use the starting coordinates of all the straight lines in the line channel to calculate the center point coordinates of the line channel;

[0035] Use the point coordinates of the nearest points from the center point coordinates to all the straight lines to calculate the eccentric coordinate;

[0036] Calculate the distances from each straight line to the eccentric coordinate.

[0037] Preferably, the step of voting and scoring the distances from each straight line to the eccentric coordinate by the above preset voting score mechanism includes:

[0038] Substitute the three-dimensional distance, height, and averaged distance from each straight line to the eccentric coordinate into the straight line score formula corresponding to the preset voting score mechanism to obtain the score corresponding to each straight line;

[0039] Select the straight line with the highest score as the target tracking straight line.

[0040] Preferably, in the above power line locking and tracking processing method, the steps of constraining the target tracking straight line to obtain the straight line equation of the target tracking straight line include:

[0041] Using a preset line priority principle to select the target tracking straight line in real time, and obtaining multiple frames of straight lines corresponding to the target tracking straight line;

[0042] Using the straight line of the previous frame in the multiple frames of straight lines to constrain the straight line of the current frame to obtain the straight line equation of the straight line of the current frame, which is used as the straight line equation of the target tracking straight line.

[0043] According to the second aspect of the present invention, the present invention further provides a system for using lidar to perform power line locking and tracking processing, which is characterized by including:

[0044] A point cloud acquisition module, configured to use lidar to collect laser point clouds in real time to obtain the point cloud data;

[0045] A point cloud segmentation module, configured to use a deep learning model to segment the laser point clouds included in the point cloud data to obtain power line points and tower points corresponding to the current line channel;

[0046] An extraction and clustering module, configured to cluster the tower points in the current line channel, and use the clustered tower points to calculate and generate the central point coordinates of the tower points;

[0047] A straight line extraction module, configured to use a RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines in the current line channel according to the central point coordinates of the tower points, and obtain the straight line equations of all straight lines in the current line channel;

[0048] A target tracking straight line selection module, configured to select a target tracking straight line according to the straight line equations of all straight lines in the current line channel;

[0049] A deviation angle judgment module, configured to judge whether the deviation angle of the target tracking straight line is less than or equal to a preset standard deviation threshold;

[0050] A flight prediction module, configured to, when it is determined that the deviation angle is less than or equal to the preset standard deviation threshold, use the target tracking straight line to predict the flight prediction position of the aircraft.

[0051] According to the third aspect of the present invention, the present invention further provides a system for using lidar to perform power line locking and tracking processing, including:

[0052] A memory, a processor, and a program for using lidar to perform wire-locking tracking processing of power lines and stored on the memory and executable on the processor. When the program for using lidar to perform wire-locking tracking processing of power lines is executed by the processor, the steps of the method for using lidar to perform wire-locking tracking processing of power lines according to any one of the above technical solutions are implemented.

[0053] In summary, the power line wire-locking tracking processing solution provided by the above technical solution of the present application uses lidar to collect laser point clouds in real time to obtain point cloud data, and uses a deep learning model to segment the laser point clouds included in the point cloud data to obtain power line points and tower points corresponding to the line channel. Then, the tower points in the current line channel are extracted and clustered, and the center point coordinates of the tower points are generated using the clustered tower points. Then, a RANSAC multi-line extraction algorithm with constraint conditions is used to extract a straight line from the center point coordinates of the tower points, and the target tracking straight line in the line channel can be obtained. Using this target tracking straight line, the flight prediction position of the aircraft can be accurately predicted. Since the technical solution of the present application uses lidar to obtain point cloud data, and then obtains power line points and tower points corresponding to the current line channel, it can ensure stable wire-locking tracking. In addition, the random sample consensus RANSAC multi-line extraction algorithm is used to sample the center point coordinates of the tower points to extract a straight line in the line channel, and with constraint conditions, the straight line can be extracted quickly and accurately. And the RANSAC multi-line extraction algorithm has higher efficiency and less memory occupancy compared with the hough transform algorithm. Therefore, it can solve the problem that the hough algorithm in the prior art has high requirements for the computing power and computing resources of the computing unit. Description of the Drawings

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0055] Figure 1 It is a schematic flowchart of the first method for using lidar to perform wire-locking tracking processing of power lines provided by an embodiment of the present invention;

[0056] Figure 2 is Figure 1 It is a schematic flowchart of a method for selecting a line channel provided by the shown embodiment;

[0057] Figure 3 is Figure 1 It is a schematic flowchart of a method for extracting and clustering tower points provided by the shown embodiment;

[0058] Figure 4 is Figure 1 A schematic flowchart of a method for extracting a straight line of the center point coordinates of a pole tower point provided by the illustrated embodiment;

[0059] Figure 5 A schematic flowchart of a second method for processing power line locking and tracking using lidar provided by an embodiment of the present invention;

[0060] Figure 6 is Figure 5 A schematic flowchart of a method for selecting a target tracking straight line provided by the illustrated embodiment;

[0061] Figure 7 is Figure 6 A schematic flowchart of a straight line scoring method provided by the illustrated embodiment;

[0062] Figure 8 is Figure 5 A schematic flowchart of a method for obtaining a straight line equation provided by the illustrated embodiment;

[0063] Figure 9 A schematic structural diagram of a power line locking and tracking processing system based on the RANSAC multi-line extraction algorithm provided by the first embodiment of the present invention;

[0064] Figure 10 A schematic structural diagram of a power line locking and tracking processing system based on the RANSAC multi-line extraction algorithm provided by the second embodiment of the present invention.

[0065] The realization, functional features and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the accompanying drawings. Detailed implementation manners

[0066] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] The main solution of the embodiment of the present invention is:

[0068] Most of the existing straight line extraction algorithms use the hough transform algorithm. However, the hough transform algorithm has low efficiency, large memory occupation, and high requirements for the computing power and computing resources of the computing unit, which limits the application scope of hough. For example, the computing unit carried by the unmanned aerial vehicle platform has relatively weak computing power and extremely precious computing resources, and using the hough transform is not conducive to the deployment of the unmanned aerial vehicle platform.

[0069] To solve the above problems, the following embodiments of the present invention provide a power line locking and tracking processing solution based on the RANSAC multi-line extraction algorithm. By using the random sample consensus (RANSAC) multi-line extraction algorithm to sample the power line points in the line channel, the straight lines in the line channel are extracted, and the extraction efficiency is relatively high. Moreover, due to the small memory occupation of random sampling, it can solve the problem that the Hough algorithm in the prior art has relatively high requirements for the computing power and computing resources of the computing unit.

[0070] To achieve the above object, please refer to Figure 1 , Figure 1 which is a schematic flowchart of the first lidar power line locking and tracking processing method provided by the embodiments of the present invention. As Figure 1 shown, the power line locking and tracking processing method includes:

[0071] S110: Use a lidar to collect lidar point clouds in real time to obtain point cloud data. Using a lidar to collect point clouds is not affected by light, and can achieve long-distance point cloud collection, solving the problems that the existing camera image collection is easily affected by light and requires close-range shooting, otherwise the collected image is insufficient or unclear and the lines in the image cannot be recognized.

[0072] S120: Use a deep learning model to segment the lidar point clouds included in the point cloud data to obtain the power line points and tower points corresponding to the current line channel.

[0073] Because there are often some noises in the line channels predicted by deep learning (such as the line channels of power lines), and multiple lines in the scene may be predicted and extracted. At this time, it is necessary to track and lock the lines in the line channel, and this requires line imitation initialization. Specifically, determine the general direction of the UAV flight, select the line (such as a power line) that is closest to the current position and direction of the UAV, and if the same line channel is locked twice in a row, the initialization is successful, and the flight direction and line channel are determined.

[0074] Specifically, as a preferred embodiment, as Figure 2 shown, the step of obtaining the power line points and tower points corresponding to the current line channel includes:

[0075] S121: Obtain the current position and flight direction of the aircraft;

[0076] S122: Select the line channel in the three-dimensional space that is closest to the current position and flight direction of the aircraft as the current line channel;

[0077] S123: Select the power line points and tower points corresponding to the current line channel from the point cloud data.

[0078] By selecting the line that is closest to the current position and flight direction of the aircraft, for example, the line passing through the coordinates of the current position of the aircraft and having an included angle with the flight direction not exceeding a predetermined angle threshold, when the same line channel is locked twice in a row, the line channel is determined.

[0079] In addition, after selecting the line channel in the three-dimensional space, in Figure 1 the lidar power line locking and tracking processing method shown also includes:

[0080] S130: Cluster the tower pole points in the current line channel, and use the clustered tower pole points to calculate and generate the central point coordinates of the tower pole points.

[0081] Taking the power line channel as an example, the embodiments of the present application can extract multiple frames of power line channels, and each frame of power line channel contains multiple tower pole points. In this way, the power line channel extracted in the previous frame can be used to constrain the power line channel in the next frame to obtain the coordinates of almost all the tower pole points on the power line channel. After obtaining the coordinates of a large number of tower pole points, the tower pole points can be clustered. The embodiments of the present application use the DBSCAN algorithm to cluster the tower pole points. After successful clustering, using the coordinates of a large number of tower pole points, according to a certain algorithm (such as weighted average, or selecting the tower pole point with the closest coordinate distance to the average value), the central point coordinates of the tower pole points can be generated.

[0082] Specifically, as a preferred embodiment, as Figure 3 shown, the steps of extracting and clustering the tower pole points in the line channel, and generating the central point coordinates of the tower pole points using the coordinates of the clustered tower pole points include:

[0083] S131: Extract the tower pole points within a predetermined distance range of the current line channel, where the tower pole points are the tower pole points detected and output by the deep learning model for the line channel.

[0084] S132: Use the DBSCAN algorithm to perform clustering operations on the tower pole points, and select the category with the largest number of tower pole points.

[0085] S133: Calculate the average of all the tower pole point coordinates in the category with the largest number of tower pole points to obtain the central point coordinates of the tower pole points.

[0086] Among them, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a relatively representative density-based clustering algorithm. Different from partitioning and hierarchical clustering methods, it defines a cluster as the largest set of density-connected points, can divide the area with sufficient high density into clusters, and can discover clusters of any shape in a spatial database with noise.

[0087] Taking the extraction and clustering of tower points as an example, in the embodiment of the present application, the power line channel extracted from the previous frame is used to constrain the tower points output by the deep learning model for each subsequent frame, and the tower points beyond a certain distance range are filtered out, leaving the points of the current line. Then, the remaining tower points are accumulated and clustered using the DBSCAN algorithm, and the largest category is selected as the tower point category. After the number of clustered points reaches a certain threshold, it is confirmed that the tower is recognized and valid. At this time, the center point coordinates of the tower are calculated using the coordinates of the points after clustering.

[0088] After generating the center point coordinates of the tower points using the clustered tower points, Figure 1 The power line locking and tracking processing method provided by the embodiment shown further includes the following steps:

[0089] S140: Use the RANSAC multi-line extraction algorithm with constraint conditions to extract all the straight lines of the current line channel according to the center point coordinates of the tower points, and obtain the straight line equations of all the straight lines in the current line channel.

[0090] RANSAC multi-line extraction algorithm with constraint conditions:

[0091] The random sample consensus (RANSAC) multi-line extraction algorithm is an algorithm that calculates the mathematical model parameters of the data based on a set of sample data sets containing abnormal data to obtain valid sample data. This algorithm has the characteristics of high sampling efficiency and small resource consumption. Using the RANSAC multi-line extraction algorithm to extract straight lines from the power line points (including some noise points) output by the deep learning model, straight lines in the three-dimensional space scene can be obtained, and the corresponding straight line equations can be generated based on these straight lines. Taking the power channel as an example, due to the possible presence of noise points in the point cloud of the power channel and when passing through the tower, the points on both sides of the tower are on different straight lines. Using the traditional RANSAC multi-line extraction algorithm, it is easy to count the points that are not on the same straight line as the same straight line, resulting in certain errors. In addition, the classic RANSAC multi-line extraction algorithm often easily extracts the same straight line in the line as multiple straight lines. Therefore, the embodiment of the present application proposes a RANSAC multi-line extraction algorithm with constraint conditions, which uses the straight lines and tower coordinates of the previous frame to constrain the current straight line, and at the same time screens during the extraction of multiple straight lines to prevent the same straight line from being detected as multiple straight lines, and finally realizes the accurate extraction of the straight lines of the line.

[0092] Specifically, as a preferred embodiment, as Figure 4 shown, the steps of using the RANSAC multi-line extraction algorithm with constraint conditions to extract straight lines include:

[0093] S141: Use the RANSAC multi-line extraction algorithm to extract the power line points corresponding to the current line channel to obtain the straight lines of the current frame;

[0094] S142: If the included angle between the two-frame straight lines exceeds the preset angle threshold, discard the current-frame straight line and remove the power line points from the current line channel.

[0095] S143: If the included angle between the two-frame straight lines does not exceed the preset angle threshold, determine that the straight line segment corresponding to the current-frame straight line is a valid segment.

[0096] S144: Obtain the central coordinates of the nearest tower pole point, and constrain the valid segment according to the positional relationship between the central coordinates of the nearest tower pole point and the valid segment.

[0097] S145: Constrain the valid segment according to the distance between the valid segment and the power line points in the current line channel.

[0098] S146: Extract the constrained valid segment and repeat the above steps until all straight lines are extracted.

[0099] In summary, the steps for extracting all the straight lines of the current line channel based on the central point coordinates of the tower pole points are as follows:

[0100] 1. Use the traditional RANSAC multi-line extraction algorithm to extract a straight line from the original power line points (including some noise points) given by the model.

[0101] 2. Constrain the current straight line using the direction of the previous-frame straight line (if there is no previous-frame straight line, do not perform this operation). If the included angle between the two straight lines exceeds the threshold, it is considered that the current straight line has a large deviation and may be too affected by noise. This straight line is discarded, and the corresponding points are removed as outliers from the original power line points, and RANSAC straight line extraction is no longer performed, and step 1 is repeated. If the included angle does not exceed the limit, then this straight line segment is a valid segment.

[0102] 3. Calculate the point on the straight line equation that is closest to the tower pole. If the tower pole point falls in the middle of the straight line segment, then the current line is actually two lines, and it is necessary to use the tower pole to divide the straight line segment into two segments, select one of them, and put the points of the remaining segment into the original power line points for the next extraction. If the tower pole is not in the middle of the straight line segment but is far away on both sides, then the current straight line is considered to be actually only one line and does not require additional processing.

[0103] 4. Using the current straight line equation, set a threshold d, and calculate the distance from the original power line points to the current straight line. If the distance is less than the distance d, then the point is considered a point close to the straight line or a point on the straight line, and use step 3 to complete the line segment division to obtain the points near the current straight line. Remove the points near the current straight line from the original power line points and no longer participate in the straight line extraction, so as to prevent a single straight line from being extracted as multiple straight lines.

[0104] 5. Save the current straight line and repeat the above steps until all straight lines are extracted.

[0105] Note: The above is the process of the RANSAC multi-line extraction algorithm. If the corresponding calculation formulas and calculation methods are needed, they can be provided. (Such as the calculation of the nearest point of a straight line segment, the calculation of the included angle between straight lines, the calculation of the distance from a point to a line, etc.).

[0106] In the technical solution provided by the embodiment of the present application, during the process of randomly selecting points by RANSAC and continuously adding points, the straight line equation is calculated, and the nearest coordinate P of the tower pole points obtained in the above steps on this straight line equation is judged. If this coordinate P is in the middle of the straight line segment after adding the tower pole points, then the subsequent points that are not on the effective line segment (the range from the starting position of the unmanned aerial vehicle to the tower pole position is the effective range) should not be used as straight line points. Otherwise, they are added to the straight line and used as inliers.

[0107] Then, the direction of the straight line in the previous frame is used to constrain the extraction of the current straight line. If the included angle between the two straight lines does not exceed a certain angle threshold, then when randomly selecting power line points for straight line extraction in the current frame, the power line points that fall on the straight line are selected and added to the straight line as inliers. Otherwise, they should not be added to the straight line to prevent large errors caused by excessive noise of channel points.

[0108] Since there are often multiple straight lines in the line channel, for example, in the power line channel, after each straight line is extracted, the straight line points corresponding to this straight line are removed, and the remaining points are left to repeat the above process again until all straight lines are extracted.

[0109] S150: According to the preset voting score mechanism, use the straight line equations of all straight lines in the current line channel, and select the straight line with the highest voting score as the target tracking straight line.

[0110] S160: Judge whether the deviation angle and distance of the target tracking straight line are less than or equal to the preset standard deviation threshold; if so, execute step S170; if not, return to execute step S110 to start the processing of the straight line in the next frame.

[0111] S170: If it is determined that the deviation angle and distance are less than or equal to the preset standard deviation threshold, then use the target tracking straight line to predict the flight prediction position of the aircraft.

[0112] The flight prediction position, that is, the prediction method of the flight trajectory points, is specifically as follows:

[0113] In order to realize the autonomous flight of the unmanned aerial vehicle, after using the unmanned aerial vehicle lidar point cloud to extract the best line straight line equation, the next flight position can be calculated according to the straight line equation. Let the expression of the space straight line equation be:

[0114] X=nx ·t + X0

[0115] Y = n y ·t + Y0

[0116] Z = n z ·t + Z0

[0117] where (n x , n y , n z ) is the direction of the straight line, (X0, Y0, Z0) is a point on the straight line, and t is an arbitrary scalar.

[0118] Then, based on the current point position coordinates of the aircraft, the straight line direction, and the predicted distance t, the position of the next point, i.e., the flight prediction position, can be calculated.

[0119] In summary, for the power line locking and tracking processing method provided by the above technical solution of the present application, by selecting a line channel in three-dimensional space, then extracting and clustering the tower pole points in the line channel, using the clustered tower pole points to generate the central point coordinates of the tower pole points, and then using the RANSAC multi-line extraction algorithm with constraint conditions to extract straight lines from the power line points in the channel, the straight lines in the line channel can be obtained. Since the technical solution of the present application uses the random sample consensus RANSAC multi-line extraction algorithm to sample the power line points in the channel to extract the straight lines in the line channel and has constraint conditions, it can quickly and accurately extract straight lines, and the RANSAC multi-line extraction algorithm has higher efficiency and less memory occupancy compared with the hough transform algorithm. Therefore, it can solve the problem that the hough algorithm in the prior art has high requirements for the computing power and computing resources of the computing unit.

[0120] Because the line channel contains a large number of straight lines, after extracting the straight lines of the line channel, the straight line with the smallest deviation angle and distance can be selected for aircraft navigation applications.

[0121] Specifically, as a preferred embodiment, as Figure 5 shown, for the power line locking and tracking processing method provided by the embodiment of the present application, the step of selecting the straight line equation of all straight lines in the current line channel according to the preset voting score mechanism and selecting the straight line with the highest voting score as the target tracking straight line specifically includes:

[0122] S210: Using the straight line equations of all straight lines in the current line channel, calculate the distances from each straight line to the eccentric coordinates.

[0123] Among them, as Figure 6 shown, the step of using the straight line equations of all straight lines in the current line channel to calculate the distances from each straight line to the eccentric coordinates includes:

[0124] S211: Filter the straight lines in the line channel according to the position information of the line channel to remove the straight lines not in the line channel;

[0125] S212: Calculate the center point coordinates of the line channel using the starting coordinates of all the straight lines in the line channel;

[0126] S213: Calculate the eccentricity coordinates using the point coordinates of the closest points from the center point coordinates to all the straight lines;

[0127] S214: Calculate the distances from all the straight lines to the eccentricity coordinates.

[0128] In the embodiment of the present application, it is necessary to filter the straight lines in the scene. Usually, the extracted straight lines are the channels of all the existing ground features in the ground scene, such as the channels in the parallel lines. It is necessary to filter out the straight lines in other irrelevant channels, and use the information such as the distance and angle of the line channel locked by the above-mentioned aircraft for constraint to remove the lines not in the current channel, so as to obtain all the straight lines in the current line.

[0129] After obtaining all the straight lines in the current line, it is necessary to select and lock a specific line. Because there are multiple straight lines in a channel, it is necessary to lock the same line during the flight of the aircraft to maintain stable flight.

[0130] Therefore, after calculating the distances from each straight line to the eccentricity coordinates, Figure 5 The method shown also includes the following steps:

[0131] S220: Vote and score the distances from each straight line to the eccentricity coordinates according to a preset vote score mechanism, and select the straight line with the highest vote score as the target tracking straight line.

[0132] Specifically, as a preferred embodiment, as Figure 7 shown, the step of scoring the distances from each straight line to the eccentricity coordinates according to the preset vote score mechanism includes:

[0133] S221: Substitute the three-dimensional distance, height, and averaged distance from each straight line to the eccentricity coordinates into the straight line score formula corresponding to the preset vote score mechanism to obtain the score corresponding to each straight line;

[0134] S222: Select the straight line with the highest score as the target tracking straight line.

[0135] The specific line selection, line locking, and tracking algorithm process is as follows:

[0136] During the flight of the drone, real-time tracking of the line requires locking and tracking the same power line in the channel. Otherwise, during the switching process among multiple lines, large deviations will occur in the predicted flight position, ultimately leading to unstable flight. Therefore, a line-locking tracking method based on a preset voting score mechanism is proposed to calculate the score of each line, and the line with the highest score is used as the best line in the current frame.

[0137] The detailed process of the line-locking tracking algorithm is as follows:

[0138] Calculate the average center point coordinate Pc based on the starting and ending coordinates of all straight line segments in the line channel.

[0139] Using the center point coordinate Pc, calculate the coordinate Pf of the nearest point from the center point coordinate to all straight lines.

[0140] Calculate the azimuth angle using the center point coordinate Pc and the direction from the center point coordinate to the nearest point Pf on the straight line, obtain the azimuth angle from the center point coordinate to each straight line, and sort all azimuth angles from smallest to largest.

[0141] After the above sorting, calculate the average of the nearest point coordinates Pf in the first half to obtain the eccentric coordinate Pe. At this time, the line division is completed, and only one side of the lines on the left and right sides of the channel center will be selected.

[0142] Calculate the 3D distance d from all straight lines to the eccentric coordinate Pe, and obtain dmin and dmax. Calculate the score Sc of each straight line as Sc = (1 - (d - dmin) / (dmax - dmin)) * 100. The closer the line is to the channel center, the higher the score, and vice versa. That is, the line closer to the center is tracked preferentially.

[0143] Calculate the height H of all straight lines at the eccentric coordinate Pe, and obtain Hmin and Hmax. Calculate the score Sh of all straight lines as Sh = (H - Hmin) / (Hmax - Hmin) * 100. The higher the line in the line, the higher the score, and vice versa. That is, the higher line is tracked preferentially.

[0144] Calculate the distances from N points at the eccentric coordinate Pe of all straight lines and equidistant from Pe along the straight line direction to the straight line in the previous frame (if it exists), take the average to obtain the distance D (i.e., the above-mentioned averaged distance), and obtain Dmin and Dmax. Calculate the score Sr of each straight line as Sr = (1 - (D - Dmin) / (Dmax - Dmin)) * 100. The closer the straight line segment in the line is to the previous frame, the higher the score, and vice versa. That is, the straight line closest to the previous frame is tracked preferentially, and this process realizes the tracking of the same straight line.

[0145] Calculate the score S of each straight line as S = Sc + Sh + Sr, and select the straight line with the highest score as the line to be locked for flight.

[0146] Calculate the deviation between the current straight line and the straight line in the previous frame, calculate the included angle between the two straight lines, and equally spaced sample within the line segment range to calculate the distance between the two straight lines. If the included angle and the distance exceed the threshold range, it is considered that the current best straight line is not acceptable. Otherwise, obtain the best straight line for the next trajectory prediction.

[0147] Note: The above process is the thread tracking algorithm.

[0148] After selecting the target tracking straight line as the target tracking straight line of the aircraft, Figure 5 The straight line extraction method in the three-dimensional space shown further includes the following steps:

[0149] S230: Constrain the target tracking straight line to obtain the straight line equation of the target tracking straight line.

[0150] As a preferred embodiment, as Figure 8 shown, the steps of constraining the target tracking straight line to obtain the straight line equation of the target tracking straight line include:

[0151] S231: Use the preset line priority principle to select the target tracking straight line in real time to obtain multiple frames of straight lines corresponding to the target tracking straight line;

[0152] S232: Use the straight line of the previous frame to constrain the straight line of the current frame to obtain the straight line equation of the current frame as the straight line equation of the target tracking straight line.

[0153] The embodiment of the present application adopts the preset line priority principle of higher height and always giving priority to the left or right line. After successful initialization, the straight line of the previous frame is used to constrain and lock the straight line of the current frame to obtain the straight line equation of the current frame.

[0154] Based on the same concept of the above method embodiment, the embodiment of the present invention also proposes a power line thread tracking processing system based on the RANSAC multi-line extraction algorithm to implement the above method of the present invention. Since the principle of solving problems by this system embodiment is similar to the method, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be repeated here one by one.

[0155] See Figure 9 , Figure 9 is a schematic structural diagram of a power line thread tracking processing system based on the RANSAC multi-line extraction algorithm provided by the embodiment of the present invention. As Figure 9 shown, the system for using lidar to perform power line thread tracking processing is characterized by including:

[0156] A point cloud acquisition module 110, configured to use lidar to collect lidar point clouds in real time to obtain the point cloud data;

[0157] A point cloud segmentation module 120, configured to segment the laser point cloud included in the point cloud data by using a deep learning model, and obtain power line points and tower points corresponding to the current line channel;

[0158] An extraction and clustering module 130, configured to cluster the tower points in the current line channel, and calculate and generate the central point coordinates of the tower points by using the clustered tower points;

[0159] A straight line extraction module 140, configured to use a RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines of the current line channel according to the central point coordinates of the tower points, and obtain the straight line equations of all straight lines in the current line channel;

[0160] A target tracking straight line selection module 150, configured to select a target tracking straight line according to the straight line equations of all straight lines in the current line channel;

[0161] A deviation angle judgment module 160, configured to judge whether the deviation angle of the target tracking straight line is less than or equal to a preset standard deviation threshold;

[0162] A flight prediction module 170, configured to, when it is determined that the deviation angle is less than or equal to the preset standard deviation threshold, use the target tracking straight line to predict the flight prediction position of the aircraft.

[0163] The power line locking and tracking processing system provided by the embodiment of the present application selects a line channel in a three-dimensional space, then extracts and clusters the tower points in the line channel, uses the clustered tower points to generate the central point coordinates of the tower points, and then uses a RANSAC multi-line extraction algorithm with constraint conditions to perform straight line extraction on the power line points predicted by the deep learning model, so as to obtain the straight lines in the line channel. Since the technical solution of the present application uses the random sample consensus RANSAC multi-line extraction algorithm to sample the power line points to extract the straight lines in the line channel and has constraint conditions, it can quickly and accurately extract the straight lines, and the RANSAC multi-line extraction algorithm has higher efficiency and less memory occupation compared with the hough transform algorithm. Therefore, it can solve the problem that the hough algorithm in the prior art has high requirements for the computing power and computing resources of the computing unit.

[0164] In addition, as Figure 10 shown, the embodiment of the present invention also provides a structural schematic diagram of a power line locking and tracking processing system. The power line locking and tracking processing system includes:

[0165] A communication line 1002, a communication module 1003, a memory 1004, a processor 1001, and a power line lock tracking processing program stored in the memory 1004 and executable on the processor 1001. When the power line lock tracking processing program is executed by the processor 1001, it implements the steps of the power line lock tracking processing method provided by any of the above technical solutions.

[0166] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0168] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0170] It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim listing several devices, several of these devices may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0171] Although the preferred embodiments of the present invention have been described, additional changes and modifications can be made by those skilled in the art once they learn the basic creative concept. Therefore, the appended claims are intended to be construed to cover the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0172] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for using lidar to perform wire-locking tracking processing on power lines, characterized in that, Including: Using a lidar to collect laser point clouds in real time to obtain point cloud data; Using a deep learning model to segment the laser point clouds included in the point cloud data to obtain power line points and tower points corresponding to the current line channel; Clustering the tower points in the current line channel, and using the clustered tower points to calculate and generate the central point coordinates of the tower points; Using a RANSAC multi-line extraction algorithm with constraint conditions, and extracting all straight lines of the current line channel according to the central point coordinates of the tower points to obtain the straight line equations of all straight lines in the current line channel; According to a preset voting score mechanism, using the straight line equations of all straight lines in the current line channel, selecting the straight line with the highest voting score as the target tracking straight line; Judging whether the deviation angle and distance of the target tracking straight line are less than or equal to a preset standard deviation threshold; If it is determined that the deviation angle and distance are less than or equal to the preset standard deviation threshold, then using the target tracking straight line to predict the flight prediction position of the aircraft; The step of using a RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines of the current line channel according to the central point coordinates of the tower points includes: Using the RANSAC multi-line extraction algorithm to extract the power line points corresponding to the current line channel to obtain the straight lines of the current frame; Obtaining the straight lines of the previous frame of the straight lines of the current frame, and constraining the straight lines of the current frame according to the direction of the straight lines of the previous frame to judge whether the included angle between the two frames of straight lines exceeds a preset angle threshold; If the included angle between the two frames of straight lines exceeds the preset angle threshold, then discarding the straight lines of the current frame and removing the power line points from the current line channel; If the included angle between the two frames of straight lines does not exceed the preset angle threshold, then determining the straight line segment corresponding to the straight lines of the current frame as a valid line segment; Obtaining the central coordinates of the nearest tower point, and constraining the valid line segment according to the positional relationship between the central coordinates of the nearest tower point and the valid line segment; Constraining the valid line segment according to the distance between the valid line segment and the power line points of the current line channel; Extracting the constrained valid line segment and repeating the above steps until all straight lines are extracted.

2. The method for power line locking and tracking processing according to claim 1, characterized in that The step of clustering the tower points in the current line channel and using the clustered tower points to calculate and generate the central point coordinates of the tower points includes: Extracting the tower points located within a predetermined distance range of the current line channel; Using the DBSCAN algorithm to perform a clustering operation on the tower points, and selecting the category with the largest number of tower points; Using the coordinates of all tower points in the category with the largest number of tower points to perform an average calculation to obtain the central point coordinates of the tower points.

3. The method for power line locking and tracking processing according to claim 1, characterized in that The step of obtaining the power line points and tower points corresponding to the current line channel includes: Obtaining the current position and flight direction of the aircraft; Selecting the line channel in the three-dimensional space that is closest to the current position and flight direction of the aircraft; Selecting the power line points and tower points corresponding to the current line channel from the point cloud data.

4. The method for power line wire-locking tracking processing according to claim 1 or 3, characterized in that The step of selecting the straight line with the highest voting score as the target tracking straight line by using the straight line equations of all the straight lines in the current line channel according to the preset voting score mechanism includes: Calculating the distances from each straight line to the eccentric coordinates by using the straight line equations of all the straight lines in the current line channel; Voting and scoring the distances from each straight line to the eccentric coordinates according to the preset voting mechanism, and selecting the straight line with the highest voting score as the target tracking straight line; Constraining the target tracking straight line to obtain the straight line equation of the target tracking straight line.

5. The method for power line lock-in tracking processing according to claim 4, wherein The step of calculating the distances from each straight line to the eccentric coordinates includes: Calculating the center point coordinates of the line channel by using the starting coordinates of all the straight lines in the line channel; Calculating the eccentric coordinates by using the point coordinates of the nearest points from the center point coordinates to all the straight lines; Calculating the distances from each straight line to the eccentric coordinates.

6. The method for power line lock-wire tracking processing according to claim 5, characterized in that The step of voting and scoring the distances from each straight line to the eccentric coordinates according to the preset voting score mechanism includes: Substituting the three-dimensional distance, height and averaged distance from each straight line to the eccentric coordinates into the straight line score formula corresponding to the preset voting score mechanism to obtain the scores corresponding to each straight line; Selecting the straight line with the highest score as the target tracking straight line.

7. The method for power line lock-in tracking processing according to claim 5, characterized in that, The step of constraining the target tracking straight line to obtain the straight line equation of the target tracking straight line includes: Real-time selecting the target tracking straight line by using the preset line priority principle to obtain multiple frames of straight lines corresponding to the target tracking straight line; Using the straight line of the previous frame in the multiple frames of straight lines to constrain the straight line of the current frame to obtain the straight line equation of the current frame as the straight line equation of the target tracking straight line.

8. A system for using lidar to perform wire locking and tracking processing on power lines, characterized in that, Including: A point cloud acquisition module for using a lidar to real-time collect a laser point cloud to obtain point cloud data; A point cloud segmentation module for using a deep learning model to segment the laser point cloud included in the point cloud data to obtain the power line points and tower points corresponding to the current line channel; An extraction and clustering module for clustering the tower points in the current line channel and calculating and generating the center point coordinates of the tower points by using the clustered tower points; A straight line extraction module for using a RANSAC multi-line extraction algorithm with constraint conditions to extract all the straight lines of the current line channel according to the center point coordinates of the tower points to obtain the straight line equations of all the straight lines in the current line channel; A target tracking straight line selection module for selecting a target tracking straight line according to the straight line equations of all the straight lines in the current line channel; A deviation angle judgment module for judging whether the deviation angle of the target tracking straight line is less than or equal to a preset standard deviation threshold; A flight prediction module for predicting the flight prediction position of the aircraft by using the target tracking straight line when it is determined that the deviation angle is less than or equal to the preset standard deviation threshold. The straight line extraction module, when performing the step of "using the RANSAC multi-line extraction algorithm with constraint conditions to extract all straight lines of the current line channel according to the central coordinates of the tower pole points", is specifically used to use the RANSAC multi-line extraction algorithm to extract the power line points corresponding to the current line channel to obtain the straight lines of the current frame; obtain the straight lines of the previous frame of the current frame straight lines, and constrain the current frame straight lines according to the direction of the previous frame straight lines to determine whether the included angle between the two frames of straight lines exceeds a preset angle threshold; If the included angle between the two frames of straight lines exceeds the preset angle threshold, discard the current frame straight lines and remove the power line points from the current line channel; If the included angle between the two frames of straight lines does not exceed the preset angle threshold, determine that the straight line segment corresponding to the current frame straight lines is a valid line segment; Obtain the central coordinates of the nearest tower pole point, and constrain the valid line segment according to the positional relationship between the central coordinates of the nearest tower pole point and the valid line segment; constrain the valid line segment according to the distance between the valid line segment and the power line points of the current line channel; extract the constrained valid line segment and repeat the above steps until all straight lines are extracted.

9. A system for using lidar to perform wire-locking tracking processing on power lines, characterized in that, It includes: A memory, a processor, and a program for using lidar to perform line locking and tracking processing of power lines stored on the memory and executable on the processor. When the program for using lidar to perform line locking and tracking processing of power lines is executed by the processor, it implements the steps of the method for using lidar to perform line locking and tracking processing of power lines according to any one of claims 1 to 7.

Images