Method and device for detecting energized main electric wire based on point cloud

Abstract

The present invention discloses a method and device for detecting live main wires based on point clouds, which have strong anti-interference ability and high recognition accuracy. The method for detecting live main wires of the present invention comprises the following steps: (10) point cloud pre-processing: performing through filtering, outlier removal and voxel filtering pre-processing on the three-dimensional point cloud obtained from the main cable area, and extracting the target point cloud without noise within the operation range; (20) point cloud Hough transform filtering: using Hough transform, further filtering the pre-processed point cloud, and extracting the effective point cloud with main cable characteristics; (30) effective point cloud post-processing: segmenting the effective point cloud, performing DBASCAN clustering on each segment of the point cloud, performing straight line fitting on each type of point cloud, performing fitting voting on the fitted straight line, and according to the fitting voting results, counting the associated point clouds and fitting the main line.

Description

Technical Field

[0001] The present invention belongs to the technical field of on-line detection of high-voltage cables, and particularly relates to a method and device for detecting a live main cable based on point cloud. Background Art

[0002] A live working robot is a special robot that replaces manual labor to complete a series of high-risk operations such as high-voltage cable connection and disassembly in the air. Compared with traditional manual operations, it greatly improves the operation safety and efficiency. During the operation process, the live working robot needs to obtain the accurate direction and position of the main cable of the operation, which depends on the equipped area array laser. However, when the area array laser irradiates the main cable, it will be interfered by sunlight, and the collected point cloud data will contain a large amount of point cloud noise, affecting the accurate detection of the main cable.

[0003] The existing laser point cloud main cable detection algorithms mainly include: a main cable detection method based on the region growing algorithm, a straight cable detection method based on the point cloud straight line fitting algorithm, and a main point cloud detection method based on deep learning.

[0004] For the main cable detection method based on the region growing algorithm, a seed point cloud continuously spreads and grows to find the point cloud within a connected region, and the point cloud within this segment is fitted using a spline curve. However, when there are large connected point clouds near the main line, this method will still consider the interfering point cloud block as part of the main line.

[0005] For the straight cable detection method based on the point cloud straight line fitting algorithm, the least squares method or the RANSC algorithm is mainly used to fit the straight line. This method can fit well when there are few noise point clouds. However, when there is too much noise data, when using the least squares method or the RANSC algorithm to fit the straight line, the fitted straight line will deviate towards the position of the noise.

[0006] For the main point cloud detection method based on deep learning, this method uses deep learning to train a point cloud segmentation model to complete the extraction of the main line. However, training a deep learning point cloud segmentation model requires collecting a large amount of training data and requires a lot of time to train a model with good results. In addition, during the model inference process, it consumes a lot of computing resources, resulting in an overly slow detection time.

[0007] In summary, the problems existing in the prior art are: when a live working robot uses an area array laser to identify the main line of a high-voltage cable, due to the interference of sunlight, noise point clouds appear near the main line, resulting in low recognition accuracy of the main cable. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for detecting a live main cable based on point cloud, which has strong anti-interference ability and high recognition accuracy.

[0009] Another object of the present invention is to provide a point cloud-based live main wire detection device with strong anti-interference ability and high recognition accuracy.

[0010] The technical solution to achieve the purpose of the present invention is:

[0011] A method for detecting live main electric wires based on point cloud comprises the following steps:

[0012] (10) Point cloud pre-processing: Perform through filtering, outlier removal, and voxel filtering pre-processing on the three-dimensional point cloud acquired from the main cable area to extract the target point cloud without noise within the operating range;

[0013] (20) Point cloud Hough transform filter: Use Hough transform to further filter the pre-processed point cloud and extract the valid point cloud with main line cable features;

[0014] (30) Post-processing of valid point clouds: segment the valid point clouds, perform DBASCAN clustering on each segment of the point cloud, perform straight line fitting on each type of point cloud, perform fitting voting on the fitted straight lines, and count the related point clouds and fit the main line based on the fitting voting results.

[0015] The technical solution to achieve another purpose of the present invention is:

[0016] A point cloud-based live main electric line detection device, comprising:

[0017] Point cloud pre-processing module, used to perform through filtering, outlier removal and voxel filtering pre-processing on the three-dimensional point cloud acquired from the main line cable area, and extract the target point cloud without noise within the operation range;

[0018] The point cloud Hough transform filter module is used to further filter the pre-processed point cloud using Hough transform to extract the valid point cloud with main line cable features;

[0019] The effective point cloud post-processing module is used to segment the effective point cloud, perform DBASCAN clustering on each segment of the point cloud, perform straight line fitting on each type of point cloud, perform fitting voting on the fitted straight line, and count the associated point clouds and fit the main line according to the fitting voting results. Compared with the prior art, the present invention has the following significant advantages:

[0020] 1. Strong anti-interference ability and high recognition accuracy: The present invention adopts point cloud pre-processing to improve the quality of extracted point cloud; by extracting only point cloud with main line cable characteristics for main line fitting, the direction of the main line cable can be accurately identified in the case of solar radiation interference, which effectively improves the operational reliability of the live working robot, has strong anti-interference ability and high recognition accuracy.

[0021] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the main flow chart of the live main electric line detection method based on point cloud of the present invention.

[0023] Figure 2 It is a schematic diagram of the structure of the coordinate system of the area array laser.

[0024] Figure 3 yes Figure 1 Flowchart of the point cloud pre-processing steps.

[0025] Figure 4 yes Figure 1 Flowchart of the Hough transform filtering steps in the point cloud.

[0026] Figure 5 yes Figure 1 Flowchart of the steps for effective point cloud post-processing.

[0027] Figure 6 It is an example of the main line position fitted using the existing region growing algorithm and the least squares method.

[0028] Figure 7 This is an example of the main line position detected by using the live main line detection method based on point cloud of the present invention. DETAILED DESCRIPTION

[0029] like Figure 1 The figure shows the main flow chart of the live main line detection method based on point cloud of the present invention. The method of the present invention is applicable to a live working robot, which is equipped with a surface array laser for collecting point cloud data of the main line cable. The surface array laser is installed on the robot arm. In the working scene, the main line cable is above the robot. The robot arm is controlled to raise to collect point cloud data, and the position information of the main line is obtained by using the method of the present invention, and the grabbing actuator is called for execution. Figure 2 This is the coordinate system of the area array laser. The irradiation angle is below the main line and is easily disturbed by the sun.

[0030] like Figure 1 As shown, the method for detecting live main electric wires based on point cloud of the present invention comprises the following steps:

[0031] (10) Point cloud pre-processing: Perform through filtering, outlier removal, and voxel filtering pre-processing on the three-dimensional point cloud acquired from the main cable area to extract the target point cloud without noise within the operating range;

[0032] In this step, the original point cloud data is subjected to straight-through filtering to extract the point cloud within the effective operating range, and then the density of the point cloud is counted to remove the interference of outliers. Finally, voxel filtering is used to reduce the scale of the point cloud.

[0033] As Figure 3 shown, the (10) point cloud pre - processing steps include:

[0034] (11) Pass - through filtering: Obtain the three - dimensional point cloud within the main cable region through an area array laser, perform pass - through filtering using a pass - through filter, and extract the region of interest R, where R ∈ (-x, -y, -z) ~ (x, y, z).

[0035] (12) Outlier removal: Remove the interfering points from the existing point cloud. Count the number s of point clouds within the neighborhood of each point cloud r with a radius in the R region. When s < n, consider this point as an outlier and remove it from R (note: r and n are hyperparameters).

[0036] (13) Voxel filtering: Perform voxel filtering on R using a grid of size (leaf, leaf, leaf) to obtain the V point cloud, where leaf is the size of the voxel;

[0037] Through voxel filtering, the number of point clouds can be reduced, and the operation speed can be improved.

[0038] (20) Point cloud Hough transform filtering: Use the Hough transform to further filter the pre - processed point cloud and extract the effective point cloud with the characteristics of the main cable.

[0039] In this step, the point cloud is compressed onto the xOy plane, converted to the image coordinate system to obtain a grayscale image. The grayscale value is the number of point clouds within the range of this pixel. Then, use the Progressive Probabilistic Hough Transform (PPHT) to detect lines on the grayscale image. For the multiple detected lines, considering that the main cable has a certain width and there are multiple parallel lines detected, all the detected lines are clustered by DBSCAN according to the slope. Extract the lines in each category where there are more than two adjacent parallel lines and add them to the main cable list. Select the longest line in the list as the best main cable. Convert the selected line into a plane in three - dimensional space, calculate the distance from the point cloud to this plane, and extract the point cloud within the neighborhood of this plane.

[0040] As Figure 4 shown, the (20) point cloud Hough transform filtering step includes:

[0041] (21) Point cloud conversion: Compress the three - dimensional V point cloud after voxel filtering into a two - dimensional image and convert the point cloud into a grayscale image;

[0042] Specifically, it includes: Project the V point cloud onto the xOy plane (as shown in the figure), compress the three - dimensional point cloud into a two - dimensional image, set the size of the image to (w, h), and convert the coordinates (x i , y i , z i ) of each point in the point cloud to the coordinates in the image coordinate system Count the number of point clouds p contained in each coordinate (i, j) of the statistical image ij , p max is the maximum value of p ij , calculate the pixel value of the current image coordinate (i, j) as In this way, the point cloud is converted into a grayscale image.

[0043] (22) Optimal straight line detection of grayscale image: Use the cumulative probability Hough transform method to detect the grayscale image, obtain multiple straight lines, then perform clustering and statistics, and select the longest straight line among all the main lines obtained by statistics as the optimal straight line.

[0044] Specifically:[[]]

[0045] Use the cumulative probability Hough transform (PPHT) to detect straight lines for the two-dimensional image, use the DBSACAN algorithm to cluster the slopes of the multiple detected straight lines for the main lines, and set the angle threshold angle thresh and the distance threshold angle thresh , calculate the angle angle between every two straight lines in each category ij , when angle ij ≤angle thresh , it is considered that the two straight lines are approximately parallel, and then calculate the distance d between the two straight lines ij , when d ij ≤d thresh , it is considered that these two straight lines are similar parallel straight lines. When the number of approximate parallel straight lines in each category exceeds two, it is considered that these similar parallel main lines belong to one category of main lines, and the longest straight line among all the main lines obtained by statistics is selected as the optimal straight line.

[0046] (23) Extract effective point cloud: Extract the effective point cloud near the straight line to form an effective point cloud set H.

[0047] Specifically: Extract the effective point cloud near the straight line, convert the detected straight line line in the image to a plane ax + by + d = 0 in the actual coordinate system, and calculate the distance from the point (x i , y i , z i ) in the V point cloud to the plane When d < d thresh , extract the point cloud to form a new point cloud set H.

[0048] (30) Post-processing of effective point cloud: Segment the effective point cloud, perform DBASCAN clustering on each segment of the point cloud, perform straight line fitting on each category of point cloud, perform fitting voting on the fitted straight line, and according to the fitting voting result, count the associated point cloud and fit the main line.

[0049] Perform segmented processing on the effective point cloud extracted in step (20) along the plane direction, and calculate the variance σ of each segment of the point cloud along the z direction, and remove the interfering point cloud with a large variance.

[0050] Cluster the segmented point cloud processed in the previous step using the DBSCAN algorithm, and then use the least squares method to fit the direction vector and starting point of the straight line for each category of point cloud respectively.

[0051] Then vote on all the fitted straight lines in the previous step. The voting score is the weighted sum of the angles and z-direction distances between any two straight lines. When the score is greater than the threshold, each of the two straight lines gets one vote. After statistics, select the point cloud of the straight line with the highest number of votes and the straight lines whose voting scores with it are greater than the threshold.

[0052] Perform a least squares fit on the overall point cloud statistically obtained in the previous step to obtain the direction and position of the main cable, which is used for subsequent operation processes.

[0053] As Figure 5 shown, the post-processing steps of the (30) effective point cloud include:

[0054] (31) Effective point cloud segmentation: According to the orientation and attributes of the main cable, divide the H point cloud into n segments along the plane direction, calculate the variance σ of each segment of the point cloud in the z direction. When σ > σ thresh , remove this segment of the point cloud from H, and then perform the following processing on each segment of the point cloud respectively.

[0055] (32) Fitting of segmented straight lines: Use the DBSCAN algorithm with a radius of radius and a minimum number of clustering points of n for each segment of the point cloud for clustering to obtain m clusters of point clouds. Respectively use the least squares method to fit a straight line to these m clusters of point clouds to obtain the parameters of the straight line (line min , line start , k), where line end is the starting point of the fitted straight line, line start is the ending point of the fitted straight line, and k is the direction vector of the straight line. After calculation, obtain end groups of parameters.

[0056] (33) Fitting of the main line: Vote according to the k of groups of parameters, and use the weighted sum of the cosine distance and the deviation in the z direction between two categories of point clouds to measure the similarity between two straight lines, and calculate the scores between each pair of these straight lines

[0057]

[0058] When score > score thresh ​When this happens, the vote counts of the two lines are incremented by one. The line point cloud with the highest vote count and the line point clouds whose scores calculated with it reach the threshold are used to form a new point cloud S. The least squares method is used to fit the final line direction and position for the point cloud S.

[0059] The fitted main line direction and position are passed to the robot to complete the precise positioning of the main line.

[0060] Figure 6 、 Figure 7 The white points in [Figure number] are the point cloud, and the straight lines are the detected main lines. As shown in [Figure number], it is the position of the main line fitted by the region growing algorithm and the least squares method. It can be seen that the large block of point cloud near the main line has bent the correct direction of the main line. Figure 6 shown, is the detection result of the method of the present invention, which can better fit the direction and position of the main line. Figure 7 is the detection result of the method of the present invention, which can better fit the direction and position of the main line.

[0061] The charged main wire detection device based on point cloud of the present invention is used to implement the aforementioned charged main wire detection method based on point cloud, and includes:

[0062] A point cloud preprocessing module for performing preprocessing of passing through a filter, removing outliers, and voxel filtering on the three-dimensional point cloud obtained from within the main wire cable area range, and extracting the target point cloud without noise within the operation range;

[0063] The point cloud preprocessing module includes:

[0064] A passing through filter unit for obtaining a three-dimensional point cloud within the main wire cable area range through a planar array laser, performing passing through filtering using a passing through filter, and extracting the region of interest R, where R ∈ (-x, -y, -z) ~ (x, y, z);

[0065] An outlier removal unit for removing interfering points from the existing point cloud, counting the number s of point clouds within the neighborhood of each point cloud r with a radius within the R region. When s < n, the point is considered an outlier and removed from R, where r and n are hyperparameters;

[0066] A voxel filtering unit for performing voxel filtering on R using a grid of (leaf, leaf, leaf) size to obtain the V point cloud, where leaf is the size of the voxel.

[0067] A point cloud Hough transform filtering module for further filtering the preprocessed point cloud using the Hough transform and extracting the effective point cloud with the characteristics of the main wire cable;

[0068] The point cloud Hough transform filtering module includes:

[0069] A point cloud conversion unit for compressing the three-dimensional V point cloud after voxel filtering into a two-dimensional image and converting the point cloud into a grayscale image;

[0070] The grayscale image optimal straight line detection unit is used to detect the grayscale image by using the cumulative probability Hough transform method, obtain multiple straight lines, then perform clustering and statistics, and select the longest straight line among all the main lines obtained by statistics as the optimal straight line;

[0071] The effective point cloud extraction unit is used to extract the effective point cloud near the straight line to form an effective point cloud set H. The effective point cloud post-processing module is used to segment the effective point cloud, perform DBSCAN clustering on each segment of the point cloud, perform straight line fitting on each class of point cloud, perform fitting voting on the fitted straight lines, and according to the fitting voting results, count the associated point cloud and fit out the main line.

[0072] The effective point cloud post-processing module includes:

[0073] The effective point cloud segmentation unit is used to divide the H point cloud into n segments along the direction of the plane according to the direction and attributes of the main line cable, calculate the variance σ of the z direction of each segment of the point cloud. When σ > σ thresh , remove this segment of the point cloud from H, and then perform the following processing on each segment of the point cloud respectively;

[0074] The segmented straight line fitting unit is used to cluster each segment of the point cloud using the DBASCAN algorithm with a radius of radius and a minimum number of clustering points of n min to obtain m clusters of point cloud, and respectively use the least squares method to fit out a straight line for these m clusters of point cloud to obtain the parameters of the straight line (line start , line end , k), where line start is the starting point of the fitted straight line, line end is the ending point of the fitted straight line, and k is the direction vector of the straight line. After calculation, groups of parameters are obtained;

[0075] The main line fitting unit is used to vote according to the k of the groups of parameters, use the cosine distance and the weighted deviation of the z direction of two classes of point cloud to measure the similarity of two straight lines, and calculate the scores between each pair of these straight lines

[0076]

[0077] When score > score thresh , the votes of the two straight lines are incremented by one respectively. Select the line point cloud with the most votes and the line point cloud whose score calculated with it reaches the threshold to form a new point cloud S, and use the least squares method to fit out the final straight line direction and position for the point cloud S.

Claims

1. A method for detecting live main electric wires based on point cloud, comprising the following steps: (10) Point cloud pre-processing: Perform through filtering, outlier removal, and voxel filtering pre-processing on the three-dimensional point cloud acquired from the main cable area to extract the target point cloud without noise within the operating range; (20) Point cloud Hough transform filter: Use Hough transform to further filter the pre-processed point cloud and extract the valid point cloud with main line cable features; (30) Post-processing of valid point clouds: segment the valid point clouds, perform DBASCAN clustering on each segment of the point clouds, perform straight line fitting on each type of point clouds, perform fitting voting on the fitted straight lines, and count the associated point clouds and fit the main line based on the fitting voting results; The (10) point cloud pre-processing step includes: (11) Through-filtering: The three-dimensional point cloud within the main cable area is obtained by using a planar laser array, and a through-filter is used to perform through-filtering to extract the region of interest R, R∈(-x, -y, -z)~(x, y, z); (12) Outlier removal: Remove interference points from the existing point cloud, count the number of point clouds s within the radius of each point cloud in region R, and when s < n, the point is considered an outlier and removed from R, where r and n are hyperparameters; (13) Voxel filtering: Use a grid of size (leaf, leaf, leaf) to perform voxel filtering on R to obtain a point cloud V, where leaf is the size of the voxel; The (20) point cloud Hough transform filtering step comprises: (21) Point cloud conversion: compress the three-dimensional V point cloud after voxel filtering into a two-dimensional image and convert the point cloud into a grayscale image; (22) Grayscale image best straight line detection: The grayscale image is detected using the cumulative probability Hough transform method to obtain multiple straight lines, which are then clustered and counted, and the longest straight line among all the main lines of the class obtained is selected as the best straight line; (23) Valid point cloud extraction: Extract valid point clouds near the straight line to form a valid point cloud set H; It is characterized in that the (30) effective point cloud post-processing step includes: (31)Effective point cloud segmentation: According to the orientation and properties of the main cable, the H point cloud is divided into n segments along the surface direction, and the variance σ of the z direction of each segment of the point cloud is calculated. When σ > σ thresh , remove this segment of the point cloud from H, and then perform the following processing on each segment of the point cloud respectively; (32) Piecewise linear fitting: For each segment of point cloud, use the radius of radius and n min The DBSCAN algorithm with the minimum number of clustering points is used for clustering to obtain m clusters of point clouds. The least squares method is used to fit a straight line to each of these m clusters of point clouds to obtain the parameters of the straight line (line start , line end , k), where line start is the starting point of the fitted straight line, line end is the ending point of the fitted straight line, and k is the direction vector of the straight line. After the calculation, groups of parameters are obtained; (33) Main line fitting: According to Vote based on the k of the group of parameters, measure the similarity of two lines with the weighted cosine distance and the deviation of the two types of point clouds in the z direction, and calculate the scores between each pair of these lines When score > score thresh At this time, the vote counts of the two lines are incremented by one respectively. The line point cloud with the most votes and the line point cloud whose calculated score reaches the threshold with it form a new point cloud S. The least squares method is used to fit the final line direction and position for the point cloud S.

2. A point cloud-based live main wire detection device, comprising: Point cloud pre-processing module, used to perform through filtering, outlier removal and voxel filtering pre-processing on the three-dimensional point cloud acquired from the main line cable area, and extract the target point cloud without noise within the operation range; The point cloud Hough transform filter module is used to further filter the pre-processed point cloud using Hough transform to extract the valid point cloud with main line cable features; The effective point cloud post-processing module is used to segment the effective point cloud, perform DBASCAN clustering on each segment of the point cloud, perform straight line fitting on each type of point cloud, perform fitting voting on the fitted straight line, and count the associated point clouds and fit the main line according to the fitting voting results; The point cloud pre-processing module includes: A through-filter unit is used to obtain a three-dimensional point cloud within the main cable area through a planar array laser, and to extract an area of interest R using a through-filter, R∈(-x, -y, -z)~(x, y, z); An outlier removal unit is used to remove interfering points from the existing point cloud, count the number s of point clouds within the neighborhood of each point cloud r with a radius of r in the R region. When s < n, the point is considered an outlier and is removed from R. r and n are hyperparameters; A voxel filtering unit is used to perform voxel filtering on R using a grid of size (leaf, leaf, leaf) to obtain the V point cloud. leaf is the size of the voxel; The point cloud Hough transform filtering module includes: A point cloud conversion unit is used to compress the three-dimensional V point cloud after voxel filtering into a two-dimensional image and convert the point cloud into a grayscale image; A grayscale image optimal line detection unit is used to detect the grayscale image using the cumulative probability Hough transform method to obtain multiple lines, then perform clustering and statistics, and select the longest line among all the main lines of the statistically obtained classes as the optimal line; An effective point cloud extraction unit is used to extract the effective point cloud near the line to form an effective point cloud set H; It is characterized in that the effective point cloud post-processing module includes: An effective point cloud segmentation unit, which is used to divide the H point cloud into n segments along the direction of the plane according to the trend and attributes of the main cable, calculate the variance σ of each segment of the point cloud in the z direction, and when σ > σ thresh , remove this segment of the point cloud from H, and then perform the following processing on each segment of the point cloud respectively; The piecewise linear fitting unit is used to perform clustering on each segment of point cloud using the DBASCAN algorithm with a radius of radius and a minimum number of cluster points of n to obtain m clusters of point cloud. Then, the least squares method is used to fit a straight line to each of these m clusters of point cloud to obtain the parameters of the straight line (line min , line start , line end , k), where line start is the starting point of the fitted straight line, line end is the ending point of the fitted straight line, and k is the direction vector of the straight line. After the calculation, groups of parameters are obtained; The main line fitting unit is used to vote according to the k of a group of parameters, measure the similarity between two lines by the weighted cosine distance and the deviation of the two types of point clouds in the z direction, and calculate the scores between every two of these lines When score > score thresh At this time, the vote counts of the two lines are incremented by one respectively. The line point cloud with the most votes and the line point cloud whose calculated score reaches the threshold with it form a new point cloud S. The least squares method is used to fit the final line direction and position for the point cloud S.

Images