Power line three-dimensional reconstruction method based on binocular depth estimation

Abstract

The invention discloses a power line three-dimensional reconstruction method based on binocular depth estimation. The method comprises the following steps: obtaining depth information of an image based on binocular depth estimation; removing abnormal outliers in the depth information by using a statistical filtering algorithm; extracting a power line by using a linear model and a clustering algorithm; and performing multi-dimensional fitting reconstruction on the power line based on an extraction result, and establishing a three-dimensional channel of the power line. According to the method, computer vision and a digital image processing technology are combined, the images captured by the binocular camera are subjected to analysis binocular depth estimation calculation, the power line channel is reconstructed in a three-dimensional mode, and the method has the advantages of being high in universality, low in cost, high in accuracy and the like.

Description

Technical Field

[0001] The invention relates to the field of computer vision, and in particular to a method for three-dimensional reconstruction of power lines based on binocular depth estimation. Background Art

[0002] The 3D reconstruction of power lines based on binocular depth estimation uses image processing and computer vision technology to estimate the depth of power lines using images obtained by binocular cameras, thereby establishing a 3D point cloud model of power lines. Binocular depth estimation reconstructs the 3D structure of power lines by capturing two views, left and right, and calculating depth information using the principle of parallax.

[0003] At present, the transmission line inspection of my country's power grid is mainly based on manual inspection. This inspection method has high labor intensity, difficult working conditions, low inspection efficiency, and is difficult to manage. It can no longer meet the needs of the development and safe operation of modern power grids, and the cost of use and maintenance is high. With the advancement of computer vision technology and the development of ranging devices such as binocular cameras, image-based 3D reconstruction of power lines has become an effective method.

[0004] In this method, the image is first processed using a binocular depth estimation algorithm, and the depth information of the image is obtained through parallax calculation, and then the three-dimensional point cloud information of the power line is constructed. The binocular depth estimation technology can provide accurate power line position and shape information, thereby accurately reconstructing the three-dimensional structure of the power line. Image-based computer vision technology can perform three-dimensional reconstruction of power lines in real time to ensure the accuracy and reliability of the power line structure.

[0005] In order to improve the accuracy and real-time performance of 3D reconstruction of power lines, a feasible method is to obtain the image of the power line in real time through the binocular camera module, and then calculate the depth information of the image through the binocular depth estimation algorithm, combine the obtained depth information with the power line image, calculate the 3D point cloud coordinates of the power line, and more accurately reconstruct the 3D structure of the power line in 3D space. This method can establish 3D point cloud data of the power line in real time through the binocular camera module, which is lower cost and faster. Summary of the invention

[0006] In order to improve the accuracy of tree barrier intrusion detection and enhance anti-interference capability, the present invention provides a tree barrier intrusion detection method based on semantic segmentation. The tree area is obtained based on an image segmentation algorithm, and the image depth information obtained by the depth estimation algorithm is combined to obtain the three-dimensional point cloud area coordinates of the tree. The three-dimensional coordinates of the power lines are obtained by scanning the power lines with a laser radar. The three-dimensional point cloud coordinates of the trees and the three-dimensional coordinates of the power lines are combined to more accurately detect the tree barrier intrusion phenomenon.

[0007] To achieve the above object, the present invention discloses a method for three-dimensional reconstruction of power lines based on binocular depth estimation, the steps comprising:

[0008] Step S10: obtaining depth information of the input image based on a binocular depth estimation algorithm;

[0009] Step S101: First, by comparing the similarity between the to-be-matched point and the candidate matching point, a search range D is set for each pixel to be matched, and the search range is limited to D. A matrix of size W×H×D (W and H are the width and height of the image) is used to record the matching cost value of each pixel with the pixel within the disparity range;

[0010] Step S102: performing a cost aggregation operation on the initial matching cost to optimize the matching result, thereby reducing the sensitivity to noise and obtaining a more accurate match;

[0011] Step S103: Using the winner-take-all (WTA) algorithm, select the disparity value corresponding to the minimum aggregation cost value for each pixel as the final disparity. Generate a disparity map of the same size as the original image based on the calculation results, and convert the disparity map into a depth map when the internal and external parameters of the image are known, indicating the position of each pixel in the three-dimensional space;

[0012] Step S104: Optimize the disparity through the left-right consistency method to reduce matching errors and improve matching accuracy. Compare the disparity values ​​of the corresponding points of the left and right images. If the difference between the two is less than a certain threshold, it is considered a successful match. Otherwise, it is considered as a disparity that does not meet the uniqueness constraint and is excluded;

[0013] Step S20: using a statistical filtering algorithm to remove abnormal outliers in the depth information;

[0014] Step S201: Determine a spherical neighborhood range N(p) with a fixed radius r for each depth point p:

[0015] N(P)={q∈D|||pq||≤r}

[0016] Among them, q is the pixel point in the neighborhood range, and D is the set of all pixels in the depth map.

[0017] Within the neighborhood range N(p), count the depth values ​​Z of all depth points and calculate the mean μ and standard deviation σ:

[0018]

[0019]

[0020] Among them, Z(q) is the depth value of point q in the neighborhood, and |N(p) is the number of neighborhood points.

[0021] Step S202: Calculate the difference between the depth value Z(p) of each depth point p and the neighborhood depth mean μ:

[0022] ΔZ(p)=|Z(p)-μ|

[0023] Compare the difference ΔZ(p) with the preset threshold T. If the difference exceeds the threshold T, the depth point is marked as an error outlier and the depth value of the depth point is replaced by the neighborhood mean μ.

[0024] Step S30: extracting power lines using a straight line model and a clustering algorithm;

[0025] Step S301: First, use the Canny edge detector to extract edge features to obtain an edge map, and then use the Hough transform to detect straight lines from the edge map. The Hough transform can map points in the image space to straight lines in the parameter space, and detect straight lines by finding local maxima in the parameter space:

[0026] ρ=xcosθ+ysinθ

[0027] Where ρ is the distance from the line to the origin, θ is the angle of the line, and (x, y) are the coordinates in the image.

[0028] Step S302: Cluster the straight lines detected in S301 using the density-based DBSCAN clustering algorithm to effectively process noise points and automatically determine the number of clusters:

[0029] DBSCAN(∈,MinPts)

[0030] Among them, ∈ is the distance threshold and MinPts is the minimum number of points in each cluster.

[0031] Step S303: Use the least squares method to fit a straight line model to each cluster obtained in S02, and calculate the sum of squared errors:

[0032]

[0033] Among them, m, b are the slope and intercept of the straight line, (x i ,y i ) are points in the cluster.

[0034] Solve for m and b by minimizing the sum of squared errors:

[0035]

[0036]

[0037] Step S40: Perform multi-dimensional fitting reconstruction on the power lines based on the extraction results to establish a three-dimensional channel of the power lines.

[0038] Step S401: convert the power lines extracted in step S30 into three-dimensional point cloud data, and convert the two-dimensional image coordinates into three-dimensional space coordinates using the depth information in the depth map and the internal and external parameters of the image.

[0039] Step S402: Fit the three-dimensional point cloud data using the least squares polynomial fitting method:

[0040]

[0041] Among them, P(t) is the parametric equation of the fitting curve, a i is the fitting coefficient and t is the curve parameter.

[0042] Step S403: establishing a three-dimensional channel model of electric power lines according to the fitted three-dimensional curve. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The present invention is a flowchart of the steps of a method for three-dimensional reconstruction of power lines based on binocular depth estimation. DETAILED DESCRIPTION

[0044] The following describes the implementation of the present invention through specific examples and in conjunction with the accompanying drawings. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific examples, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention.

[0045] Step S10: obtaining depth information of the input image based on a binocular depth estimation algorithm;

[0046] Step S101: First, by comparing the similarity between the to-be-matched point and the candidate matching point, a search range D is set for each pixel to be matched, and the search range is limited to D. A matrix of size W×H×D (W and H are the width and height of the image) is used to record the matching cost value of each pixel with the pixel within the disparity range;

[0047] Step S102: performing a cost aggregation operation on the initial matching cost to optimize the matching result, thereby reducing the sensitivity to noise and obtaining a more accurate match;

[0048] Step S103: Using the winner-take-all (WTA) algorithm, select the disparity value corresponding to the minimum aggregation cost value for each pixel as the final disparity. Generate a disparity map of the same size as the original image based on the calculation results, and convert the disparity map into a depth map when the internal and external parameters of the image are known, indicating the position of each pixel in the three-dimensional space;

[0049] Step S104: Optimize the disparity through the left-right consistency method to reduce matching errors and improve matching accuracy. Compare the disparity values ​​of the corresponding points of the left and right images. If the difference between the two is less than a certain threshold, it is considered a successful match. Otherwise, it is considered as a disparity that does not meet the uniqueness constraint and is excluded;

[0050] Step S20: using a statistical filtering algorithm to remove abnormal outliers in the depth information;

[0051] Step S201: Determine a spherical neighborhood range N(p) with a fixed radius r for each depth point p:

[0052] N(P)={q∈D|||pq||≤r}

[0053] Among them, q is the pixel point in the neighborhood range, and D is the set of all pixels in the depth map.

[0054] Within the neighborhood range N(p), count the depth values ​​Z of all depth points and calculate the mean μ and standard deviation σ:

[0055]

[0056]

[0057] Among them, Z(q) is the depth value of point q in the neighborhood, and |N(p) is the number of neighborhood points.

[0058] Step S202: Calculate the difference between the depth value Z(p) of each depth point p and the neighborhood depth mean μ:

[0059] ΔZ(p)=|Z(p)-μ|

[0060] Compare the difference ΔZ(p) with the preset threshold T. If the difference exceeds the threshold T, the depth point is marked as an error outlier and the depth value of the depth point is replaced by the neighborhood mean μ.

[0061] Step S30: extracting power lines using a straight line model and a clustering algorithm;

[0062] Step S301: First, use the Canny edge detector to extract edge features to obtain an edge map, and then use the Hough transform to detect straight lines from the edge map. The Hough transform can map points in the image space to straight lines in the parameter space, and detect straight lines by finding local maxima in the parameter space:

[0063] ρ=xcosθ+ysinθ

[0064] Where ρ is the distance from the line to the origin, θ is the angle of the line, and (x, y) are the coordinates in the image.

[0065] Step S302: Cluster the straight lines detected in S301 using the density-based DBSCAN clustering algorithm to effectively process noise points and automatically determine the number of clusters:

[0066] DBSCAN(∈,MinPts)

[0067] Among them, ∈ is the distance threshold and MinPts is the minimum number of points in each cluster.

[0068] Step S303: Use the least squares method to fit a straight line model to each cluster obtained in S02, and calculate the sum of squared errors:

[0069]

[0070] Among them, m, b are the slope and intercept of the straight line, (x i ,y i ) are points in the cluster.

[0071] Solve for m and b by minimizing the sum of squared errors:

[0072]

[0073]

[0074] Step S40: Perform multi-dimensional fitting reconstruction on the power lines based on the extraction results to establish a three-dimensional channel of the power lines.

[0075] Step S401: convert the power lines extracted in step S30 into three-dimensional point cloud data, and convert the two-dimensional image coordinates into three-dimensional space coordinates using the depth information in the depth map and the internal and external parameters of the image.

[0076] Step S402: Fit the three-dimensional point cloud data using the least squares polynomial fitting method:

[0077]

[0078] Among them, P(t) is the parametric equation of the fitting curve, a i is the fitting coefficient and t is the curve parameter.

[0079] Step S403: A three-dimensional channel model of the power line is established according to the fitted three-dimensional curve, and the curve is discretized into a plurality of small segments, each of which is represented by a pipeline structure to form a continuous three-dimensional channel.

Claims

1. A method for 3D reconstruction of power lines based on binocular depth estimation, characterized in that: The following steps are involved: S10: Obtaining depth information of the input image based on a binocular depth estimation algorithm; S20: using a statistical filtering algorithm to remove abnormal outliers in the depth information; S30: extracting power lines using a straight line model and a clustering algorithm; S40: Perform multi-dimensional fitting reconstruction on the power lines based on the extraction results to establish a three-dimensional channel of the power lines.

2. The method for obtaining image depth information using a binocular depth estimation algorithm according to claim 1, characterized in that: Using the binocular depth estimation network, after a certain amount of training, the power transmission line and tree obstacle scenes are learned so that it can output accurate depth information.

3. The method of removing abnormal outliers in depth information by using a statistical filtering algorithm according to claim 1, characterized in that: The statistical filtering method is used to perform statistical analysis on the neighborhood of each point, and the points exceeding the threshold are regarded as error outliers. Outlier replacement is used to change the value of the abnormal outlier to the mean of its neighborhood.

4. The method for extracting power lines using a straight line model and a clustering algorithm according to claim 1, characterized in that: The linear features of power lines are used to distinguish them from other interferences in the image. Edge detection and Hough transform are used to preliminarily extract linear distribution features and roughly extract power lines. Then, a clustering algorithm is used to separate single power lines, especially the top and bottom power lines.

Images