Power line reconstruction method and device based on laser point cloud and image fusion

By fusing the point cloud and image data of the power lines, virtual three-dimensional wire points are generated, which solves the problem of lidar obtaining incomplete power line data in complex scenarios, and achieves high-precision and high-efficiency three-dimensional reconstruction.

CN118918247APending Publication Date: 2024-11-08CHINA UNIV OF GEOSCIENCES (WUHAN)
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410939890.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In complex scenarios, it is difficult for lidar to accurately obtain distance information of power lines on dark or smooth surfaces, resulting in incomplete power line point clouds, making it difficult to achieve high-precision three-dimensional reconstruction, and relying solely on images is difficult to achieve high-precision three-dimensional reconstruction.

Method used

By obtaining the original point cloud data and image data of the power line, filtering and reconstruction, real three-dimensional wire points are generated, and projecting them to the image coordinate system, and using depth information fusion to generate virtual three-dimensional wire points, thereby determining the target three-dimensional parameters of the power line.

Benefits of technology

Three-dimensional high-precision reconstruction of incomplete power lines is realized, and the reconstruction efficiency is improved. Through the complementarity of images and point cloud data, the accuracy and efficiency of power line identification detection are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918247A_ABST
    Figure CN118918247A_ABST
Patent Text Reader

Abstract

The invention provides a power line reconstruction method and device based on laser point cloud and image fusion, and relates to the technical field of graphic image processing, and the method comprises the steps: obtaining original point cloud data and original image data of a to-be-detected power line, screening and reconstructing the original point cloud data, generating a real three-dimensional power line point, and sampling the original image data, generating two-dimensional wire points, projecting the real three-dimensional wire points to a preset image coordinate system, determining a target real three-dimensional wire point closest to each two-dimensional wire point, fusing the corresponding two-dimensional wire points by using depth information of the target real three-dimensional wire points, generating virtual three-dimensional wire points, and displaying the virtual three-dimensional wire points. And determining a target three-dimensional parameter of the to-be-detected power line based on the real three-dimensional wire point and the virtual three-dimensional wire point. Fusion analysis is carried out based on the two-dimensional image and the three-dimensional point cloud data, complementation of the two kinds of data is achieved, and the three-dimensional high-precision reconstruction effect of the incomplete power line is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of graphic image processing, and in particular to a method and device for reconstructing power lines based on laser point cloud and image fusion. Background Art

[0002] With the large-scale development of smart grids, power lines, as the main equipment of power transmission channels, play an important role in transmitting electricity. Tower detection is one of the commonly used power line safety monitoring methods. It obtains power line scene data by installing sensors such as lidar on the tower. However, the reflectivity of lidar may be different for objects of different materials and colors. In particular, for objects with dark or smooth surfaces, lidar may not be able to accurately obtain their distance information. In the face of obscured objects, the laser may not be able to directly illuminate the obscured objects and thus fail to obtain accurate information. Therefore, the power line point cloud actually collected is often missing or incomplete, making it difficult to perform high-precision three-dimensional reconstruction over long distances. In terms of environmental perception and scene understanding, cameras can provide rich visual information, including background, lighting, texture, etc., which helps to perceive and understand the environment, but lacks depth information, and it is difficult to achieve high-precision three-dimensional reconstruction based on images alone. How to strike a balance between high precision and high efficiency when identifying and detecting power lines in complex scenes has become an urgent problem to be solved. Summary of the invention

[0003] The purpose of the present invention is to solve the problem that it is difficult to achieve both high precision and high efficiency when identifying and detecting power lines in complex scenes. The present invention provides a power line reconstruction method and device based on laser point cloud and image fusion.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] A first aspect of an embodiment of the present application provides a method for reconstructing power lines based on laser point cloud and image fusion, comprising:

[0006] Obtaining original point cloud data and original image data of the power line to be tested;

[0007] Screening and reconstructing the original point cloud data to generate real three-dimensional wire points, and sampling the original image data to generate two-dimensional wire points;

[0008] Projecting the real three-dimensional wire points to a preset image coordinate system, determining the target real three-dimensional wire point closest to each of the two-dimensional wire points, and fusing the corresponding two-dimensional wire points using the depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point;

[0009] A target three-dimensional parameter of the power line to be measured is determined based on the real three-dimensional power line point and the virtual three-dimensional power line point.

[0010] Optionally, the obtaining of original point cloud data and original image data of the power line to be tested includes:

[0011] The original coordinates corresponding to the original point cloud data are converted to the direction of the maximum principal component based on the maximum principal component analysis method.

[0012] Optionally, the converting the original coordinates corresponding to the original point cloud data to the maximum principal component direction based on the maximum principal component analysis method includes:

[0013] Obtaining the mean value of the original point cloud data after filtering in each dimension;

[0014] Performing de-centroiding processing on the original point cloud data based on the mean value to obtain a covariance matrix;

[0015] Perform eigenvalue decomposition on the covariance matrix, determine the eigenvectors corresponding to the three largest eigenvalues, and determine the eigenvectors as the maximum principal component directions.

[0016] Optionally, the screening and reconstruction of the original point cloud data to generate real three-dimensional wire points includes:

[0017] Performing a rough extraction on the original point cloud data using a power line point cloud extraction method based on a dynamic threshold to obtain first point cloud data;

[0018] Using a Euclidean clustering algorithm to precisely extract the first point cloud data of each of the power lines to be tested, to obtain second point cloud data;

[0019] The second point cloud data is reconstructed using a model combining a straight line and a parabola, and after solving the model parameters based on the least squares method, equidistant sampling and completion are performed to generate real three-dimensional power line points.

[0020] Optionally, the sampling of the original image data to generate two-dimensional wire points includes:

[0021] Performing data enhancement on the original image data to obtain optimized image data;

[0022] Performing power line edge detection on the optimized image data, and performing two-dimensional power line equation fitting based on edge points obtained by edge detection to obtain a fitting equation;

[0023] Equidistant sampling is performed based on the fitting equation to generate two-dimensional electric wire points.

[0024] Optionally, projecting the real three-dimensional wire points to a preset image coordinate system, determining a target real three-dimensional wire point that is closest to each of the two-dimensional wire points, and fusing corresponding two-dimensional wire points using depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point includes:

[0025] Projecting the real three-dimensional wire point to a preset image coordinate system to obtain a pixel coordinate set and constructing a KD tree;

[0026] Filtering out pixel coordinates closest to each of the two-dimensional electric wire points from the KD tree, and determining the real three-dimensional electric wire point corresponding to the pixel coordinates as the target real three-dimensional electric wire point;

[0027] The corresponding two-dimensional electric wire points are fused using the depth information of the target real three-dimensional electric wire point to generate a virtual three-dimensional electric wire point.

[0028] Optionally, determining the target three-dimensional parameters of the power line to be measured based on the real three-dimensional power line point and the virtual three-dimensional power line point includes:

[0029] Constructing a real three-dimensional model and a virtual three-dimensional model based on the real three-dimensional wire points and the virtual three-dimensional wire points respectively;

[0030] Determining real model parameters and virtual model parameters of each power line from the real three-dimensional model and the virtual three-dimensional model;

[0031] A weighted processing is performed based on the difference between the real model parameter and the virtual model parameter to determine the target three-dimensional parameters of the power line to be measured.

[0032] A second aspect of the embodiment of the present application provides a power line reconstruction device based on laser point cloud and image fusion, comprising: an acquisition module, a generation module, a fusion module and a determination module, wherein:

[0033] The acquisition module is configured to acquire original point cloud data and original image data of the power line to be tested;

[0034] The generating module is configured to screen and reconstruct the original point cloud data to generate real three-dimensional wire points, and to sample the original image data to generate two-dimensional wire points.

[0035] The fusion module is configured to project the real three-dimensional wire points to a preset image coordinate system, determine the target real three-dimensional wire point closest to each of the two-dimensional wire points, and fuse the corresponding two-dimensional wire points using the depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point;

[0036] The determination module is configured to determine a target three-dimensional parameter of the power line to be measured based on the real three-dimensional power line point and the virtual three-dimensional power line point.

[0037] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the power line reconstruction method based on laser point cloud and image fusion as described in the first aspect.

[0038] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0039] Compared with the prior art, the technical solution provided by this application has the following beneficial effects:

[0040] The present invention provides a method and device for reconstructing electric lines based on laser point cloud and image fusion, which obtains the original point cloud data and original image data of the electric line to be tested, screens and reconstructs the original point cloud data, generates real three-dimensional electric line points, samples the original image data, generates two-dimensional electric line points, projects the real three-dimensional electric line points to a preset image coordinate system, determines the target real three-dimensional electric line points closest to each two-dimensional electric line point, fuses the corresponding two-dimensional electric line points using the depth information of the target real three-dimensional electric line points, generates virtual three-dimensional electric line points, and then determines the target three-dimensional parameters of the electric line to be tested based on the real three-dimensional electric line points and the virtual three-dimensional electric line points. By fusing and analyzing the two-dimensional image and three-dimensional point cloud data, the complementarity of the two data is achieved, and the three-dimensional high-precision reconstruction accuracy and reconstruction efficiency of the incomplete electric line are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a power line reconstruction method based on laser point cloud and image fusion provided in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of the structure of a power line reconstruction device based on laser point cloud and image fusion provided in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] Below, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present application.

[0045] The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "include", "comprising", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0046] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0047] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some blocks or combinations thereof in the block diagrams and / or flow charts may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions, when executed by the processor, may create a device for implementing the functions / operations described in these block diagrams and / or flow charts.

[0048] In some embodiments, see Figure 1 , Figure 1 A schematic flow chart of a method for reconstructing power lines based on laser point cloud and image fusion provided in an embodiment of the present application; the method for reconstructing power lines based on laser point cloud and image fusion provided in an embodiment of the present application includes:

[0049] S110, obtaining original point cloud data and original image data of the power line to be tested.

[0050] Here, the original point cloud data and original image data can be derived from actual on-site shooting. By installing the laser radar and camera on the power tower, the shooting direction is taken as the extension direction of the power line, and the radar installation angle is taken as a reference to the overall trend of the power line, and the angle can be adjusted within a small range. The relative position of the camera and radar is fixed, and the shooting is side by side. The camera shooting angle is consistent with the radar, so as to collect data.

[0051] In some embodiments, S110, obtaining original point cloud data and original image data of the power line to be tested includes:

[0052] Based on the maximum principal component analysis method, the original coordinates corresponding to the original point cloud data are converted to the maximum principal component direction.

[0053] The principal component analysis (PCA) method is essentially a basis transformation, which makes the transformed data have the largest variance. That is, by rotating the coordinate axes and translating the coordinate origin, the variance between one of the axes (the principal axis) and the data points is minimized. After the coordinate transformation, the orthogonal axes with high variance are removed to obtain a reduced-dimensional data set.

[0054] In this embodiment, the original coordinates corresponding to the original point cloud data are transformed based on the maximum principal component analysis method, three maximum principal component directions can be determined, three-dimensional coordinates are established, and the coordinate transformation is then completed.

[0055] In some embodiments, converting the original coordinates corresponding to the original point cloud data to the maximum principal component direction based on the maximum principal component analysis method includes:

[0056] Get the mean value of the original point cloud data after filtering in each dimension;

[0057] De-centroiding the original point cloud data based on the mean to obtain a covariance matrix;

[0058] Perform eigenvalue decomposition on the covariance matrix, determine the eigenvectors corresponding to the three largest eigenvalues, and determine the eigenvectors as the direction of the maximum principal component.

[0059] In one example, the original point cloud data is first filtered to avoid the interference of discrete point clouds on subsequent algorithms, and then the mean of each dimension of the point cloud data is calculated, as shown in the following formula (1-3), where n represents the number of point clouds, (x i ,y i ,z i ) represents the point cloud coordinates.

[0060]

[0061] By subtracting the corresponding dimension mean from each point cloud data, the centroid-removed matrix Xc can be constructed, as shown in formula (4).

[0062]

[0063] On this basis, the covariance matrix C after removing the centroid can be calculated, as shown in formula (5). The eigenvector and eigenvalue of the matrix C represent the principal component direction and the variance size in the principal component direction of the point cloud respectively.

[0064]

[0065] Perform eigenvalue decomposition on the covariance matrix and obtain the eigenvalue λ K and the eigenvector ν K , as shown in formula (6)

[0066] C K =λ K ν K (6);

[0067] By arranging the eigenvalues ​​obtained by eigendecomposition in order, the corresponding eigenvectors are the principal component directions of the data. Take the eigenvectors corresponding to the first three largest eigenvalues. These three principal components are the x-axis, y-axis, and z-axis directions in the new coordinate system from large to small.

[0068] S120, screening and reconstructing the original point cloud data to generate real three-dimensional wire points, and sampling the original image data to generate two-dimensional wire points.

[0069] Here, the original point cloud data is screened to remove noise points in the point cloud. If the point cloud data or the original image data is too dense, the data volume can be reduced by methods such as voxel grid downsampling to improve processing efficiency. According to the characteristics of the wire and the required reconstruction accuracy, select the appropriate point cloud reconstruction method.

[0070] In some embodiments, S120, screening and reconstructing the original point cloud data to generate real three-dimensional wire points includes:

[0071] A power line point cloud extraction method based on a dynamic threshold performs rough extraction on the original point cloud data to obtain first point cloud data;

[0072] The first point cloud data of each power line to be tested is precisely extracted using the Euclidean clustering algorithm to obtain the second point cloud data;

[0073] The second point cloud data is reconstructed using a model combining straight lines and parabolas, and after solving the model parameters based on the least squares method, equidistant sampling and completion are performed to generate real three-dimensional wire points.

[0074] In this embodiment, the rough extraction can be to realize the self-extraction of the power line point cloud based on the histogram analysis method for the rotated point cloud. The histogram analysis is performed along the z-axis to analyze the relationship between the elevation and the number of point clouds to determine the range of the elevation filter, and then the elevation filter is performed based on the parameter to realize the self-extraction of the power line. Specifically, the point cloud data is obtained in the new coordinate system along the z-axis direction threshold (Z min ,Z max ), divide the threshold interval into k intervals, and the length of each interval is N, as shown in formula (7).

[0075] N=(Zmax -Z min ) / k(7);

[0076] Record the number of point clouds in each interval and select the interval with the largest decrease in the number of point clouds (N imin ,N imax ), get the median value N of the interval mid , as shown in formula (8).

[0077]

[0078] The N obtained in the above steps mid and Z max The range of elevation filtering is used to extract the point cloud within the range, thereby realizing the extraction of power line point cloud based on dynamic threshold, avoiding the algorithm's dependence on empirical parameters, and realizing parameter adaptation.

[0079] After the rough extraction is completed to obtain the first point cloud data, the single power line point cloud is finely extracted by Euclidean clustering. The similarity of the points is determined by calculating the Euclidean distance between the points in the point cloud, and then similar points are clustered together to form clusters, thereby distinguishing each power line and obtaining the second point cloud data. After obtaining the power line point cloud, long-distance three-dimensional reconstruction is performed on each power line separately. Here, a model combining straight lines and parabolas is selected to reconstruct the second point cloud data. Straight line segments are usually located in the xy plane, while parabolic segments are used to describe the drooping or curved parts of the wires. The straight line segments are fitted using the least squares method, and the parameters of the straight line (such as slope and intercept) are determined by the center points of all the straight line segment sampling points. Based on the straight line segments, the parabola segments are fitted. According to the density and total length of the three-dimensional wire points to be generated, the appropriate sampling interval is calculated. Finally, the sampling points on the straight line segments and the parabola segments are merged to generate the real three-dimensional wire points.

[0080] In terms of model selection, the model is constructed as shown in formula (9-10).

[0081] y=k0x+b0(9);

[0082]

[0083] In the parameter solution part, the least square method is used to solve the parameters A, B, C, b0, and k. First, the error function E is constructed as shown in formula (11). Then the error function is differentiated and the derivative is set to 0 to obtain the optimal solution of the parameters b0 and k. The obtained expression is shown in formula (12-13). Where n represents the number of power line point clouds.

[0084]

[0085] After obtaining the fitting parameters of the xy plane of the power line, the point cloud coordinate system is rotated, and the x direction of the new coordinate system is set to the direction of the linear model of the power line. The parameters of formula (10) are solved by sampling the least squares method in the new coordinate system.

[0086] In some embodiments, S120, sampling the original image data to generate two-dimensional wire points includes:

[0087] Perform data enhancement on the original image data to obtain optimized image data;

[0088] Performing power line edge detection on the optimized image data, and performing two-dimensional power line equation fitting based on edge points obtained by edge detection to obtain a fitting equation;

[0089] Equidistant sampling is performed based on the fitting equation to generate two-dimensional wire points.

[0090] In this embodiment, the three channels of the power line image are firstly grayed to obtain a gray image, as shown in formula (14). Gray (x, y) is a gray image, and R, G, and B represent the three channel values ​​respectively.

[0091] Gray(x,y)=0.2989*R+0.587*G+0.114*B(14);

[0092] Then, the Canny edge detection is used to obtain the power line edges of the optimized image data. The main steps include:

[0093] (1) Filtering: The grayscale image is denoised by Gaussian filtering.

[0094] (2) Calculate the gradient: Calculate the gradient magnitude and direction through the Sobel filter to find the gradient of the image.

[0095] (3) Non-maximum suppression: Perform non-maximum suppression on the gradient amplitude map, retain the pixels with the largest local gradient, and suppress non-edge pixels.

[0096] (4) Dual threshold edge detection: Edge detection is performed on the image after non-maximum suppression by setting high and low thresholds.

[0097] The edge points of each power line are extracted and fitted with a two-dimensional parabola based on the least square method. The specific implementation steps are similar to the above-mentioned fitting polynomial equation. The fitted parabola is sampled at equal intervals to obtain the two-dimensional power line point set p on the two-dimensional image. v (x v ,y v ).

[0098] S130, projecting the real three-dimensional wire points to a preset image coordinate system, determining the target real three-dimensional wire point closest to each two-dimensional wire point, and fusing the corresponding two-dimensional wire points using the depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point.

[0099] Here, the data fusion of the camera and lidar is realized based on joint calibration. By finding the positional relationship between the lidar coordinate system and the camera coordinate system, the data of different sensors can be accurately matched in space.

[0100] In some embodiments, S130, projecting the real 3D wire point to a preset image coordinate system, determining the target real 3D wire point closest to each 2D wire point, and fusing the corresponding 2D wire points using the depth information of the target real 3D wire point to generate a virtual 3D wire point, including:

[0101] Project the real three-dimensional wire points to the preset image coordinate system, obtain the pixel coordinate set, and construct the KD tree;

[0102] Filter out the pixel coordinates closest to each two-dimensional electric wire point from the KD tree, and determine the real three-dimensional electric wire point corresponding to the pixel coordinates as the target real three-dimensional electric wire point;

[0103] The depth information of the target real 3D wire point is used to fuse the corresponding 2D wire point to generate a virtual 3D wire point.

[0104] In this embodiment, the above-mentioned power line point cloud coordinates P are obtained. L (X L , Y L , Z L ) to the camera coordinate system P C (X C , Y C , Z C ), then the conversion relationship between PL and PC is shown in formula (15). 3x3 is the rotation matrix, T 3x1 is the translation matrix, through the external parameter matrix M 4x4 The spatial synchronization of multiple sensors is achieved. Then PC in the camera coordinate system is converted to the image coordinate system, and then to the pixel coordinate system to obtain point P p (u l ,v l ), the conversion relationship is shown in formula (16), where K is the internal parameter matrix, (u0, v0) is the coordinate of the image coordinate origin in the pixel coordinate system, f is the focal length, and dx and dy are the sizes of a single pixel on the X and Y coordinate axes, respectively. This realizes the conversion of power line point cloud data to two-dimensional wire coordinates on the image.

[0105]

[0106] Based on the pixel coordinate set P p (u l ,v l ) constructs a KD tree, for the set p v Every point P in p (u l ,v l ) finds the point closest to it in the KD tree, and implements p according to this step v Search for the nearest point of each point in the p (u i ,v i )'s depth information is synchronized to p v , construct the set p v2 (x v ,y v ,dep i ), where dep i is the depth information. Based on the above formula (15-16), p v2 The pixel coordinate system is converted to the radar coordinate system, thus realizing the virtual three-dimensional wire point p 3D (x k ,y k ,z k ) is generated.

[0107] S140, determining target three-dimensional parameters of the power line to be measured based on the real three-dimensional power line points and the virtual three-dimensional power line points.

[0108] In this embodiment, the virtual wire is projected back to the point cloud coordinate system, and the virtual wire is modeled in the point cloud coordinate system to obtain corresponding parameters. The actual wire point cloud 3D model and the virtual wire 3D model parameters are weighted and fused to obtain the final model parameters, thereby realizing long-distance 3D reconstruction of the power line.

[0109] In some embodiments, S140, determining a target three-dimensional parameter of the power line to be measured based on the real three-dimensional power line point and the virtual three-dimensional power line point includes:

[0110] Based on the real three-dimensional wire points and the virtual three-dimensional wire points, a real three-dimensional model and a virtual three-dimensional model are respectively constructed;

[0111] determining a real model parameter and a virtual model parameter of each power line from the real three-dimensional model and the virtual three-dimensional model;

[0112] The target three-dimensional parameters of the power line to be tested are determined by performing weighted processing based on the difference between the real model parameters and the virtual model parameters.

[0113] For example, the virtual three-dimensional electric wire points are fitted with electric wires to obtain a reconstruction model of the virtual electric wire in the point cloud coordinate system. i ,B i ,C i ),i∈(0,n) and the model parameters of the virtual wire (a j ,b j ,c j ),j∈(0,n) to expand the analysis and take the average values ​​of the two groups of parameters to obtain and Then, two sets of parameters of the same power line are obtained. If the corresponding parameters differ within 1.8 times, the average value of the corresponding values ​​of the two sets of parameters is taken as the final model parameters of the power line; if there is a large difference between the corresponding parameters in the two sets of parameters, the parameter values ​​in the two sets of parameters that are closer to the mean are taken as the final parameters of the power line, thereby realizing the optimization of the power line model parameters.

[0114] The embodiment of the present invention achieves the complementarity of the two types of data by fusing and analyzing the two-dimensional image and the three-dimensional point cloud data, thereby improving the accuracy and efficiency of the three-dimensional high-precision reconstruction of the incomplete power line.

[0115] In some embodiments, see Figure 2 , Figure 2 A schematic diagram of the structure of a power line reconstruction device based on laser point cloud and image fusion provided in an embodiment of the present application; the power line reconstruction device 200 based on laser point cloud and image fusion provided in an embodiment of the present application includes: an acquisition module 210, a generation module 220, a fusion module 230 and a determination module 240, wherein:

[0116] An acquisition module 210 is configured to acquire original point cloud data and original image data of the power line to be tested;

[0117] The generating module 220 is configured to screen and reconstruct the original point cloud data to generate real three-dimensional wire points, and to sample the original image data to generate two-dimensional wire points.

[0118] A fusion module 230 is configured to project the real 3D wire points to a preset image coordinate system, determine the target real 3D wire point closest to each 2D wire point, fuse the corresponding 2D wire points using the depth information of the target real 3D wire point, and generate a virtual 3D wire point;

[0119] The determination module 240 is configured to determine the target three-dimensional parameters of the power line to be measured based on the real three-dimensional power line points and the virtual three-dimensional power line points.

[0120] In some embodiments, the acquisition module 210 is specifically configured as follows:

[0121] Based on the maximum principal component analysis method, the original coordinates corresponding to the original point cloud data are converted to the maximum principal component direction.

[0122] In some embodiments, the acquisition module 210 is specifically configured as follows:

[0123] Get the mean value of the original point cloud data after filtering in each dimension;

[0124] De-centroiding the original point cloud data based on the mean to obtain a covariance matrix;

[0125] Perform eigenvalue decomposition on the covariance matrix, determine the eigenvectors corresponding to the three largest eigenvalues, and determine the eigenvectors as the direction of the maximum principal component.

[0126] In some embodiments, the generation module 220 is specifically configured as follows:

[0127] A power line point cloud extraction method based on a dynamic threshold performs rough extraction on the original point cloud data to obtain first point cloud data;

[0128] The first point cloud data of each power line to be tested is precisely extracted using the Euclidean clustering algorithm to obtain the second point cloud data;

[0129] The second point cloud data is reconstructed using a model combining straight lines and parabolas, and after solving the model parameters based on the least squares method, equidistant sampling and completion are performed to generate real three-dimensional wire points.

[0130] In some embodiments, the generation module 220 is specifically configured as follows:

[0131] Perform data enhancement on the original image data to obtain optimized image data;

[0132] Performing power line edge detection on the optimized image data, and performing two-dimensional power line equation fitting based on edge points obtained by edge detection to obtain a fitting equation;

[0133] Equidistant sampling is performed based on the fitting equation to generate two-dimensional wire points.

[0134] In some embodiments, the fusion module 230 is specifically configured as follows:

[0135] Project the real three-dimensional wire points to the preset image coordinate system, obtain the pixel coordinate set, and construct the KD tree;

[0136] Filter out the pixel coordinates closest to each two-dimensional electric wire point from the KD tree, and determine the real three-dimensional electric wire point corresponding to the pixel coordinates as the target real three-dimensional electric wire point;

[0137] The depth information of the target real 3D wire point is used to fuse the corresponding 2D wire point to generate a virtual 3D wire point.

[0138] In some embodiments, the determination module 240 is specifically configured as follows:

[0139] Based on the real three-dimensional wire points and the virtual three-dimensional wire points, a real three-dimensional model and a virtual three-dimensional model are respectively constructed;

[0140] determining a real model parameter and a virtual model parameter of each power line from the real three-dimensional model and the virtual three-dimensional model;

[0141] The target three-dimensional parameters of the power line to be tested are determined by performing weighted processing based on the difference between the real model parameters and the virtual model parameters.

[0142] The power line reconstruction device based on laser point cloud and image fusion provided in the embodiment of the present application is based on the above-mentioned power line reconstruction method based on laser point cloud and image fusion, which will not be described again here to avoid repetition.

[0143] It should be noted that the power line reconstruction device based on laser point cloud and image fusion provided in the embodiment of the present application and the power line reconstruction method based on laser point cloud and image fusion provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned power line reconstruction method based on laser point cloud and image fusion, and the repeated parts will not be repeated.

[0144] In some embodiments, see Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. An electronic device 300 provided in an embodiment of the present application includes a processor 310 and a memory 320; the memory 320 stores a computer program, wherein the computer program implements the above-mentioned power line reconstruction method based on laser point cloud and image fusion when executed by the processor.

[0145] Specifically, the processor 310 may include, for example, a general-purpose microprocessor, an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 310 may also include an onboard memory for cache purposes. The processor 310 may be a single processing unit or multiple processing units for executing different actions of the method flow according to the embodiment of the present application.

[0146] The memory 320 may be any medium capable of containing, storing, conveying, propagating or transmitting instructions. For example, the memory 320 may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared or semiconductor system, device, component or propagation medium. Specific examples of the memory 320 include: a magnetic storage device, such as a magnetic tape or a hard disk (HDD); an optical storage device, such as a compact disk (CD-ROM); a random access memory (RAM) or flash memory; and / or a wired / wireless communication link.

[0147] The present application also provides a computer-readable medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned power line reconstruction method based on laser point cloud and image fusion is implemented. The computer-readable medium may be included in the device / apparatus / system described in the above-mentioned embodiment; or it may exist independently without being assembled into the device / apparatus / system. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed, the method according to the embodiment of the present application is implemented.

[0148] According to an embodiment of the present application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, optical cable, radio frequency signal, etc., or any suitable combination of the above.

[0149] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present application. Therefore, the scope of the present application should not be limited to the above-described embodiments, but should be determined not only by the appended claims, but also by the equivalents of the appended claims.

Claims

1. A method for power line reconstruction based on laser point cloud and image fusion, characterized in that: include: Obtaining original point cloud data and original image data of the power line to be tested; Screening and reconstructing the original point cloud data to generate real three-dimensional wire points, and sampling the original image data to generate two-dimensional wire points; Projecting the real three-dimensional wire points to a preset image coordinate system, determining the target real three-dimensional wire point closest to each of the two-dimensional wire points, and fusing the corresponding two-dimensional wire points using the depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point; A target three-dimensional parameter of the power line to be measured is determined based on the real three-dimensional power line point and the virtual three-dimensional power line point.

2. The power line reconstruction method based on laser point cloud and image fusion according to claim 1, characterized in that: The obtaining of original point cloud data and original image data of the power line to be tested includes: The original coordinates corresponding to the original point cloud data are converted to the maximum principal component direction based on the maximum principal component analysis method.

3. The power line reconstruction method based on laser point cloud and image fusion according to claim 2 is characterized in that: The method of converting the original coordinates corresponding to the original point cloud data into the direction of the maximum principal component based on the maximum principal component analysis method includes: Obtaining the mean value of the original point cloud data after filtering in each dimension; Performing de-centroiding processing on the original point cloud data based on the mean value to obtain a covariance matrix; Perform eigenvalue decomposition on the covariance matrix, determine the eigenvectors corresponding to the three largest eigenvalues, and determine the eigenvectors as the maximum principal component directions.

4. The power line reconstruction method based on laser point cloud and image fusion according to claim 1, characterized in that: The screening and reconstruction of the original point cloud data to generate real three-dimensional wire points includes: Performing a rough extraction on the original point cloud data using a power line point cloud extraction method based on a dynamic threshold to obtain first point cloud data; Using a Euclidean clustering algorithm to precisely extract the first point cloud data of each of the power lines to be tested, to obtain second point cloud data; The second point cloud data is reconstructed using a model combining a straight line and a parabola, and after solving the model parameters based on the least squares method, equidistant sampling and completion are performed to generate real three-dimensional wire points.

5. The power line reconstruction method based on laser point cloud and image fusion according to claim 1, characterized in that: The step of sampling the original image data to generate two-dimensional wire points includes: Performing data enhancement on the original image data to obtain optimized image data; Performing power line edge detection on the optimized image data, and performing two-dimensional power line equation fitting based on edge points obtained by edge detection to obtain a fitting equation; Equidistant sampling is performed based on the fitting equation to generate two-dimensional electric wire points.

6. The method for power line reconstruction based on laser point cloud and image fusion according to claim 1, characterized in that: The method of projecting the real three-dimensional wire points to a preset image coordinate system, determining a target real three-dimensional wire point that is closest to each of the two-dimensional wire points, and fusing the corresponding two-dimensional wire points using depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point includes: Projecting the real three-dimensional wire point to a preset image coordinate system to obtain a pixel coordinate set and constructing a KD tree; Filtering out pixel coordinates closest to each of the two-dimensional electric wire points from the KD tree, and determining the real three-dimensional electric wire point corresponding to the pixel coordinates as the target real three-dimensional electric wire point; The corresponding two-dimensional electric wire points are fused using the depth information of the target real three-dimensional electric wire point to generate a virtual three-dimensional electric wire point.

7. The power line reconstruction method based on laser point cloud and image fusion according to claim 1, characterized in that: The step of determining the target three-dimensional parameters of the power line to be measured based on the real three-dimensional power line point and the virtual three-dimensional power line point comprises: Constructing a real three-dimensional model and a virtual three-dimensional model based on the real three-dimensional wire points and the virtual three-dimensional wire points respectively; Determining real model parameters and virtual model parameters of each power line from the real three-dimensional model and the virtual three-dimensional model; A weighted processing is performed based on the difference between the real model parameter and the virtual model parameter to determine the target three-dimensional parameters of the power line to be measured.

8. A power line reconstruction device based on laser point cloud and image fusion, characterized in that: include: Acquisition module, generation module, fusion module and determination module, among which, The acquisition module is configured to acquire original point cloud data and original image data of the power line to be tested; The generating module is configured to screen and reconstruct the original point cloud data to generate real three-dimensional wire points, and to sample the original image data to generate two-dimensional wire points. The fusion module is configured to project the real three-dimensional wire points to a preset image coordinate system, determine the target real three-dimensional wire point closest to each of the two-dimensional wire points, and fuse the corresponding two-dimensional wire points using the depth information of the target real three-dimensional wire point to generate a virtual three-dimensional wire point; The determination module is configured to determine a target three-dimensional parameter of the power line to be measured based on the real three-dimensional power line point and the virtual three-dimensional power line point.

9. An electronic device comprising a processor and a memory; the memory stores a computer program, wherein: When the computer program is executed by the processor, the computer program implements the power line reconstruction method based on laser point cloud and image fusion according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Model state measurement method and system in water forced landing test

    CN119958560A

  • Electric power facility three-dimensional reconstruction method and device based on multi-source point cloud cooperation

    CN121544811A