High-voltage transmission line three-dimensional reconstruction method and system based on visual radar fusion and monitoring method

Abstract

The invention discloses a high-voltage transmission line three-dimensional reconstruction method and system based on visual radar fusion and a monitoring method. The method comprises the following steps: acquiring visual image data and scanning point cloud data of a transmission line; preprocessing the visual image data to obtain power transmission line image data; preprocessing the scanning point cloud data to obtain a power transmission line point cloud set, and constructing a power transmission line initial three-dimensional model based on the power transmission line point cloud set; and performing coordinate transformation on the power transmission line point cloud set, optimizing parameters of the initial three-dimensional model based on data after coordinate transformation and power transmission line image data, and constructing a final power transmission line three-dimensional model by using the optimized parameters. The scheme of the invention is more suitable for the actual work requirement of power transmission line inspection, and the time and economic cost are obviously reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of real-scene modeling, and in particular relates to a three-dimensional reconstruction method, system and monitoring method of a high-voltage transmission line based on visual radar fusion. Background Art

[0002] As an important part of power transmission, the safety monitoring methods for high-voltage transmission lines have always been the research focus of power grid companies and related research institutions. When the transmission distance is long, due to the deadweight of the conductor, a slight sag will be formed, making the conductor in the shape of a catenary. The size of the sag is not fixed, but is affected by multiple factors such as conductor material, cross-sectional dimensions, temperature changes, span length, and meteorological conditions at the time. Therefore, in actual transmission line monitoring, it is essential to obtain a complete three-dimensional model of the transmission line.

[0003] At present, there are many means of state monitoring for high-voltage transmission lines, among which drone inspection modeling, video stream monitoring and synthetic aperture radar (SAR) technology monitoring are the main ones, but the above methods all have their own defects. Among them, drone inspection modeling can obtain complete and high-precision point cloud information of transmission lines, but the manpower and material costs are high and the inspection process is complicated; while video stream monitoring methods are mostly based on deep learning methods to detect and locate special signs on transmission lines: such as insulators, so as to monitor the spatial state of the complete transmission line, but the method has high requirements for visual observation and is greatly affected by the external environment, and the monitoring stability is difficult to guarantee; SAR technology can extract the deformation displacement of the transmission line, so as to monitor the state of the transmission line in advance, but the technology is costly, has low temporal resolution, and is subject to many external factors, and cannot guarantee the instant response required for transmission line monitoring. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings of the above-mentioned background technology and to provide a three-dimensional reconstruction method of high-voltage transmission lines based on visual radar fusion with fast response and high stability.

[0005] The technical solution adopted by the present invention is: a three-dimensional reconstruction method of high-voltage transmission lines based on visual radar fusion,

[0006] Acquire visual image data and scanned point cloud data of power lines;

[0007] Preprocessing the visual image data to obtain transmission line image data;

[0008] Preprocessing the scanned point cloud data to obtain a transmission line point cloud set, and constructing an initial three-dimensional model of the transmission line based on the transmission line point cloud set;

[0009] The coordinate transformation of the transmission line point cloud set is performed, and the parameters of the initial three-dimensional model are optimized based on the data after coordinate transformation and the transmission line image data, and the final transmission line three-dimensional model is constructed with the optimized parameters.

[0010] Furthermore, visual image data and scanning point cloud data of the transmission line are acquired respectively by means of a camera and a laser radar, wherein the camera and the laser radar are installed on the towers around the transmission line.

[0011] Furthermore, the preprocessing of the visual image data includes:

[0012] A number of control points are selected in the image, and the transmission line image data is extracted through the B-spline function interpolation method according to the pixel coordinates of the control points.

[0013] Furthermore, the preprocessing of the scanned point cloud data includes:

[0014] The scanning point cloud data is filtered to obtain a transmission line point cloud set, wherein the filtering is any one of straight-through filtering, voxel filtering, statistical filtering, Gaussian filtering, and bilateral filtering.

[0015] Furthermore, an initial three-dimensional model of the transmission line is constructed based on the transmission line point cloud set by adopting a curve fitting method.

[0016] Furthermore, the coordinate transformation of the transmission line point cloud set includes:

[0017] Based on the device calibration parameters, the transmission line point cloud coordinates are first converted from the radar coordinate system to the camera coordinate system, and then converted to the pixel plane coordinate system.

[0018] Furthermore, the optimization of the parameters of the initial three-dimensional model based on the coordinate transformed data and the transmission line image data includes:

[0019] For each pixel point in the data after coordinate transformation, a number of pixel points closest to the pixel point are searched in the transmission line image data, and a straight line fitting is performed on the searched pixel points to obtain the equation of the closest point straight line on the pixel plane;

[0020] Calculate the distance from each pixel to the corresponding closest point straight line according to the closest point straight line method;

[0021] The objective function is constructed based on the distance, and the objective function is iteratively solved to obtain the optimal value of the parameter.

[0022] Furthermore, the objective function is

[0023]

[0024] Among them, F(w) is the objective function, N is the number of pixels after the transformation projection, and ei is the distance from the ith pixel to the corresponding nearest point straight line, w is the transmission line model parameter, k and h are the first-order term parameters and constant parameters of the straight line equation corresponding to the transmission line model, f, g, d are the second-order term parameters, first-order term parameters and constant parameters of the curve equation corresponding to the transmission line model, u and v are the horizontal and vertical coordinates of the projected point on the pixel plane, m and n are the first-order term parameters and constant parameters of the nearest point straight line equation, respectively.

[0025] A 3D reconstruction system for high-voltage transmission lines based on vision radar fusion, including

[0026] A data acquisition module, used to acquire visual image data and scanned point cloud data of the transmission line;

[0027] A data preprocessing module, used for preprocessing visual image data to obtain transmission line image data, and for preprocessing scanned point cloud data to obtain a transmission line point cloud set;

[0028] A model building module, used to build an initial three-dimensional model of the transmission line based on the transmission line point cloud set, and used to build a final three-dimensional model of the transmission line with optimized parameters;

[0029] The model optimization module is used to perform coordinate transformation on the transmission line point cloud set and optimize the parameters of the initial three-dimensional model based on the data after coordinate transformation and the transmission line image data.

[0030] A method for monitoring a high-voltage transmission line uses a three-dimensional model of the transmission line constructed by the three-dimensional reconstruction method of the high-voltage transmission line as described above to monitor the state of the transmission line.

[0031] The beneficial effects of the present invention are:

[0032] The present invention obtains the initial value of the three-dimensional model of the transmission line through radar point cloud observation, uses a camera to realize the acquisition of pixel points on the transmission line image plane, and then constructs a reprojection matching relationship between the two observations, and uses this error as the final objective function to optimize the parameters of the initial three-dimensional model, construct a final three-dimensional model of the transmission line, and then uses the high-precision three-dimensional model to sample the spatial points to obtain the three-dimensional coordinates of the transmission line point cloud, so as to complete the inspection of the transmission line. The scheme proposed by the present invention is more in line with the actual needs of the transmission line inspection work, and significantly reduces the time and economic costs. Compared with the traditional method, it has the following obvious advantages:

[0033] (1) No additional manpower or material resources are required. After the initial arrangement and debugging of sensors (industrial cameras and lidars), only data collection and processing are required in the background during subsequent use to complete the modeling and inspection of power lines. Compared with drone inspection and modeling, this greatly reduces manpower, material resources and time costs.

[0034] (2) It can realize real-time response of transmission line inspection. Compared with UAV inspection technology and SAR monitoring technology, the present invention can realize real-time response of transmission line inspection. When it is necessary to inspect the transmission line, the constructed three-dimensional model of the transmission line can realize instant real-time monitoring, which has obvious advantages in transmission line inspection.

[0035] (3) Compared with the video stream method, the present invention does not rely solely on visual observation and neural network extraction results. By extracting the power lines on the image, it achieves higher accuracy and stability at a minimal labor cost, allowing the system to be put into use under more environmental conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The present invention is a flow chart of the method.

[0037] Figure 2 It is a system principle diagram of the present invention. DETAILED DESCRIPTION

[0038] The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0039] like Figure 1 As shown, the present invention provides a method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion, comprising the following steps:

[0040] Step 1: Collect visual image data and scan point cloud data of the transmission line;

[0041] Step 2: Preprocess the visual image data to obtain the transmission line image data; preprocess the scanned point cloud data to obtain the transmission line point cloud set, and construct the initial three-dimensional model of the transmission line based on the transmission line point cloud set;

[0042] Step three, according to the calibrated equipment parameters, the transmission line point cloud set is transformed into coordinates, the parameters of the initial three-dimensional model are optimized based on the data after coordinate transformation and the transmission line image data, and the final transmission line three-dimensional model is constructed with the optimized parameters.

[0043] It can be understood that the visual image data of the transmission line in the above step 1 is the original observation information of the transmission line obtained by the industrial camera, which is in the picture format, that is, the pictures collected by the camera are single-frame images. The industrial camera is installed on the towers around the transmission line and should be set up in a position with minimal interference from other environments to achieve complete observation of the transmission line; at the same time, attention should be paid to the shooting angle of the industrial camera to avoid the reduction of the transmission line image resolution and blurring due to the increase in distance. One or more industrial cameras can be installed at one or more locations as needed, and the selection of industrial cameras should ensure the effectiveness of transmission line observation.

[0044] The scanning point cloud data of the transmission line in the above step 1 is obtained by the laser radar installed on the tower pole around the transmission line. One or more laser radars can be installed at one or more locations as needed. The installation of the laser radar needs to ensure its effectiveness in observing the transmission line. The selection of the laser radar needs to ensure that the transmission line point cloud range is 10-100m, and the initial point cloud range is 0-80m. Due to the limitations of the installation position and characteristics of the laser radar, the scanning point cloud data obtained is limited and is only a part of the entire cable. Therefore, the three-dimensional modeling method of the present invention does not require point clouds of all ranges. Only the scanned part of the point cloud can be used to reconstruct the entire cable model, and the amount of data used is less.

[0045] It is understandable that, due to the high difficulty of curve extraction in a complex observation environment, the present embodiment pre-processes the visual image data by using the method of selecting control points to extract the image of the transmission line, that is, selecting several control points in the image, and extracting the transmission line image data by the B-spline function interpolation method according to the pixel coordinates of the several control points. The transmission line image data is in the form of a curve or function, and the data contains the pixel coordinates of all pixel points on the transmission line. The specific processing process is as follows:

[0046] Select several control points in the image (the number is greater than 4) and record the pixel coordinates of these points as:

[0047]

[0048] In the formula, p control Represents the set of all control points in the visual image data, p i Represents the pixel coordinates of the selected i-th control point, u i 、v i Represents the horizontal and vertical coordinates of the i-th control point.

[0049] The use process of the cubic B-spline function with four control points is as follows:

[0050] A cubic B-spline curve can be determined by four plane discrete points, where the matrix expression of the cubic B-spline function is:

[0051]

[0052] In the formula, p(t) is the matrix of the cubic B-spline function, p1-p4 represent the coordinates of the four specific control points, and t is a proportional function in the range of 0 to 1, which is related to the number of final interpolation points. Further, the expression can be written in component form:

[0053]

[0054] Where:

[0055]

[0056] Among them, u(t) and v(t) are the horizontal and vertical coordinates of the control point under the corresponding proportional function, a0~a3 are the polynomial coefficients of the horizontal coordinate curve of the control point, b0~b3 are the polynomial coefficients of the vertical coordinate curve of the control point, u0~u3 are the horizontal coordinates of the four control points, and v0~v3 are the vertical coordinates of the four control points.

[0057] Thus, the cubic B-spline function with four control points can be obtained; when there are multiple control points, it is only necessary to select four control points in sequence from the multiple points to obtain the B-spline function curve within the corresponding range, and then all the curves are spliced ​​to obtain the complete curve sampling points, that is, the image data of the transmission line.

[0058] It can be understood that in step 2, the original point cloud data obtained by the laser radar scanning is filtered to extract the transmission line point cloud set therein, and the original laser radar scanning point cloud is recorded as:

[0059]

[0060] In the formula, p lidar Represents the entire point cloud set scanned by the LiDAR. Represents the 3D coordinates of the i-th 3D point in the laser radar coordinate system. Furthermore, by realizing the known range of the radar point cloud, all point clouds are filtered to obtain the partial point cloud set of the transmission line:

[0061] p curve =filter(p lidar )

[0062] In the formula, p curve Represents the set of transmission line point clouds after filtering, and filter represents the filtering operation performed on the point cloud.

[0063] It should be noted that the filtering may be any one of straight-through filtering, voxel filtering, statistical filtering, Gaussian filtering, and bilateral filtering. The specific steps of filtering processing generally include:

[0064] Load point cloud data and use professional point cloud processing software (such as CloudCompare, PCL, etc.) to load the point cloud data that needs to be processed.

[0065] Select a filtering method and choose an appropriate filtering method based on the characteristics of the point cloud data and processing requirements.

[0066] Set the filter parameters. According to the selected filter method, set the corresponding parameters. For example, the through filter needs to set the attribute range and the conditions for retention or removal; the voxel filter needs to set the voxel size; the statistical filter needs to set the number of neighboring points and the standard deviation multiple, etc.

[0067] Apply filtering, perform filtering operations in the software, and obtain filtered point cloud data.

[0068] Check the filtering effect and check the filtered point cloud data to ensure that the filtering effect meets the expectations. If the filtering effect is not satisfactory, you can adjust the filtering parameters and reapply the filter.

[0069] It should be noted that in step 2, the incomplete point cloud set after filtering will be used for the initial modeling of the transmission line. Combined with the spatial characteristics of the transmission line, the transmission line is regarded as a curve in space, and the curve equation is assumed to be:

[0070]

[0071] Among them, x, y, and z are the spatial three-axis coordinates of the point cloud in the transmission line model, k and h are the first-order term parameters and constant parameters of the straight line equation corresponding to the transmission line model, and f, g, and d are the second-order term parameters, first-order term parameters, and constant parameters of the curve equation corresponding to the transmission line model.

[0072] Summarizing all point clouds, we can obtain overdetermined equations for the two equations:

[0073]

[0074] In the formula, A1 and A2 are the parameter matrix of the first equation and the parameter matrix of the second equation after summarizing multiple points, respectively. b1 and b2 can be regarded as the residual vector of the first equation and the residual vector of the second equation after summarizing multiple points. For this overdetermined equation, matrix decomposition or least squares can be used to solve it, so as to obtain the initial value of the parameters of the curve equation, that is, the initial three-dimensional model, which is recorded as:

[0075]

[0076] Where k0 and h0 are the initial first-order term parameters and initial constant parameters of the linear equation corresponding to the transmission line model, respectively; a0, b0, and c0 are the initial second-order term parameters, initial first-order term parameters, and initial constant parameters of the curve equation corresponding to the transmission line model, respectively.

[0077] It can be understood that in step 3, the external parameters between the sensors (i.e., the laser radar and the camera) obtained by calibration and the camera internal parameters are used to realize the transformation of the transmission line point cloud from the radar coordinate system to the pixel plane coordinate system, and the three-dimensional point coordinates p in the radar coordinate system are calculated. l First use the external parameters to get its coordinates p in the camera coordinate system c :

[0078]

[0079] In the formula, The rotation matrix that represents the point transformed from the radar coordinate system to the camera coordinate system, represents the translation vector from the radar coordinate system to the camera coordinate system, both of which can be obtained by offline calibration in advance. c It can be projected to the camera pixel plane according to the camera intrinsic parameter matrix:

[0080]

[0081] Where P m Represents the coordinates of the 3D point in the camera normalized plane coordinate system, x c ,y c 、z c represents the 3D coordinate value of the 3D point in the camera coordinate system, K represents the intrinsic parameter matrix of the camera, which can be obtained by offline calibration in advance. It is composed of two parameters: the focal length and the optical center of the camera. The coordinate p of the 3D point in the pixel plane coordinate system can be obtained. img .

[0082] It should be noted that in step three, the error relationship (i.e., the objective function) will be constructed by converting the distance relationship between the projected pixel points and the points on the image curve obtained in step two. Since the initial curve model may have a large deviation, the projected points may deviate greatly from the image curve points, and the matching relationship between the points cannot be directly constructed. To this end, it is necessary to use the nearest neighbor search to find the 5 points closest to the pixel distance between the projected point and the original image curve, and perform straight line fitting on these points. The specific fitting method is similar to the curve fitting method in step two, and the equation of the nearest point on the pixel plane is obtained: u=mv+n, where u and v are the horizontal and vertical coordinates of the projected point on the pixel plane, respectively, and m and n are the first-order term parameters and constant parameters of the nearest point straight line equation, respectively.

[0083] For each pixel obtained by projection, calculate the distance e from the point to the corresponding nearest straight line i , which can be viewed as a residual:

[0084]

[0085] It can be obtained that the final constructed least squares objective function F(w) can be written as:

[0086] N is the number of pixels after the transformation projection, and w is the transmission line model parameter.

[0087] The function is solved. During the solution process, the parameters of the three-dimensional curve equation are regarded as parameters to be estimated, and other values ​​are regarded as known values. Through nonlinear optimization methods such as the Gauss-Newton method, the optimal values ​​of the parameters are iteratively solved as the final curve equation parameters, and then the final complete three-dimensional model of the transmission line is obtained.

[0088] To realize the above-mentioned 3D reconstruction method of high voltage transmission line, Figure 2 As shown, the present invention also provides a high-voltage transmission line three-dimensional reconstruction system, comprising

[0089] A data acquisition module, used to acquire visual image data and scanned point cloud data of the transmission line;

[0090] A data preprocessing module, used for preprocessing visual image data to obtain transmission line image data, and for preprocessing scanned point cloud data to obtain a transmission line point cloud set;

[0091] A model building module, used to build an initial three-dimensional model of the transmission line based on the transmission line point cloud set, and used to build a final three-dimensional model of the transmission line with optimized parameters;

[0092] The model optimization module is used to perform coordinate transformation on the transmission line point cloud set and optimize the parameters of the initial three-dimensional model based on the data after coordinate transformation and the transmission line image data.

[0093] It should be noted that the specific implementation process of the above-mentioned functional modules has been described in detail in the aforementioned transmission line three-dimensional reconstruction method, and will not be repeated here.

[0094] The present invention also provides a monitoring method for high-voltage transmission lines. By utilizing the three-dimensional model of the transmission line finally obtained above, sampling points are selected according to needs within a known spatial range to obtain the three-dimensional coordinates of the corresponding points, thereby realizing real-time monitoring of the three-dimensional state of the transmission line.

[0095] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of protection of the present disclosure. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0096] In order to make the description of the present disclosure more detailed and complete, the above description is provided for the implementation and specific embodiments of the present invention; however, this is not the only form of implementing or using the specific embodiments of the present invention. The implementation covers the features of multiple specific embodiments and the method steps and sequences used to construct and operate these specific embodiments. However, other specific embodiments can also be used to achieve the same or equivalent functions and step sequences.

[0097] Those skilled in the art may also appreciate that the various illustrative logic blocks, units, and steps listed in the embodiments of the present invention may be implemented by electronic hardware, computer software, or a combination of the two. To clearly demonstrate the interchangeability of hardware and software, the various illustrative components, units, and steps described above have generally described their functions. Whether such functions are implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement the described functions for each specific application, but such implementation should not be understood as exceeding the scope of protection of the embodiments of the present invention.

[0098] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in the field.

Claims

1. A method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion, characterized in that: Acquire visual image data and scanned point cloud data of power lines; Preprocessing the visual image data to obtain transmission line image data; Preprocessing the scanned point cloud data to obtain a transmission line point cloud set, and constructing an initial three-dimensional model of the transmission line based on the transmission line point cloud set; The coordinate transformation of the transmission line point cloud set is performed, and the parameters of the initial three-dimensional model are optimized based on the data after coordinate transformation and the transmission line image data, and the final transmission line three-dimensional model is constructed with the optimized parameters.

2. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: Visual image data and scanning point cloud data of the transmission line are acquired respectively by using a camera and a laser radar, wherein the camera and the laser radar are installed on the towers around the transmission line.

3. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: The preprocessing of the visual image data comprises: A number of control points are selected in the image, and the transmission line image data is extracted through the B-spline function interpolation method according to the pixel coordinates of the control points.

4. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: The preprocessing of the scanned point cloud data comprises: The scanning point cloud data is filtered to obtain the transmission line point cloud set.

5. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: Based on the transmission line point cloud set, the initial three-dimensional model of the transmission line is constructed by curve fitting.

6. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: The coordinate transformation of the transmission line point cloud set comprises: Based on the device calibration parameters, the transmission line point cloud coordinates are first converted from the radar coordinate system to the camera coordinate system, and then converted to the pixel plane coordinate system.

7. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 1 is characterized in that: The optimization of the parameters of the initial three-dimensional model based on the data after coordinate transformation and the transmission line image data includes: For each pixel point in the data after coordinate transformation, a number of pixel points closest to the pixel point are searched in the transmission line image data, and a straight line fitting is performed on the searched pixel points to obtain the equation of the closest point straight line on the pixel plane; Calculate the distance from each pixel to the corresponding closest point straight line according to the closest point straight line method; The objective function is constructed based on the distance, and the objective function is iteratively solved to obtain the optimal value of the parameter.

8. The method for 3D reconstruction of high-voltage transmission lines based on visual radar fusion according to claim 7 is characterized in that: The objective function is Among them, F(w) is the objective function, N is the number of pixels after the transformation projection, and e i is the distance from the ith pixel to the corresponding nearest point straight line, w is the transmission line model parameter, k and h are the first-order term parameters and constant parameters of the straight line equation corresponding to the transmission line model, f, g, d are the second-order term parameters, first-order term parameters and constant parameters of the curve equation corresponding to the transmission line model, u and v are the horizontal and vertical coordinates of the projected point on the pixel plane, m and n are the first-order term parameters and constant parameters of the nearest point straight line equation, respectively.

9. A 3D reconstruction system for high-voltage transmission lines based on visual radar fusion, characterized in that: include A data acquisition module, used to acquire visual image data and scanned point cloud data of the transmission line; A data preprocessing module, used for preprocessing visual image data to obtain transmission line image data, and for preprocessing scanned point cloud data to obtain a transmission line point cloud set; A model building module, used to build an initial three-dimensional model of the transmission line based on the transmission line point cloud set, and used to build a final three-dimensional model of the transmission line with optimized parameters; The model optimization module is used to perform coordinate transformation on the transmission line point cloud set and optimize the parameters of the initial three-dimensional model based on the data after coordinate transformation and the transmission line image data.

10. A method for monitoring a high voltage transmission line, characterized in that: The state of the transmission line is monitored by using a three-dimensional model of the transmission line constructed using the three-dimensional reconstruction method of the high-voltage transmission line as described in any one of claims 1 to 8.

Images