Rapid and high-accuracy reconstruction method for power lines based on multi-source data fusion

Abstract

A rapid and high-accuracy reconstruction method for power lines based on multi-source data fusion, wherein a flyaway patrol robot collects multi-source data for power lines in a process of walking alo

Description

TECHNICAL FIELD

[01] The present invention belongs to the field of power lines for intelligent inspection, and particularly relates to a rapid and high-accuracy reconstruction method for power lines based on multi-source data fusion. BACKGROUND ART

[02] The overhead transmission line is a main component of China’s power system, and is also an important infrastructure of China’s power system. In recent years, with the continuous acceleration of the development of electric power in China, the power grid has also been rapidly developed, the voltage level for operation of a power grid system has continued to improve, the network scale has continued to expand, and a complete long-distance transmission grid framework has been basically formed. Both at home and abroad, the failure of a power line will have a great influence on the lives of people, so the importance of stable operation of the power system is more and more significant. A long-distance power line needs to cross deserts, mountains, forests, rivers and other harsh environments, and is extremely damaged by nonhuman factors such as rain and snow, material aging, external short circuit, continuous tension, etc. In order to ensure the efficient and reliable transmission of electric energy, the power line must be regularly overhauled to discover faults in time and conduct repair. However, the traditional manual inspection mode adopted by the existing power system mainly relies on human perception and experience, which has the problems of low inspection efficiency, inadequate inspection, high work intensity and difficult work. With the continuous development of science and technology, the intelligent inspection technology has been increasingly mature and has become an important means to ensure the safe and reliable operation of the transmission lines. There are mainly two intelligent inspection methods. The first method is the UAV patrol, and the second method is climbing robot patrol, wherein a UAV has good maneuverability and high detection accuracy, but its endurance is poor; the flight distance is difficult to meet the needs of long-distance inspection; and it is easy to be affected by weather, air traffic control and other factors. The robot patrol can go deep into inaccessible places such as mountain forests along the power line to conduct detailed inspection for the line, and can also use downhill power generation, solar charging and other ways to ensure the online supply of the electric energy. However, the robot gets on and off the line artificially or by a special lifting mechanism installed on a power tower, and a complex obstacle crossing mechanism and a control system need to be designed to ensure that the obstacle crosses the robot on the line. Therefore, we propose a flyaway patrol robot, which can retain the advantages of the above two inspection modes and make up for the shortcomings of both. Dual mode refers to a robot mode that can switch between two motion modes of flying up and down the line or flying over a tower head, and walking along a ground wire. The flyaway patrol robot is equipped with a laser LiDAR device and a matching synchronous camera. LiDAR point cloud data and related image data are obtained during flight and walking along the ground wire. Through the fusion of the LiDAR point cloud data and the image data, the prior information is used to conduct rapid 3D reconstruction of the power line, and a high-accuracy power line model is generated by combining image texture information. The power line after 3D reconstruction can be used for: dangerous point detection, conductor interphase height difference measurement, distance measurement between conductors, distance measurement from conductors to ground, 3D visualization, sag analysis of conductors, ice analysis of conductors, wind deviation analysis of conductors, etc.

[03] At present, the 3D reconstruction of the power line based on optical images faces the problems of matching difficulty and low reconstruction accuracy, and the 3D reconstruction technology of the power line based on LiDAR point cloud technology also has problems of low accuracy of point cloud classification and power line point cloud extraction. Therefore, the present invention uses the flyaway patrol robot equipped with the laser LiDAR device and the matching synchronous camera to conduct 3D power line reconstruction in the process of flight and walking along the ground wire. On the one hand, the problem of matching difficulty and low reconstruction accuracy of power line reconstruction based on images only is solved. On the other hand, by combining RGB image data to assist point cloud classification and power line point extraction, the problem of low accuracy of point cloud classification and power line point cloud extraction of laser point cloud is solved. This method can completely reconstruct the power line texture model in the same field with fast reconstruction speed and high reconstruction accuracy. The model can also be used for conductor interphase height difference measurement, distance measurement between conductors, distance measurement from conductors to ground, sag analysis of conductors, ice analysis of conductors, wind deviation analysis of conductors, dangerous point detection and investigation in the future.

[04] The difficulty in 3D power line reconstruction work is the processing work of a large number of point cloud data, which will directly affect the efficiency and the accuracy of reconstruction. The ingenious feature of the present invention is that the point cloud data in a power line channel can be selected by using POS prior information in the process of walking along the ground wire by tire flyaway patrol robot and in combination with laser point cloud depth information and image RGB information, so that the point cloud data is lightweight and the point cloud extraction speed of the power line is increased. In addition, the present invention proposes a reconstruction method of local centroid proportional fitting to carry out 3D reconstruction of the power line, selects key nodes by extracting the locally weighted centroid, and carries out fitting reconstruction of key nodes by proportional polynomial equations, thereby not only greatly improving the reconstruction efficiency of the power line and avoiding the influence of noise points on the reconstruction quality, but also considering the errors of the point cloud in x and y directions, greatly improving the reconstruction accuracy of the power line, achieving the establishment of a complete high-accuracy 3D model of the power line and making the whole process fast and efficient. SUMMARY

[05] The technical problem to be solved by the present invention is to provide a rapid and high-accuracy reconstruction method for power lines based on multi-source data fusion, so as to solve two major problems of 3D reconstruction of power lines in the above technology: (1) large point cloud data and low reconstruction efficiency; and (2) reconstruction and matching difficulty and low 3D reconstruction accuracy.

[06] In order to solve the above technical problems, the present invention provides a technical solution for rapid and high-accuracy reconstruction of power lines based on multisource data fusion. The specific technical solution is as follows.

[07] A rapid and high-accuracy reconstruction method for power lines based on multisource data fusion is provided, wherein a flyaway patrol robot collects multi-source data for power lines in a process of walking along a ground wire, prior information is used to conduct rapid 3D reconstruction of the power lines by fusing laser point cloud data and image data, and a high-accuracy texture model of the power lines is generated by combining image texture information. The method comprises the following steps:

[08] step 1: patrolling, by a flyaway patrol robot, according to a planned route and obtaining LiDAR point cloud data of a 3D environment of power lines to be reconstructed through an airborne radar;

[09] step 2: initially setting a focal length of a synchronous camera according to spacing information of the power lines, controlling the focal length of the synchronous camera through LiDAR depth information feedback, alternatively shooting and collecting RGB image data of power lines in different layers and an overall environment, and establishing a corresponding dataset;

[10] step 3: classifying and grading transmission lines by using the prior information and determining LiDAR points of each grade;

[11] step 4: reconstructing a ground wire by using POS information and extracting a power line point cloud model roughly according to structured working conditions of the power lines;

[12] step 5: screening and quickly refining colored point cloud by using RGB information of a power line image;

[13] step 6: clustering single-grade power line point cloud by using a K-means clustering method;

[14] step 7: sampling key nodes of each power line in a single grade, and performing fitting calculation to complete rapid reconstruction of the power lines; and

[15] step 8: integrating surface texture information of the power lines to generate the high-accuracy texture model of the power lines.

[16] In the above technical solution, the synchronous camera mentioned in the step 2 is a zoom camera, and has been jointly calibrated with LiDAR. Ure focal length of the synchronous camera can be initially set by using the power line spacing information, and the focal length of the synchronous camera can be controlled to alternatively shooting the RGB image data of the power lines and the overall environment through the LiDAR depth information feedback. The following steps are specifically included:

[17] step 2.1: transmitting the point cloud data collected by the LiDAR to an image processing end, and converting the point cloud data into a depth map;

[18] step 2.2: retrieving, by a camera control end, whether a known zoom instruction is loaded; if there is prior information (spacing information of power lines in different layers), then proceeding to step 2.3; and if not, selecting a middle depth as an initial focus and proceeding directly to step 2.4;

[19] step 2.3: converting, by the image processing end, the spacing information of power lines in different layers into corresponding depth information in the depth map through an algorithm, setting a corresponding initial focus distance through the depth information, and issuing an instruction to control the synchronous camera to adjust the focal length;

[20] step 2.4: shooting an image, and transmitting the image to the image processing end; performing focal length feedback through definition analysis and depth information comparison; and proceeding to a zoom step for adjusting the focal length of the synchronous camera in real time; and

[21] step 2.5: performing real-time zoom of the synchronous camera, alternatively shooting and collecting the RGB image data of power lines in each layer and the overall environment, and storing into different datasets.

[22] In the above technical solution, the classifying and grading transmission lines by using the prior information in the step 3 comprises:

[23] (1) determining locations and structures of the transmission lines according to geographic information system (GIS) data, comprising geographical coordinates, heights, line routes and the like of a power tower and the power lines;

[24] (2) determining connections and structures of the transmission lines according to a line map and topological information, comprising voltage levels of the lines, the number of conductors, the number of phases, and the like; and

[25] (3) according to line file data of a power company, determining attributes of the transmission lines, such as voltage levels, pole types, conductor transmission types, and the like.

[26] In the above technical solution, in step 4, POS data is used to obtain the motion trajectory of the flyaway patrol robot, and deformation caused by the weight of a UAV is taken into consideration to obtain a ground wire trajectory according to a correction function, and the structured working conditions (the consistency of trends and shapes) of the power lines are used to roughly extract point cloud models for other power lines.

[27] In the above technical solution, in step 5, colored point cloud is screened and quickly refined by using the RGB information of a power line image, specifically comprising the following steps:

[28] step 5.1: defining an RGB color range of the power lines by using high-definition power line images in a mapping dataset;

[29] step 5.2: traversing the entire colored point cloud dataset and accessing the RGB color information of each point;

[30] step 5.3: deciding whether to keep or discard each point according to conditions of the defined color range of the power lines; and

[31] step 5.4: marking screening results as power line feature points to complete the rapid refinement of the point cloud.

[32] In the above technical solution, a local centroid scaling projection fitting method is proposed in the step 7 to carry out fitting reconstruction of the power lines. In this method, key nodes are selected by extracting a local weighted centroid to realize th e sampling of the power line point cloud, and the fitting calculation of the scaling projection model is carried out only on the sampled key nodes to realize the power line reconstruction, thereby improving the reconstruction efficiency of the power lines and avoiding the influence of noise points on the reconstruction quality. The errors of the point cloud in x and y directions are also considered in the scaling projection reconstruction model, which can improve the reconstruction accuracy of the power lines. The specific scaling projection reconstruction model can be expressed in the following mode: setting a reconstruction straight line projected on an x-O-y plane by a power line point cloud as d = x cos a + y sin a, and performing local reconstruction for a local power line point cloud by a polynomial equation z = A + Bs + Cs2. In the formula, s is a scale factor of a horizontal coordinate Q(%o>yo) of a point on the polynomial equation corresponding to a corresponding point Q' (Xq, y„) on the reconstruction straight line projected on the x-O-y plane, in which

[33] ——, Isin a\ Icos a\ -sm a Isin al <Icos al xosa 1 11 1

[34] Coordinates of two endpoints of the power line can be obtained by traversing the point cloud, namely (xi, y i) and (xr, yr). The point cloud of the power line is divided into N segments along the trend of the power line, and then a separation interval is:

[35]

[36] A calculation formula of a local centroid is:

[37] (x, y, z) fflLiVW g=imizA

[38] In the formula, i represents a point number; n represents the total number of points; m represents the weight of the point, which is calculated by a reciprocal of a distance from the point to the reconstruction straight line, that is, the farther the distance from the reconstruction straight line is, the smaller the weight is. After the key points of a single power line are extracted, the power line is reconstructed. According to the least squares principle, a reconstruction process can be transformed into the following S-minimum problem. S represents the sum of squared errors between an actual value and a calculated value of z coordinate in a 3D coordinate system:

[39] S = 2k -( / 4 + BSi + Cs?)]2

[40] i.e.: g = -22^-(A+ Bsf + Cs?)]2 = 0

[41] = -22k^ - (ASi + Bs? + Cs^Y = 0 = -22ks2 - (As? + Bsf + Cs4)]2 = 0

[42] In the formulas, A, B and C represent polynomial coefficients respectively; Z[ and Si represent the coordinate of the reconstruction node on z axis in the 3D coordinate system and the scale factor on the corresponding point of the reconstruction straight line projected on the x-O-y plane respectively; dS / dA, dS / dB and dS / dC are first-order partial derivatives.

[43] In the above technical solution, in step 8, the point cloud data in the power line point cloud model is discretized into a voxel grid; texture coordinates are mapped after coordinate alignment is completed; and finally, the texture information is applied to a voxel crystal grid to form texture information, to obtain the high-accuracy texture model of the power lines.

[44] Compared with the prior art, the present invention has the following advantages:

[45] (1) Compared with the 3D reconstruction method using an oblique photography technology, the present invention improves the matching accuracy of power line reconstruction, and can reconstruct the complete 3D model of the power lines quickly and accurately;

[46] (2) Compared with the 3D reconstruction method using the LiDAR technology, the present invention can obtain the surface texture color information of spatial objects, eliminates some false information caused by the loss of fine structure, and entirely improves the speed and the accuracy of power line point cloud classification and point cloud extraction;

[47] (3) The combination of the prior information of multi-source data, the robot walking motion trajectory and the structured working conditions is closer to the practical application and suitable for the intelligent inspection field of the flyaway patrol robot. In the future, the investigation of dangerous points of the power lines can be realized through the integrity detection of the high-precision model of the power lines and the detection of foreign body points. BRIEF DESCRIPTION OF THE DRAWINGS

[48] FIG. 1 is a flow block diagram of a rapid and high-accuracy reconstruction method for power lines based on multi-source data fusion in the present invention;

[49] FIG. 2 is a schematic diagram of a motion mode for collecting data by a flyaway patrol robot in the present invention;

[50] FIG. 3 is a schematic diagram of a ground wire model based on POS and rough extraction of point cloud in the present invention;

[51] FIG. 4 is an effect diagram of point cloud after denoising in the present invention;

[52] In the figures: 1. phase line; 2. ground wire; 3. anti-vibration hammer; 4. flyaway patrol robot; 5. actual motion trajectory; 6. real ground wire trajectory; 7. remaining surface points; 8. trajectory channel extraction model; 9. noise point. DETAILED DESCRIPTION OF THE EMBODIMENTS

[53] The present invention is further described in detail below in combination with test examples and specific embodiments. However, it should not be understood that the scope of the above subject matter of the present invention is limited to the following embodiments, and all technologies realized based on the contents of the present invention fall within the scope of the present invention.

[54] A rapid and high-accuracy reconstruction method for power lines based on multisource data fusion is provided, wherein a flyaway patrol robot collects multi-source data for power lines in a process of walking along a ground wire, prior information is used to conduct rapid 3D reconstruction of the power lines by fusing laser point cloud data and image data, and a high-accuracy texture model of the power lines is generated by combining image texture information. The method comprises the following steps.

[55] Step 1: a flyaway patrol robot patrols according to a planned route and obtains LiDAR point cloud data of a 3D environment of power lines to be reconstructed through an airborne radar.

[56] Specifically, according to the existing power tower construction specifications, the take-off, landing and other steps of the flyaway patrol robot are carried out within a safe range, wherein the data collection mode by the motion of the flyaway patrol robot is show n in Fig. 2. The LiDAR device is turned on to collect data and transmit the depth information to a camera control end.

[57] Step 2: a focal length of a synchronous camera is initially set according to spacing information of the power lines, the focal length of the synchronous camera is controlled through LiDAR depth information feedback, and RGB image data of power lines in different layers and an overall environment are alternatively shoot and collected, and a corresponding dataset is established.

[58] Specifically, firstly, it is necessary to ensure that the synchronous camera is a zoom camera, and has been jointly calibrated with LiDAR. After a data collection instruction is issued, the camera control end starts to operate. If there is prior information (spacing information of power lines in different layers), the initial focal length is set according to the demand, and the instruction is issued to control the synchronous camera for zoom. If there is no prior data, the depth information provided by the LiDAR feeds back the distance information, and enters the zoom step through the distance feedback to adjust the focal length of the camera in real time. Here, the above step can be implemented using an automatic regression algorithm:

[59] (I) Depth data processing: the point cloud data is converted into a depth map, wherein the value of each pixel represents the distance from the corresponding point to the camera, which can be obtained by mapping the point cloud data to the image coordinate system, and then setting the depth value of each pixel as the distance from the closest point.

[60] (2) Focus initialization: the initial focus distance is set, which can be usually initialized by selecting a middle depth or according to the needs of an application scenario; and if there is prior information (spacing information of power lines in different layers), the initial focus distance is set according to the distance between the existing ground wire and a lower phase line.

[61] (3) Image capturing: an image is captured by using a camera, and the image can be used for subsequent definition evaluation.

[62] (4) Calculation of image definition: in order to evaluate the definition of the image, the gradient amplitude of tire image can be calculated, and the gradient amplitude represents the change rate of the pixel value in the image and is used to indicate die edges and details of objects in the image.

[63] (5) Depth information feedback: the current focus distance and the image definition are compared. If the image definition is lower than a preset threshold, it indicates that the image is not clear enough and the focus distance needs to be adjusted;

[64] (6) Automatic adjustment of focal length: according to the depth information, the focus distance is automatically adjusted, which can be achieved by PID (proportional-integral-differential) control, wherein the output of a controller is the adjustment amount of the focal length.

[65] (7) Repeating of steps 3-6: during continuous shooting, the above steps are repeated continuously to monitor the depth information and adjust the focus in real time to ensure that the power lines maintain optimal clarity.

[66] (8) The image RGB data of the overall environment shot synchronously with the LiDAR is stored together with the point cloud into the colored point cloud dataset. This step needs to ensure that an appropriate data format that can store the RGB color information is selected. One of the common point cloud data formats is PLY (Polygon File) format, which supports coordinate and color information for points; and other formats, such as XYZRGB, can also be used to store point cloud data containing RGB information. The latter is selected here.

[67] The image data of power lines in different layers are stored in the corresponding mapping dataset, which can provide more accurate texture information for the subsequent texture mapping of the power lines.

[68] Step 3: transmission lines are classified and graded by using the prior information and determining LiDAR points of each grade.

[69] Specifically, the prior information that can be used comprises:

[70] (1) Geographic information system (GIS) data: the GIS data contains fault and attribute information that can be used for determining locations and structures of the transmission lines, comprising geographical coordinates, heights, line routes and the like of a power tower and the power lines.

[71] (2) Line map and topological description information: the line map and the topological information explain the connection and the structures of the transmission lines, comprising voltage levels of the lines, the number of conductors, the number of phases, and the like.

[72] (3) Transmission line attribute data: the attribute information of the transmission lines, such as voltage levels, pole types, conductor transmission types and the like, can be used for determining the features of the lines. These attributes are usually recorded in the archives of the power company.

[73] Classifying and grading transmission lines and determining LiDAR points of each grade can be achieved by the following steps:

[74] (1) Data obtaining: the GIS data, the line map and the topological information are obtained from the power company or related agencies. This needs to ensure the accuracy of the geographic coordinates and the line attribute data.

[75] (2) Matching of the GIS data with the point cloud coordinates: the point cloud is matched with the GIS data coordinates by using geographic coordinates. This can be done through Geographic Information system (GIS) software.

[76] In addition, the line map and the topological information can be matched with the geographic coordinates. For example, geographic coordinate relationships can be used to determine the paths and the connection of output lines.

[77] In this step, the total number of power towers in the transmission lines, the total number of grades of the power lines and the specific geographic coordinates of the power towers can be obtained, and through coordinate conversion, the geographic coordinates of the power towers can be converted into point cloud coordinates and the central point coordinates of the power towers can be obtained.

[78] (3) Classification based on spatial association: the transmission lines are identified by using the laser point cloud data. Based on the geographic coordinates in the GIS data, the approximate locations of power tower points and power line points of the transmission lines in the laser point cloud data can be determined.

[79] Classification according to the line map and the topological information: the transmission lines are further classified according to the matching results of the line map and the topological information. Information such as line types and voltage levels can be determined. The spacing of the outermost power lines at both ends of the tower is determined according to the tower construction specifications to further determine the two-dimensional space range of each grade of the transmission lines.

[80] (4) Grading: the transmission line points are graded according to the classification results. For example, the transmission lines of different voltage levels are divided into different grades. Since this method is the reconstruction of the power lines, the transmission lines are switched once for each grade segment.

[81] (5) Integration of the results: the classification and grading results are integrated into the point cloud data, and the classified transmission line points is associated with the attribute information. It is worth noting that the subsequent steps are conducted for the single-grade transmission line point cloud data based on the integration results.

[82] It should be noted that these steps show how to use the prior information and different data sources to classify and grade the transmission lines. Practical application may require more complex algorithms and more data processing, depending on the data and needs.

[83] Step 4: a ground wire is reconstructed by using POS information and a power line point cloud model is extracted roughly according to structured working conditions of the power lines.

[84] Specifically, the POS system (position and orientation system) carried on the flyaway patrol robot provides an accurate motion trajectory of the flyaway patrol robot during patrol. Considering the deformation caused by the weight of the UAV, the trajectory of a vertical plane of the ground wire can be obtained according to a correction function when the ground wire is overhanging naturally.

[85] Tire correction function of the ground wire trajectory in the vertical plane:

[86] y = Ax2 + Bx + C + 5(x)

[87] wherein 8 is an additional sag, with an expression as follows:

[88] 5(%) =

[89] In the formula: r / is an increasing coefficient of a shock effect; Lisa horizontal span; W is the mass of tire robot; Oq is horizontal stress; and S is the cross-sectional area of the walking ground wire.

[90] Tower construction specification in the prior information: the width of the outermost power line and the fixed distance of the power line can initially determine the distance between the power lines, and the width of the outermost power line and the distance between the ground wire and the lowest power line can be used for determining the width and height thresholds of the space channel of the power lines.

[91] The trajectories of the power lines in the space channel are preliminarily determined by adding the spacing information of the power lines through the structured working condition of the power lines (consistency of trend and shape) according to the above ground wire correction trajectory. The power line points of each layer are extracted by fitting the POS-based model (the above ground wire trajectory correction function), as shown in Fig. 3, to complete the rough extraction of the power line model of the colored point cloud. Remaining surface points (e.g., phase spacers) and noise points (e.g., tree tops) can be further removed.

[92] The effect diagram of the point cloud after denoising is shown in Fig. 4.

[93] Step 5: colored point cloud is screened and quickly refined by using RGB information of a power line image.

[94] The following steps need to ensure that each point in the colored point cloud contains RGB color information.

[95] Step 5 . 1: an RGB color range of the power lines is defined by using high-definition power line images in a mapping dataset. The colors of the power lines may vary from situations, so it is necessary to determine the specific RGB color value or color range of the power lines.

[96] Specifically, the high-definition power line image in the mapping dataset is loaded, the RGB values of sample power lines are extracted and the color range of the power lines is determined. A minimum RGB value and a maximum RGB value can be defined to create the color range.

[97] For example, if the color samples of the power lines have RGB values (R, G, B), then the color range can be defined as:

[98] Minimum color value: (R-AR, G-AG, B-AB)

[99] Maximum color value: (R+AR, G+AG, B+AB)

[100] AR, AG and AB are tolerable color deviations. This will allow to capture color change within a certain range, rather than just exact matching.

[101] Step 5.2: the entire colored point cloud dataset is traversed and the RGB color information of each point is accessed.

[102] Step 5.3: whether to keep or discard each point is decided according to the defined color range conditions of the power lines; checking whether the color of each point falls within the color range of the power lines; using ‘‘meets _power_line_color_criteria” function to check whether the color of each point meets the color range conditions of the power lines; and if so, adding the point to the screening result.

[103] Step 5.4: screening results are marked as power line feature points to complete the rapid refinement of the point cloud.

[104] Step 6: single-grade power line point cloud is clustered by using a K-means clustering method;

[105] Specifically, the clustering steps are as follows:

[106] (1) randomly selecting n seed centers, wherein n represents the number of single-grade power lines;

[107] (2) calculating the distance between other objects and the n seed centers by an Euclidean distance method, and dividing other objects into a cluster corresponding to the corresponding cluster center, which represents a cluster;

[108] the 3D space expression of the Euclidean distance method:

[109] p = ; and

[110] (3) recalculating the seed center of each cluster (that is, moving each seed center to the center position of the cluster in which the seed center is located), wherein this position is the position of tire updated seed center; [Hl] recalculating the cluster center each time, wherein the mean value of all other objects in the corresponding cluster is the updated seed center.

[112] A calculation formula for the updated seed center is:

[113] Centerk=^XECXt

[114] Ck represents the number of objects contained in the cluster k, wherein each property of all objects belonging to the cluster is summed and averaged. So Centerk is also an object containing multiple properties.

[115] Steps (2) and (3) are repeated until the termination condition is met:

[116] One condition is when the position of the updated seed center does not change from the previous one (that is, the position of the seed center is already at the center position of other objects in the cluster), and the other method is to terminate the iteration when the sum of squared errors is the local minimum.

[117] Finally, the laser point cloud data of the single-grade power line with clear boundary is obtained.

[118] Step 7: key nodes of each power line in a single grade are sampled, and fitting calculation is performed to complete rapid reconstruction of the power lines;

[119] Specifically, a local centroid scaling projection fitting method is used to carry out fitting reconstruction of the power lines. In this method, key nodes are selected by extracting a local weighted centroid to realize the sampling of the power line point cloud, and the fitting calculation of the scaling projection model is carried out only on the sampled key nodes to realize the power line reconstruction, thereby greatly improving the reconstruction efficiency of the power lines and avoiding the influence of noise points on the reconstruction quality. The specific scaling projection reconstruction model can be expressed in the following mode:

[120] setting a reconstruction straight line projected on an x-O-y plane by a power line point cloud as d = x cos a + y sin a, wherein a represents an angle between the projection straight line and the positive direction of x-axis.

[121] Coordinates of two endpoints of the power line can be obtained by traversing the point cloud, namely (xi, y i) and (xr, yr). The point cloud of the power line is divided into N segments along the trend of the power line, and then a separation interval is:

[122]

[123] I N I A calculation formula of a local centroid is:

[124] (x,y,z) =

[125] In the formula, i represents a point number; n represents the total number of points; m represents the weight of the point, which is calculated by a reciprocal of a distance from the point to the reconstruction straight line, that is, the farther the distance from the reconstruction straight line is, the smaller the weight is.

[126] After the key point (local centroid) of a single power line is extracted, the power line is reconstructed: the point cloud of the local power line is locally reconstructed by the polynomial equation z = A + Bs + Cs2 . In the formula, s is a scale factor of a point Q(x0,y0) on the polynomial equation corresponding to a corresponding point Q'Cx^yp) 011 the reconstruction straight line projected on tire x-O-y plane, in which

[127] —2—, |sin a\ Icos a\ -sin a |sjn ai <icos ai .cos a

[128] In this model, a projection model of a 3D conductor on the vertical plane of the overfitted straight line is regarded as a parabola, and the errors of x value and y value of the point cloud are considered.

[129] According to the least squares principle, a reconstruction process can be transformed into the following S-minimum problem. S represents the sum of squared errors between an actual value and a calculated value of z coordinate in a 3D coordinate system:

[130] S = — (A + Bst + Cs2)]2

[131] i.e.: - = -2^ - (A + Bx, + Cx2}}2 = 0

[132] g = -2^1 ~ (As, + Bs2 + Cs^W = 0 = - (As2 + Bsf + Csf}\2 = 0

[133] In the formulas, A, B and C represent polynomial coefficients respectively; z, and s, represent the coordinate of the reconstruction node on z axis in the 3D coordinate system and the scale factor on the corresponding point of the reconstruction straight line projected on the x-O-y plane respectively; dS / dA, dS / dB and dS / dC are first-order partial derivatives.

[134] Finally, the model is used to complete the fitting reconstruction of the point cloud of the power line.

[135] Step 8: surface texture information of the power lines is integrated to generate a high-accuracy texture model of the power lines.

[136] Tire point cloud data in the power line point cloud model is discretized into a voxel grid; texture coordinates are mapped after coordinate alignment is completed; and finally, the texture information is applied to a voxel crystal grid to form texture information, to obtain the high-accuracy texture model of the power lines. Specifically, the above method can be implemented by the following steps:

[137] (1) a lattice grid is prepared, and it is ensured that the lattice grid covers the region of a required mapping texture. The lattice grid can be generated by using a voxel grid generation algorithm to ensure that the resolution is detailed enough.

[138] Setting a voxel size: determining the voxel size. The voxel size is the side length of a discretized cube unit. The voxel size can be selected according to the application needs and the density of the point cloud data.

[139] Creating the voxel grid: creating a voxel grid by using a function in a point cloud library. The ‘o3d.geometiy.VoxelGrid.createJfomj)oint_cloud’ function in Open3D can be used to create the voxel grid.

[140] (2) Aligning coordinate: a corresponding power line image in the mapping dataset is extracted, and it is ensured that the image and the lattice grid are in the same coordinate system. They should have tire same coordinate origin and scale. If the image is taken by a camera, it needs to ensure that the parameters of the camera are known and that the photo and the lattice grid are aligned in the camera coordinate system.

[141] It can be understood that the LiDAR and the synchronous camera in this method have completed joint calibration before data collection, comprising the registration of the position and angle relationship between the two, so that the 3D point cloud data of the LiDAR and the image taken by the camera can be accurately matched and aligned with the pixel of the corresponding position in the RGB image of the camera. The calibration data will be used to map the 3D point cloud data and the image data of the camera obtained in real time in the 3D scenario to be reconstructed.

[142] (3) Mapping texture coordinate: for each voxel in the lattice grid, the following operation is performed: obtaining the 3D coordinate of the voxel center (usually the center of the eight comer points of the voxel); according to the coordinate system of the lattice grid, converting the 3D coordinate into the coordinate in the camera coordinate system so that the coordinate is mapped onto the photo; projecting the 3D coordinate onto a two-dimensional coordinate on the photo by using the camera parameters (internal and external parameters); and according to the projection results, calculating the UV coordinate on the photo.

[143] (4) Interpolating of texture information: interpolating is performed on the photo to obtain a corresponding texture color value by using the obtained UV coordinate. This assigns a texture color to each voxel. Interpolation can be implemented by using nearest neighbor interpolation, bilinear interpolation, or more complex interpolation techniques, depending on the application needs.

[144] (5) Application of the texture information to the voxel lattice grid: the texture color information obtained by interpolation is applied to each voxel in the lattice grid to obtain a high-accuracy texture model of the power line with color texture information.

[145] At this point, the reconstruction of the high-accuracy texture model of the power line is completed. The model can be used for power line dangerous point detection and investigation (aging, damage, mulch covering, etc.), conductor interphase height difference measurement, distance measurement between conductors, distance measurement from conductors to ground, 3D visualization, sag analysis of conductors, ice analysis of conductors, wind deviation analysis of conductors, etc.

Claims

1. A reconstruction method for power lines based on multi-source data fusion, wherein a flyaway patrol robot collects multi-source data for power lines in a process of walking along a ground wire, prior information is used to conduct rapid 3D reconstruction of the power lines by fusing laser point cloud data and image data, and a high-accuracy texture model of the power lines is generated by combining image texture information, the method comprising the following steps:step 1: patrolling, by a flyaway patrol robot, according to a planned route and obtaining LiDAR point cloud data of a 3D environment of power lines to be reconstructed through an airborne radar;step 2: initially setting a focal length of a synchronous camera according to spacing information of the power lines, controlling the focal length of the synchronous camera through LiDAR depth information feedback, alternatively shooting and collecting RGB image data of power lines in different layers and an overall environment, and establishing a corresponding dataset;step 3: classifying and grading transmission lines by using the prior information and determining LiDAR points of each grade;step 4: reconstructing a ground wire by using position and orientation system information and extracting a power line point cloud model roughly according to structured working conditions of the power lines;step 5: screening and quickly refining colored point cloud by using RGB information of a power line image;step 6: clustering single-grade power line point cloud by using a K-means clustering method;step 7: sampling key nodes of each power line in a single grade, and performing fitting calculation to complete rapid reconstruction of the power lines; andstep 8: integrating surface texture information of the power lines to generate the high-accuracy texture model of the power lines.

2. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein the synchronous camera mentioned in step 2 is a zoom camera, and has been jointly calibrated with LiDAR; the focal length of the synchronous camera can be initially set by using the spacing information of power lines in different layers, and the focal length of the synchronous camera iscontrolled to alternatively shooting the RGB image data of the power lines and the overall environment through the LiDAR depth information feedback; and step 2 comprises the following steps:step 2.1: transmitting the point cloud data collected by the LiDAR to an image processing end, and converting the point cloud data into a depth map;step 2.2: retrieving, by a camera control end, whether a known zoom instruction is loaded; if there is prior information related to spacing information of power lines in different layers, then proceeding to step 2.3; and if not, selecting a middle depth as an initial focus and proceeding directly to step 2.4;step 2.3: converting, by the image processing end, the spacing information of power lines in different layers into corresponding depth information in the depth map through an algorithm, setting a corresponding initial focus distance through the depth information, and issuing an instruction to control the synchronous camera to adjust the focal length;step 2.4: shooting an image, and transmitting the image to the image processing end; performing focal length feedback through definition analysis and depth information comparison; and proceeding to a zoom step for adjusting the focal length of the synchronous camera in real time; andstep 2.5: performing real-time zoom of the synchronous camera, alternatively shooting and collecting the RGB image data of power lines in each layer and the overall environment, and storing into different datasets.

3. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein the classifying and grading transmission lines by using the prior information in step 3 comprises:(1) determining locations and structures of the transmission lines according to geographic information system (GIS) data, comprising geographical coordinates, heights, line routes and the like of a power tower and the power lines;(2) determining connections and structures of the transmission lines according to a line map and topological information, comprising voltage levels of the lines, the number of conductors, the number of phases, and the like; and(3) according to line file data of a power company, determining attributes of the transmission lines, such as voltage levels, pole types, conductor transmission types, and the like.

4. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein in step 4, position and orientation system data is used to obtain the motion trajectory of the flyaway patrol robot, and deformation caused by the weight of a UAV is taken into consideration to obtain a ground wire trajectory according to a correction function, and the structured working conditions of the power lines are used to roughly extract point cloud models for other power lines.

5. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein colored point cloud is screened and quickly refined by using the RGB information of a power line image, wherein step 5 specifically comprises the following steps:step 5.1: defining an RGB color range of the power lines by using high-definition power line images in a mapping dataset;step 5.2: traversing the entire colored point cloud dataset and accessing the RGB color information of each point;step 5.3: deciding whether to keep or discard each point according to conditions of the defined color range of the power lines; andstep 5.4: marking screening results as power line feature points to complete the rapid refinement of the point cloud.

6. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein a local centroid scaling projection fitting method is proposed in step 7 to carry out fitting reconstruction of the power lines; in the method, key nodes are selected by extracting a local weighted centroid to realize the sampling of the power line point cloud, and the fitting calculation of the scaling projection model is carried out only on the sampled key nodes to realize the power line reconstruction, thereby greatly improving the reconstruction efficiency of the power lines and also avoiding the influence of noise points on the reconstruction quality; the specific scaling projection reconstruction model is expressed in the following mode: setting a reconstruction straight line projected on an x-O-y plane by a power line point cloud as d = x cos a + y sin a, and performing local reconstruction for a local power line point cloud by a polynomial equation z - A + Bs + Cs2‘, wherein s is a scale factor of a horizontal coordinate Q(x0, To) °f a point on the polynomial equation corresponding to a corresponding point Q'(Xo,yo) on the reconstruction straight line projected on the x-O-y plane, wherein------Jsin al >Icos al-sm ay'o-y । . ... .-----, sm a <cos acos acoordinates of two endpoints of the power line are obtained by traversing the point cloud, namely (xi,yi) and (xr,yr); the point cloud of the power line is divided into N segments along the trend of the power line, and then a separation interval is:a calculation formula of a local centroid is: , .(x, y, z) — I n , n , n Iin the formula, i represents a point number; n represents the total number of points; m represents the weight of the point, which is calculated by a reciprocal of a distance from the point to the reconstruction straight line, that is, the farther the distance from the reconstruction straight line is, the smaller the weight is; after the key points of a single power line are extracted, the power line is reconstructed; according to the least squares principle, a reconstruction process is transformed into the following S-minimum problem; S represents the sum of squared errors between an actual value and a calculated value of z coordinate in a 3D coordinate system:S = Efe -(4 + Bst + Cs?)]2i.e.:dS57=-25:(2,-(^ + 8^ + ^)12 = 0O / idS— = ^2V[zls, - (As£ + Bs- + Cs3)]2 = 0 uBdS= -2ZM - G4st2 + Bsf + CX4)]2 = 0in the formulas, A, B and C represent polynomial coefficients respectively; and S( represent the coordinate of the reconstruction node on z axis in the 3D coordinate system and the scale factor on the corresponding point of the reconstruction straight line projected on the x-O-y plane respectively; dS / dA, dS dB and dS dC are first-order partial derivatives.J. The reconstruction method for power lines based on multi-source data fusion according to claim 1, wherein in step 8, the point cloud data in the power line point cloud model is discretized into a voxel grid; texture coordinates are mapped after coordinate alignment is completed; and finally, the texture information is applied to a voxel crystal grid to form texture information, to obtain the high-accuracy texture model of the power lines.

Images