A fast reconstruction method for power transmission corridor based on fly-away dual-mode visual information

Abstract

The present invention discloses a method for rapid reconstruction of a power transmission corridor based on fly-away dual-mode visual information, that is, a method for three-dimensional reconstruction of a power transmission corridor in units of sections, which is established by combining multi-source data such as POS information during the movement of a fly-away dual-mode robot in a walking mode along a ground line; the special movement modes of the fly-away dual-mode robot are flying up and down the line, flying over the tower head and walking along the ground line, and the two modes are switchable. The present invention integrates multi-source data, robot motion trajectory and structured working conditions, and combines static and dynamic route reconstruction strategies, which can not only achieve lightweight reconstruction data, quickly establish a scene model of the fly-away dual-mode robot walking on the line, but also reduce the overall reconstruction cost. Compared with the traditional power transmission corridor reconstruction method in flight mode, the present invention has the advantage of being able to utilize prior information such as POS during the walking process along the line, use low-cost equipment, and relatively quickly reconstruct the complete power transmission corridor.

Description

Technical Field

[0001] The invention belongs to the field of intelligent inspection of power lines, and in particular relates to a method for rapid reconstruction of a power transmission corridor based on fly-away dual-mode visual information. Background Art

[0002] In recent years, with the development of the national economy and the improvement of people's living standards, the continuous increase in electricity consumption in social life and production has put forward higher requirements for the stable operation of the power grid. Since long-distance transmission lines have to pass through areas with harsh environments (such as deserts, mountains, forests and rivers), they are easily affected by non-human factors such as rain and snow, material aging, foreign body short circuit, and continuous tension. In order to transmit electricity efficiently and reliably, transmission lines need to be inspected daily to detect faults early and perform maintenance. However, the traditional manual inspection method currently adopted by the power grid mainly relies on human senses and experience, and there are problems such as low inspection efficiency, inadequate inspection, high work intensity, and high difficulty in operation under bad weather. With the development of science and technology, intelligent inspection technology has become mature, and intelligent inspection has become an important way to ensure the safe and reliable operation of transmission lines. Intelligent inspection methods are mainly divided into two types: drone line inspection and climbing robot line inspection. Among them, drones have good flexibility and high test accuracy, but their endurance is poor and the flight distance cannot meet the needs of long-line inspections. They are also easily affected by climate and air traffic control. Robotic line inspection can go deep into the mountains and forests and other inaccessible areas along the power lines to lock the lines for detailed inspections, and ensure the online supply of electricity through downhill power generation and solar charging. However, it is necessary to manually or install a special lifting mechanism on the power tower to complete the robot's up and down the line, and design a complex obstacle crossing mechanism and control system to ensure the robot's online obstacle crossing. Therefore, we proposed a flying dual-mode robot that can retain the advantages of the above two inspection methods and make up for the shortcomings of the two. The flying dual-mode robot performs line inspections by carrying various high-resolution sensors and other equipment. Dual mode refers to a robot mode that can switch between the two motion modes of flying up and down the line, flying over the tower head, and walking along the ground line. The flying dual-mode robot captures data sets during flight and walking along the ground line, obtains point cloud data during the process, and then analyzes and post-processes the point cloud data based on the spatial clustering and texture features of other lines, towers, hardware, other ancillary equipment and surrounding terrain in the transmission corridor, which can achieve scene reconstruction of the transmission corridor environment. The three-dimensionally reconstructed transmission corridor can be used for: spatial measurement, scene simulation, line wind deviation verification and detection and prediction analysis of dangerous points.

[0003] At present, the methods of obtaining point clouds can be mainly divided into two categories: laser radar technology and oblique photography technology. Laser radar technology and oblique photography technology. The former has high data acquisition efficiency, high modeling quality, high measurement accuracy, and low weather and time requirements, but it is expensive and not suitable for large-scale applications; the latter has low cost and good three-dimensional visualization effect, but it is difficult to reconstruct power lines. Therefore, the present invention uses a flying dual-mode robot equipped with a binocular camera to reconstruct the three-dimensional transmission corridor during flight and walking along the ground line. On the one hand, it solves the problem of the high price of the laser radar data acquisition device, and on the other hand, it uses the characteristics of walking on the line to solve the problem that oblique photography technology is difficult to reconstruct power lines. This method can meet the needs of reconstructing a complete transmission corridor at a lower cost in the same field, so that the reconstruction results can be used for the analysis of the inspection process of the transmission corridor, and can also be used for the spatial position prediction of dangerous points. It can also lay the foundation for the refined three-dimensional reconstruction of the transmission corridor.

[0004] Another major difficulty in the reconstruction of the three-dimensional power transmission corridor is the processing of a large amount of point cloud data. The cleverness of this invention lies in the use of prior information such as POS during the walking process of the flying dual-mode robot along the ground line, and the adoption of a static and dynamic segmentation strategy for two route reconstruction. This not only makes the point cloud data lightweight, but also solves the problem that the sequence images taken due to slight jitter of the walking line are partially blurred, resulting in the inability to fully model, and realizes the establishment of a complete power transmission corridor model, and also makes the whole process fast and efficient. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information, so as to solve the two major problems of three-dimensional reconstruction of power transmission corridors in the above-mentioned technology: (1) high modeling cost and difficulty in extending the scope of application; and (2) inability to fully model.

[0006] In order to solve the above technical problems, the present invention provides Figure 1 The technical solution shown in the flowchart of the three-dimensional reconstruction method, and the motion mode diagram of the flying dual-mode robot are shown in Figure 2 , the specific technical solution is as follows:

[0007] A method for fast reconstruction of power transmission corridors based on fly-away dual-mode visual information, characterized in that the point cloud obtained from the data acquired during the walk of the fly-away dual-mode robot along the ground line is post-processed to reconstruct a complete three-dimensional power transmission corridor, including the following steps:

[0008] Step 1: Use a binocular camera to photograph the transmission corridor and collect images of single-stage near-line transmission lines;

[0009] Step 2: Use the signal transmission module to transmit the IMU, GPS and captured image data to the ground terminal, and align the obtained data in time and space;

[0010] Step 3: Use the left and right eye images to perform camera calibration, stereo correction, stereo matching, etc. in turn. The depth information and color information of each pixel are calculated and transformed into the same coordinate system to obtain the point cloud information of the transmission corridor. Combined with the POS data, the accurate motion trajectory of the camera can be obtained.

[0011] Step 4: Use K-means clustering method to cluster the collected point cloud;

[0012] Step 5: Determine whether the point cloud is a static point cloud, that is, segment the moving object (walking line) and the static background (metal fittings, other lines, towers, etc.). If so, proceed to step 6; otherwise, proceed to step 7;

[0013] Step 6: Keep the point cloud information of the static background (metal fittings, other wires, towers, etc.);

[0014] Step 7: For the point cloud of the moving object, upsampling is used to obtain as many point clouds as possible, and then the POS information of the camera motion trajectory in step 3 is used to establish a motion trajectory curve pipeline model; the established curve pipeline model is compared and corrected with the catenary model of the power line to restore the walking trajectory point cloud;

[0015] Step 8: Fuse the point cloud information of the static background in step 6 and the point cloud information of the walking line in step 7, and perform gridding and mapping operations on the fused results to achieve rapid reconstruction of the complete transmission corridor.

[0016] In the above technical solution, step 1 includes the following steps:

[0017] Step 1.1: There are certain requirements for the collected data objects: generally, the 110kV transmission line is divided into orderly sections, the length of a single section is generally 300-500m, the power lines in the single section are uniformly distributed, the line type is the same, and the structural characteristics are prominent;

[0018] Step 1.2: Use a binocular camera to shoot the process of walking along the ground line and record the data. The front camera obtains the images of the power tower, hardware, and power lines in the front field of view to the maximum extent; the side camera collects the side information of the power tower when flying over the tower head; and thus establishes the walking process data set.

[0019] In the above technical solution, the processing of the data set in step 2 is mainly to align the left and right eye images, depth information, IMU and GPS timestamps and spatial positions to produce the data set source format. See the schematic diagram of the image file style after the data set is unpacked and time-space aligned. Figure 3 .

[0020] In the above technical solution, step 3 includes the following steps:

[0021] Step 3.1: Use the left and right eye image data in step 2 to perform camera calibration and stereo correction;

[0022] Step 3.2: Perform stereo matching on the left and right images, generate a disparity matrix, calculate the depth of each pixel, and calculate the 3D coordinates of each pixel through the camera model;

[0023] Step 3.3: Use color image matching and depth images corresponding to the matched feature points to form two point clouds. Use ICP to find the initial pose, and BA optimization to get a more accurate pose. After estimating the pose, perform pose transformation to transform all points into the same coordinate system to obtain the point cloud information of the transmission corridor. See the schematic diagram of the discrete points of the original transmission line point cloud data for details. Figure 4 ;

[0024] Step 3.4: Use the image data in step 2 to fuse the POS information generated by IMU and GPS to restore the accurate motion trajectory of the camera.

[0025] In the above technical solution, in step 4, the number of seed points of the K-means clustering algorithm is set to 7, that is, the point cloud is classified into towers, other wires, insulators, hardware, running wires, tower foundations and grounding devices.

[0026] In the above technical solution, step 5 includes the following steps:

[0027] Step 5.1: Establish the objective functions of the seven models by combining the geometric error based on dense iterative closest point (ICP) alignment with the photometric cost based on the color difference between points in the current live frame and the stored 3D model, and minimize the objective function of each of them separately to track the rigid body pose of each object in the current image frame;

[0028] Step 5.2: Group the points in the 3D model that move consistently together for motion segmentation, i.e. segment the current real-time frame and associate each pixel with a moving model;

[0029] Step 5.3: Assume that the static object moves The motion of a dynamic object is The threshold is if If it is a static object, go to step 6; otherwise, go directly to step 7;

[0030] In the above technical solution, step 6 retains the static background point cloud information for three-dimensional reconstruction in step 8.

[0031] In the above technical solution, step 7 includes the following steps:

[0032] Step 7.1: Upsample the moving object point cloud and expand the currently collected walking line point cloud data pairs by interpolating the point cloud;

[0033] Step 7.2: Fuse the high-definition image sequence file in step 2 with the camera's POS information to obtain the camera motion trajectory, which mainly includes world position coordinates, inclination, motion direction, etc. The camera motion trajectory diagram when the experimental flying dual-mode robot moves along the line is shown in Figure 5 ;

[0034] Step 7.3: Using the obtained camera motion trajectory, establish a pipeline model with the shooting point as the pipeline centerline and the cross-section circle radius as r;

[0035] Step 7.4: Fit the pipeline model using a polynomial function, and use the root mean square error (RMSE) of the distance between the original walking line point and its corresponding fitting point as the evaluation standard for determining the degree of fit of the reconstructed model. After the number of fitting times is greater than or equal to 5, the RMSE value between the original point and the fitting point of the walking line does not change significantly, which can ensure the fitting accuracy of the walking line point cloud. After fully considering the saving of computing resources and taking into account computing efficiency, the number of fitting times of the present invention is set to 5, which is more appropriate.

[0036] In the above technical solution, step 8 includes the following steps:

[0037] Step 8.1: Fusion of the static background point cloud (metal fittings, other lines, towers, etc.) from step 6 and the moving object point cloud (walking line) from step 7;

[0038] Step 8.2: Perform grid surface reconstruction on all fused point clouds and use grid filters with appropriate parameters to filter out irrelevant information;

[0039] Step 8.3: Perform texture mapping on the 3D grid. At this point, the complete transmission corridor can be quickly reconstructed.

[0040] Compared with the prior art, the present invention has the following advantages:

[0041] (1) Compared with the 3D reconstruction method using oblique photography technology, it can quickly reconstruct the complete 3D model of the transmission corridor, reproduce the real scene of the flying dual-mode robot walking along the ground line, and achieve data lightweight;

[0042] (2) Compared with the 3D reconstruction method using LiDAR technology, it can reduce equipment costs, operating costs, and maintenance costs overall;

[0043] (3) The combination of multi-source data prior information, robot walking motion trajectory and structured working conditions is closer to practical applications and is suitable for the intelligent inspection field of flying dual-mode robots. It can also be considered to be expanded to more application occasions in the future. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 This is a flowchart of the method for three-dimensional reconstruction of a power transmission corridor based on fly-away dual-mode visual information of the present invention.

[0045] Figure 2 It is a schematic diagram of the motion mode of the flying dual-mode robot based on the present invention.

[0046] Figure 3 Schematic diagram of the image file style after the dataset is unpacked and spatially aligned.

[0047] Figure 4 Schematic diagram of discrete points of the original transmission line point cloud data.

[0048] Figure 5 This is the camera motion trajectory diagram when the experimental flying dual-mode robot moves along the line. DETAILED DESCRIPTION

[0049] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0050] A method for fast reconstruction of power transmission corridors based on fly-away dual-mode visual information, characterized in that the point cloud obtained from the data acquired during the walk of the fly-away dual-mode robot along the ground line is post-processed to reconstruct a complete three-dimensional power transmission corridor, including the following steps:

[0051] Step 1: Use a binocular camera to photograph the transmission corridor and collect images of single-stage near-line transmission lines;

[0052] Specifically, when building a walking process data set by taking a video of walking along the ground line with a binocular camera, it is necessary to ensure that there are other objects in the field of view in addition to the walking line, such as towers, hardware, other lines, etc. It is necessary to ensure that the transmission corridor being photographed is a 110kV transmission line divided into orderly sections, with a single section length of generally 300 to 500m, and the power lines in the single section are uniformly distributed, have the same line shape, and have prominent structural features;

[0053] Step 2: Use the signal transmission module to transmit the IMU, GPS and captured image data to the ground terminal, and align the obtained data in time and space;

[0054] Specifically, the video data taken in step 1 and the robot's IMU, GPS data, etc. are transmitted to the ground workstation using the signal transmission module for offline operation. By parsing the data set, the video is output as a high-definition image sequence at 5 frames / unit, including two sets of left and right eye images. The left and right eye color images, IMU and GPS timestamps and spatial positions are aligned to produce the data set source format. Pay attention to the storage method.

[0055] The reason why we align the obtained data in time and space is that the timestamps and spatial positions of the left and right images are not strictly aligned one by one, but there is a certain time difference and spatial position difference;

[0056] When making source files, it is recommended to name the source format files according to the corresponding format. The naming format is:

[0057] rName="frame-"+str(count).zfill(6)+".color.jpg"#rename rgb

[0058] dName="frame-"+str(count).zfill(6)+".depth.png"#rename depth

[0059] pName="frame-"+str(count).zfill(6)+".pose.txt"#name pose

[0060] Step 3: Use the left and right eye images to perform camera calibration, stereo correction, stereo matching, etc. in turn. The depth information and color information of each pixel are calculated and transformed into the same coordinate system to obtain the point cloud information of the transmission corridor. Combined with the POS data, the accurate motion trajectory of the camera can be obtained.

[0061] Specifically, step 3.1: Use the left and right eye image data in step 2 to perform camera calibration and stereo correction: Only by knowing the relative position relationship between the two cameras can the distance calculation and depth information estimation be performed; first, the least squares method is used to minimize the error to obtain the best estimated rotation and translation matrices to complete the camera calibration; then use these two matrices to perform epipolar correction;

[0062] Step 3.2: Perform stereo matching on the left and right images, generate a disparity matrix, and calculate the depth of each pixel. According to the principle of similar triangles of the pinhole imaging model and the binocular camera model, calculate the 3D coordinates corresponding to each pixel.

[0063] For a point P in space, its coordinates in the camera coordinate system are (x, y, z), and its coordinates in the pixel coordinate system are (u, v); given that the focal length of the camera is f, the offsets from the origin of the pixel coordinate system to the origin of the imaging plane coordinate system on the x-axis and y-axis are C x and C y , the baseline length of the binocular camera is b, the x-axis and y-axis of the imaging plane coordinate system and the camera coordinate system are consistent, and the z-axis of the camera coordinate system is the outward direction of the camera lens;

[0064] According to the pinhole imaging model, we know that:

[0065] x=(uC x ) / f·z;

[0066] y=(uC y ) / f·z;

[0067] Similar triangles of the stereo camera model:

[0068] Simplified:

[0069]

[0070] Step 4: Use K-means clustering method to cluster the collected point cloud;

[0071] Specifically, the clustering steps are as follows:

[0072] (1) Randomly select 7 seed centers (tower, other wires, insulators, hardware, running wires, tower foundation, and grounding device);

[0073] (2) Use the Euclidean distance method to calculate the distance between other objects and the seven seed centers, and divide other objects into clusters corresponding to the corresponding cluster centers, which represents a cluster;

[0074] The three-dimensional space expression of the Euclidean distance method:

[0075]

[0076] (3) Recalculate the seed center of each cluster (i.e. move each seed center to the center of the cluster to which it belongs), and this position is the updated position of the seed center; each time the cluster center is recalculated, the mean of all other objects in the corresponding cluster is the updated seed center.

[0077] The calculation formula for updating the seed center is:

[0078]

[0079] C krepresents the number of objects contained in the kth cluster. Here, each attribute of all objects belonging to the cluster is summed and averaged. k It is also an object containing multiple properties.

[0080] Repeat steps (2) and (3) until the termination condition is met:

[0081] One method is to terminate the iteration when the position of the updated seed center is the same as the last one (that is, the position of the seed center is already the center position of other objects in the cluster); the other method is to terminate the iteration when the sum of squared errors is locally minimized.

[0082] Step 5: Determine whether the point cloud is a static point cloud, that is, segment the moving object (walking line) and the static background (metal fittings, other lines, towers, etc.). If so, proceed to step 6; otherwise, proceed to step 7;

[0083] Specifically, step 5.1: For each surface map M m , minimize a cost function that includes geometric terms and photometric color terms based on point-surface ICP:

[0084]

[0085] For each surfel map M m , we want to minimize the point-to-surface ICP registration error between (i) the 3D back-projection of the current depth map and (ii) the predicted depth map from the previous frame t-1:

[0086]

[0087] in is the point in the depth map of the current frame, v i and n i is the predicted depth point at time t-1, T m is the transformation that aligns the models at time t-1 and t.

[0088]

[0089] In order to improve efficiency, a four-layer pyramid strategy is adopted.

[0090] Where T m is to align the activity model M between the previous frame t-1 and the current frame m The rigid transformation, I t-1 (π(KT m π -1 (u,D t )) 2 ) is a function that provides the color of the vertices attached to the model in the previous frame t-1.

[0091] In step 5.2, the current real-time frame is segmented in the 3D model, and each pixel is associated with a moving model. This is done to distinguish between static point clouds and dynamic point clouds;

[0092] In step 5.3, we assign a score γ to each point cloud cluster i ∈[0,1], where γ i represents the probability that a cluster belongs to a static object, γ i =0 indicates a moving object, γ i =1 indicates a static object.

[0093] The reason for performing point cloud segmentation of moving objects and static background in step 5 is that the moving objects will interfere with the complete modeling of the background. The moving objects are generally walking lines, which are caused by slight shaking of the camera and shaking of the power lines when the dual-mode robot walks along the ground line.

[0094] Specifically, in step 6, the point cloud information of moving objects is removed: the point cloud clusters of moving objects will be detected and identified as outliers, and pixel-level segmentation and removal will be performed; when removing, the regularization of spatiotemporal distribution should be considered to ensure the smoothness of segmentation. At this point, the point cloud information of static objects will be preserved intact;

[0095] Step 7: Upsample and expand the currently collected walking line point cloud data by interpolating the point cloud method; then use the POS information of the camera motion trajectory in step 3 to establish a trajectory curve pipeline model; fit and correct the established curve pipeline model with the catenary model of the power line to restore the walking line point cloud;

[0096] Specifically, in step 7.4, the point cloud data in the pipeline is used for the experiment, and the number of times the polynomial function and the pipeline model fit is assumed to be 1, 2, ..., n, and the root mean square RMSE of the distance between the shooting point of the original camera and the point of the power line 3D reconstruction model is calculated in the corresponding case. The polynomial function that meets the requirements of the corresponding experimental data (meets the number of times of the fitting degree) is the fitting function; secondly, when the power line point cloud is fitted with the polynomial function of the highest order of 1, 2, ..., p, the size of p should be appropriate, which can reflect the change law of the root mean square error with the number of fittings. Finally, considering multiple factors, the p value of 5 is the most appropriate.

[0097] Step 8: Fuse the point cloud information of the static background in step 6 and the point cloud information of the walking line in step 7, and perform gridding and mapping operations on the fusion results to achieve rapid reconstruction of the complete transmission corridor.

[0098] Specifically, in step 8.2, a non-uniform box grid filter is used to sample the point cloud. The value of the maximum number of points in the grid box, maxNumPoints, is set to the minimum requirement of 6. For the processing of the seams at the texture boundary, the triangular grid strip is placed in the parameter domain and color interpolation is applied to achieve a natural transition of colors between the texture areas on both sides of the boundary.

[0099] In step 8.3, texture mapping rendering is performed, that is, the color image from any camera of the left or right eye is mapped onto the grid;

[0100] So far, the three-dimensional reconstruction result of the complete power transmission corridor is achieved according to the present invention.

Claims

1. A method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information, characterized in that The point cloud obtained from the flying dual-mode robot walking along the ground line is post-processed to reconstruct a complete three-dimensional transmission corridor, including the following steps: Step 1: Use a binocular camera to photograph the transmission corridor and collect images of single-stage near-line transmission lines; Step 2: Use the signal transmission module to transmit the IMU, GPS and captured image data to the ground terminal, and align the obtained data in time and space; Step 3: Use the left and right eye images to perform camera calibration, stereo correction, and stereo matching in turn. The depth information and color information of each pixel are calculated and transformed into the same coordinate system to obtain the point cloud information of the transmission corridor. Combined with POS data, the accurate movement trajectory of the camera can be obtained; Step 4: Use K-means clustering method to cluster the collected point cloud; Step 5: Determine whether the point cloud is a static point cloud, that is, segment the moving object and the static background, where the moving object is the walking line, and the static background is the hardware, other lines, and towers. If so, go to step 6, otherwise go to step 7; Step 6: Keep the point cloud information of the static background, i.e. the point cloud information of the hardware, other wires, and towers; Step 7: For the point cloud of the moving object, upsampling is used to obtain as many point clouds as possible, and then the POS information of the camera motion trajectory in step 3 is used to establish a motion trajectory curve pipeline model; the established curve pipeline model is compared and corrected with the catenary model of the power line to restore the walking trajectory point cloud; Step 8: Fuse the point cloud information of the static background in step 6 and the point cloud information of the walking line in step 7, and perform gridding and mapping operations on the fused results to achieve rapid reconstruction of the complete transmission corridor.

2. A method for rapid reconstruction of a power transmission corridor based on fly-away dual-mode visual information according to claim 1, characterized in that The point cloud information of the single-stage near-line transmission line is obtained by using a binocular camera. Step 1 includes the following steps: Step 1.1: The following requirements are imposed on the collected data objects: 110kV transmission lines divided into ordered sections, with a single section length of 300-500m, consistent distribution of power lines within the single section, the same line type, and prominent structural features; Step 1.2: Use a binocular camera to shoot the process of walking along the ground line and record the data; the front camera obtains the images of the power tower, hardware, and power lines in the front field of view to the maximum extent; the side camera collects the side information of the power tower when flying over the tower head; This is used to establish a walking process dataset.

3. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that Process the data set in step 2, align the left and right eye images, depth information, IMU and GPS timestamps and spatial positions, and produce the data set source format.

4. The method for rapid reconstruction of a power transmission corridor based on fly-away dual-mode visual information according to claim 1 is characterized in that To obtain the point cloud information and the accurate motion trajectory of the camera, step 3 includes the following steps: Step 3.1: Use the left and right eye image data in step 2 to perform camera calibration and stereo correction; Step 3.2: Perform stereo matching on the left and right images, generate a disparity matrix, calculate the depth of each pixel, and calculate the 3D coordinates of each pixel through the camera model; Step 3.3: Use color image matching and the depth information corresponding to the matched feature points to form two point clouds. Use ICP to find the initial pose, and BA optimization to get a more accurate pose. After estimating the pose, perform pose transformation to transform all points into the same coordinate system to obtain the point cloud information of the transmission corridor. Step 3.4: Use the image data in step 2 to fuse the POS information generated by IMU and GPS to restore the accurate motion trajectory of the camera.

5. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that The number of seed points of the K-means clustering algorithm in step 4 is set to 7, that is, the point cloud is classified into towers, other wires, insulators, hardware, running wires, tower foundations, and grounding devices.

6. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that The dynamic point cloud and the static point cloud are distinguished, and two static and dynamic reconstruction routes that do not interfere with each other are established; the step 5 includes the following steps: Step 5.1: Establish the objective functions of the seven models by combining the geometric error based on dense iterative closest point, i.e., ICP, alignment with the photometric cost based on the color difference between points in the current live frame and the stored 3D model, and minimize the objective function of each of them separately to track the rigid body pose of each object in the current image frame; Step 5.2: Group the points in the 3D model that move consistently together for motion segmentation, i.e. segment the current real-time frame and associate each pixel with a moving model; Step 5.3: Assume that the static object moves The motion of a dynamic object is The threshold is if If it is a static object, go to step 6; otherwise, go directly to step 7.

7. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that The static background point cloud information in step 6 is retained for 3D reconstruction in step 8.

8. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that The curved pipeline model is established by using the camera motion trajectory; the catenary model of the power line is used to correct the fitting pipeline model in combination with the upsampled point cloud data, and finally the walking line point cloud is restored. The step 7 includes the following steps: Step 7.1: Upsample the moving object point cloud and expand the currently collected walking line point cloud data pairs by interpolating the point cloud; Step 7.2: Fuse the high-definition image sequence file in step 2 with the camera's POS information to obtain the camera motion trajectory, including world position coordinates, inclination angle, and motion direction; Step 7.3: Using the obtained camera motion trajectory, establish a pipeline model with the shooting point as the pipeline centerline and the cross-section circle radius as r; Step 7.4: Use the polynomial function to fit the pipeline model, and use the root mean square error (RMSE) of the distance between the original walking line point and its corresponding fitting point as the evaluation standard to determine the degree of fit of the reconstructed model. After the RMSE value between the original point and the fitting point of the walking line is greater than or equal to 5 times of fitting, there is no obvious change, which can ensure the fitting accuracy of the walking line point cloud. After fully considering the saving of computing resources and taking into account computing efficiency, the number of fittings is 5.

9. The method for rapid reconstruction of power transmission corridors based on fly-away dual-mode visual information according to claim 1 is characterized in that The static background point cloud and the moving object point cloud are fused, and the fused result is meshed and mapped. The step 8 includes the following steps: Step 8.1: Fusion of the static background point cloud of step 6, i.e., hardware, other lines, and towers, and the moving object point cloud of step 7, i.e., walking lines; Step 8.2: Perform grid surface reconstruction on all fused point clouds and use grid filters with appropriate parameters to filter out irrelevant information; Step 8.3: Perform texture mapping on the three-dimensional grid; at this point, the complete transmission corridor can be quickly reconstructed.

Images