Automatic detection method and system for missing transmission tower components based on multi-source data

Abstract

The present invention discloses an automatic detection method and system for missing components of a power transmission tower based on multi-source data, and relates to the technical field of power transmission tower detection. The present invention uses multiple laser scanners and panoramic cameras to synchronously scan the tower structure in real time, obtain actual point cloud data and panoramic image data of the tower, provide a detailed view of the tower surface, create an actual point cloud model of the tower, and import it into a spatial data model for comparison with an ideal point cloud model of the tower through an HBB algorithm to identify structural missing and damage of the power transmission tower, and obtain component missing data of the power transmission tower; find a panoramic image of a rigid body component corresponding to the component missing data according to the synchronous coordinates, perform secondary verification through a visible light recognition algorithm, accurately identify minor changes and missing of the tower structure, and generate a detailed analysis report, thereby realizing non-contact, high-precision automatic detection of the power transmission tower structure, and greatly improving the detection speed and safety.

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission tower detection, and in particular to a method and system for automatically detecting missing power transmission tower components based on multi-source data. Background Art

[0002] The traditional method of transmission tower inspection mainly relies on manual inspection, which has problems such as low efficiency, low safety and inaccurate test results. Inspectors need to go to the site in person for inspection, which not only consumes a lot of time and human resources, but also poses great safety hazards during high-altitude operations. At the same time, due to the subjectivity and experience differences of manual inspection, different inspectors may have different judgments on the same problem, resulting in inaccurate test results. These problems have gradually exposed the limitations and shortcomings of traditional manual inspection methods in modern transmission tower inspection.

[0003] With the advancement of science and technology and the development of automation technology, automated detection methods have gradually been introduced into the field of transmission tower detection. Automated detection methods can not only improve the accuracy and efficiency of detection, but also greatly reduce labor costs and safety risks. At present, some automated detection systems have emerged. However, these systems mainly rely on a single detection method, mainly focusing on the tilt and torsion detection of the tower. They can only identify simple changes in the tower outline and cannot effectively detect specific problems such as missing, damaged and incorrect installation of tower components. Whether the tower components are consistent with the design, whether the components are missing, whether the components are installed incorrectly, and whether there are deviations in the components during construction are important factors affecting the stability and safety of transmission towers. However, the existing technology lacks effective processing capabilities for these problems, and the scope and depth of detection are insufficient, which cannot fully guarantee the safety and stability of the tower.

[0004] Therefore, there is an urgent need for a more advanced automated detection method that can comprehensively and accurately detect various structural problems of transmission towers and provide detailed detection reports to help maintenance personnel discover and solve potential problems in a timely manner. Summary of the invention

[0005] In view of the problems existing in the existing automatic detection of transmission towers, such as single detection means, insufficient detection range and depth, the present invention proposes an automatic detection method and system for missing transmission tower components based on multi-source data, which can not only effectively improve the accuracy and efficiency of detection, but also comprehensively detect various structural problems of the tower, providing reliable technical guarantee for the maintenance and safe operation of the transmission tower.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] An automatic detection method for missing components of a transmission tower based on multi-source data, the method comprising:

[0008] Use multiple laser scanners and panoramic cameras to scan the tower structure in real time and synchronously to obtain the tower's actual point cloud data and panoramic image data;

[0009] The actual point cloud data of the tower and the panoramic image data are fused to generate an actual point cloud model of the tower;

[0010] Based on the design file of the transmission tower, the ideal point cloud data of the tower is obtained, and an ideal point cloud model of the tower is generated;

[0011] Importing the actual point cloud model of the iron tower and the ideal point cloud model of the iron tower into the spatial data model, performing comparisons through the HBB algorithm, and obtaining the missing data of the components of the transmission iron tower;

[0012] Find the panoramic image of the rigid component corresponding to the missing data of the component according to the synchronous coordinates, and verify whether the missing data is established by combining the visible light recognition algorithm. If established, output the missing data of the component; if not established, re-scan;

[0013] The component missing data is further sorted out to form an analysis report and output.

[0014] As a preferred solution of the present invention, the tower structure is scanned in real time by a panoramic camera, and the collected images are stitched using an image stitching algorithm to form panoramic image data of the tower; the image stitching algorithm is specifically:

[0015] Use SIFT algorithm or ORB algorithm to extract image feature points from images collected by panoramic cameras;

[0016] Use the BF algorithm or FLANN algorithm to match feature points. The matching criterion is to minimize the Euclidean distance between matching feature point pairs.

[0017] The RANSAC algorithm is used to estimate the homography matrix between images, and the Homography matrix is ​​used to transform and stitch images to form panoramic image data of the tower.

[0018] As a preferred solution of the present invention, the actual point cloud data of the tower and the panoramic image data are fused to generate an actual point cloud model of the tower, and the method includes:

[0019] Preprocessing the actual point cloud data of the tower, including noise removal, downsampling and coordinate transformation;

[0020] The pre-processed actual point cloud data of the tower is fused with the panoramic image data using a multi-view stereo matching algorithm to generate an actual point cloud model of the tower;

[0021] The actual point cloud model of the tower is optimized based on the PointNet++ algorithm.

[0022] As a preferred solution of the present invention, the multi-view stereo matching algorithm is specifically:

[0023] Use the SGM algorithm to perform stereo matching on the panoramic image data and calculate the depth map. The SGM algorithm calculates the matching cost by minimizing the energy function. The calculation formula is:

[0024] ;

[0025] In the formula, is the total energy function of the disparity map D; is the matching cost of pixel p, is the disparity value of pixel p, is the disparity value of pixel q; represents a neighborhood pixel pair consisting of adjacent pixels p and q, and N is a set of neighborhood pixel pairs; and is the penalty parameter, It represents the penalty term when the disparity value between adjacent pixels differs by 1. Indicates the penalty term when the disparity value between adjacent pixels is greater than 1;

[0026] The depth map is converted into three-dimensional point cloud data, the formula is:

[0027] ;

[0028] Where (X, Y, Z) represents the coordinates of a three-dimensional point, and the depth value corresponding to the image pixel point (u, v) is d(u, v); f x and f y is the focal length, c x and c y are the principal point coordinates;

[0029] Find the corresponding pixel (u, v) of each 3D point (X, Y, Z) in the panoramic image data, map the color information of the pixel (u, v) to the 3D point cloud data, and fuse it with the actual point cloud data of the tower to generate the actual point cloud model of the tower. The fusion formula is:

[0030] ;

[0031] In the formula, is the actual point cloud model of the tower after fusion. is the actual point cloud data of the tower. is the three-dimensional point cloud data containing color information; α is the weight coefficient.

[0032] As a preferred solution of the present invention, the method of obtaining the ideal point cloud data of the tower based on the design file of the transmission tower and generating the ideal point cloud model of the tower includes:

[0033] Read and parse the design file, extract the geometric structure data of the transmission tower in the design file, including the node coordinates, component types and connection relationships of the tower components;

[0034] Preprocessing and optimizing the geometric structure data, removing redundant points using a neighborhood search algorithm, and performing geometric repair using a greedy algorithm or a least squares method;

[0035] Generate the 3D geometric model of the tower using the optimized geometric structure data, optimize the 3D geometric model of the tower using reverse engineering technology, and repair noise and redundant data;

[0036] Performing Delaunay triangulation on the three-dimensional geometric model using a Delaunay triangulation algorithm, and then converting mesh vertices into ideal point cloud data of the tower using a sampling algorithm;

[0037] The ideal point cloud data of the iron tower is organized into an ideal point cloud model of the iron tower and coordinate transformation is performed, and the ideal point cloud model of the iron tower is transformed into the coordinate system of the actual point cloud model of the iron tower.

[0038] As a preferred solution of the present invention, the method of obtaining the missing data of the components of the transmission tower by comparing through the HBB algorithm includes:

[0039] The alignment matrix T is calculated using the ICP algorithm to preliminarily align the actual point cloud model of the tower with the ideal point cloud model of the tower. The calculation formula of the alignment matrix T is:

[0040] ;

[0041] In the formula, Indicates the actual point cloud model of the tower Points, represents the i-th point in the ideal point cloud model of the tower;

[0042] Applying the preliminary alignment matrix T to the actual point cloud model of the tower to obtain the actual point cloud model of the tower after preliminary alignment;

[0043] A hierarchical bounding box is constructed for the actual point cloud model of the tower and the ideal point cloud model of the tower after preliminary alignment. The octree structure is used to organize the actual point cloud data of the tower and the ideal point cloud data of the tower in layers and regions, which can be expressed as:

[0044] ;

[0045] ;

[0046] In the formula, The octree structure of the actual point cloud model of the tower after preliminary alignment. The octree structure representing the ideal point cloud model of the tower; represents the jth node in the actual point cloud node, represents the jth node in the ideal point cloud node; N is the total number of octree nodes;

[0047] Starting from the root node of the octree, compare the bounding boxes of each layer and calculate the difference of the points inside each bounding box. The formula is:

[0048] ;

[0049] In the formula, is the actual point cloud node, is an ideal point cloud node, is the difference between the actual point cloud node and the ideal point cloud node;

[0050] In the comparison process at each level, the gradient descent algorithm is used to gradually adjust the alignment matrix of the actual point cloud model of the tower, and the search range is narrowed through iterative optimization. The optimization formula is as follows:

[0051] ;

[0052] In the formula, is the alignment matrix for the kth iteration, is the alignment matrix for the k+1th iteration; is the learning rate; is the gradient of the difference function between the actual point cloud model of the tower and the ideal point cloud model of the tower;

[0053] According to the actual point cloud model of the tower and the ideal point cloud model of the tower that are finally aligned after iterative optimization, the difference of each point is calculated to generate a difference matrix;

[0054] A difference threshold is set, and it is determined whether the value in the difference matrix exceeds the difference threshold. The actual point cloud data of the tower greater than the difference threshold is marked as missing points. All missing points are counted and the missing components are determined according to the positions of the missing points, and the missing component data is output.

[0055] As a preferred solution of the present invention, the method of verifying whether the missing is established by combining the visible light recognition algorithm includes:

[0056] Extracting a panoramic image of the rigid component corresponding to the missing data of the component according to the synchronous coordinates of the missing data of the component, and obtaining corresponding image data;

[0057] Performing preprocessing operations of grayscale conversion, noise reduction, and edge detection on the image data;

[0058] Extract feature points from the preprocessed image data using a feature extraction algorithm;

[0059] The extracted feature points are matched with the standard feature points in the tower design file using a feature matching algorithm, and based on the result of the feature matching, it is determined whether there are any missing components.

[0060] The automatic detection system for transmission tower component missing based on multi-source data is applied to the automatic detection method for transmission tower component missing based on multi-source data as described above, and the system comprises:

[0061] A data acquisition module is used to scan the tower structure in real time and synchronously through multiple laser scanners and panoramic cameras to obtain actual point cloud data and panoramic image data of the tower;

[0062] The actual point cloud model module is used to fuse the actual point cloud data of the tower and the panoramic image data to generate an actual point cloud model of the tower;

[0063] The ideal point cloud model module is used to obtain the ideal point cloud data of the tower from the design file of the transmission tower and generate the ideal point cloud model of the tower;

[0064] A point cloud comparison module is used to import the actual point cloud model of the tower and the ideal point cloud model of the tower into a spatial data model, compare them using an HBB algorithm, and obtain missing data of components of the transmission tower;

[0065] A component missing verification module is used to find the panoramic image of the rigid component corresponding to the component missing data according to the synchronous coordinates, and verify whether the missing exists in combination with the visible light recognition algorithm;

[0066] The output module is used to output the missing data of components and further organize them into analysis report output.

[0067] The beneficial effects of the present invention are as follows: using 3D laser scanning technology to obtain accurate 3D point cloud data of the tower, combining panoramic image technology to provide a detailed view of the tower surface, creating an actual point cloud model of the tower, and comparing it with the ideal point cloud model of the tower to identify structural deficiencies and damages of the transmission tower; performing secondary verification through a visible light recognition algorithm, it is possible to accurately identify minor changes and deficiencies in the tower structure, and generate a detailed analysis report, thereby realizing non-contact, high-precision automated detection of the transmission tower structure, and greatly improving detection speed and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0069] Figure 1 is a flow chart of the method of the present invention;

[0070] Figure 2 It is a system modular structure diagram of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0072] like Figure 1 As shown in FIG. 1 , an embodiment of the present invention is provided, which provides an automatic detection method for missing components of a transmission tower based on multi-source data, comprising the following steps:

[0073] S1: Use multiple laser scanners and panoramic cameras to synchronously scan the tower structure in real time to obtain the actual point cloud data and panoramic image data of the tower;

[0074] The tower structure is scanned in real time by a panoramic camera, and the collected images are stitched together using an image stitching algorithm to form panoramic image data of the tower;

[0075] In one embodiment, the image stitching algorithm is specifically:

[0076] Among them, SIFT (Scale-invariant feature transform) or ORB (Oriented FAST and Rotated BRIEF) algorithm is used to extract image feature points from images collected by the panoramic camera;

[0077] Use BF (Brute Force) or FLANN (Fast Library for Approximate Nearest Neighbors) algorithm to match feature points. The matching criterion is to minimize the Euclidean distance between matching feature point pairs.

[0078] The RANSAC (RANdom SAmple Consensus) algorithm is used to estimate the homography matrix between images, and the Homography matrix is ​​used to transform and stitch images to form panoramic image data of the tower.

[0079] S2: Fusing the actual point cloud data of the tower and the panoramic image data to generate an actual point cloud model of the tower;

[0080] In one embodiment, step S2 comprises:

[0081] S21: Preprocessing the actual point cloud data of the tower, including noise removal, downsampling and coordinate transformation;

[0082] Statistical filtering or radius filtering is used to remove noise points in the actual point cloud data of the tower. Voxel filtering or random sampling is used to downsample the actual point cloud data of the tower to reduce the amount of data. The actual point cloud data of the tower is converted into a unified coordinate system through a rigid body transformation matrix.

[0083] S22: using a multi-view stereo matching algorithm to fuse the pre-processed actual point cloud data of the tower with the panoramic image data to generate an actual point cloud model of the tower;

[0084] Preferably, the multi-view stereo matching algorithm is specifically:

[0085] Use the SGM (Semi-Global Matching) algorithm to perform stereo matching on the panoramic image data and calculate the depth map. The SGM algorithm calculates the matching cost by minimizing the energy function. The calculation formula is:

[0086] ;

[0087] In the formula, is the total energy function of the disparity map D; is the matching cost of pixel p, is the disparity value of pixel p, is the disparity value of pixel q; represents the neighborhood pixel pair composed of adjacent pixels p and q, and M is the set of neighborhood pixel pairs; and is the penalty parameter, It represents the penalty term when the disparity value between adjacent pixels differs by 1. Indicates the penalty term when the disparity value between adjacent pixels is greater than 1;

[0088] The depth map is converted into three-dimensional point cloud data, the formula is:

[0089] ;

[0090] Where (X, Y, Z) represents the coordinates of a three-dimensional point, and the depth value corresponding to the image pixel point (u, v) is d(u, v); f x and f y is the focal length, c x and c y are the principal point coordinates;

[0091] Find the corresponding pixel (u, v) of each 3D point (X, Y, Z) in the panoramic image data, map the color information of the pixel (u, v) to the 3D point cloud data, and fuse it with the actual point cloud data of the tower to generate the actual point cloud model of the tower. The fusion formula is:

[0092] ;

[0093] In the formula, The actual point cloud model of the tower is finally integrated. is the actual point cloud data of the tower. is the three-dimensional point cloud data containing color information; α is the weight coefficient.

[0094] S23: Optimize the actual point cloud model of the tower based on the PointNet++ algorithm.

[0095] S3: Acquire the ideal point cloud data of the tower based on the design file of the transmission tower and generate an ideal point cloud model of the tower;

[0096] In one embodiment, step S3 comprises:

[0097] S31: reading and parsing the design file, extracting the geometric structure data of the transmission tower in the design file, including the node coordinates, component types, connection relationships, etc. of the tower components;

[0098] The design file contains the ideal design structure information of the transmission tower. Common formats include DWG, DXF (CAD) and GIM (Geospatial Information Model). Use appropriate CAD / GIM file parsing tools or libraries (such as AutoCADAPI, Open Cascade, GDAL, etc.) to read and parse the design file. When parsing the file, use the corresponding parsing method to convert the file content into geometric structure data according to the format and structure of the design file. For example, for DXF format files, you can use the gdal.ReadDXF(filename) function of the GDAL library; when parsing GIM files, use the GIM data standard parsing method to extract geometric structure data.

[0099] S32: Preprocess and optimize the geometric structure data, including removing redundant data, repairing geometric errors (such as overlapping or discontinuous parts), etc. For example, using a neighborhood search algorithm (such as Kd-tree) to remove redundant points, and using a greedy algorithm or a least squares method to perform geometric repair;

[0100] S33: Generate a three-dimensional geometric model of the tower using the optimized geometric structure data, optimize the three-dimensional geometric model of the tower using reverse engineering technology, repair noise and redundant data, and improve the accuracy and precision of the model;

[0101] S34: Use a meshing algorithm (such as Delaunay triangulation) to divide the 3D geometric model into fine meshes, and then use a sampling algorithm to convert the mesh vertices into ideal point cloud data of the tower;

[0102] S35: Organizing the tower ideal point cloud data into a tower ideal point cloud model, and performing coordinate conversion, converting the tower ideal point cloud model into the coordinate system of the actual point cloud model of the tower, and saving it in a point cloud file format (such as PLY, PCD).

[0103] S4: Import the actual point cloud model of the tower and the ideal point cloud model of the tower into the spatial data model, compare them through the HBB (Hierarchical Bounding Box) algorithm, and obtain the missing data of the components of the transmission tower;

[0104] In a specific embodiment, the missing data of components of a transmission tower is obtained by comparison using an HBB algorithm, and the method includes:

[0105] S41: Use the ICP (Iterative Closest Point) algorithm or the least squares method to calculate the alignment matrix T, and preliminarily align the actual point cloud model of the tower with the ideal point cloud model of the tower;

[0106] The calculation formula of the alignment matrix T is:

[0107] ;

[0108] In the formula, Indicates the actual point cloud model of the tower Points, represents the i-th point in the ideal point cloud model of the tower;

[0109] S42: applying a preliminary alignment matrix T to the actual point cloud model of the tower to obtain the actual point cloud model of the tower after preliminary alignment;

[0110] S43: A hierarchical bounding box is constructed for the actual point cloud model of the tower and the ideal point cloud model of the tower after preliminary alignment. The actual point cloud data of the tower and the ideal point cloud data of the tower are organized hierarchically and regionally using the Octree structure, which is expressed as:

[0111] ;

[0112] ;

[0113] In the formula, The octree structure of the actual point cloud model of the tower after preliminary alignment. The octree structure representing the ideal point cloud model of the tower; represents the jth node of the actual point cloud model of the tower after preliminary alignment, represents the jth node of the ideal point cloud model of the tower; N is the total number of octree nodes;

[0114] S44: Starting from the root node of the octree, compare the bounding boxes of each layer, calculate the difference of the points inside each bounding box, and refine the comparison area layer by layer. The formula is:

[0115] ;

[0116] In the formula, is the actual point cloud node, is an ideal point cloud node, is the difference between the actual point cloud node and the ideal point cloud node;

[0117] The actual point cloud model of the tower and the ideal point cloud model of the tower are compared hierarchically. Through this hierarchical comparison, the areas with large differences can be preliminarily screened out, the amount of calculation can be reduced, and the comparison efficiency can be improved;

[0118] S45: In the comparison process at each level, the alignment matrix of the actual point cloud model of the tower is gradually adjusted using optimization algorithms such as gradient descent and Newton's method to narrow the search range. Through iterative optimization, the matching accuracy between the actual point cloud model of the tower and the ideal point cloud model of the tower is continuously improved. The optimization formula is as follows:

[0119] ;

[0120] In the formula, is the alignment matrix for the kth iteration, is the alignment matrix for the k+1th iteration; is the learning rate; is the gradient of the difference function between the actual point cloud model of the tower and the ideal point cloud model of the tower;

[0121] S46: according to the actual point cloud model of the tower and the ideal point cloud model of the tower finally aligned after iterative optimization, calculating the difference of each point and generating a difference matrix;

[0122] S47: Set a difference threshold, determine whether the value in the difference matrix exceeds the difference threshold, mark the actual point cloud data of the tower that is greater than the difference threshold as missing points, count all missing points and determine the missing components according to the positions of the missing points, and output the component missing data.

[0123] S5: Find the panoramic image of the rigid component corresponding to the missing data of the component according to the synchronous coordinates, and verify whether the missing data is established by combining the visible light recognition algorithm. If so, output the missing data of the component; if not, re-scan;

[0124] In one embodiment, the method of verifying whether the missing is established by combining the visible light recognition algorithm includes:

[0125] S51: extracting a panoramic image of a rigid component corresponding to the missing component data according to the synchronous coordinates of the missing component data, and obtaining corresponding image data;

[0126] S52: Perform pre-processing operations such as grayscale conversion, noise reduction, and edge detection on the image data for subsequent processing

[0127] S53: Extract feature points from the preprocessed image data using feature extraction algorithms (such as SIFT, SURF (Speeded Up Robust Features), ORB, etc.);

[0128] S54: Use a feature matching algorithm (such as FLANN, BFMatcher, etc.) to match the extracted feature points with the standard feature points in the tower design file, and determine whether there is a missing component based on the result of the feature matching.

[0129] S6: Further organize the missing component data into an analysis report and output it.

[0130] like Figure 2 As shown, another embodiment of the present invention provides an automatic detection system for missing components of a power transmission tower based on multi-source data, which is applied to the automatic detection method for missing components of a power transmission tower based on multi-source data as described above, comprising:

[0131] A data acquisition module is used to scan the tower structure in real time and synchronously through multiple laser scanners and panoramic cameras to obtain actual point cloud data and panoramic image data of the tower;

[0132] The actual point cloud model module is used to fuse the actual point cloud data of the tower and the panoramic image data to generate the actual point cloud model of the tower;

[0133] The ideal point cloud model module is used to obtain the ideal point cloud data of the tower from the design file of the transmission tower and generate the ideal point cloud model of the tower;

[0134] A point cloud comparison module is used to import the actual point cloud model of the tower and the ideal point cloud model of the tower into the spatial data model, compare them through the HBB algorithm, and obtain the missing data of the components of the transmission tower;

[0135] The component missing verification module is used to find the panoramic image of the rigid component corresponding to the component missing data according to the synchronous coordinates, and verify whether the missing exists by combining the visible light recognition algorithm;

[0136] The output module is used to output the missing data of components and further organize them into analysis report output.

[0137] In summary, the present invention uses 3D laser scanning technology to obtain accurate 3D point cloud data of the tower, combines panoramic image technology to provide a detailed view of the tower surface, creates an actual point cloud model of the tower, and compares it with the ideal point cloud model of the tower to identify structural defects and damages of the transmission tower; secondary verification is performed through a visible light recognition algorithm, which can accurately identify minor changes and defects in the tower structure and generate a detailed analysis report, thereby realizing non-contact, high-precision automated detection of the transmission tower structure, and greatly improving the detection speed and safety.

[0138] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0139] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0140] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An automatic detection method for missing transmission tower components based on multi-source data, characterized in that: The method comprises: Use multiple laser scanners and panoramic cameras to scan the tower structure in real time and synchronously to obtain the tower's actual point cloud data and panoramic image data; The actual point cloud data of the tower and the panoramic image data are fused to generate an actual point cloud model of the tower, the method comprising: Preprocessing the actual point cloud data of the tower, including noise removal, downsampling and coordinate transformation; The pre-processed actual point cloud data of the tower is fused with the panoramic image data using a multi-view stereo matching algorithm to generate an actual point cloud model of the tower; The multi-view stereo matching algorithm is specifically: Use the SGM algorithm to perform stereo matching on the panoramic image data and calculate the depth map. The SGM algorithm calculates the matching cost by minimizing the energy function. The calculation formula is: ； In the formula, is the total energy function of the disparity map D; is the matching cost of pixel p, is the disparity value of pixel p, is the disparity value of pixel q; represents a neighborhood pixel pair consisting of adjacent pixels p and q, and N is a set of neighborhood pixel pairs; and is the penalty parameter, It represents the penalty term when the disparity value between adjacent pixels differs by 1. Indicates the penalty term when the disparity value between adjacent pixels is greater than 1; The depth map is converted into three-dimensional point cloud data, the formula is: ； Where (X, Y, Z) represents the coordinates of a three-dimensional point, and the depth value corresponding to the image pixel point (u, v) is d(u, v); f x and f y is the focal length, c x and c y are the principal point coordinates; Find the corresponding pixel (u, v) of each 3D point (X, Y, Z) in the panoramic image data, map the color information of the pixel (u, v) to the 3D point cloud data, and fuse it with the actual point cloud data of the tower to generate the actual point cloud model of the tower. The fusion formula is: ； In the formula, is the actual point cloud model of the tower after fusion. is the actual point cloud data of the tower. is the three-dimensional point cloud data containing color information; α is the weight coefficient; Optimizing the actual point cloud model of the tower based on the PointNet++ algorithm; Based on the design file of the transmission tower, the ideal point cloud data of the tower is obtained, and an ideal point cloud model of the tower is generated; Importing the actual point cloud model of the iron tower and the ideal point cloud model of the iron tower into the spatial data model, performing comparisons through the HBB algorithm, and obtaining the missing data of the components of the transmission iron tower; The method of obtaining the missing data of the components of the transmission tower by comparing with the HBB algorithm includes: The alignment matrix T is calculated using the ICP algorithm to preliminarily align the actual point cloud model of the tower with the ideal point cloud model of the tower. The calculation formula of the alignment matrix T is: ； In the formula, Indicates the actual point cloud model of the tower Points, Represents the ideal point cloud model of the tower points; Apply the preliminary alignment matrix T to the actual point cloud model of the tower to obtain the actual point cloud model of the tower after preliminary alignment; A hierarchical bounding box is constructed for the actual point cloud model of the tower and the ideal point cloud model of the tower after preliminary alignment. The octree structure is used to organize the actual point cloud data of the tower and the ideal point cloud data of the tower in layers and regions, which can be expressed as: ； ； In the formula, The octree structure of the actual point cloud model of the tower after preliminary alignment. The octree structure representing the ideal point cloud model of the tower; represents the jth node in the actual point cloud node, represents the jth node in the ideal point cloud node; N is the total number of octree nodes; Starting from the root node of the octree, compare the bounding boxes of each layer and calculate the difference of the points inside each bounding box. The formula is: ； In the formula, is the actual point cloud node, is an ideal point cloud node, is the difference between the actual point cloud node and the ideal point cloud node; In the comparison process at each level, the gradient descent algorithm is used to gradually adjust the alignment matrix of the actual point cloud model of the tower, and the search range is narrowed through iterative optimization. The optimization formula is as follows: ； In the formula, is the alignment matrix for the kth iteration, is the alignment matrix for the k+1th iteration; is the learning rate; is the gradient of the difference function between the actual point cloud model of the tower and the ideal point cloud model of the tower; According to the actual point cloud model of the tower and the ideal point cloud model of the tower that are finally aligned after iterative optimization, the difference of each point is calculated to generate a difference matrix; Setting a difference threshold, determining whether the value in the difference matrix exceeds the difference threshold, marking the actual point cloud data of the tower greater than the difference threshold as missing points, counting all missing points and determining the missing components according to the positions of the missing points, and outputting the component missing data; Find the panoramic image of the rigid component corresponding to the missing data of the component according to the synchronous coordinates, and verify whether the missing data is established by combining the visible light recognition algorithm. If established, output the missing data of the component; if not established, re-scan; The component missing data is further sorted out to form an analysis report and output.

2. The method for automatically detecting missing components of a power transmission tower based on multi-source data according to claim 1 is characterized in that: The tower structure is scanned in real time by a panoramic camera, and the collected images are stitched using an image stitching algorithm to form panoramic image data of the tower; the image stitching algorithm is specifically as follows: Use SIFT algorithm or ORB algorithm to extract image feature points from images collected by panoramic cameras; Use the BF algorithm or FLANN algorithm to match feature points. The matching criterion is to minimize the Euclidean distance between matching feature point pairs. The RANSAC algorithm is used to estimate the homography matrix between images, and the Homography matrix is ​​used to transform and stitch images to form panoramic image data of the tower.

3. The automatic detection method for missing components of a power transmission tower based on multi-source data according to claim 1 is characterized in that: The method of obtaining the ideal point cloud data of the tower based on the design file of the transmission tower and generating the ideal point cloud model of the tower comprises: Read and parse the design file, extract the geometric structure data of the transmission tower in the design file, including the node coordinates, component types and connection relationships of the tower components; Preprocessing and optimizing the geometric structure data, removing redundant points using a neighborhood search algorithm, and performing geometric repair using a greedy algorithm or a least squares method; Generate the 3D geometric model of the tower using the optimized geometric structure data, optimize the 3D geometric model of the tower using reverse engineering technology, and repair noise and redundant data; Performing Delaunay triangulation on the three-dimensional geometric model using a Delaunay triangulation algorithm, and then converting mesh vertices into ideal point cloud data of the tower using a sampling algorithm; The ideal point cloud data of the iron tower is organized into an ideal point cloud model of the iron tower and coordinate transformation is performed, and the ideal point cloud model of the iron tower is transformed into the coordinate system of the actual point cloud model of the iron tower.

4. The method for automatically detecting missing components of a power transmission tower based on multi-source data according to claim 1, characterized in that: The method of verifying whether the missing is established by combining the visible light recognition algorithm includes: Extracting a panoramic image of the rigid component corresponding to the missing data of the component according to the synchronous coordinates of the missing data of the component, and obtaining corresponding image data; Performing preprocessing operations of grayscale conversion, noise reduction, and edge detection on the image data; Extract feature points from the preprocessed image data using a feature extraction algorithm; The extracted feature points are matched with the standard feature points in the tower design file using a feature matching algorithm, and based on the result of the feature matching, it is determined whether there are any missing components.

5. An automated detection system based on the method for automated detection of missing components of a transmission tower based on multi-source data according to any one of claims 1 to 4, characterized in that: The system comprises: A data acquisition module is used to scan the tower structure in real time and synchronously through multiple laser scanners and panoramic cameras to obtain actual point cloud data and panoramic image data of the tower; The actual point cloud model module is used to fuse the actual point cloud data of the tower and the panoramic image data to generate an actual point cloud model of the tower; The ideal point cloud model module is used to obtain the ideal point cloud data of the tower from the design file of the transmission tower and generate the ideal point cloud model of the tower; A point cloud comparison module is used to import the actual point cloud model of the tower and the ideal point cloud model of the tower into a spatial data model, compare them using an HBB algorithm, and obtain missing data of components of the transmission tower; A component missing verification module is used to find the panoramic image of the rigid component corresponding to the component missing data according to the synchronous coordinates, and verify whether the missing exists in combination with the visible light recognition algorithm; The output module is used to output the missing data of components and further organize them into analysis report output.

Images