Power transmission channel obstacle positioning method and system based on binocular vision and COLMAP

Abstract

The invention discloses a power transmission channel obstacle positioning method and system based on binocular vision and COLMAP, and relates to the technical field of intelligent inspection and three-dimensional space perception, and the method comprises the steps: collecting a left image pair and a right image pair of a power transmission channel, carrying out the preprocessing of denoising, distortion correction and image enhancement, generating a depth map through parallax calculation based on the preprocessed image pair, and carrying out the positioning of the obstacle in the power transmission channel. Performing feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, outputting a three-dimensional dense point cloud model of the power transmission channel, segmenting an obstacle region from the dense point cloud model, and calculating the spatial position and size of an obstacle and the distance between the obstacle and the power transmission line through geometric model fitting. And the obstacle risk level is evaluated according to the distance and the size to generate a positioning result. According to the invention, full-process automation from data acquisition to risk early warning is realized, the problems of low efficiency and large error of traditional manual inspection are solved, and reliable technical support is provided for safe operation and maintenance of the power transmission line.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection and three-dimensional spatial perception technology, specifically to a method and system for locating obstacles in power transmission channels based on binocular vision and COLMAP. Background Technology

[0002] With the continuous development of the power system, especially the continuous expansion of the scale of transmission lines, the safe operation of transmission lines has become one of the challenges that the power industry urgently needs to address. The maintenance of transmission lines not only requires regular inspection of the line's operating status, but also timely detection of various obstacles that may exist in the line corridor, especially tall trees and buildings. These obstacles may cause collisions or obstructions to the transmission lines, or even cause power equipment failures, which may lead to large-scale power outages in severe cases. Therefore, effective detection and spatial positioning of obstacles in the transmission line corridor are particularly important.

[0003] Traditional power transmission line inspection methods mainly rely on manual inspection, helicopter inspection, and drone inspection. While these methods have improved inspection efficiency to some extent, they also have many limitations. Manual inspection is labor-intensive, costly, and easily affected by external factors such as weather and geographical conditions. Although helicopter inspection is fast, its high cost makes it unsuitable for large-scale, long-term inspections. Drone inspection has the advantage of low cost and flexibility, but in complex environments, especially in areas with complex terrain such as forests or mountains, drones cannot achieve efficient and accurate obstacle location due to limitations in flight altitude, field of view, and image quality. In addition, most existing obstacle detection methods are based on single sensor data acquisition. Due to limitations in sensor accuracy and the complexity of the working environment, traditional methods have insufficient adaptability and accuracy in complex environments. Summary of the Invention

[0004] In view of the above-mentioned existing problems, the present invention provides a method and system for locating obstacles in power transmission channels based on binocular vision and COLMAP, in order to solve the problems of lack of comprehensive perception of three-dimensional spatial structure and great susceptibility to environmental influence in the prior art.

[0005] To address the aforementioned technical problems, a method for locating obstacles in power transmission channels based on binocular vision and COLMAP is proposed, including:

[0006] The system acquires left and right image pairs of the power transmission channel and performs preprocessing such as denoising, distortion correction, and image enhancement. Based on the preprocessed image pairs, a depth map is generated through disparity calculation and optimized. The optimized depth map is then input into the COLMAP system to perform feature extraction and matching, sparse point cloud generation, and dense point cloud reconstruction, outputting a 3D dense point cloud model of the power transmission channel. Obstacle regions are segmented from the dense point cloud model, and the spatial location, size, and distance from the power transmission line of the obstacles are calculated through geometric model fitting. The obstacle risk level is assessed based on the distance and size, generating the positioning result.

[0007] As a preferred embodiment of the power transmission channel obstacle localization method based on binocular vision and COLMAP described in this invention, the preprocessing includes: acquiring left and right image pairs of the power transmission channel and performing preprocessing such as denoising, distortion correction and image enhancement.

[0008] Image noise is removed by sorting neighboring pixels and taking the median value. Based on camera calibration parameters, image distortion is corrected by radial and tangential distortion models. Image contrast is enhanced by redistributing pixel grayscale value ranges.

[0009] As a preferred embodiment of the power transmission channel obstacle localization method based on binocular vision and COLMAP described in this invention, the disparity calculation includes generating a depth map based on preprocessed image pairs through disparity calculation, and optimizing the depth map.

[0010] The horizontal offset is calculated by the difference in pixel values ​​between the left and right images. Combined with the camera focal length and baseline distance, the disparity value is converted into a depth value. A matching algorithm that combines local and global optimization is then used to refine the disparity accuracy of the tree area.

[0011] As a preferred embodiment of the obstacle localization method for power transmission channels based on binocular vision and COLMAP described in this invention, the output three-dimensional dense point cloud model of the power transmission channel includes: inputting the optimized depth map into the COLMAP system, performing feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, and outputting the three-dimensional dense point cloud model of the power transmission channel.

[0012] Scale-invariant feature points are extracted from the viewpoint image. Feature points are matched by nearest neighbor search, and matching pairs with a distance ratio exceeding a threshold are removed. Sparse point clouds are generated using camera pose estimation and triangulation. The point cloud density is optimized by multi-view stereo matching to generate dense point clouds containing details of obstacle surfaces.

[0013] As a preferred embodiment of the power transmission channel obstacle localization method based on binocular vision and COLMAP described in this invention, the segmentation of the obstacle region includes: fitting the obstacle geometric model using a sampling consistency algorithm, calculating the obstacle center coordinates, calculating the height using the vertical coordinate range of the obstacle point cloud, calculating the diameter using the model radius, and calculating the safety distance based on the shortest distance formula from the obstacle center to the power transmission line equation.

[0014] As a preferred embodiment of the power transmission channel obstacle localization method based on binocular vision and COLMAP described in this invention, the multi-sensor data fusion includes introducing GPS and IMU data to assist camera pose estimation, and fusing lidar point clouds and binocular vision point clouds through a filtering algorithm to correct data errors in low light and rain / snow environments.

[0015] Real-time optimizations include using parallel computing for point cloud segmentation and geometric fitting steps, setting the drone flight interval to 50 meters to acquire images, and ensuring a 70% overlap rate between adjacent images.

[0016] As a preferred embodiment of the power transmission channel obstacle localization method based on binocular vision and COLMAP described in this invention, the obstacle risk level assessment includes setting a tree height safety threshold and a distance safety threshold. When the obstacle height exceeds the tree height threshold and the distance is less than the distance threshold, it is marked as a high-risk obstacle.

[0017] The generation of positioning results includes fitting a cylindrical model using a random sampling consensus algorithm, calculating the coordinates of the tree trunk center, calculating the tree height and trunk diameter, and calculating the safe distance based on the power transmission line equation.

[0018] The formula for calculating the coordinates of the tree trunk center is expressed as follows:

[0019]

[0020] Among them, (x i ,y i (x0, y0) represents the coordinates of the i-th point in the point cloud, (x0, y0) represents the coordinates of the center point of the tree trunk, r represents the radius of the tree trunk, N represents the total number of points in the point cloud, and ∈ represents the error in the fitting process.

[0021] As a preferred embodiment of the power transmission channel obstacle localization system based on binocular vision and COLMAP described in this invention, it is characterized by including a data acquisition and preprocessing module, a depth map generation and optimization module, a dense point cloud reconstruction module, and an obstacle localization and risk assessment module.

[0022] The data acquisition and preprocessing module is used to acquire left and right image pairs of the power transmission channel in real time, and to remove environmental noise by median filtering with the median value of the neighboring pixels sorted. Based on the camera calibration parameters, the module corrects lens distortion through radial distortion model and tangential distortion model, and enhances image contrast by histogram equalization.

[0023] The depth map generation and optimization module is used to generate depth information based on the preprocessed image pairs through disparity calculation. First, it calculates the initial depth value according to the disparity-depth conversion formula, and then uses an algorithm that combines local matching and global optimization to improve the disparity accuracy of the tree area. The depth map is smoothed by median filtering, and noise is removed based on the depth threshold to eliminate dynamic interference.

[0024] The dense point cloud reconstruction module is used to convert the depth map into a 3D model, extract scale-invariant feature points by constructing a scale space through Gaussian blur, match feature points by nearest neighbor search and remove low-quality matching pairs, calculate the camera pose based on the essential matrix and generate a sparse point cloud through triangulation, and generate a dense point cloud containing tree trunk texture by multi-view stereo matching optimization.

[0025] The obstacle localization and risk assessment module is used to extract obstacle spatial information from point cloud and assess risk. It calculates the trunk center coordinates by fitting a cylindrical model with RANSAC, calculates the tree height using the vertical range formula, calculates the trunk size using the diameter formula, calculates the safe distance based on the transmission line equation using the shortest distance formula, and marks the risk level according to the dual threshold conditions.

[0026] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a method for locating power transmission channel obstacles based on binocular vision and COLMAP.

[0027] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for locating power transmission channel obstacles based on binocular vision and COLMAP.

[0028] The beneficial effects of this invention are as follows: Preprocessing with median filtering, distortion correction, and histogram equalization improves the sharpness of tree edges by over 40%, providing high-quality image input for subsequent processing; the disparity calculation algorithm combining local matching and global optimization reduces disparity error in tree areas by 62%, significantly improving depth map accuracy; and through multi-view stereo matching and feature optimization, it generates a density exceeding 10^5 points / m. 2The dense point cloud reconstruction achieves a 2.1-fold improvement in completeness compared to traditional methods. RANSAC cylindrical fitting is used to precisely quantify tree trunk spatial parameters, achieving millimeter-level precision with tree height measurement errors of <2cm and distance errors of <1cm. Combined with GPS / IMU data fusion and a 70% overlap acquisition strategy, point cloud completeness remains >90% even under complex weather conditions, and images are processed in real-time at a flight speed of 5m / s. A dual-threshold judgment mechanism for tree height and distance improves risk assessment accuracy to 98%, achieving full automation from data acquisition to risk warning. This solves the problems of low efficiency and large errors in traditional manual inspections, providing reliable technical support for the safe operation and maintenance of power transmission lines. Attached Figure Description

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating the overall process of a power transmission channel obstacle localization method based on binocular vision and COLMAP, as provided in one embodiment of the present invention.

[0031] Figure 2 The left and right views are schematic diagrams obtained by a binocular vision system for a method for locating obstacles in a power transmission channel based on binocular vision and COLMAP, which is an embodiment of the present invention.

[0032] Figure 3 A schematic diagram of point cloud fusion based on spatial volume for a method of locating obstacles in power transmission channels based on binocular vision and COLMAP, provided in an embodiment of the present invention.

[0033] Figure 4 The flowchart shows a system scheme for a power transmission channel obstacle localization system based on binocular vision and COLMAP, as provided in one embodiment of the present invention. Detailed Implementation

[0034] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0035] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0036] Secondly, the term "an embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places throughout this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive, either alone or selectively, with other embodiments.

[0037] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0038] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0039] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0040] Example 1, referring to Figures 1-3 This is the first embodiment of the present invention, which provides a method for locating obstacles in power transmission channels based on binocular vision and COLMAP, including:

[0041] S1: Acquire left and right image pairs of the power transmission channel, and perform preprocessing such as denoising, distortion correction and image enhancement. Based on the preprocessed image pairs, generate a depth map by parallax calculation and optimize the depth map.

[0042] The acquisition of left and right image pairs of the power transmission channel involves the UAV acquiring image pairs from the left and right cameras in real time during flight (each image pair is spaced 0.5 seconds apart). The UAV automatically captures an image pair every 50 meters of flight. The image acquisition frequency of 2 frames per second during flight ensures continuous image data. During flight, the horizontal overlap between adjacent images is approximately 70%, ensuring the accuracy of subsequent image matching and 3D reconstruction. All image data is uploaded to the ground station in real time and stored on a high-speed memory card for subsequent processing. Each image pair has a resolution of 6000×4000 pixels, and the spacing between the left and right images is set to 0.3 meters to ensure sufficient parallax between each image pair for generating accurate depth maps. To avoid time errors caused by changes in UAV attitude, all image captures are synchronized with GPS timestamps to ensure accurate registration of subsequent images.

[0043] During flight, real-time image quality monitoring is conducted to ensure that the acquired images meet requirements. Monitoring points include checking the exposure level of the images in real time through the UAV control system to avoid loss of detail due to overexposure or underexposure, especially in low-light and unevenly lit forest environments where exposure control is crucial; and confirming that the imaging pairs of the binocular cameras are aligned to ensure that the images are parallel and have a consistent perspective, avoiding image distortion caused by camera misalignment.

[0044] The preprocessing includes real-time image acquisition by left and right binocular cameras during UAV flight, and image quality improvement through preprocessing operations such as denoising, distortion correction, and image enhancement. The preprocessing algorithms include denoising, distortion correction, image enhancement, and histogram equalization to optimize image details.

[0045] Denoising processing includes using a median filtering algorithm to denoise the image. For each pixel, the values ​​of its surrounding pixels are sorted, and the median value is selected to replace the current pixel value. This effectively removes noise while preserving edge information. The formula is expressed as:

[0046]

[0047] Where f(x′,y′) represents the pixel value of the original image, and N(x,y) is the set of neighborhood pixels centered at pixel (x,y). out (x,y) represents the pixel value after denoising, and (x′,y′) represents the coordinates after distortion correction;

[0048] Distortion correction involves applying the camera's distortion model (radial and tangential distortion) to correct the image using the camera's intrinsic parameters and distortion parameters. Radial distortion mainly manifests as magnification and reduction between the image center and edges, and is corrected using a perspective distortion model. The distortion model is represented as follows:

[0049] x′=x+(k1d2 +k2d 4 +k3d 6 )x

[0050] y′=y+(k1d 2 +k2d 4 +k3d 6 )y

[0051] Where (x,y) are the pixel coordinates in the original image, (x′,y′) are the coordinates after distortion correction, d is the distance of the pixel from the center of the image, and k1, k2 and k3 are the distortion coefficients.

[0052] Image enhancement involves redistributing the grayscale range of an image to enhance its contrast, making it easier to extract detailed information, especially the texture and contours of trees. The histogram equalization formula is expressed as:

[0053]

[0054] Among them, H eq (x,y) represents the enhanced pixel value, p(j) represents the frequency of occurrence of pixel value j in the original image, I(x,y) represents the pixel value in the original image, M represents the total number of pixels in the image, and A represents the maximum grayscale value of the pixel value.

[0055] Image noise is removed by sorting neighboring pixels and taking the median value. Based on camera calibration parameters, image distortion is corrected by radial and tangential distortion models. Image contrast is enhanced by redistributing pixel grayscale value ranges.

[0056] like Figure 2 The process of generating a depth map through disparity calculation includes: generating a depth map based on preprocessed image pairs through disparity calculation, and optimizing the depth map; after completing data preprocessing, calculating a disparity map, which describes the offset of the same scene point in two images on the horizontal coordinate; the larger the disparity value, the farther away from the camera, and vice versa; the left and right views (i.e., the left and right images) obtained by the binocular vision system can calculate the depth information of each pixel based on disparity, involving disparity matching, depth estimation, and disparity map generation.

[0057] Calculating a disparity map involves determining the difference in pixel position between the same object in two images (left and right). By calculating disparity, depth information of the object is obtained. The relationship between disparity D and depth C is as follows:

[0058]

[0059] Where f′ is the focal length of the camera, B is the baseline distance of the binocular camera, D is the disparity value of the corresponding point of the object in the left and right images, which is obtained through image registration and feature matching, and C is the depth.

[0060] In tree-covered areas, the parallax of a stereo camera is typically large, meaning the trees are close to the camera. To improve the depth estimation accuracy in tree areas, the SGBM (Semi-Global Block Matching) algorithm is used for parallax calculation, expressed by the following formula:

[0061]

[0062] Among them, I L (m,n) represents the pixel values ​​in the left image, I R (m,n) represents the pixel value in the right image, D(m,n) represents the disparity of pixel (m,n), and λ represents the smoothing parameter.

[0063] The horizontal offset is calculated by the difference in pixel values ​​between the left and right images. Combined with the camera focal length and baseline distance, the disparity value is converted into a depth value. A matching algorithm that combines local and global optimization is then used to refine the disparity accuracy of the tree area.

[0064] In the embodiments of this application, disparity optimization includes using an algorithm that combines local matching (enhancing tree texture) and global optimization (disparity continuity constraint) to calculate a disparity map and convert it into a depth value.

[0065] In one alternative implementation, disparity optimization includes constructing a pixel-level matching cost cube (using Census transform to calculate texture differences), performing cost aggregation along eight directions, enhancing the vertical linear features of the tree trunk, dynamically programming to solve for the optimal disparity, enabling a penalty factor for the canopy region (to suppress mismatches of branches and leaves), and refining the disparity value through sub-pixel interpolation.

[0066] In another alternative implementation, disparity optimization includes inputting preprocessed binocular image pairs, extracting multi-scale tree features through convolutional layers (the backbone network uses a lightweight MobileNetV3), constructing a cost volume and performing 3D convolutional regularization (to suppress rain and fog noise), regressing the output disparity map, and calculating depth by combining focal length and baseline.

[0067] S2: Input the optimized depth map into the COLMAP system, perform feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, and output a three-dimensional dense point cloud model of the power transmission channel.

[0068] Furthermore, the three-dimensional dense point cloud model of the output power transmission channel includes inputting the optimized depth map into the COLMAP system, performing feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, and outputting the three-dimensional dense point cloud model of the power transmission channel.

[0069] like Figure 3The dense point cloud reconstruction includes generating a high-precision three-dimensional point cloud of the tree region by using disparity information obtained from binocular images. Specifically, it includes feature extraction and matching, sparse point cloud generation, and dense point cloud reconstruction, which can obtain the three-dimensional structure of the trees and their surrounding environment, providing accurate data for subsequent tree positioning and analysis.

[0070] In this embodiment of the application, the dense point cloud reconstruction includes generating a sparse point cloud through SIFT feature extraction, FLANN matching, and triangulation, and then generating a dense point cloud through multi-view stereo optimization.

[0071] In one alternative implementation, FAST corner detection is combined with BRIEF descriptors to generate ORB features, and Hamming distance matching is performed. Matching pairs with a distance ratio <0.8 are retained. The fundamental matrix is ​​solved using RANSAC to replace the essential matrix, and a dense point cloud is generated based on patch multiview stereo (PMVS).

[0072] In another alternative implementation, the dense point cloud reconstruction includes performing bilateral filtering to denoise the binocular depth map, backprojecting the depth map into a 3D point cloud, fusing the multi-view point cloud using TSDF (truncated symbolic distance function), and extracting the surface mesh using the moving cube algorithm.

[0073] Feature extraction and matching include using the SIFT feature matching algorithm to extract and match feature points between two images. The SIFT (Scale Invariant Feature Transform) algorithm can effectively detect and describe key points in an image, enabling matching even when the images are rotated, scaled, or have changes in lighting, thus exhibiting strong robustness.

[0074] The SIFT algorithm includes applying a multi-scale Gaussian blur to the image to find local extrema. Within each scale space, potential keypoints are located by detecting these extrema. Descriptors are generated by calculating the gradient direction and intensity around each keypoint. These descriptors form the basis for subsequent matching. The formula is as follows:

[0075] L(x,y,σ)=G(x,y,σ)*I(x,y)

[0076]

[0077] Where (x,y) are the pixel coordinates in the original image, I(x,y) are the pixel values ​​in the original image, * is the convolution operation, σ is the scale-adaptive control parameter for feature extraction, G(x,y,σ) is the Gaussian filter function used to construct the scale space, and L(x,y,σ) is the tree feature response map of the power transmission channel image in the scale space.

[0078] Feature matching involves extracting feature points from the left and right images, finding corresponding feature points in the two images through feature matching, and reconstructing the three-dimensional space using the feature points. This allows for rapid feature matching on large-scale datasets, reducing computation time.

[0079] Scale-invariant feature points are extracted from the viewpoint image. Feature points are matched by nearest neighbor search, and matching pairs with a distance ratio exceeding a threshold are removed. Sparse point clouds are generated using camera pose estimation and triangulation. The point cloud density is optimized by multi-view stereo matching to generate dense point clouds containing details of obstacle surfaces.

[0080] Generating a sparse point cloud involves constructing a preliminary 3D point cloud containing the scene structure. The sparse point cloud is mainly composed of feature points in the image. Although the point cloud density is low, it can effectively reflect the basic geometric shape of objects and scenes. Camera pose estimation is also performed. COLMAP uses the essential matrix and fundamental matrix to estimate the external pose of the camera. After calculating the camera position for each image, COLMAP uses a triangulation algorithm to estimate the precise position of each feature point in 3D space. For each pair of matching feature points, the 3D coordinates are calculated using the camera projection matrix and image coordinates to generate the sparse point cloud.

[0081] It should be noted that the formula for camera pose estimation is:

[0082] V = [t] × R

[0083] Where V is the essential matrix, [t] × Let R be the antisymmetric matrix of the translation vector, and R be the rotation matrix;

[0084] The formula for the triangulation algorithm is expressed as follows:

[0085]

[0086] Where J is the position of the 3D point, P1 and P2 are the projection matrices of the camera, and X1 and X2 are the matched feature points in the image.

[0087] S3: Segment the obstacle region from the dense point cloud model, calculate the spatial location, size, and distance from the power transmission line of the obstacle by fitting the geometric model, assess the obstacle risk level based on the distance and size, and generate the positioning result.

[0088] Furthermore, the segmentation of the obstacle region includes fitting the obstacle geometric model using a sampling consistency algorithm, calculating the obstacle center coordinates, calculating the height using the vertical coordinate range of the obstacle point cloud, calculating the diameter using the model radius, and calculating the safe distance based on the shortest distance formula from the obstacle center to the transmission line equation.

[0089] Multi-sensor data fusion includes introducing GPS and IMU data to assist in camera pose estimation, and fusing LiDAR point clouds and binocular visual point clouds through filtering algorithms to correct data errors in low light and rain / snow environments.

[0090] Real-time optimizations include using parallel computing for point cloud segmentation and geometric fitting steps, setting the drone flight interval to 50 meters to acquire images, and ensuring a 70% overlap rate between adjacent images.

[0091] Furthermore, the assessment of obstacle risk level includes setting a tree height safety threshold and a distance safety threshold. When the obstacle height exceeds the tree height threshold and the distance is less than the distance threshold, it is marked as a high-risk obstacle.

[0092] The generation of positioning results includes fitting a cylindrical model using a random sampling consensus algorithm, fitting a cylindrical model based on the RANSAC algorithm, calculating the geometric center of the tree trunk, calculating the tree height and trunk diameter, and calculating the safe distance according to the power transmission line equation.

[0093] The RANSAC algorithm formula is expressed as follows:

[0094] (a-x0) 2 +(b-y0) 2 =r 2

[0095] The formula for calculating the coordinates of the tree trunk center is expressed as follows:

[0096]

[0097] Among them, (x i ,y i Let (x0, y0) be the coordinates of the i-th point in the point cloud, (x0, y0) be the coordinates of the center point of the tree trunk, r be the radius of the tree trunk, N be the total number of points in the point cloud, ∈ be the error in the fitting process, and (a, b) be the two-dimensional projection coordinates of the cross-section of the tree trunk in the dense point cloud.

[0098] Calculating the spatial location, size, and distance from power lines of obstacles includes extracting different parts of trees from the point cloud to calculate tree height, crown width, and trunk diameter. The formula for calculating tree height is as follows:

[0099] H tree =Z max -Z min

[0100] Among them, Z max The maximum Z-coordinate in the tree point cloud, Z min H is the minimum Z-coordinate in the tree point cloud. tree The actual height of the tree;

[0101] The formula for calculating the diameter of a tree trunk is expressed as follows:

[0102] D tree =2r tree

[0103] Among them, D tree To calculate the obtained trunk diameter, r tree The trunk radius after fitting the point cloud of the trunk;

[0104] The formula for calculating the width of a tree crown is as follows:

[0105]

[0106] Where (a1,b1) and (a2,b2) are the coordinates of the two farthest points in the canopy point cloud, W crown This refers to the width of the tree crown.

[0107] Furthermore, the calculation of the distance between trees and power transmission lines includes obtaining the distance by calculating the perpendicular distance from the center point of the tree to the straight line, expressed by the formula:

[0108]

[0109] Where (x0, y0) are the coordinates of the tree trunk center point, S, U, and O are the coefficients of the straight line equation of the transmission line in a two-dimensional plane (parallel to the ground), obtained by fitting UAV aerial images or lidar point clouds, d line The shortest horizontal distance from the center of the tree to the power transmission line is used to determine whether the tree constitutes a potential obstacle based on the minimum distance between the tree and the power transmission line and the size of the tree. If the height of the tree is greater than the minimum safe distance from the power transmission line and the distance between the tree and the line is less than the safe distance, the tree is determined to be an obstacle and needs to be pruned and removed; otherwise, the tree is determined not to be an obstacle.

[0110] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0111] Example 2, refer to Figure 4 The second embodiment of the present invention provides a power transmission channel obstacle localization system based on binocular vision and COLMAP, including a data acquisition and preprocessing module, a depth map generation and optimization module, a dense point cloud reconstruction module, and an obstacle localization and risk assessment module.

[0112] The data acquisition and preprocessing module is used to acquire left and right image pairs of the power transmission channel in real time, and to remove environmental noise by median filtering with the median value of the neighboring pixels sorted. Based on the camera calibration parameters, the module corrects lens distortion through radial distortion model and tangential distortion model, and enhances image contrast by histogram equalization.

[0113] The depth map generation and optimization module is used to generate depth information based on the preprocessed image pairs through disparity calculation. First, it calculates the initial depth value according to the disparity-depth conversion formula, and then uses an algorithm that combines local matching and global optimization to improve the disparity accuracy of the tree area. The depth map is smoothed by median filtering, and noise is removed based on the depth threshold to eliminate dynamic interference.

[0114] The dense point cloud reconstruction module is used to convert the depth map into a 3D model, extract scale-invariant feature points by constructing a scale space through Gaussian blur, match feature points by nearest neighbor search and remove low-quality matching pairs, calculate the camera pose based on the essential matrix and generate a sparse point cloud through triangulation, and generate a dense point cloud containing tree trunk texture by multi-view stereo matching optimization.

[0115] The obstacle localization and risk assessment module is used to extract obstacle spatial information from point cloud and assess risk. It calculates the trunk center coordinates by fitting a cylindrical model with RANSAC, calculates the tree height using the vertical range formula, calculates the trunk size using the diameter formula, calculates the safe distance based on the transmission line equation using the shortest distance formula, and marks the risk level according to the dual threshold conditions.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

[0117] Example 3, the third embodiment of the present invention, differs from the previous two embodiments in that:

[0118] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0119] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0120] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0121] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

Claims

1. A method for locating obstacles in power transmission channels based on binocular vision and COLMAP, characterized in that: include, The left and right image pairs of the power transmission channel are acquired and preprocessed with denoising, distortion correction and image enhancement. Based on the preprocessed image pairs, a depth map is generated by parallax calculation and the depth map is optimized. The optimized depth map is input into the COLMAP system to perform feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, and output a three-dimensional dense point cloud model of the power transmission channel. The obstacle region is segmented from the dense point cloud model. The spatial location, size, and distance from the power transmission line of the obstacle are calculated by fitting a geometric model. The risk level of the obstacle is assessed based on the distance and size, and the positioning result is generated.

2. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 1, characterized in that: The preprocessing includes acquiring left and right image pairs of the power transmission channel and performing preprocessing such as denoising, distortion correction, and image enhancement. Image noise is removed by sorting neighboring pixels and taking the median value. Based on camera calibration parameters, image distortion is corrected by radial and tangential distortion models. Image contrast is enhanced by redistributing pixel grayscale value ranges.

3. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 2, characterized in that: The disparity calculation includes generating a depth map based on the preprocessed image pairs through disparity calculation, and optimizing the depth map; The horizontal offset is calculated by the difference in pixel values ​​between the left and right images. Combined with the camera focal length and baseline distance, the disparity value is converted into a depth value. A matching algorithm that combines local and global optimization is then used to refine the disparity accuracy of the tree area.

4. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 3, characterized in that: The three-dimensional dense point cloud model of the output transmission channel includes inputting the optimized depth map into the COLMAP system, performing feature extraction and matching, sparse point cloud generation and dense point cloud reconstruction, and outputting the three-dimensional dense point cloud model of the transmission channel. Scale-invariant feature points are extracted from the viewpoint image. Feature points are matched by nearest neighbor search, and matching pairs with a distance ratio exceeding a threshold are removed. Sparse point clouds are generated using camera pose estimation and triangulation. The point cloud density is optimized by multi-view stereo matching to generate dense point clouds containing details of obstacle surfaces.

5. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 4, characterized in that: The segmentation of the obstacle region includes fitting the obstacle geometric model using a sampling consistency algorithm, calculating the obstacle center coordinates, calculating the height using the vertical coordinate range of the obstacle point cloud, calculating the diameter using the model radius, and calculating the safe distance based on the shortest distance formula from the obstacle center to the transmission line equation.

6. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 5, characterized in that: Multi-sensor data fusion includes introducing GPS and IMU data to assist in camera pose estimation, and fusing LiDAR point clouds and binocular visual point clouds through filtering algorithms to correct data errors in low light and rain / snow environments. Real-time optimizations include using parallel computing for point cloud segmentation and geometric fitting steps, setting the drone flight interval to 50 meters to acquire images, and ensuring a 70% overlap rate between adjacent images.

7. The method for locating obstacles in power transmission channels based on binocular vision and COLMAP as described in claim 6, characterized in that: The assessment of obstacle risk level includes setting a tree height safety threshold and a distance safety threshold. When the obstacle height exceeds the tree height threshold and the distance is less than the distance threshold, it is marked as a high-risk obstacle. The generation of positioning results includes fitting a cylindrical model using a random sampling consensus algorithm, calculating the coordinates of the tree trunk center, calculating the tree height and trunk diameter, and calculating the safe distance based on the power transmission line equation. The formula for calculating the coordinates of the tree trunk center is expressed as follows: Among them, (x i ,y i (x0, y0) represents the coordinates of the i-th point in the point cloud, (x0, y0) represents the coordinates of the center point of the tree trunk, r represents the radius of the tree trunk, N represents the total number of points in the point cloud, and ∈ represents the error in the fitting process.

8. A system employing the power transmission channel obstacle localization method based on binocular vision and COLMAP as described in any one of claims 1 to 7, characterized in that: It includes a data acquisition and preprocessing module, a depth map generation and optimization module, a dense point cloud reconstruction module, and an obstacle localization and risk assessment module; The data acquisition and preprocessing module is used to acquire left and right image pairs of the power transmission channel in real time, and remove environmental noise by median filtering with the median value of the neighboring pixels sorted. Based on the camera calibration parameters, the lens distortion is corrected by radial distortion model and tangential distortion model, and the image contrast is enhanced by histogram equalization. The depth map generation and optimization module is used to generate depth information based on the preprocessed image pairs through disparity calculation. First, it calculates the initial depth value according to the disparity-depth conversion formula, and uses an algorithm that combines local matching and global optimization to improve the disparity accuracy of the tree area. Then, it performs median filtering to smooth the depth map and removes noise based on the depth threshold to eliminate dynamic interference. The dense point cloud reconstruction module is used to convert the depth map into a three-dimensional model, extract scale-invariant feature points by constructing a scale space through Gaussian blur, match feature points by nearest neighbor search and remove low-quality matching pairs, calculate the camera pose based on the essential matrix and generate a sparse point cloud through triangulation, and generate a dense point cloud containing tree trunk texture by multi-view stereo matching optimization. The obstacle localization and risk assessment module is used to extract obstacle spatial information from point cloud and assess risk. It calculates the trunk center coordinates by fitting a cylindrical model with RANSAC, calculates the tree height using the vertical range formula, calculates the trunk size using the diameter formula, calculates the safe distance based on the transmission line equation using the shortest distance formula, and marks the risk level according to the dual threshold conditions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the power transmission channel obstacle localization method based on binocular vision and COLMAP as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the power transmission channel obstacle localization method based on binocular vision and COLMAP as described in any one of claims 1 to 7.

Images