A split-to-face binocular distance measuring method and system for power transmission lines

Abstract

This invention relates to the field of power transmission line technology. To address the problem of object occlusion and blind spots affecting the perception of monitoring scenes, it provides a split-type binocular ranging method and system for power transmission lines. The method includes: predicting a corresponding dense depth map based on jointly calibrated original images; obtaining a true depth map from two original images and using it to calibrate the two dense depth maps to obtain a corresponding monocular depth map; identifying potential 3D objects from any original image and calculating a sparse 3D point cloud; selecting any original image as a projection surface image and extracting its semantic and texture features, then combining the sparse depth map with a global structural prior for depth completion to obtain a completed dense depth map of the 3D object region; and fusing the completed dense depth map of the 3D object region with the background region of its corresponding monocular depth map to generate a final dense depth map covering the entire monitoring scene. This enables dense depth perception across the entire scene.

Description

Technical Field

[0001] This invention relates to the field of power transmission line technology, and in particular to a split-type binocular ranging method and system for power transmission lines. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Traditional monocular vision systems, employing a unidirectional chain layout (one camera per tower, one viewpoint), monitor in the same direction along the power line. Because a single camera struggles to estimate depth information, monocular vision systems cannot accurately locate intruding objects, leading to frequent misjudgments and missed detections. In contrast, a binocular stereo vision system, using two cameras per tower, monitors in the same direction along the power line. This system simulates human vision, utilizing parallax for binocular ranging to reconstruct the scene's 3D coordinates. Theoretically, it addresses the limitations of monocular vision in spatial perception, enabling accurate modeling and identification of potential hazards, and reducing misjudgments and missed detections.

[0004] Currently, the binocular system of the "one tower, two cameras, one field of view" mode for transmission lines is limited by the tower structure. The baseline distance between the two cameras is relatively short, resulting in a mismatch between the baseline length and the measurement distance, which makes it difficult to meet the accuracy requirements of long-distance measurement. Transmission lines have large spans and thin conductor diameters. In order to achieve accurate positioning of small targets, the system requires extremely high camera resolution and high magnification focal length, which greatly increases the system cost. At the same time, the existence of objects obstructing the field of view and blind spots affects the accuracy of depth perception. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a split-type binocular ranging method and system for power transmission lines, which can achieve dense depth perception of conductors, the ground, and key three-dimensional hidden danger targets across the entire scene.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a split-view binocular ranging method for power transmission lines.

[0007] In one or more embodiments, a split-type binocular ranging method for power transmission lines is provided, comprising: Joint calibration is performed on two cameras deployed on two adjacent towers in a facing-to-facing configuration. The original images captured by the two cameras after joint calibration are obtained, and the corresponding dense depth maps are predicted by the pre-trained monocular depth estimation neural network. Epipolar correction, confidence-based feature region extraction and matching are performed on both original images sequentially to obtain the true depth map, and the two dense depth maps are then used to calibrate the corresponding monocular depth map. Identify potential 3D objects from any original image and obtain their 2D bounding boxes. Perform feature point matching within the 2D bounding boxes to calculate a sparse 3D point cloud. Select any original image as the projection plane image, backproject the sparse 3D point cloud onto the projection plane image to generate a sparse depth map; obtain the predicted value of the 3D object region in the monocular depth map corresponding to the projection plane image, and use it as the global structure prior for depth completion; extract the semantic and texture features of the projection plane image, and then combine the sparse depth map and the global structure prior for depth completion to obtain the completed dense depth map of the 3D object region. The dense depth map of the completed 3D object region is merged with the background region of its corresponding monocular depth map to generate the final dense depth map covering the entire monitoring scene.

[0008] As one implementation method, the confidence feature area includes the transmission line area and the flat ground area.

[0009] As one implementation method, in the process of training a monocular depth estimation neural network, its total loss function consists of L1 loss, depth loss, multi-scale gradient loss, and wire sensing loss, and its corresponding expression is: ; in, This is the total loss function; For L1 loss; This is a deep loss; Multi-scale gradient loss; Loss sensing for the conductor; 、 and It is a constant coefficient.

[0010] As one implementation method, depth loss The expression is: ; ; ; in, The total number of valid pixels; and The first The and the first Logarithmic depth residual of each effective pixel; and The first The and the first Predicted depth map of 1,000 valid pixels; and The first The and the first The true depth map of each effective pixel; These are the coordinates of the valid pixels.

[0011] As one implementation method, multi-scale gradient loss To different downsampling levels The sum of the first-order gradient differences is expressed as: ; ; in, Representative process Log-depth residual after average pooling; This represents the total number of downsampling levels, which is a positive integer greater than 1. For the first Logarithmic depth residual of each effective pixel; For the first Predicted depth map of 1,000 valid pixels; For the first The true depth map of each effective pixel; These are the coordinates of the valid pixels; The total number of valid pixels; and These represent the two-dimensional spatial gradient operators for the x and y coordinates of the effective pixels, respectively. It represents the 1-norm.

[0012] As one implementation method, wire sensing loss The expression is: ; in, Use the edge confidence mask for the original image; For the first Spatial gradient magnitude of the predicted depth map for each valid pixel; For the first Spatial gradient magnitude of the true depth map corresponding to each valid pixel; It represents the 1-norm.

[0013] As one implementation method, the process of calibrating a dense depth map using a real depth map to obtain a corresponding monocular depth map is as follows: Based on each pixel on the wire depth truth value , obtain the truth-prediction pairs for all conductors { ( , )};in, For the first root wire number The predicted depth value for each pixel; ; ; This refers to the number of wires; It is the first The number of pixels in each wire; Construct a weighted least squares objective function. ; The global calibration parameters are obtained by solving for them and then applied to the dense depth map to obtain the monocular depth map: ; in, For the first The fitting residual is the average distance from the fitting point of the root conductor to the catenary. As weight, , The minimum positive number is preset; and These are global calibration parameters; This is the final calibrated monocular depth map.

[0014] A second aspect of the present invention provides a split-type binocular ranging system for power transmission lines.

[0015] In one or more embodiments, a split-type binocular ranging system for power transmission lines includes: The joint calibration module is used to jointly calibrate two cameras deployed on two adjacent towers in a facing-to-facing layout. The dense depth map prediction module is used to acquire the original images captured by the two jointly calibrated cameras, and then predict the corresponding dense depth maps through a pre-trained monocular depth estimation neural network. The monocular depth map calculation module is used to sequentially perform epipolar correction, set confidence feature region extraction and matching on two original images to obtain the true depth map, and to calibrate two dense depth maps to obtain the corresponding monocular depth map. The sparse 3D point cloud computing module is used to identify potential 3D objects from any original image and obtain their 2D bounding boxes. Feature point matching is performed within the 2D bounding boxes to calculate a sparse 3D point cloud. The dense depth map completion module is used to select any original image as the projection plane image, backproject the sparse 3D point cloud onto the projection plane image to generate a sparse depth map; obtain the predicted value of the 3D object region in the monocular depth map corresponding to the projection plane image, and use it as the global structure prior for depth completion; extract the semantic and texture features of the projection plane image, and then combine the sparse depth map and the global structure prior for depth completion to obtain the completed dense depth map of the 3D object region. The final dense depth map generation module is used to merge the completed dense depth map of the 3D object region with the background region of its corresponding monocular depth map to generate a final dense depth map covering the entire monitoring scene.

[0016] A third aspect of the present invention provides a computer-readable storage medium.

[0017] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the split-view binocular ranging method for transmission lines as described above.

[0018] A fourth aspect of the present invention provides an electronic device.

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the split-view binocular ranging method for power transmission lines as described above.

[0020] Compared with the prior art, the beneficial effects of the present invention are: This invention provides a split-type binocular ranging method for power transmission lines. It predicts corresponding dense depth maps from jointly calibrated original images, then processes the original images to obtain a true depth map and a sparse 3D point cloud. The dense depth map is calibrated using the true depth map to obtain a corresponding monocular depth map. The predicted values ​​of 3D object regions in the monocular depth map are used as global structural priors and semantic and texture features for depth completion, resulting in a completed dense depth map of the 3D object regions. Finally, the background region of the corresponding monocular depth map is fused to generate a final dense depth map covering the entire monitoring scene. This solves the core ranging problem in peer-to-peer layouts, achieving a balance between accuracy and robustness. Binocular stereo matching provides ground truth accuracy in high-confidence areas such as conductors and flat ground. Calibration of the monocular network ensures the absolute scale accuracy of the overall depth estimation. Simultaneously, the monocular network provides reliable depth priors for low-texture and occluded areas. Depth information reconstruction is performed on the most critical 3D objects in hazard detection, effectively acquiring their height and volume information, meeting the core spatial positioning requirements for hazard identification and early warning. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of a split-type binocular ranging method for power transmission lines according to an embodiment of the present invention; Figure 2 This is a schematic diagram of camera deployment according to an embodiment of the present invention; Figure 3 This is a comparison of the single dense depth map estimation performance of an embodiment of the present invention; Figure 4 This is a comparison of the monocular depth map estimation performance of embodiments of the present invention; Figure 5 This is a comparison of the completed dense depth map effect according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a split-type binocular ranging system for power transmission lines according to an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Figure 1 A flowchart of a split-type binocular ranging method for power transmission lines according to an embodiment of the present invention is provided. Figure 1 The split-type binocular ranging method for transmission lines in this embodiment may include the following steps S101 to S106.

[0027] The specific implementation process of steps S101 to S106 is as follows: Step S101: Perform joint calibration on two cameras deployed on two adjacent towers in a facing-to-facing layout.

[0028] according to Figure 2As shown, two cameras (defined as camera A and camera B) deployed on adjacent towers in a facing configuration are jointly calibrated to obtain their respective intrinsic parameter matrices, distortion coefficients, and the rotation matrix R and translation vector T between the two cameras. Two original images, simultaneously acquired by camera A and camera B and covering the same span area of ​​the transmission line, are then acquired in real-time or at regular intervals. and .

[0029] Step S102: Obtain the original images captured by the two cameras after joint calibration, and predict the corresponding dense depth maps by the pre-trained monocular depth estimation neural network.

[0030] The captured images and Each input is fed into a pre-trained monocular depth estimation neural network. In this context, a monocular depth estimation neural network can predict the depth value of each pixel from a single image, outputting two preliminary dense depth maps. and .

[0031] Monocular depth estimation neural network Any monocular depth estimation model based on convolutional neural networks (CNN) or visual transformers (ViT), such as DepthFormer, can be used. By learning on large-scale datasets, it can understand the semantic and geometric priors of the scene and estimate a relatively reasonable depth even without stereo matching.

[0032] Specifically, in the process of training a monocular depth estimation neural network, its total loss function consists of L1 loss, depth loss, multi-scale gradient loss, and wire sensing loss, and its corresponding expression is: ; in, This is the total loss function; For L1 loss; This is a deep loss; Multi-scale gradient loss; Loss sensing for the conductor; 、 and It is a constant coefficient.

[0033] Specifically, depth loss This ensures that the final relative proportion of the predicted depth is correct.

[0034] The expression is: ; ; ; in, The total number of valid pixels; and The first The and the first Logarithmic depth residual of each effective pixel; and The first The and the first Predicted depth map of 1,000 valid pixels; and The first The and the first The true depth map of each effective pixel; These are the coordinates of the valid pixels.

[0035] Multiscale gradient loss To different downsampling levels The sum of the first-order gradient differences is expressed as: ; ; in, Representative process Log-depth residual after average pooling; This represents the total number of downsampling levels, which is a positive integer greater than 1, for example... , ; For the first Logarithmic depth residual of each effective pixel; For the first Predicted depth map of 1,000 valid pixels; For the first The true depth map of each effective pixel; These are the coordinates of the valid pixels; The total number of valid pixels; and These represent the two-dimensional spatial gradient operators for the x and y coordinates of the effective pixels, respectively. Represents the 1-norm. Two-dimensional spatial gradient operator. .

[0036] To force the depth estimation network to pay attention to easily overlooked conductor edges and tower profiles, the conductor-aware loss directly applies supervision in the gradient domain of the depth map. The expression is: ; in, Use the edge confidence mask for the original image; For the first Spatial gradient magnitude of the predicted depth map for each valid pixel; For the first Spatial gradient magnitude of the true depth map corresponding to each valid pixel; It represents the 1-norm.

[0037] Figure 3 (a) in the image is the original image; Figure 3 (b) in the image is the dense depth map predicted by an untrained monocular depth estimation neural network from the original image; Figure 3 In the above comparison, (c) is the dense depth map predicted by the pre-trained monocular depth estimation neural network for the original image. It can be seen that the dense depth map predicted by the pre-trained monocular depth estimation neural network for the original image in the embodiment of the present invention provides an initial full-coverage depth estimate for the entire scene, especially for the depth inference of areas that are difficult to match due to occlusion in the binocular view.

[0038] Step S103: Perform epipolar correction, confidence-based feature region extraction and matching on both original images sequentially to obtain the true depth map, and use it to calibrate the two dense depth maps to obtain the corresponding monocular depth map.

[0039] Although the eye-to-eye layout makes overall matching difficult, traditional binocular matching methods can still provide high-precision depth values ​​for certain areas with clear and stable features in transmission line scenarios. This step aims to use these "anchor points" to calibrate dense depth maps.

[0040] In this embodiment, high-precision feature matching is performed on two images after epipolar correction for a specific target. The confidence feature regions include the power transmission line region and the flat ground region.

[0041] Transmission conductor region: The pixel region of the conductor is extracted using conductor segmentation algorithms (such as semantic segmentation networks based on deep learning). Since the conductor is a thin, elongated linear structure in space and is clearly visible in both views, a point feature matching algorithm based on line features or sub-pixel precision can be used to obtain a series of high-precision corresponding point pairs.

[0042] Flat ground areas: In flat ground areas (such as roads and grass) that meet the co-visibility condition and have rich texture, reliable matching point pairs can be obtained by using traditional local feature matching algorithms or deep learning-based feature matching networks.

[0043] Based on the matched point pairs and camera calibration parameters, the precise coordinates of these points in 3D space are calculated using triangulation. By setting consistency checks and geometric constraints, mismatched points are filtered out, resulting in a set of high-confidence sparse 3D point clouds. These high-confidence point clouds are then back-projected onto the imaging planes of cameras A and B to obtain the true depth values ​​at these known pixel locations. . These positions With monocular depth map and The predicted values ​​at corresponding locations are compared to calculate systematic biases (such as scale shift and global scaling factor). Using this bias information, the original model is fine-tuned through global linear scaling. and Perform overall calibration to obtain preliminary monocular depth maps corresponding to camera A and camera B. and This step effectively solves the scale uncertainty problem commonly encountered in monocular depth estimation and improves the accuracy of the model.

[0044] Preliminary monocular depth map and Scale error has been largely eliminated, but local biases may still exist due to uneven distribution of matching points. To address this issue, each wire in the original image is assigned an independent label. Connectivity analysis is performed on the wire segmentation mask, and combined with curve orientation clustering, pixels belonging to different wires are separated.

[0045] Perform the following operations for each individually identified wire: Point cloud extraction: The 3D point cloud corresponding to the traverse is obtained by backprojecting from the preliminary monocular depth map; Determine the natural coordinate system: Perform principal component analysis on the 3D point cloud to determine the direction (X-axis) and vertical direction (Y-axis) of the traverse, and rotate the point cloud to a local coordinate system with the direction as the X-axis and the vertical direction as the Y-axis.

[0046] Catenary fitting: Fitting a three-dimensional spatial curve in a local coordinate system; Generating ground truth depth values: Calculate the ground truth depth value for each pixel on the guide wire using the fitted curve. .

[0047] Specifically, the process of calibrating a dense depth map using a real depth map to obtain a corresponding monocular depth map is as follows: Based on each pixel on the wire depth truth value , obtain the truth-prediction pairs for all conductors { ( , )};in, For the first root wire number The predicted depth value for each pixel; ; ; This refers to the number of wires; It is the first The number of pixels in a single wire.

[0048] Construct a weighted least squares objective function. ; The global calibration parameters are obtained by solving for them and then applied to the dense depth map to obtain the monocular depth map: ; in, For the first The fitting residual is the average distance from the fitting point of the root conductor to the catenary. As weight, ε is a pre-defined minimum positive number; and These are global calibration parameters; This is the final calibrated monocular depth map. Figure 4 (a) in the image is the original image; Figure 4 (b) in the image is the dense depth map corresponding to the original image; Figure 4 (c) in the image is the monocular depth map after calibration of the dense depth map corresponding to the original image; Figure 4 In the image, (d) represents the true depth map corresponding to the original image.

[0049] Step S104: Identify potential 3D objects from any original image and obtain their 2D bounding boxes. Perform feature point matching within the 2D bounding boxes to calculate a sparse 3D point cloud.

[0050] Using object detection algorithms from any view (such as the view of camera A) Potential 3D objects are identified in the image, and their 2D bounding boxes are obtained. Within the object region, a robust feature matching algorithm (such as a feature matching network incorporating semantic information) is used to match between stereo image pairs. Since the top, edges, and other parts of the object may still have similar textures or structures from the opposite viewpoint, this step can obtain a set of sparse, high-precision corresponding feature point pairs. Using triangulation, the precise coordinates of these matching point pairs in 3D space are calculated, thus obtaining a series of sparse 3D point clouds with known precise depth values ​​on the 3D object.

[0051] Step S105: Select any original image as the projection plane image, backproject the sparse 3D point cloud onto the projection plane image to generate a sparse depth map; obtain the predicted value of the 3D object region in the monocular depth map corresponding to the projection plane image, and use it as the global structure prior for depth completion; extract the semantic and texture features of the projection plane image, and then combine the sparse depth map and the global structure prior for depth completion to obtain the completed dense depth map of the 3D object region.

[0052] A sparse 3D point cloud is back-projected onto the image plane of camera A (front view) to generate a sparse depth map. In this map, only a small number of pixels within the 3D object region have high-precision depth values, while the depth values ​​of the remaining pixels are unknown (marked as 0 or invalid values). Simultaneously, the calibrated monocular depth map... The predicted value for the corresponding object region, as a global structural prior for depth completion, provides a preliminary estimate of the overall depth distribution of the object, although there may be local biases.

[0053] The following multimodal information is used as input and processed by a pre-trained deep completion neural network to obtain a dense depth map of the completed 3D object region.

[0054] (1) Guiding information: The sparse depth map (from the common points identified by the binocular depth estimation) is the core constraint for the deep completion neural network learning, ensuring that the completion result is consistent with the true value of the binocular measurement at the known points.

[0055] (2) Prior information: The depth value of the calibrated monocular depth prediction in the object region (from the calibrated monocular depth estimate) serves as the initial estimate and context for completion.

[0056] (3) Semantic and texture information: The original image helps the network understand the structure of objects in order to reasonably infer the depth of unknown areas.

[0057] After training, a deep completion neural network can correct the bias of monocular priors under strong supervision of sparse depth points and infer the complete, smooth, and geometrically consistent depth of an object's surface. The output of the deep completion neural network is a completed dense depth map of the 3D object region. It should be noted that the aforementioned multimodal information can be added or removed depending on the actual structure of the deep completion neural network.

[0058] Figure 5 (a) in the image is the original image; Figure 5 (b) in the image is the dense depth map before completion; Figure 5 (c) in the image is the completed dense depth map; it can be seen that, compared with the dense depth map before completion, the object surfaces in the completed dense depth map are complete, smooth and geometrically consistent.

[0059] Step S106: Merge the completed dense depth map of the 3D object region with the background region of its corresponding monocular depth map to generate the final dense depth map covering the entire monitoring scene.

[0060] The final dense depth map covering the entire monitoring scene combines high precision for conductors and flat areas with reasonable, complete, and significantly improved distance measurement accuracy for three-dimensional hidden objects.

[0061] This invention solves the core ranging problem in the face-to-face layout. By introducing monocular depth estimation as the basis, it bypasses the fundamental bottleneck of low feature matching rate of traditional binocular methods under ultra-large parallax and viewpoint differences, enabling the system to perform depth perception in the face of complex terrain and three-dimensional objects.

[0062] This invention achieves a balance between accuracy and robustness. It utilizes binocular stereo matching to provide ground truth level accuracy in high-confidence areas such as wires and flat ground, and calibrates the monocular network to ensure the absolute scale accuracy of the overall depth estimation. At the same time, the monocular network provides reliable depth priors for low-texture and occluded areas.

[0063] This invention specifically repairs the depth of three-dimensional objects, reconstructing the depth information of the three-dimensional objects most critical in hazard detection, effectively obtaining their height and volume information, and meeting the core spatial positioning requirements of hazard identification and early warning.

[0064] This invention is highly practical, with a clearly modular approach. Monocular depth estimation, object detection, and depth restoration can all utilize currently mature deep learning models, making it easy to implement and deploy. Furthermore, it offers flexibility in terms of computing power requirements, allowing for the selection of models with varying levels of complexity based on available hardware conditions.

[0065] This invention expands the application boundaries of binocular systems, not only applicable to line-of-sight systems for power transmission lines, but also providing new technical ideas for other binocular vision applications with fixed ultra-long baselines and large-angle observation requirements (such as monitoring of large bridges and reservoir dams).

[0066] like Figure 6 As shown, the split-type binocular ranging system for power transmission lines provided in this embodiment of the invention can be implemented in software. The split-type binocular ranging system for power transmission lines includes the following software modules: The joint calibration module 601 is used to perform joint calibration on two cameras deployed on two adjacent towers in a facing-to-facing layout. The dense depth map prediction module 602 is used to acquire the first image and the second image captured by the two cameras after joint calibration, and predict the first dense depth map and the second dense depth map by a pre-trained monocular depth estimation neural network respectively. The dense depth map calibration module 603 is used to sequentially perform epipolar correction on the first image and the second image, extract and match high-confidence feature regions to obtain a true depth map, and use it to calibrate the first dense depth map and the second dense depth map to obtain a calibrated first monocular depth map and second monocular depth map. The sparse 3D point cloud computing module 604 is used to identify potential 3D objects from the first image / second image and obtain their 2D bounding boxes, perform feature point matching of the binocular image pairs within the 2D bounding boxes, and calculate the sparse 3D point cloud. The dense depth map completion module 605 is used to backproject the sparse 3D point cloud onto the first image / second image plane to generate a sparse depth map; the predicted values ​​of the calibrated first monocular depth map / second monocular depth map in the corresponding object region are used as the global structural prior for depth completion; the semantic and texture features of the first image / second image are extracted, and combined with the sparse depth map and the global structural prior for depth completion, the completed dense depth map of the 3D object region is obtained. The final dense depth map generation module 606 is used to merge the completed 3D object depth map with the background area of ​​the corresponding monocular depth map to generate a final dense depth map covering the entire monitoring scene.

[0067] It should be noted that each module in the split-type binocular ranging system for transmission lines in this embodiment of the invention corresponds one-to-one with each step in the split-type binocular ranging method for transmission lines in the above embodiment, and their specific implementation processes are the same, so they will not be repeated here.

[0068] The structure of the electronic device according to embodiments of the present invention will be described in detail below. The electronic device provided in the embodiments of the present invention includes: at least one processor, a memory, a user interface, and at least one network interface. The various components in the split-type binocular ranging system for power transmission lines are coupled together through a bus system. It can be understood that the bus system is used to realize the connection and communication between these components. In addition to a data bus, the bus system also includes a power bus, a control bus, and a status signal bus. The user interface may include a display, keyboard, mouse, trackball, click wheel, buttons, a touchpad, or a touch screen, etc.

[0069] It is understood that the memory can be volatile memory or non-volatile memory, or both. The memory in this embodiment of the invention is capable of storing data to support the operation of the terminal. Examples of this data include any computer programs used to operate on the terminal, such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0070] In some embodiments, the split-type binocular ranging system for power transmission lines provided in this invention can be implemented using a combination of hardware and software. As an example, the split-type binocular ranging system for power transmission lines provided in this invention can be a processor in the form of a hardware decoding processor, programmed to execute the split-type binocular ranging method for power transmission lines provided in this invention. For example, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0071] As an example, a processor can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where a general-purpose processor can be a microprocessor or any conventional processor, etc.

[0072] As an example of the hardware implementation of the split-type binocular ranging system for power transmission lines provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the split-type binocular ranging method for power transmission lines provided in this embodiment of the invention.

[0073] The memory in this embodiment of the invention is used to store various types of data to support the operation of a split-type binocular ranging system for power transmission lines, or to store data for execution. Figure 1The program code for the method shown. Examples of this data include: any executable instructions for operation on a split-type binocular ranging system oriented towards transmission lines, such as executable instructions that can be included in the executable instructions to implement the split-type binocular ranging method for transmission lines according to embodiments of the present invention.

[0074] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including functions for executing... Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by the central processing unit, it performs the various functions defined in the apparatus of this application.

[0075] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0076] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A split-type binocular ranging method for power transmission lines, characterized in that, include: Joint calibration is performed on two cameras deployed on two adjacent towers in a facing-to-facing configuration. The original images captured by the two cameras after joint calibration are obtained, and the corresponding dense depth maps are predicted by the pre-trained monocular depth estimation neural network. Epipolar correction, confidence-based feature region extraction and matching are performed on both original images sequentially to obtain the true depth map, and the two dense depth maps are then used to calibrate the corresponding monocular depth map. Identify potential 3D objects from any original image and obtain their 2D bounding boxes. Perform feature point matching within the 2D bounding boxes to calculate a sparse 3D point cloud. Select any original image as the projection plane image, backproject the sparse 3D point cloud onto the projection plane image, and generate a sparse depth map. Obtain the predicted value of the 3D object region in the monocular depth map corresponding to the projection plane image, and use it as the global structural prior for depth completion; Semantic and texture features of the projection surface image are extracted, and then combined with the sparse depth map and the global structural prior of depth completion to obtain the dense depth map after the completion of the 3D object region. The dense depth map of the completed 3D object region is merged with the background region of its corresponding monocular depth map to generate the final dense depth map covering the entire monitoring scene.

2. The split-type binocular ranging method for transmission lines as described in claim 1, characterized in that, The confidence level feature areas include transmission line areas and flat ground areas.

3. The split-type binocular ranging method for transmission lines as described in claim 1, characterized in that, In the process of training a monocular depth estimation neural network, its total loss function consists of L1 loss, depth loss, multi-scale gradient loss, and wire sensing loss, and its corresponding expression is: ； in, This is the total loss function; For L1 loss; This is a deep loss; Multi-scale gradient loss; Loss sensing for the conductor; 、 and It is a constant coefficient.

4. The split-type binocular ranging method for transmission lines as described in claim 3, characterized in that, Deep loss The expression is: ； ； ； in, The total number of valid pixels; and The first The and the first Logarithmic depth residual of each effective pixel; and The first The and the first Predicted depth map of 1,000 valid pixels; and The first The and the first The true depth map of each effective pixel; These are the coordinates of the valid pixels.

5. The split-type binocular ranging method for transmission lines as described in claim 3, characterized in that, Multiscale gradient loss To different downsampling levels The sum of the first-order gradient differences is expressed as: ； ； in, Representative process Log-depth residual after average pooling; This represents the total number of downsampling levels, which is a positive integer greater than 1. For the first Logarithmic depth residual of each effective pixel; For the first Predicted depth map of 1,000 valid pixels; For the first The true depth map of each effective pixel; These are the coordinates of the valid pixels; The total number of valid pixels; and These represent the two-dimensional spatial gradient operators for the x and y coordinates of the effective pixels, respectively. It represents the 1-norm.

6. The split-type binocular ranging method for transmission lines as described in claim 3, characterized in that, Wire sensing loss The expression is: ； in, Use the edge confidence mask for the original image; For the first Spatial gradient magnitude of the predicted depth map for each valid pixel; For the first Spatial gradient magnitude of the true depth map corresponding to each valid pixel; It represents the 1-norm.

7. The split-type binocular ranging method for transmission lines as described in claim 1, characterized in that, The process of calibrating a dense depth map using a real depth map to obtain a corresponding monocular depth map is as follows: Based on each pixel on the wire depth truth value , obtain the truth-prediction pairs for all conductors { ( , )};in, For the first root wire number The predicted depth value for each pixel; ; ; This refers to the number of wires; It is the first The number of pixels in each wire; Construct a weighted least squares objective function. ; The global calibration parameters are obtained by solving for them and then applied to the dense depth map to obtain the monocular depth map: ； in, For the first The fitting residual is the average distance from the fitting point of the root conductor to the catenary. As weight, , The minimum positive number is preset; and These are global calibration parameters; This is the final calibrated monocular depth map.

8. A split-type binocular ranging system for power transmission lines, characterized in that, The split-type binocular ranging method for transmission lines as described in any one of claims 1-7 includes: The joint calibration module is used to jointly calibrate two cameras deployed on two adjacent towers in a facing-to-facing layout. The dense depth map prediction module is used to acquire the original images captured by the two jointly calibrated cameras, and then predict the corresponding dense depth maps through a pre-trained monocular depth estimation neural network. The monocular depth map calculation module is used to sequentially perform epipolar correction, set confidence feature region extraction and matching on two original images to obtain the true depth map, and to calibrate two dense depth maps to obtain the corresponding monocular depth map. The sparse 3D point cloud computing module is used to identify potential 3D objects from any original image and obtain their 2D bounding boxes. Feature point matching is performed within the 2D bounding boxes to calculate a sparse 3D point cloud. The dense depth map completion module is used to select any original image as the projection plane image, backproject the sparse 3D point cloud onto the projection plane image to generate a sparse depth map; obtain the predicted value of the 3D object region in the monocular depth map corresponding to the projection plane image, and use it as the global structure prior for depth completion; extract the semantic and texture features of the projection plane image, and then combine the sparse depth map and the global structure prior for depth completion to obtain the completed dense depth map of the 3D object region. The final dense depth map generation module is used to merge the completed dense depth map of the 3D object region with the background region of its corresponding monocular depth map to generate a final dense depth map covering the entire monitoring scene.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the split-view binocular ranging method for power transmission lines as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the split-type binocular ranging method for power transmission lines as described in any one of claims 1-7.

Images