Power transmission line external damage prevention monitoring system and method based on improved DepthPro depth estimation

Abstract

The invention discloses a power transmission line external force damage prevention monitoring system and method based on improved DepthPro depth estimation. The monitoring system comprises a target detection module which is used for inputting a continuous image It of a monitoring area and outputting spatial positions and masks of a crane boom, a lifting hook, a tower and a power transmission line; the depth estimation module is used for inputting a pixel-level depth estimation result map Dt, detecting a lifting arm and a power transmission line area and outputting a minimum space distance dmin; the minimum space distance calculation module is used for inputting an original image frame It and a detection result Bt and outputting a pixel-level depth estimation result image Dt and pixel-level uncertainty Ut; and the risk assessment and grading early warning module is used for inputting the minimum space distance dmin and the pixel-level uncertainty Ut and outputting a risk level and an alarm signal. According to the method, a YOLOv11 target detection algorithm is adopted, a DepthPro depth estimation model is improved, a time sequence Transform module is introduced to optimize depth data, and the problems of false detection, missing detection, unstable depth estimation and the like in the prior art can be effectively solved.

Description

Technical Field

[0001] This invention relates to the field of power transmission line safety monitoring technology, specifically to a power transmission line protection monitoring system and method based on improved DepthPro depth estimation. Background Technology

[0002] The dynamic evolution of urban landscapes has a profound impact on urban planning, environmental monitoring, and resource management. Similarly, the issue of external damage to power transmission lines directly affects the safety and stability of power systems. During construction, the risk of collisions between large construction machinery (such as crane booms) and power transmission lines is constantly increasing, especially in complex environments. Accurately monitoring the spatial relationship between construction equipment and power transmission lines and providing timely early warnings has become a critical issue in the operation of modern power systems. With the continuous advancement of remote sensing technology, particularly the widespread application of high-resolution satellites and drones, changes in cities and surrounding areas can be captured with unprecedented precision and frequency, especially regarding the interaction between buildings and power transmission lines.

[0003] Currently, depth estimation technology has made significant progress in image processing and computer vision, especially in object recognition and scene understanding. However, depth estimation still faces many challenges in power transmission line monitoring. Particularly at construction sites, the relative positions of large equipment such as crane booms and power transmission lines need to be accurately captured and estimated. Traditional depth estimation methods often rely on single-frame images or traditional sensors, but these methods are easily affected by changes in lighting, weather interference, and distance variations between targets in complex construction environments, leading to unstable depth estimation results and large errors. Existing depth estimation methods generally suffer from insufficient acquisition of global contextual information and inadequate modeling of temporal dependencies, resulting in inaccurate depth predictions for small targets (such as slender power transmission lines) and complex structures (such as crane booms).

[0004] The Chinese patent "A Depth Estimation Network Training Method, Depth Estimation Method, and Electronic Device" (application number: CN202311718735.3) discloses a depth estimation network training method aimed at improving the accuracy of image depth information based on monocular depth estimation, especially for human depth estimation in complex scenes. This method combines a human body parsing subnetwork and a monocular depth estimation subnetwork, integrating human semantic information with depth information to train a depth estimation network that can better distinguish different parts of a person from the background. During training, the loss weights are dynamically adjusted for different regions in the training image (such as the human figure and the background) to achieve more accurate depth estimation results. Furthermore, the depth estimation model also employs motion blur enhancement and low-light enhancement processing, resulting in superior depth estimation performance in night scenes and motion scenarios. Ultimately, the trained network can provide more accurate human depth information for blurring processing, making it particularly suitable for complex scenes such as night scenes and motion scenarios. However, its limitations lie in its specific application scope, insufficient adaptability to non-human targets (especially slender structures), lack of temporal modeling capabilities, and the singularity of its technical objectives.

[0005] Chinese patent "A Lightweight Depth Estimation Method and System" (application number: 202510261430.7) discloses a lightweight depth estimation method and system, aiming to solve the problems of high computational load and high memory consumption in existing binocular depth estimation methods in practical applications. This method combines 2D depth-separable convolution and 3D pointwise convolution to extract features and process depth information from binocular depth images, thereby generating multi-depth feature information and reducing the computational complexity and number of parameters of the model. By performing depth estimation processing in a multi-layer stacked hourglass network, this method can significantly reduce the computational burden and improve the practical application efficiency of the model while maintaining depth estimation accuracy. This technology is particularly suitable for resource-constrained scenarios such as mobile devices, smart wearable devices, and drones, optimizing the storage, memory, and computational resource utilization of the depth estimation network. However, the dependence on binocular hardware limits the application scope, the lightweight goal may inherently contradict the high-precision requirements, and there is a lack of support for temporal dynamic scenes and complete monitoring chains.

[0006] Existing deep learning-based depth estimation methods have achieved success in some applications, but due to the limitations of convolutional neural networks, these methods often struggle to capture global temporal information and perform poorly in small target detection. For example, convolutional operations cannot effectively extract spatial relationships between distant or slender targets (such as power transmission lines), leading to significant errors in accurately measuring target spacing. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a power transmission line protection monitoring system and method based on improved DepthPro depth estimation. Combining target detection and depth estimation techniques, it aims to improve the monitoring accuracy of the spatial relationship between crane booms and power transmission lines at construction sites and issue real-time warnings when potential risks occur. This method employs the YOLOv11 target detection algorithm, an improved DepthPro depth estimation model, and introduces a temporal Transformer module to optimize depth data, effectively solving problems such as false detections, missed detections, and unstable depth estimation in existing technologies.

[0008] The technical solution adopted in this invention is as follows: A power transmission line protection monitoring system based on improved DepthPro depth estimation, comprising: Target detection module: Input continuous image of the monitoring area I t Output the spatial location and mask of the crane boom, hook, tower, and power transmission line; Depth estimation module: Input pixel-level depth estimation result image D t The minimum spatial distance d is output for the detected boom and power line areas. min ; Minimum spatial distance calculation module: Input original image frame I t Test result B t Output pixel-level depth estimation results in image D t and pixel-level uncertainty U t ; Risk assessment and graded early warning module: Input minimum spatial distance d min Pixel-level uncertainty U t It outputs the risk level and alarm signal.

[0009] The target detection module includes: a1: Input image sequence of the monitoring area {I t This includes crane booms, hooks, towers, and power transmission lines; a2: Input the image into the YOLOv11 network, and output the bounding box (x_center, y_center, width, height) and class label ci of the target. The specific calculation is as follows: ; Among them, B t This represents the detection result, specifically the set of targets detected at time t, which includes information such as label category, bounding box, and confidence score si; specifically: B t=(ci,si,x_center,y_center,width,height) where:x_center andy_center represent the coordinates of the target's center point; width represents the horizontal distance from the center point to the target bounding box; and height represents the vertical distance from the center point to the target bounding box. This represents the YOLOv11 network model; b i This indicates that the image contains spatial coordinates and category information, i=1,2,…,n, where n represents that the image has n bounding boxes.

[0010] a3: For the power transmission line section, further output pixel-level masks. This is used for subsequent depth estimation and spatial relationship modeling. ; in, `Proto` represents the Sigmoid function; `Proto` represents the prototype mask, a set of general shape templates learned during training, typically smaller than the original image (e.g., 32x32); `Coef_i` represents the mask coefficients: for the i-th detected transmission line instance, the model predicts a specific coefficient vector. This vector can be understood as how to linearly combine those general shape templates to form the precise shape of this specific transmission line instance.

[0011] The depth estimation module includes: b1: Transfer image I t Inputting the improved DepthPro model yields pixel-level depth estimation results (Figure D). t (ci, si, x_center, y_center, width, height,Mt): ; Where Mt is the pixel-level mask, (x_center, y_center, width, height) is the bounding box, and ci is the category label.

[0012] Improved DepthPro models, such as Figure 6 As shown, the improved DepthPro model includes the description in section b3 below.

[0013] b2: Introduce a temporal Transformer module at the prediction end of the DepthPro model. The temporal Transformer module is as follows: Figure 7 As shown, adjacent image frames (I) are used t-1 I t The context information is fused to alleviate single-frame depth drift.

[0014] b3: Output pixel-level uncertainty U t For subsequent risk assessment; ; Among them, D t This represents the pixel-level depth estimation result image, U t This represents the pixel-level uncertainty corresponding to depth estimation; TTS stands for Temporal Transformer; Concat is the concatenation operation; This represents the depth estimation model.

[0015] The minimum spatial distance calculation module includes: c1: Based on the test results B t Image D shows the pixel-level depth estimation results. t The image coordinates of each pixel within the detected monitoring area are combined with its corresponding depth value to form a three-dimensional point, thus creating a set of three-dimensional points representing the local area of ​​the boom and conductor, i.e., a pseudo-point cloud (x). t ,y t ,z t ), for subsequent spatial analysis: ; Where d is the estimated depth value of the pixel, K is the intrinsic parameter matrix, and (u,v) are the two-dimensional pixel coordinates; c2: Apply physical priors to geometric corrections for transmission line areas; ; Where z' represents the estimated height of the transmission line at location x'. This is the output of the fitted curve. x' represents the horizontal distance along the direction of the transmission line (connecting the two towers). The projection point of the first tower is usually set as the origin of the coordinate system. a, b, c (fitting parameters): These are the key parameters that need to be solved using the least squares method.

[0016] c3: Use instruments to extract point cloud datasets for the boom as P. arm ; c4: Based on the above pseudo-point cloud set and boom point cloud set, calculate the minimum spatial distance d between the boom and the guide wire. min The expression is as follows: ; Where: d min P represents the minimum spatial distance between the boom and the power transmission line. arm P represents the set of three-dimensional coordinates of key points of the boom; line Represents a set of 3D points in a pseudo-point cloud; symbol Represents the Euclidean distance operation; For set P armThe point above, For the corresponding P line A little bit above.

[0017] The risk assessment and tiered early warning module includes: d1: Minimum spatial distance d min Additional uncertainty correction: ; in: This represents the estimated distance with a confidence interval; This represents the standard deviation corresponding to the uncertainty. d2: Set the risk level based on dest: When d est When the depth is greater than 3m, it is considered safe. When 2m <d est If the depth is ≤3m, it is considered a slight risk. When 1m <d est When the depth is ≤2m, it is judged as moderate risk; When d est If the depth is ≤1m, it is considered a serious risk; d3: When the risk level reaches the threshold, trigger an audible and visual alarm and remote push notification.

[0018] A monitoring method for power transmission lines to resist external damage based on improved DepthPro depth estimation includes the following steps: Step 1: Define image IA, which is the image of the crane boom and the power transmission line in the monitoring area at time t; Define an image IB, which is the image of the same region at time t+1; At the same time, risk labels are defined for training and validating the monitoring model; Step 2: Use the YOLOv11 network to perform target detection on images IA and IB, identify targets such as crane booms, hooks, towers, and power transmission lines, and output bounding boxes and pixel-level masks; the detection results serve as input for subsequent depth estimation and spatial modeling. Step 3: Use the improved DepthPro model to predict the depth of image IA and image IB. This model introduces a temporal Transformer module at the output end to model cross-frame features to alleviate the scale drift and jitter problems of single-frame prediction, thereby obtaining a more stable depth map and uncertainty map.

[0019] Step 4: Based on the detection results and depth map, a pseudo-point cloud is generated, and the sag equation is introduced into the transmission line section for fitting and correction to ensure the physical rationality of the conductor's spatial morphology. At the same time, the crane boom is modeled as a skeleton, and the key point set is extracted as the spatial representation of the boom.

[0020] Step 5: Based on the pseudo-point cloud and the fitted traverse curve, calculate the minimum spatial distance between the key points of the boom and the traverse point set, and obtain the minimum spatial distance d. min ; Step 6: Combine the uncertainty propagation method to analyze d min After correction, the estimated distance d with confidence intervals is obtained. est Risk is classified according to thresholds. If a risk is detected, audible and visual alarms and remote warnings are triggered.

[0021] This invention discloses a monitoring system and method for preventing external force damage to power transmission lines based on improved DepthPro depth estimation, with the following technical advantages: 1) This invention uses the YOLOv11 network for target detection. Compared with traditional methods, it can accurately identify crane booms and power transmission lines in construction environments with complex lighting, occlusion and multi-target interference, thus improving detection robustness.

[0022] 2) This invention introduces a temporal Transformer module into the DepthPro model, which uses cross-frame context information to enhance the stability of depth estimation, reduce scale drift and jitter in single-frame prediction, and make ranging more reliable.

[0023] 3) This invention combines pseudo-point cloud and conductor sag model correction to achieve high-precision fitting of the spatial position of transmission lines, effectively improving the physical rationality and engineering applicability of 3D modeling.

[0024] 4) By calculating the minimum spatial distance and introducing an uncertainty propagation method, this invention can provide a confidence interval while giving a distance estimate, thereby enhancing the reliability and interpretability of risk assessment.

[0025] 5) The hierarchical early warning mechanism proposed in this invention, which combines audible and visual alarms with remote platform push notifications, can realize real-time on-site risk protection and intelligent operation and maintenance linkage in the background, thereby improving the safety management level of power transmission line construction. Attached Figure Description

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a diagram showing the overall structure of the monitoring method of the present invention.

[0027] Figure 2 This is a diagram of the target detection module.

[0028] Figure 3 This is a diagram of the depth estimation process.

[0029] Figure 4 This is a flowchart for risk assessment.

[0030] Figure 5This is a diagram of the Yolov11 network model.

[0031] Figure 6 To improve the DepthPro model diagram.

[0032] Figure 7 This is a time-series Transformer diagram. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments: 1) First, the system acquires image sequences I of the construction area. t , among which, I t Images representing time t. These images are acquired by sensors or cameras and processed using the YOLOv11 object detection model. This model can detect key targets such as crane booms, hooks, towers, and power transmission lines in real time, and output the bounding box coordinates and category labels of the targets. After object detection, all relevant targets in the construction area are identified, providing accurate location and category information. This data will serve as input for subsequent steps, such as depth estimation and spatial modeling. Figure 2 As shown.

[0034] 2) The system compares the target detection results with images I from adjacent time points. t-1 and I t Input the DepthPro depth estimation model. This model utilizes temporal Transformer technology to fuse contextual information from preceding and following frames, thereby generating more stable depth estimation results, such as... Figure 3 As shown. In this process, the system not only acquires the spatial coordinates of each pixel, but also generates an uncertainty map of the depth estimation, reflecting the reliability of the prediction. The output of the depth estimation model includes a depth map D. t and uncertainty plot U t This provides high-quality depth data for subsequent spatial modeling and distance calculation.

[0035] 3) The system combines the results of target detection and depth estimation to generate 3D point cloud data of the crane boom and power transmission lines. Key points of the crane boom are obtained through skeleton extraction and form a key point set P. arm The transmission lines, on the other hand, are fitted using a sag model to obtain a three-dimensional point set P. line Using these point sets, the system calculates the minimum spatial distance d between the boom and the transmission line. min .like Figure 4 The process shown can accurately assess the spatial relationship between construction machinery and power transmission lines, providing basic data for further risk assessment.

[0036] 4) After obtaining the minimum spatial distance, the system combines the uncertainty diagram U of the depth estimation. t The minimum spatial distance is corrected to obtain a distance estimate d with a confidence interval. est By introducing uncertainty correction, the system can better handle the uncertainty caused by depth estimation errors, providing a more stable and reliable basis for risk assessment. At this point, the system classifies risk levels according to a set threshold: when d... est When the depth is greater than 3m, it is considered safe; when the depth is less than 2m... <d est When the depth is ≤3m, it is judged as a slight risk; when the depth is ≤1m <d est When the risk level is ≤2m, it is classified as moderate risk; when the risk level is d... est If the distance is less than 1m, it is considered a serious risk.

[0037] 5) Finally, when the risk level reaches mild or above, the system triggers an audible and visual alarm and pushes the alarm information through a remote platform. The alarm mechanism, through on-site audible and visual equipment and a remote monitoring platform, enables real-time linkage between the construction site and the operation and maintenance center, ensuring that construction personnel can take necessary safety precautions in a timely manner.

[0038] Verification Case: To verify the effectiveness of the model, this invention selects the NuScenes dataset as the core testing and validation benchmark. This choice is based on the high degree of compatibility between the dataset and the technical requirements of this invention in terms of scene complexity, data truth authority, and temporal consistency, aiming to provide highly convincing objective evidence of the method's effectiveness in real and complex environments.

[0039] Table 1 compares the quantitative performance of the method of this invention with existing mainstream depth estimation models (DepthAnything, DepthPro), with evaluation metrics covering depth error (AbsRel, Log10, SI-Log) and depth accuracy. And the three-dimensional geometric consistency indices (PC-CD, PC-F, PC-IoU). The comparison results show that the method of this invention achieves the best performance across all key indices, with AbsRel decreasing from 0.45 / 0.29 to 0.24. The accuracy was improved to 92.4%, and the PC-CD was reduced to 21.54, with overall performance significantly better than the comparison model. This indicates that the present invention, by introducing a temporal Transformer, regional priors, and physical constraints on conductor sag, effectively improves the scale drift and structural noise in single-frame depth estimation, resulting in a significant improvement in both global consistency and local detail recovery of predicted depth. Especially on slender or complex structures such as power transmission lines and large machinery, the depth prediction of the present invention is more stable and continuous, and can generate 3D point clouds that are closer to the real structure, providing more reliable and accurate basic data for calculating the minimum spatial distance between conductors and booms and for risk warning.

[0040]

[0041] This invention proposes a depth estimation method based on YOLOv11 and an improved DepthPro, which optimizes the depth results by incorporating a temporal Transformer module. This method effectively addresses the problems of insufficient global context information, unstable depth estimation, and large errors in existing technologies. It is particularly useful for accurately estimating the spatial relationship between crane booms and power transmission lines in complex construction environments, thus providing high-precision depth data support for real-time early warning systems.

Claims

1. A power transmission line protection monitoring system based on improved DepthPro depth estimation, characterized in that... The monitoring system includes: Target detection module: Input continuous image of the monitoring area I t Output the spatial location and mask of the crane boom, hook, tower, and power transmission line; Depth estimation module: Input pixel-level depth estimation result image D t The minimum spatial distance d is output for the detected boom and power line areas. min ; Minimum spatial distance calculation module: Input original image frame I t Test result B t Output pixel-level depth estimation results in image D t and pixel-level uncertainty U t ; Risk assessment and graded early warning module: Input minimum spatial distance d min Pixel-level uncertainty U t It outputs the risk level and alarm signal.

2. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 1, characterized in that: The target detection module includes: a1: Input image sequence of the monitoring area {I t This includes crane booms, hooks, towers, and power transmission lines; a2: Input the image into the YOLOv11 network and output the bounding box and class label ci of the target; b i This indicates that the image contains spatial coordinates and category information, i=1,2,…,n, where n represents that the image has n bounding boxes; a3: For the power transmission line section, further output pixel-level masks. This is used for subsequent depth estimation and spatial relationship modeling.

3. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 2, characterized in that: The specific calculations for the bounding box and category label CI are as follows: ； Among them, B t This represents the detection result, specifically the set of targets detected at time t, which includes label category, bounding box, and confidence score si; specifically, B. t =(ci,si,x_center,y_cneter,width,height); Where: x_center, y_center represent the coordinates of the target's center point; width represents the horizontal distance from the center point to the target bounding box; height represents the vertical distance from the center point to the target bounding box. This represents the YOLOv11 network model; b i This indicates that the image contains spatial coordinates and category information, i=1,2,…,n, where n represents that the image has n bounding boxes.

4. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 3, characterized in that: Pixel-level mask The specific calculations are as follows: ； in, represents the Sigmoid function; Proto represents the prototype mask; Coef_i represents the mask coefficient.

5. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 4, characterized in that: The depth estimation module includes: b1: Transfer image I t Inputting the improved DepthPro model yields pixel-level depth estimation results (Figure D). t (ci, si, x_center, y_center, width, height,Mt): ； Where: Mt is the pixel-level mask, (x_center, y_center, width, height) is the bounding box and ci is the category label; b2: Introduce a temporal Transformer module at the prediction end of the DepthPro model, utilizing adjacent image frames (I t-1 I t By fusing contextual information, the depth drift of a single frame can be mitigated; b3: Output pixel-level uncertainty U t For subsequent risk assessment; ； Among them, D t This represents the pixel-level depth estimation result image, U t This represents the pixel-level uncertainty corresponding to depth estimation; TTS stands for Temporal Transformer; Concat is the concatenation operation; This represents the depth estimation model.

6. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 5, characterized in that: The minimum spatial distance calculation module includes: c1: Based on the test results B t Image D shows the pixel-level depth estimation results. t The image coordinates of each pixel within the detected monitoring area are combined with its corresponding depth value to form a three-dimensional point, thus creating a set of three-dimensional points representing the local area of ​​the boom and conductor, i.e., a pseudo-point cloud (x). t ,y t ,z t ), for subsequent spatial analysis: ； Where d is the estimated depth value of the pixel, K is the intrinsic parameter matrix, and (u,v) are the two-dimensional pixel coordinates; c2: Apply physical priors to geometric corrections for transmission line areas; ； Where z' represents the estimated height of the transmission line at position x'; this is the output of the fitted curve; x' represents the horizontal distance along the direction of the transmission line; usually, the projection point of the first tower is set as the origin of the coordinate system; a, b, c are the fitting parameters; c3: Use instruments to extract point cloud datasets for the boom as P. arm ; c4: Based on the above pseudo-point cloud set and boom point cloud set, calculate the minimum spatial distance d between the boom and the guide wire. min .

7. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 6, characterized in that: Minimum spatial distance d min The expression is as follows: ； Where: d min P represents the minimum spatial distance between the boom and the power transmission line. arm P represents the set of three-dimensional coordinates of key points of the boom; line Represents a set of 3D points in a pseudo-point cloud; symbol Represents the Euclidean distance operation; For set P arm The point above, For the corresponding P line A little bit above.

8. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 7, characterized in that... The risk assessment and graded early warning module includes: d1: Minimum spatial distance d min Additional uncertainty correction: ； in: This represents the estimated distance with a confidence interval; This represents the standard deviation corresponding to the uncertainty. d2: Set the risk level based on dest: d3: When the risk level reaches the threshold, trigger an audible and visual alarm and remote push notification.

9. The transmission line protection monitoring system based on improved DepthPro depth estimation according to claim 8, characterized in that: In d2, risk levels are set based on dest, including: When d est When the depth is greater than 3m, it is considered safe. When 2m <d est If the depth is ≤3m, it is considered a slight risk. When 1m <d est When the depth is ≤2m, it is judged as moderate risk; When d est If the depth is ≤1m, it is considered a serious risk.

10. A method for monitoring the protection of power transmission lines from external damage using any one of the monitoring systems described in claims 1 to 9, characterized in that... Includes the following steps: Step 1: Define image IA, which is the image of the crane boom and the power transmission line in the monitoring area at time t; Define an image IB, which is the image of the same region at time t+1; and define a risk label label for training and validating the monitoring model. Step 2: Use the YOLOv11 network to perform target detection on images IA and IB, identify targets such as crane booms, hooks, towers, and power transmission lines, and output bounding boxes and pixel-level masks; the detection results serve as input for subsequent depth estimation and spatial modeling. Step 3: Use the improved DepthPro model to predict the depth of image IA and image IB; this model introduces a temporal Transformer module at the output to model cross-frame features; Step 4: Generate a pseudo point cloud based on the detection results and depth map, and introduce the sag equation into the transmission line section for fitting and correction. At the same time, perform skeleton modeling of the crane boom and extract the key point set as the spatial representation of the boom. Step 5: Based on the pseudo-point cloud and the fitted traverse curve, calculate the minimum spatial distance between the key points of the boom and the traverse point set, and obtain the minimum spatial distance d. min ; Step 6: Combine the uncertainty propagation method to analyze d min After correction, the estimated distance d with confidence intervals is obtained. est Risk classification is based on threshold values. If a risk is detected, an audible and visual alarm and a remote warning will be triggered.

Images