Tree obstacle intrusion detection method based on deep learning

Abstract

The invention discloses a tree obstacle intrusion detection method based on deep learning. The method comprises the following steps: detecting a power line and a tree based on a YOLOv8 target detection model; depth information of the image scene is obtained through binocular depth estimation; performing segmentation processing on the image based on the depth information, and removing distant trees and non-concerned area objects in combination with a target detection result; for the retained object, analyzing the relative positions of the tree and the power line in the 3D space; and establishing a comprehensive evaluation model based on factors such as spatial distance, tree size and growth trend, and generating early warning information. According to the method, computer vision, a digital image processing technology and a laser radar technology are combined, tree obstacle intrusion is judged by analyzing the images captured by the camera, and the method has the advantages of high universality, low cost, high accuracy and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a tree barrier intrusion detection method based on deep learning. Background Art

[0002] Deep learning-based tree intrusion detection refers to the use of image processing and computer vision technology to build a tree growth model using the feature information in the input image, and combine it with the three-dimensional coordinates of the power lines obtained by depth estimation to detect whether there is tree intrusion.

[0003] The safe and reliable operation of power transmission lines is directly related to the stable development of a country's economy and people's livelihood. Currently, transmission line channel inspections in my country's power grid are primarily conducted manually. This method is labor-intensive, has harsh working conditions, is inefficient, and difficult to manage. It no longer meets the needs of modern power grid development and safe operation, and its use and maintenance costs are high. With the advancement of computer vision technology and the development of ranging devices such as depth cameras and lidar, image-based tree intrusion detection has become an effective method. In this method, object detection technology is first used to process the image to detect power lines and trees in the scene. The tree depth information obtained by the depth camera is combined to build a growth model and remove the distant scene. The tree growth model and the 3D coordinates of the power lines are combined to detect whether the trees are at risk of crossing or approaching the power lines, thereby detecting whether a tree intrusion has occurred.

[0004] A search revealed a Chinese patent application with publication number CN112466104A, which provides a tree obstacle warning method and system for power transmission lines. This method uses a drone equipped with a lidar to scan the transmission lines in a target area, collecting first point cloud data of the transmission lines and second point cloud data of specific plants surrounding the transmission lines. The minimum distance between the transmission lines and the plants is calculated using this first and second point cloud data, allowing for subsequent tree obstacle detection. However, this method requires the drone to continuously scan and update the tree point cloud data in the tree area, which is cost-prohibitive, has low universality, and poor real-time performance.

[0005] To improve the universality and real-time performance of power line tree intrusion detection, a feasible method is to use a camera module to obtain real-time images of trees and power lines. Then, using an object detection algorithm, the tree and power line areas are detected. A depth estimation network is used to estimate image depth information. The detected tree areas and depth information are combined to calculate the 3D world coordinates of the tree and power line areas in the image. This is then combined with an erosion algorithm to build a future growth model of the trees. This model can determine current and future tree growth trends, provide early warnings for the risk of tree intrusion, and improve the accuracy of tree intrusion detection. This method uses a camera module to establish the 3D coordinates of tree points in real time, which is more cost-effective and faster. Summary of the Invention

[0006] In order to improve the accuracy of tree barrier intrusion detection and enhance anti-interference ability, the present invention provides a tree barrier intrusion detection method based on deep learning. The tree and power line areas are obtained based on the target detection algorithm, the relative position of the target of interest in 3D space is obtained by combining the depth estimation algorithm, and the tree boundary is expanded by combining the corrosion algorithm to more accurately warn of tree barrier intrusion phenomena.

[0007] To achieve the above objectives, the present invention discloses a tree barrier intrusion detection method based on deep learning, comprising the following steps: Step S10: Detecting power lines and trees based on the YOLOv8 target detection model; Step S101: During the model training phase, trees, power towers, and power lines are generated using a virtual engine and pasted onto a background image. The pasted detection frames are automatically labeled. This method allows for batch production of a large number of training datasets with different backgrounds, enhancing the robustness of the model. Step S102: For the feature expression of slender targets such as power lines, the CBAM attention mechanism is added to strengthen the expression of power line-related feature channels, and the power line contour feature expression is enhanced by generating a spatial attention map: , Where F is the input feature map with dimensions of C×H×W, is the channel average pooling result, with a dimension of C×1×1, is the channel maximum pooling result, with a dimension of C×1×1, and is the weight matrix, is the sigmoid activation function; Step S103: Enhance the feature pyramid network of the original model, add more cross-layer connections, and use a weighted fusion mechanism instead of the simple weighted fusion of the traditional feature pyramid: , Where, is the i-th layer feature of the backbone network, is the i-th layer feature of the feature pyramid, is the attention weight generating function, is the weight parameter of adaptive learning, is an upsampling operation; Step S20: Obtaining depth information of the image scene using binocular depth estimation; Step S201: During the model training phase, due to the lack of ground-truth depth data from real scenes, we used a teacher-student model for model training. For the captured images, we first used a large depth estimation model to infer depth information. After manual screening, we created a dataset of pseudo-ground-truth depth values ​​for model training. Testing showed that the trained depth estimation network achieved sufficient accuracy to meet operational requirements.

[0008] Step S30: applying segmentation processing to the image based on the depth information, and removing distant trees and objects outside the area of ​​interest in combination with the target detection results; Step S301: segmenting and masking the depth map according to the determined threshold value to divide it into three regions: near, middle and far; Step S302: Assign the tree and power line targets detected by YOLOv8 to the depth map, calculate the average depth value and depth distribution characteristics for each detected target, and establish a target depth feature database; Step S40: For the retained objects, analyze the relative positions of the trees and power lines in the 3D space; Step S401: Based on the binocular camera parameters, the 2D image coordinates are converted into 3D world coordinates, and the positional relationship between the trees and the power lines in the same 3D coordinate system is established: , Where (X, Y, Z) is the 3D world coordinate, (x, y) is the image pixel coordinate, f is the camera focal length, B is the binocular camera baseline length, d is the parallax value, is the coordinate of the camera principal point; Step S402: Extract key geometric parameters of tree height, crown width, and trunk width, establish a tree morphological characteristic feature vector, and establish a tree growth simulation model based on the principle of morphological corrosion: , Where, is the tree shape at time t, is the initial tree form, is the expansion operator, It is a time-related structural element.

[0009] The tree growth simulation model is used to predict the growth profile of trees at different time periods and calculate the estimated time required for trees to reach the dangerous contact area: , Where, To predict the collision time, is the tree growth model at time t, Simulate safe areas for power lines; Step S50: Establish a comprehensive assessment model based on factors such as spatial distance, tree size, and growth trend to generate early warning information. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Flowchart of the steps of the tree barrier intrusion detection method of the present invention.

[0011] Figure 2 Flowchart of the steps of the target detection method of the present invention. DETAILED DESCRIPTION

[0012] The following describes the embodiments of the present invention using specific examples and accompanying drawings. Those skilled in the art will readily understand the other advantages and benefits of the present invention from the disclosure herein. The present invention may also be implemented or applied through other different specific examples, and the details in this specification may be modified and altered based on different viewpoints and applications without departing from the spirit of the present invention.

[0013] A tree barrier intrusion detection method based on deep learning: Step S10: Detecting power lines and trees based on the YOLOv8 target detection model; Step S101: During the model training phase, trees, power towers, and power lines are generated using a virtual engine and pasted onto a background image. The pasted detection frames are automatically labeled. This method allows for batch production of a large number of training datasets with different backgrounds, enhancing the robustness of the model. Step S102: For the feature expression of slender targets such as power lines, the CBAM attention mechanism is added to strengthen the expression of power line-related feature channels, and the power line contour feature expression is enhanced by generating a spatial attention map: , Where F is the input feature map with dimensions of C×H×W, is the channel average pooling result, with a dimension of C×1×1, is the channel maximum pooling result, with a dimension of C×1×1, and is the weight matrix, is the sigmoid activation function; Step S103: Enhance the feature pyramid network of the original model, add more cross-layer connections, and use a weighted fusion mechanism instead of the simple weighted fusion of the traditional feature pyramid: , Where, is the i-th layer feature of the backbone network, is the i-th layer feature of the feature pyramid, is the attention weight generating function, is the weight parameter of adaptive learning, is an upsampling operation; Step S20: Obtaining depth information of the image scene using binocular depth estimation; Step S201: During the model training phase, due to the lack of ground-truth depth data from real scenes, we used a teacher-student model for model training. For the captured images, we first used a large depth estimation model to infer depth information. After manual screening, we created a dataset of pseudo-ground-truth depth values ​​for model training. Testing showed that the trained depth estimation network achieved sufficient accuracy to meet operational requirements.

[0014] Step S30: applying segmentation processing to the image based on the depth information, and removing distant trees and objects outside the area of ​​interest in combination with the target detection results; Step S301: segmenting and masking the depth map according to the determined threshold value to divide it into three regions: near, middle and far; Step S302: Assign the tree and power line targets detected by YOLOv8 to the depth map, calculate the average depth value and depth distribution characteristics for each detected target, and establish a target depth feature database; Step S40: For the retained objects, analyze the relative positions of the trees and power lines in the 3D space; Step S401: Based on the binocular camera parameters, the 2D image coordinates are converted into 3D world coordinates, and the positional relationship between the trees and the power lines in the same 3D coordinate system is established: , Where (X, Y, Z) is the 3D world coordinate, (x, y) is the image pixel coordinate, f is the camera focal length, B is the binocular camera baseline length, d is the parallax value, is the coordinate of the camera principal point; Step S402: Extract key geometric parameters of tree height, crown width, and trunk width, establish a tree morphological characteristic feature vector, and establish a tree growth simulation model based on the principle of morphological corrosion: , Where, is the tree shape at time t, is the initial tree form, is the expansion operator, It is a time-related structural element.

[0015] The tree growth simulation model is used to predict the growth profile of trees at different time periods and calculate the estimated time required for trees to reach the dangerous contact area: , Where, To predict the collision time, is the tree growth model at time t, For power line safety area simulation, Step S50: Establish a comprehensive assessment model based on factors such as spatial distance, tree size, and growth trend to generate early warning information.

Claims

1. A tree barrier intrusion detection method based on deep learning, characterized in that: The following steps are involved: S10: Detect power lines and trees based on the YOLOv8 object detection model; S20: Obtaining depth information of the image scene using binocular depth estimation; S30: applying segmentation processing to the image based on the depth information, and removing distant trees and objects outside the area of ​​interest in combination with the target detection results; S40: For the retained objects, analyze the relative positions of trees and power lines in 3D space; S50: Establish a comprehensive assessment model based on factors such as spatial distance, tree size, and growth trend to generate early warning information.

2. The tree barrier intrusion detection method based on deep learning according to claim 1, characterized in that: The step S10 specifically includes: adding the CBAM attention mechanism to strengthen the expression of the power line related feature channels, and enhancing the power line contour feature expression by generating a spatial attention map: ， Where F is the input feature map with dimensions of C×H×W, is the channel average pooling result, with a dimension of C×1×1, is the channel maximum pooling result, with a dimension of C×1×1, and is the weight matrix, It is a sigmoid activation function. At the same time, the feature pyramid network of the original model is enhanced, more cross-layer connections are added, and a weighted fusion mechanism is used instead of the simple weighted fusion of the traditional feature pyramid.

3. The tree barrier intrusion detection method based on deep learning according to claim 1, characterized in that: Step S40 specifically includes: converting 2D image coordinates into 3D world coordinates based on binocular camera parameters, establishing the positional relationship between trees and power lines in the same 3D coordinate system, extracting key geometric parameters of the tree height, crown width, and trunk width, establishing a tree morphological characteristic feature vector, and establishing a tree growth simulation model based on the principles of morphological corrosion. ， Where, is the tree shape at time t, is the initial tree form, is the expansion operator, It is a time-related structural element.

4. The tree barrier intrusion detection method based on deep learning according to claim 3 is characterized in that: The tree growth simulation model is used to predict the growth profile of trees at different time periods and calculate the estimated time required for trees to reach the dangerous contact area: ， Where, To predict the collision time, is the tree growth model at time t, Simulates the safe area for power lines.

Images