Tree obstacle risk grading method and system for analyzing binocular vision point cloud based on DETR network

Abstract

The invention discloses a tree obstacle risk grading method and system for analyzing binocular vision point cloud based on a DETR network, and relates to the field of intelligent safety monitoring of electric power facilities, and the method comprises the steps: collecting a stereo image of a power transmission line corridor, and generating three-dimensional point cloud data based on the image; based on the three-dimensional point cloud data, performing point cloud segmentation to obtain feature parameters; performing projection conversion to obtain a projection image; detecting and classifying the tree target based on the projection image to obtain a tree detection result with a label; based on the characteristic parameters and the tree detection result with the label, establishing a tree obstacle risk assessment model, and performing risk grading on the tree; the tree obstacle risk assessment efficiency of different tree species is remarkably improved, the tree growth trend can be monitored in real time, the risk classification can be dynamically adjusted, an accurate decision basis is provided, and a guarantee is provided for safe operation of a power transmission line.

Description

Technical Field

[0001] This invention relates to the field of intelligent safety monitoring of power facilities, and in particular to a method and system for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds. Background Technology

[0002] In power transmission line corridors, tree growth poses a potential threat to the safety of power transmission, especially under extreme weather conditions such as wind and rain. Fallen trees or branches touching transmission lines can cause short circuits, power outages, and even large-scale blackouts. As the power industry's requirements for the safe operation of transmission lines increase, how to efficiently and accurately assess tree-related risks in transmission line corridors has become a critical issue that urgently needs to be addressed. Traditional tree-related risk assessment methods largely rely on manual inspections and visual checks, which are not only costly and inefficient, but also difficult to achieve a comprehensive and accurate assessment due to human factors. Moreover, traditional methods often rely on simple tree classification and location judgment, failing to fully consider the tree's growth status, environmental factors, and dynamic changes in surrounding facilities, leading to significant errors in tree-related risk classification.

[0003] In recent years, with the widespread application of high-tech sensors such as unmanned aerial vehicles (UAVs), LiDAR, and binocular vision, tree identification and analysis technologies based on point cloud data have gradually become an effective means of solving the problem of tree-related risks. LiDAR and binocular vision technologies can model trees through accurate 3D imaging, obtain the spatial relationship between trees and power transmission lines, and thus predict potential tree-related risks. However, the processing and analysis of point cloud data faces significant technical challenges. Point cloud data is typically sparse, unordered, and massive in volume. How to effectively extract useful features from this complex 3D data and perform accurate target detection and segmentation is one of the current technological challenges. In the field of point cloud data processing, deep learning technology, especially the DETR (Detection Transformer) network based on Transformer, has achieved significant results in target detection tasks in recent years. DETR can perform global modeling through a self-attention mechanism, thus overcoming the limitations of traditional convolutional neural networks in complex backgrounds. Although DETR performs excellently in target detection in 2D images, its application in 3D point cloud data is still in the exploratory stage. The sparsity and irregularity of point cloud data make it difficult to directly apply traditional deep learning models such as DETR to 3D data analysis. Meanwhile, binocular vision systems, as a low-cost and technologically mature method for acquiring 3D data, have been widely used in environmental perception and target detection in recent years. Using images acquired by two cameras, binocular vision can generate point cloud data through disparity calculation, providing accurate 3D information on the relative positions of trees and power transmission lines. Current point cloud analysis techniques typically focus on tree detection and segmentation, lacking in-depth analysis of the relationship between trees and their surrounding environment. Especially in the special scenario of power transmission line corridors, tree barrier risk assessment often fails to achieve dynamic and refined classification. This means that existing technologies have weak predictive capabilities for dynamic factors such as tree growth trends, seasonal changes, and weather, and cannot comprehensively and timely reflect the threat posed by trees to power transmission lines. Furthermore, traditional tree barrier risk assessment methods often rely on human experience or simple threshold judgments, lacking data-driven intelligent analysis. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is: how to accurately identify trees and their surrounding environment, comprehensively analyze the spatial relationship between trees and power transmission lines, and thus realize intelligent classification of tree obstacle risks.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds, including:

[0008] Collect stereoscopic images of the power transmission line corridor and generate 3D point cloud data based on the images;

[0009] The acquisition of stereoscopic images of the power transmission line corridor, and the generation of three-dimensional point cloud data based on the images, includes:

[0010] Stereoscopic images of the power transmission line corridor were acquired using a binocular camera system;

[0011] Based on parallax calculation and camera parameters, the stereo image is converted into three-dimensional point cloud data;

[0012] Based on 3D point cloud data, point cloud segmentation is performed to obtain feature parameters; projection transformation is then performed to obtain a projected image.

[0013] Based on the projected images, tree targets are detected and classified to obtain labeled tree detection results;

[0014] Based on feature parameters and labeled tree detection results, a tree obstacle risk assessment model is established to classify the risk of trees.

[0015] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0016] The feature parameters obtained by point cloud segmentation include:

[0017] The 3D point cloud data is segmented to extract point cloud clusters for individual trees;

[0018] The tree height, canopy width, tilt angle, and shortest distance to the power transmission line are calculated based on the point cloud cluster.

[0019] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0020] The process of performing projection conversion to obtain the projected image includes:

[0021] The 3D point cloud data is mapped onto the XY plane to generate a 2D image, where the value of each pixel represents the depth information of the corresponding spatial location; the size of the projected image is set to a preset resolution.

[0022] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0023] The process of detecting and classifying tree targets based on projection images to obtain labeled tree detection results includes:

[0024] Based on projected images, a labeled dataset containing trees to be risk-classified is constructed by annotating bounding boxes and corresponding tree species labels. This dataset is then input into the DETR network for training, and the trained DETR network outputs tree species labels and location coordinates.

[0025] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0026] The tree obstacle risk assessment model, based on feature parameters and labeled tree detection results, is used to classify trees according to their risk, including:

[0027] The risk of trees is divided into three levels: low risk, medium risk, and high risk. A weighted average model is used to calculate the comprehensive risk value of each tree, and the risk is classified according to the magnitude of the comprehensive risk value.

[0028] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0029] The weighted average model uses the following weighting formula:

[0030] R = w1·H + w2·D + w3·G + w4·T

[0031] Where R is the risk value of the tree; H is the height of the tree; D is the shortest distance between the tree and the power transmission line; G is the growth rate of the tree; T is the structural stability score of the tree; and w1, w2, w3, and w4 are the weights set.

[0032] As a preferred approach for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, wherein:

[0033] The process of establishing a tree barrier risk assessment model based on feature parameters and labeled tree detection results, and classifying the risk of trees, also includes:

[0034] Based on the comprehensive risk value R of each tree, a grading standard is set to divide the trees into different risk levels: low risk: R<5; medium risk: 5≤R<7; high risk: R≥7.

[0035] Secondly, embodiments of the present invention provide a tree obstacle risk classification system based on DETR network analysis of binocular visual point clouds, comprising:

[0036] The point cloud generation module is used to acquire stereoscopic images of the power transmission line corridor and generate three-dimensional point cloud data based on the images;

[0037] The point cloud processing module is used to perform point cloud segmentation to obtain feature parameters based on 3D point cloud data; and to perform projection transformation to obtain projected images.

[0038] The detection and classification module is used to detect and classify tree targets based on the projected image, and obtain labeled tree detection results;

[0039] The risk classification module is used to establish a tree barrier risk assessment model based on feature parameters and labeled tree detection results, and to classify the risk of trees.

[0040] Thirdly, embodiments of the present invention provide a computing device, including:

[0041] Memory and processor;

[0042] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the one or more programs are executed by the one or more processors, the one or more processors implement the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in any embodiment of the present invention.

[0043] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds.

[0044] The beneficial effects of this invention are as follows: This invention uses a DETR network for tree target detection, which greatly improves the accuracy and robustness of target detection; through automated visual data acquisition and deep learning-based target detection, it significantly improves the efficiency and accuracy of tree identification and risk assessment. The system can process large amounts of point cloud data in a short time, automatically identify and analyze tree risks, reduce the need for manual intervention, and greatly improve work efficiency, especially in the inspection of large-scale power transmission line corridors. It can not only accurately assess tree risks during initial monitoring, but also combine dynamic factors such as tree growth rate and seasonal changes to perform real-time risk prediction and updates. Through periodic point cloud data acquisition, the system can monitor tree growth in real time and promptly detect potential risks from trees. Especially for fast-growing tree species (such as eucalyptus and tung trees) and unstable tree species (such as bamboo), it can dynamically adjust risk assessment results and take measures in advance to avoid sudden tree-related risks; by comprehensively considering multiple key factors (tree height, distance from power transmission lines, tree growth rate, tree stability, etc.), a comprehensive risk value is obtained through weighted calculation, which can comprehensively and scientifically assess the risk level of trees. This method avoids the limitations of single-factor analysis, resulting in more accurate assessments. Tree height and growth rate, along with distances from transmission lines, vary significantly among different tree species. Traditional methods often fail to consider the interaction of all factors, while this invention comprehensively considers all potential risks associated with trees, providing power companies with scientific tree management and maintenance solutions. Based on the analysis of tree risk values ​​and risk levels, personalized risk management strategies can be developed. For high-risk trees, pruning or transplantation can be carried out in advance to reduce the risk of trees coming into contact with transmission lines; for medium-risk trees, regular inspections and growth monitoring plans can be developed. This data-driven decision-making method not only improves the accuracy of tree management but also reduces potential safety hazards in power line operation. The use of deep learning and automated data processing reduces human error, ensuring the objectivity and consistency of tree risk assessments. This not only improves the scientific nature of tree risk management but also significantly reduces the probability of transmission lines encountering tree-related accidents, ensuring the safe and stable operation of power lines. Through binocular visual point cloud data and DETR networks, it can effectively adapt to tree identification in complex environments, especially in dense forests and complex terrain, accurately identifying tree types and locations. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is an overall flowchart of the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds provided by the present invention;

[0047] Figure 2 This is a DETR model structure diagram of the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds provided by the present invention;

[0048] Figure 3 This invention provides a risk heatmap generated by a tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds. Detailed Implementation

[0049] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0050] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds, including:

[0051] S1: Acquire stereoscopic images of the power transmission line corridor and generate 3D point cloud data based on the images;

[0052] S2: Based on 3D point cloud data, perform point cloud segmentation to obtain feature parameters; perform projection transformation to obtain a projected image;

[0053] S3: Based on the projected image, detect and classify tree targets to obtain labeled tree detection results;

[0054] S4: Based on feature parameters and labeled tree detection results, establish a tree obstacle risk assessment model to classify the risk of trees.

[0055] It should be noted that through steps S1-S4, trees and their surrounding environment can be accurately identified, and the spatial relationship between trees and transmission lines can be comprehensively analyzed, thereby achieving intelligent classification of tree-related risks. Compared with traditional technologies, this method can combine the rich information of point cloud data to conduct dynamic and refined tree-related risk assessments, providing effective technical support for the safe operation of transmission lines.

[0056] Example 2, refer to Figures 1-3Tables 1-5 illustrate one embodiment of the present invention, providing a tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds, according to the previous embodiment. The method includes:

[0057] In this embodiment, the step S1 above, which involves acquiring a stereoscopic image of the transmission line corridor and generating three-dimensional point cloud data based on the image, includes:

[0058] Using a binocular camera system, two images of the power transmission line corridor area from different perspectives are captured, and three-dimensional point cloud data is generated through parallax calculation.

[0059] For example, to ensure the representativeness of the acquired point cloud data and its coverage of different tree species, nine common tree species (eucalyptus, camphor, paper mulberry, oak, tung tree, cypress, pine, fir, and bamboo) were selected from representative transmission line corridors for data collection. These trees exhibit differences in height, crown width, and growth status, effectively validating the adaptability of this method to different tree species.

[0060] Specifically, in order to effectively capture the spatial relationship between trees and power transmission lines in complex environments, a suitable binocular vision camera system is set up. Commonly used binocular vision systems include two camera sensors configured at appropriate baseline distances and angles to ensure coverage of the entire power transmission line corridor.

[0061] Camera Selection: Choose a high-precision binocular camera system (such as the Intel RealSense D455, with a 90° field of view and capable of acquiring image data at a resolution of up to 1280x720). This system is used for monitoring power corridors with significant ambient light variations and supports high-precision depth sensing.

[0062] Camera Installation: To maximize the accuracy of point cloud data acquisition, the relative distance and installation angle of the binocular cameras need to be precisely set. When deploying binocular cameras along the power transmission line corridor, fix two cameras horizontally or at a slight angle to the platform, ensuring that each tree is covered by the camera at least twice for imaging.

[0063] For example, the baseline distance (i.e., the horizontal distance between the two cameras) of the installed binocular cameras is 0.5 meters, and the cameras are installed at a height of 2.5 meters above the ground. This configuration is suitable for measuring the height of trees, the extent of their canopies, and their relative position to power lines.

[0064] Furthermore, when acquiring stereoscopic images of the power transmission line corridor, the camera captures the same scene from different angles, simultaneously taking two images to obtain a set of left and right images (such as the left and right images). These images contain visual information about the target area (trees, power transmission lines, and the surrounding environment). Shooting is done from the side or slightly forward of the trees to ensure a complete view of the trees and their spatial relationship with the power transmission lines. Shooting is also conducted on sunny days to avoid the impact of lighting and weather on image quality. To ensure alignment of the two images, stereo calibration is required. This is achieved by calibrating both internal camera parameters (such as focal length, principal point, and distortion parameters) and external parameters (i.e., the relative position and orientation between the two cameras) to ensure consistent visual information between the left and right images.

[0065] Image-based generation of 3D point cloud data includes:

[0066] By utilizing the parallax principle of binocular cameras, a depth map is obtained by calculating the parallax of the same object in two images.

[0067] Specifically, parallax refers to the difference in position of the same point in the same scene between two images. Let PL(xL,yL) be the point in the left image and PR(xR,yR) be the point in the right image. Then the parallax d is calculated by d = xL - xR; where d is the parallax, representing the difference in the abscissa of the corresponding point of the same object in the left and right images; xL and xR are the abscissas of the points in the left and right images, respectively.

[0068] Each pixel in the image is converted into depth information of a three-dimensional point, thereby generating point cloud data; the depth information Z of each pixel in the depth map is calculated by the following formula:

[0069]

[0070] Where f is the focal length of the camera; B is the baseline distance between the two cameras (the distance between the two cameras); and d is the depth of the camera.

[0071] After parallax and depth calculations, each pixel in the image is converted into coordinates in three-dimensional space. Let the intrinsic parameter matrix of the left camera be K, and the intrinsic parameter matrix of the right camera be K′. The three-dimensional coordinates (X, Y, Z) of each pixel are obtained through geometric transformation.

[0072] For each pixel (xL, yL), its 3D coordinates (X, Y, Z) are calculated using the depth map Z and the camera intrinsic matrix K, and are represented as follows:

[0073]

[0074] Where cx, cy are the coordinates of the camera's principal point (usually located at the center of the image); fx, fy are the camera's focal lengths (horizontal and vertical).

[0075] By processing and synthesizing left and right images from different perspectives, the calculated 3D coordinate points are combined to generate complete 3D point cloud data. This point cloud data contains spatial distribution information of trees, such as tree crowns, trunks, and relative distances to power transmission lines.

[0076] It should be noted that this step yields a three-dimensional spatial coordinate point cloud containing trees, power lines, and the environment. This enables efficient and stable tree data collection in complex environments (such as dense forests, obstructions, etc.), providing high-quality data support for subsequent target detection and risk assessment.

[0077] In this embodiment, the point cloud segmentation to obtain feature parameters in step S2 above includes:

[0078] For each tree, clustering is performed using the Euclidean clustering algorithm based on the spatial location (such as distance and density) of points in the point cloud: for each point, find points that are less than a certain threshold to form the neighborhood of the point; by setting the threshold, independent clustering regions are formed (each cluster represents a tree).

[0079] The specific formula is as follows:

[0080]

[0081] Where D is the shortest distance between the tree and the power transmission line; X tree ,Yt ree Z tree The three-dimensional coordinates of the tree; X line Y line Z line The three-dimensional coordinates of the transmission line.

[0082] In another possible implementation, if clustering methods are not suitable for complex scenarios (such as trees being too close together), deep learning methods (such as PointNet++ or VoxelNet) can be used to automatically identify and segment the 3D region of each tree from the point cloud. By spatially segmenting the trees, clear tree boundaries can be provided for subsequent risk assessment.

[0083] After extracting the point cloud data of the trees, tree features are extracted.

[0084] Different tree species have different growth forms, therefore feature extraction for each species needs to be differentiated. The following are feature extraction methods for nine major tree species:

[0085] Tree height extraction: Tree height is obtained by calculating the vertical range of the tree point cloud. Tree height is the distance from the ground to the treetop, typically the maximum value of the Z-axis coordinate in the point cloud data, expressed as:

[0086] H = max(Z) Crown width extraction: The crown width W is obtained by calculating the horizontal range of the tree point cloud (such as the maximum diameter on the horizontal plane), and is expressed as:

[0087] W = max(X - coordinate range)

[0088] Tree tilt angle extraction: The tilt angle of a tree is determined by calculating the centroid position of its point cloud. If the centroid of the tree's point cloud deviates significantly from the vertical direction, it indicates that the tree may be at risk of tilting. The tilt angle θ can be calculated using the following formula:

[0089]

[0090] Shortest distance between trees and power lines: Calculate the distance between each tree and the power line. If the location of the power line is known, the shortest distance can be calculated using the nearest point between the tree and the power line in the point cloud data.

[0091] For example, after acquiring and segmenting point cloud data, experiments were conducted to verify the results for nine tree species (eucalyptus, camphor, paper mulberry, oak, tung oil tree, cypress, pine, fir, and bamboo). The features extracted for each tree species are as follows:

[0092] Eucalyptus trees: They are relatively tall, have wide canopies, grow quickly, and are easily located near power lines, so special attention should be paid to the distance between trees and power lines.

[0093] Camphor trees and paper mulberry trees: These trees are relatively sturdy, but grow slowly, and are generally classified as low to medium risk.

[0094] Tung tree: The tree grows taller relatively quickly and has many branches, which can easily cause tree obstacles, and is rated as medium to high risk.

[0095] Chinese arborvitae and pine trees: They grow relatively evenly and are large trees, but are generally classified as medium risk.

[0096] Fir and bamboo: Although the trees are relatively short, they are rated as medium risk due to their vigorous growth and tendency to lean.

[0097] It should be noted that through this process, the 3D point cloud features of each type of tree are accurately extracted, providing data support for subsequent tree barrier risk assessment. All accurately extracted tree point cloud data will be stored in a database and transmitted and retrieved via a cloud platform or local server. To ensure the real-time nature and reliability of the data, efficient transmission protocols, such as MQTT or WebSocket, are used to ensure that the point cloud data can be quickly uploaded to the analysis platform after collection.

[0098] The projection transformation to obtain the projected image includes:

[0099] The 3D point cloud data is mapped onto the XY plane to generate a 2D image, where the value of each pixel represents the depth information of the corresponding spatial location. To ensure detection accuracy, the size of the projected image needs to be set to an appropriate resolution. In this embodiment, a 512×512 image is preferred, which is encoded into a grayscale image using depth map data (depth values ​​are normalized and mapped to a grayscale range).

[0100] In another possible implementation, to improve the robustness and generalization ability of the DETR network, data augmentation can be performed on the generated images, including operations such as rotation, scaling, and flipping. Random noise and blurring can also be applied to simulate detection tasks under different environments.

[0101] In this embodiment, step S3 above, which involves detecting and classifying tree targets based on the projected image to obtain labeled tree detection results, includes:

[0102] like Figure 2 The diagram shown is a structure diagram of the DETR model. In this embodiment, the DETR (Detection Transformer) network is used to detect and classify tree targets.

[0103] It should be noted that DETR is a Transformer-based object detection method that efficiently processes and identifies objects in images through a self-attention mechanism and end-to-end training. For the task of detecting trees along power transmission line corridors, the DETR network can convert complex 3D point cloud data into 2D or projected images for processing, thereby achieving tree object detection and classification.

[0104] Based on projected images, a labeled dataset containing trees to be risk-classified is constructed. The location and category (i.e., tree species) of the trees are labeled for each image. For each tree, the bounding box and the corresponding tree species label are manually labeled.

[0105] Examples include eucalyptus and camphor trees:

[0106] Eucalyptus: The location and size of the trees are indicated by a bounding box, and the category label is "eucalyptus";

[0107] Camphor tree: The location and size of the tree are indicated by a bounding box, and the category label is "camphor tree";

[0108] Different category labels are given for different tree species. Training sets are constructed using these labels and used for training and optimization of the DETR network.

[0109] Specifically, the DETR network uses cross-entropy loss for classification training and L1 loss and GIoU loss for bounding box regression training. The Adam optimizer is used for optimization, with a learning rate of 0.0001 and a batch size of 4.

[0110] During the training phase, the goal of the DETR network is to minimize the following loss function:

[0111] L = L cls +L bbox

[0112] Among them, L cls The classification loss represents the difference between the detected object category and the true label; L bbox The bounding box loss represents the positional difference between the predicted bounding box and the true bounding box.

[0113] The bounding box for each object detection is represented by four parameters: [x, y, w, h], where x and y are the coordinates of the center point of the bounding box, and w and h are the width and height, respectively.

[0114] After training, the DETR network can be used for the detection and classification of tree targets.

[0115] Specifically, the input image is a projected image that is propagated forward through the network.

[0116] Feature extraction: The backbone of the network (such as ResNet) extracts feature maps from the input image. Feature maps are high-level feature representations of the image, including information such as the shape, size, and color of trees.

[0117] Transformer Encoder: The feature map is passed to the Transformer encoder to model the relationships between different regions in the image. The encoder globally correlates local information, extracting contextual information about the trees and their surrounding environment.

[0118] Object detection: After processing by the Transformer decoder, the network outputs the category and location of each predicted box. Each box contains the tree category (e.g., eucalyptus, camphor, etc.) and location (bounding box coordinates).

[0119] For example, when selecting nine types of trees (eucalyptus, camphor, paper mulberry, oak, tung tree, cypress, pine, fir, bamboo, etc.) for detection, the DETR network can process and classify multiple tree targets simultaneously. Each tree is assigned a corresponding label, such as "eucalyptus" or "pine," and its location bounding box is output. The DETR network can identify these trees based on features in the input image and assign a classification label (tree species) and a location bounding box to each tree. For each tree, the DETR output is as follows:

[0120] Tree 1 (Eucalyptus): Predicted category is "Eucalyptus", bounding box coordinates are [x1, y1, w1, h1]

[0121] Tree 2 (Camphor Tree): Predicted category is "Camphor Tree", bounding box coordinates are [x2, y2, w2, h2]

[0122] Tree 3 (Paper Mulberry): Predicted category is "Paper Mulberry", bounding box coordinates are [x3, y3, w3, h3]

[0123] Tree 4 (Quercus glauca): Predicted category is "Quercus glauca", bounding box coordinates are [x4, y4, w4, h4]

[0124] Tree 5 (Tung Tree): Predicted category is "Tung Tree", bounding box coordinates are [x5, y5, w5, h5]

[0125] Tree 6 (Chinese arborvitae): Predicted category is "Chinese arborvitae", bounding box coordinates are [x6, y6, w6, h6]

[0126] Tree 7 (Pine): Predicted category is "Pine", bounding box coordinates are [x7, y7, w7, h7]

[0127] Tree 8 (Fir): Predicted category is "Fir", bounding box coordinates are [x8, y8, w8, h8]

[0128] Tree 9 (Bamboo): Predicted category is "Bamboo", bounding box coordinates are [x9, y9, w9, h9]

[0129] It should be noted that through this detection method, the DETR network can efficiently detect and classify different tree species in images, identifying the location of each tree. After training, the DETR network can detect tree targets in images relatively accurately and classify them. Taking eucalyptus trees as an example, experimental results show that:

[0130] In multiple test scenarios, DETR successfully detected the location of trees and accurately identified them as "eucalyptus" with an accuracy of 85%. For tree bounding box localization, DETR was able to control the bounding box error within 5%, ensuring that the relative position of trees and power transmission lines was accurately captured. For other tree species (such as camphor trees and pine trees), DETR's classification accuracy was also above 80%, and it could effectively handle complex situations where trees were close together or obscured.

[0131] In this embodiment, step S4 above, which establishes a tree obstacle risk assessment model based on feature parameters and labeled tree detection results, and classifies trees according to risk, includes:

[0132] It should be noted that tree barrier risk assessment models typically consider the following main factors:

[0133] Tree height: The height of trees directly affects whether they may come into contact with power transmission lines, especially under extreme weather conditions such as strong winds or heavy rain.

[0134] The minimum distance between trees and power lines: The distance between trees and power lines is a key factor affecting the risk of tree-related hazards. If the distance is too close, there is a risk that the trees may fall or grow and come into contact with the lines.

[0135] Tree growth rate and structural stability: Different tree species have different growth rates, and trees with shorter growth cycles may approach power transmission lines more quickly; the structural stability of trees is also related to whether they are prone to tilting or falling over.

[0136] Tree type and wood properties: The wood properties of different tree species affect their ability to withstand winds in extreme weather such as storms. For example, eucalyptus trees grow relatively quickly but are more susceptible to storms, while pine trees are generally more stable.

[0137] Based on the combined effects of these factors, the risk of trees is categorized into multiple levels (low risk, medium risk, and high risk). A weighted average model is used to calculate the overall risk value for each tree and classify it according to its size.

[0138] To conduct tree barrier risk assessment, the following weighted formula is used to combine risks of different characteristics:

[0139] R = w1·H + w2·D + w3·G + w4·T

[0140] Where R is the risk value of the tree; H is the height of the tree (in meters); D is the shortest distance between the tree and the transmission line (in meters); G is the growth rate of the tree (in years), with different growth rate values ​​given according to the characteristics of the tree species; and T is the structural stability score of the tree (range: 0 to 10, with 10 being the most stable). The weights w1, w2, w3, and w4 reflect the relative importance of different factors in risk assessment. These weights can be determined using historical data. In this embodiment, the weights are set as follows: w1 = 0.4 (tree height has a significant impact on risk); w2 = 0.3 (distance between the tree and the transmission line has a significant impact on risk); w3 = 0.2 (tree growth rate has a relatively small impact on risk); and w4 = 0.1 (tree stability has the least impact on risk).

[0141] Based on the feature parameters, obtain the tree height H and the shortest distance D between the tree and the power transmission line;

[0142] The distance D between the trees and the power transmission line can be obtained by calculating the shortest distance from each tree to the power transmission line in the point cloud. Assuming the power transmission line is a known geometric path (approximately a straight line), the shortest distance D between each tree can be calculated by taking the distance from the point to the line.

[0143] To obtain the tree growth rate G: Based on the characteristics of the tree species, we set a fixed value for the annual increase in tree height (or diameter).

[0144] Tree stability score (T): A stability score is assessed based on the tree's physical characteristics and environmental conditions. Factors considered include trunk diameter, branch structure, and soil quality. Tree stability can be determined through expert experience or existing literature.

[0145] Substitute the characteristic values ​​of the trees to be risk-classified into the weighted formula to calculate their risk values. The calculation is as follows:

[0146] R = w1·H + w2·D + w3·G + w4·T

[0147] Based on the calculated risk values, grading standards are set to classify trees into different risk levels.

[0148] The classification criteria set in this embodiment are as follows: Low risk: R<5; Medium risk: 5≤R<7; High risk: R≥7;

[0149] Based on the above analysis results, corresponding management recommendations are proposed for trees categorized into high-risk, medium-risk, and low-risk groups:

[0150] High-risk trees should be pruned regularly to prevent them from growing too tall and coming into contact with power lines. If trees grow too fast or are too close to power lines, consider removing them or transplanting them to a safe distance in advance. Regular inspections of these trees should be strengthened, especially before and after extreme weather events such as strong winds and thunderstorms.

[0151] Regularly prune trees at medium risk to maintain appropriate height and avoid prolonged contact with power lines; monitor their growth and pay attention to the risk of contact with lines during growth; strengthen monitoring of trees with poor stability, such as moso bamboo, and promptly address issues of lodging or weakening.

[0152] Low-risk trees may be left untouched for the time being.

[0153] For example, the risk values ​​for nine common tree species are calculated as follows:

[0154] Extracting the maximum vertical coordinate of each tree, the height of the trees is obtained as follows:

[0155] Table 1 Tree Height

[0156]

[0157]

[0158] The shortest distance between the trees and the power transmission line is obtained as follows:

[0159] Table 2 Shortest distance between trees and power lines

[0160] species Shortest distance D (meters) eucalyptus 2.8 camphor tree 4.5 Paper mulberry 3.2 Qinggang 5.0 tung tree 3.5 Chinese arborvitae 6.0 pine 3.8 cedar 4.2 Nan bamboo 7.5

[0161] The estimated tree growth rate is as follows:

[0162] Table 3 Tree growth rate

[0163] species Tree growth rate G (meters per year) eucalyptus 1.2 camphor tree 0.8 Paper mulberry 1.0 Qinggang 0.6 tung tree 1.1 Chinese arborvitae 0.4 pine 0.7 cedar 0.9 Nan bamboo 0.5

[0164] The tree stability scores are as follows:

[0165] Table 4 Tree Stability Scores

[0166] species Stability score T (0-10 points) eucalyptus 6 camphor tree 8 Paper mulberry 7 Qinggang 9 tung tree 7 Chinese arborvitae 9 pine 8 cedar 7 Nan bamboo 5

[0167] The risk values ​​of the trees are calculated as follows:

[0168] Table 5 Tree Risk Values

[0169] species Risk value R eucalyptus 0.4×15.2+0.3×2.8+0.2×1.2+0.1×6＝7.76 camphor tree 0.4×12.5+0.3×4.5+0.2×0.8+0.1×8＝7.31 Paper mulberry 0.4×10.8+0.3×3.2+0.2×1.0+0.1×7＝6.18 Qinggang 0.4×18.0+0.3×5.0+0.2×0.6+0.1×9＝9.72 tung tree 0.4×14.0+0.3×3.5+0.2×1.1+0.1×7＝7.57 Chinese arborvitae 0.4×10.0+0.3×6.0+0.2×0.4+0.1×9＝6.78 pine 0.4×12.0+0.3×3.8+0.2×0.7+0.1×8＝6.88 cedar 0.4×16.5+0.3×4.2+0.2×0.9+0.1×7＝8.74 Nan bamboo 0.4×8.0+0.3×7.5+0.2×0.5+0.1×5＝6.05

[0170] Based on the calculation results, the risk levels of the nine tree species are as follows: Eucalyptus: High risk (7.76); Camphor tree: High risk (7.31); Paper mulberry: Medium risk (6.18); Oak: High risk (9.72); Tung tree: High risk (7.57); Chinese arborvitae: Medium risk (6.78); Pine: Medium risk (6.88); Fir: High risk (8.74); Bamboo: Medium risk (6.05).

[0171] Based on the above calculation results, the following management recommendations are proposed:

[0172] Eucalyptus (Risk Value 7.76): Eucalyptus trees are relatively tall (15.2 meters) and grow close to power lines (2.8 meters), with a rapid growth rate (1.2 meters / year), but a low stability score (6 points). This makes eucalyptus a high-risk species, especially during strong winds or as the trees grow, as they may come into contact with power lines. Regular pruning and monitoring of eucalyptus trees are recommended to prevent excessive growth, or removal should be considered.

[0173] Camphor Tree (Risk Value 7.31): The camphor tree has relatively good height (12.5 meters) and stability score (8 points), but its proximity to power lines (4.5 meters) and slow growth rate (0.8 meters / year) mean that it is a potential source of risk. Despite its slow growth, the camphor tree's proximity to power lines remains a potential risk and therefore requires regular inspection and pruning.

[0174] Oak (Risk Value 9.72): Oak trees are very tall (18 meters) and located 5 meters from power lines, making them a high-risk species. Although it has a high stability score (9 points), its enormous height and slow growth rate (0.6 meters / year) still pose a risk of it falling over or touching power lines. Special attention needs to be paid to its growth and health to ensure it does not pose a threat to power lines.

[0175] Tung tree (risk value 7.57): The height (14 meters) and stability score (7 points) of the tung tree make it a relatively high-risk tree. Although its growth rate is relatively fast (1.1 meters / year), its distance from power transmission lines (3.5 meters) still poses a potential risk. Increased monitoring of the tung tree, regular pruning, and growth control are needed.

[0176] Cedar (Risk Value 8.74): The height of the cedar (16.5 meters) and its distance from power lines (4.2 meters) make it a high-risk tree. Although it grows relatively quickly (0.9 meters / year), its stability score is low (7 points), and it may be affected by extreme weather such as storms. Cedar trees require regular inspection and pruning as needed.

[0177] Paper mulberry (risk value 6.18): Paper mulberry has a moderate height and growth rate (10.8 meters, 1.0 meter / year) and is relatively far from power transmission lines (3.2 meters). Although its stability score is good (7 points), due to its relatively low height and fast growth rate, paper mulberry is classified as a medium-risk tree species. Regular checks on its growth are necessary to ensure it does not protrude near power transmission lines.

[0178] Chinese arborvitae (Risk value 6.78): Chinese arborvitae are relatively short (10 meters) and grow slowly (0.4 meters / year), and are located far from power lines (6 meters). While it scores high on stability (9 points), its slow growth and short height place it in the medium-risk category in the risk assessment. Although the risk is low, regular monitoring of its growth is still necessary.

[0179] Pine Tree (Risk Value 6.88): The pine tree is of moderate height (12 meters) and has a high stability score (8 points), but it is relatively close to the power transmission line (3.8 meters). Due to the moderate growth rate of the pine tree (0.7 meters / year), its proximity to the power transmission line may become a potential risk in the coming years. Regular pruning is necessary to prevent the pine tree from growing too tall and coming into contact with the power transmission line.

[0180] Moso bamboo (risk value 6.36): Moso bamboo is relatively short (8 meters) and far from power lines (7.5 meters). Although its growth rate is slow (0.5 meters / year), its stability score is relatively low (5 points). Due to its slow growth and relatively poor stability, its health still needs to be monitored regularly to prevent problems such as lodging.

[0181] In another possible implementation, based on the results of risk classification, heat maps and reports can be generated, and targeted management recommendations can be output.

[0182] like Figure 3 As shown, the generated risk heat map can be overlaid on the GIS map with risk levels (red / yellow / green corresponding to high / medium / low); high-risk trees are highlighted in the point cloud and their risk values ​​are marked.

[0183] If the risk assessment result is high risk, a work order will be generated to prompt pruning (e.g., oak trees need to be pruned within 3 months). If the risk assessment result is medium risk, an inspection reminder will be pushed to the mobile terminal (e.g., paper mulberry trees should be checked every 6 months). If the risk assessment result is low risk, it will be included in the annual inspection plan and no emergency treatment is required.

[0184] Set alarm thresholds and monitor them in real time;

[0185] For example, a real-time alarm is triggered when the shortest distance between a tree and a power transmission line is less than 2 meters or when the risk value increases by more than 10% per week.

[0186] It should be noted that this method provides a more comprehensive tree risk assessment model by comprehensively evaluating factors such as tree height, distance, and stability. This innovation avoids the one-sidedness of relying on a single factor and improves the accuracy of risk assessment. Tree growth rate is a key factor, especially in high-growth tree species (such as eucalyptus and tung trees), where the trees may rapidly approach power transmission lines in a short period of time, increasing the risk. By dynamically monitoring the tree growth rate, the future risk of trees can be predicted, avoiding the limitations of traditional methods that rely solely on static data for assessment.

[0187] Based on the risk assessment results of trees, the system automatically classifies risks and provides corresponding management suggestions. This intelligent risk management approach reduces human intervention and improves the efficiency of tree obstacle management. According to the risk value of trees, they are divided into three levels: high risk, medium risk, and low risk. The system can automatically determine whether trees need to be pruned, transplanted, or monitored, avoiding the subjectivity and inefficiency of manual judgment.

[0188] Personalized management measures are provided based on the risk level of different trees. For example, for high-risk trees (such as eucalyptus and oak), pruning or removal plans are provided; for medium-risk trees (such as pine and bamboo), their growth is regularly inspected and monitored. This enables power companies to have practical tree management strategies to ensure the safety of transmission lines.

[0189] Example 3: The above is an illustrative scheme of the tree barrier risk classification method based on DETR network analysis of binocular visual point clouds in this embodiment. It should be noted that the technical solution of the tree barrier risk classification system based on DETR network analysis of binocular visual point clouds belongs to the same concept as the technical solution of the tree barrier risk classification method based on DETR network analysis of binocular visual point clouds described above. Details not described in detail in the technical solution of the tree barrier risk classification system based on DETR network analysis of binocular visual point clouds in this embodiment can be found in the description of the technical solution of the tree barrier risk classification method based on DETR network analysis of binocular visual point clouds described above.

[0190] This embodiment also provides a system for tree obstacle risk classification based on DETR network analysis of binocular visual point clouds, including:

[0191] The point cloud generation module is used to acquire stereoscopic images of the power transmission line corridor and generate three-dimensional point cloud data based on the images;

[0192] The point cloud processing module is used to perform point cloud segmentation to obtain feature parameters based on 3D point cloud data; and to perform projection transformation to obtain projected images.

[0193] The detection and classification module is used to detect and classify tree targets based on the projected image, and obtain labeled tree detection results;

[0194] The risk classification module is used to establish a tree barrier risk assessment model based on feature parameters and labeled tree detection results, and to classify the risk of trees.

[0195] This embodiment also provides a computing device suitable for tree obstacle risk classification methods based on DETR network analysis of binocular visual point clouds, including:

[0196] The system includes a memory and a processor. The memory stores computer-executable instructions, and the processor executes these instructions to implement the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds, as proposed in the above embodiments.

[0197] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as proposed in the above embodiment.

[0198] The storage medium proposed in this embodiment belongs to the same inventive concept as the tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0199] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds, characterized in that, include: Collect stereoscopic images of the power transmission line corridor and generate 3D point cloud data based on the images; The acquisition of stereoscopic images of the power transmission line corridor, and the generation of three-dimensional point cloud data based on the images, includes: Stereoscopic images of the power transmission line corridor were acquired using a binocular camera system; Based on parallax calculation and camera parameters, the stereo image is converted into three-dimensional point cloud data; Based on 3D point cloud data, point cloud segmentation is performed to obtain feature parameters; projection transformation is then performed to obtain a projected image. Based on the projected images, tree targets are detected and classified to obtain labeled tree detection results; Based on feature parameters and labeled tree detection results, a tree obstacle risk assessment model is established to classify the risk of trees.

2. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 1, characterized in that, The feature parameters obtained by point cloud segmentation include: The 3D point cloud data is segmented to extract point cloud clusters for individual trees; The tree height, canopy width, tilt angle, and shortest distance to the power transmission line are calculated based on the point cloud cluster.

3. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 2, characterized in that, The process of performing projection conversion to obtain the projected image includes: The 3D point cloud data is mapped onto the XY plane to generate a 2D image, where the value of each pixel represents the depth information of the corresponding spatial location; the size of the projected image is set to a preset resolution.

4. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 3, characterized in that, The process of detecting and classifying tree targets based on projection images to obtain labeled tree detection results includes: Based on projected images, a labeled dataset containing trees to be risk-classified is constructed by annotating bounding boxes and corresponding tree species labels. This dataset is then input into the DETR network for training, and the trained DETR network outputs tree species labels and location coordinates.

5. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 4, characterized in that, The tree obstacle risk assessment model, based on feature parameters and labeled tree detection results, is used to classify trees according to their risk, including: The risk of trees is divided into three levels: low risk, medium risk, and high risk. A weighted average model is used to calculate the comprehensive risk value of each tree, and the risk is classified according to the magnitude of the comprehensive risk value.

6. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 5, characterized in that, The weighted average model uses the following weighting formula: R = w1·H + w2·D + w3·G + w4·T Where R is the risk value of the tree; H is the height of the tree; D is the shortest distance between the tree and the power transmission line; G is the growth rate of the tree; T is the structural stability score of the tree; and w1, w2, w3, and w4 are the weights set.

7. The tree obstacle risk classification method based on DETR network analysis of binocular visual point clouds as described in claim 6, characterized in that, The process of establishing a tree barrier risk assessment model based on feature parameters and labeled tree detection results, and classifying the risk of trees, also includes: Based on the comprehensive risk value R of each tree, a grading standard is set to divide the trees into different risk levels: low risk: R<5; medium risk: 5≤R<7; high risk: R≥7.

8. A tree obstacle risk classification system based on DETR network analysis of binocular visual point clouds, employing the method described in any one of claims 1 to 7, characterized in that, include: The point cloud generation module is used to acquire stereoscopic images of the power transmission line corridor and generate three-dimensional point cloud data based on the images; The point cloud processing module is used to segment the point cloud based on 3D point cloud data to obtain feature parameters; Perform projection transformation to obtain the projected image; The detection and classification module is used to detect and classify tree targets based on the projected image, and obtain labeled tree detection results; The risk classification module is used to establish a tree barrier risk assessment model based on feature parameters and labeled tree detection results, and to classify the risk of trees.

9. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method described in any one of claims 1 to 7.

Images