Transmission line tree barrier hidden danger automatic identification method based on unmanned aerial vehicle multi-source remote sensing

Abstract

This application provides an automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing. The method includes: acquiring multi-source remote sensing data; performing image feature extraction and feature fusion processing on high-resolution remote sensing images to detect all suspected tree obstruction locations; classifying airborne point cloud data to extract power line results; performing voxel-based classification on the airborne point cloud data within the coverage area of ​​the suspected tree obstruction location to obtain tree classification results; calculating the distances between trees and power lines within the coverage area based on the tree classification results and power line results to perform gradient partitioning to obtain partitioning results; determining the tree obstruction hazard level of the tree obstruction area where the suspected tree obstruction location is located when the partitioning result is a tree obstruction area; and judging all suspected tree obstruction locations to obtain all tree obstruction areas and their corresponding tree obstruction hazard levels. This method solves the problem of low inspection efficiency for tree obstructions on power transmission lines in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of power transmission line tree obstruction detection technology, and more specifically, to an automatic identification method for power transmission line tree obstruction hazards based on UAV multi-source remote sensing, an automatic identification device for power transmission line tree obstruction hazards based on UAV multi-source remote sensing, a computer-readable storage medium, and an automatic identification system for power transmission line tree obstruction hazards. Background Technology

[0002] Transmission lines are a critical infrastructure of modern power systems, bearing the heavy responsibility of transmitting electricity over long distances to end users in cities, industrial areas, and rural areas. With the growth of global electricity demand and the continuous expansion of power grids, the coverage of transmission lines is gradually extending to more complex geographical regions. These lines often traverse diverse terrains such as cities, villages, forests, mountains, and lakes. The complexity of the external natural environment poses severe challenges to the daily operation and maintenance and safety of transmission lines, especially in areas with abundant trees, such as forests and rural areas, where vegetation around transmission lines poses a potential threat to line safety. Tree growth is natural and uncontrollable; over time, trees or branches may come into contact with power lines, especially under extreme weather conditions such as strong winds, heavy rain, or snow. Once branches or trees come into contact with power lines, it can cause accidents such as short circuits, fires, or power outages, leading to power interruptions and, in severe cases, even large-scale grid failures. This not only affects residential electricity use but also causes huge losses to industrial production and social operations. Therefore, ensuring the safe and stable operation of transmission lines is a crucial task for grid operators.

[0003] To prevent power grid safety hazards caused by excessive tree growth, it is usually necessary to regularly inspect and assess tree obstructions around transmission lines to ensure that the distance between the lines and surrounding trees remains within a safe range. Once tree growth is found to threaten line safety, timely measures such as pruning or removing the trees are required to prevent accidents caused by tree obstructions.

[0004] Currently, in practice, the detection of tree obstruction hazards typically relies on manual inspections. Manual inspections require inspectors to walk along power lines or use vehicles for on-site checks, a method that is time-consuming, labor-intensive, and inefficient. Since power lines often traverse areas with complex terrain, manual inspections also pose significant safety risks. For example, inspectors may need to work in steep mountains, dense forests, or high-risk areas, not only consuming substantial time and manpower but also facing potential safety threats in the work environment. Furthermore, manual inspections depend on the inspectors' experience and subjective judgment, making it difficult to guarantee the accuracy and consistency of the results, and increasing the likelihood of missed detections or misjudgments.

[0005] To address these issues, drone technology has been increasingly introduced into power transmission line inspections in recent years. Drones, equipped with high-resolution cameras and LiDAR sensors, fly over power lines to comprehensively monitor the lines and their surrounding environment. Data collected by drones can quickly acquire high-definition images and 3D point cloud data of the area around the lines, thereby improving inspection efficiency and coverage. However, currently, the images and point cloud data from drone inspections typically still require manual analysis, relying on professionals to identify and analyze potential tree obstructions. This approach cannot achieve fully automated detection and still suffers from inefficiency and subjective results. Summary of the Invention

[0006] The main objective of this application is to provide an automatic identification method, device, computer-readable storage medium, and automatic identification system for tree obstructions on power transmission lines based on UAV multi-source remote sensing, so as to at least solve the problem of low inspection efficiency for tree obstructions on power transmission lines in the prior art.

[0007] To achieve the above objectives, according to one aspect of this application, an automatic identification method for tree obstruction hazards along power transmission lines based on UAV multi-source remote sensing is provided, comprising: acquiring multi-source remote sensing data, wherein the multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data, wherein the high-resolution remote sensing images are remote sensing images of the power lines of the power transmission line, and the airborne point cloud data includes at least spatial positioning data of the power lines, trees, and transmission towers; and using the high-resolution remote sensing images to perform image feature extraction and feature fusion processing based on an improved target detection model to detect all suspected tree obstruction locations, wherein the improved target detection model is based on YOLOv8. An improved model is obtained by introducing a channel attention mechanism based on the existing model. The airborne point cloud data is classified using an improved point cloud deep learning model to extract power line results. This improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. This dedicated classifier is specifically designed to separate linear point cloud data. A retrieval and classification step involves retrieving the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location to obtain the target point cloud data corresponding to the suspected tree obstacle location. The improved point cloud object detection model is then used to perform voxelization classification on the target point cloud data to obtain… The tree classification results indicate that the suspected tree barrier location is any one of the suspected tree barrier locations. The point cloud object detection model is an improved model obtained by replacing the ReLU function with the GELU function as the activation function based on the VoxelNet model. The calculation steps involve calculating the distance between trees and power lines within the coverage area based on the tree classification results and the power line results, and performing gradient partitioning based on the distances to obtain partitioning results. The partitioning results are either tree barrier regions or non-tree barrier regions. Tree barrier regions are areas where the distance is less than a first tree barrier safety distance, and non-tree barrier regions are areas where the distance is greater than the first tree barrier safety distance. The safe distance area; the determination step, in the case that the division result is the tree barrier area, analyze according to the corresponding distance and the power of the transmission line to determine the tree barrier hazard level of the tree barrier area where the target suspected tree barrier location is located, the tree barrier hazard level is one of the following: high hazard level, medium hazard level and low hazard level; the retrieval classification step, the calculation step and the determination step are repeated at least once until all the suspected tree barrier locations are judged, so as to obtain all the tree barrier areas and the corresponding tree barrier hazard levels, so as to inspect the corresponding tree barrier areas according to the tree barrier hazard levels.

[0008] Optionally, acquiring multi-source remote sensing data includes: planning the flight trajectory of the UAV based on the direction and location distribution of the power lines to obtain the flight trajectory of the target UAV; controlling the medium-to-high resolution remote sensing equipment on the UAV platform to acquire detailed image data of the power lines based on the flight trajectory of the target UAV; and simultaneously acquiring the airborne point cloud data using the lidar on the UAV platform.

[0009] Optionally, the improved target detection model includes a Backbone module, a Neck module, and a Head module. The Backbone module is used for image feature extraction to generate multi-scale feature maps. The Neck module is used for feature fusion processing of the multi-scale feature maps to generate fused feature maps. The Head module is used for detecting and outputting the fused feature maps. Using the high-resolution remote sensing image, image feature extraction and feature fusion processing are performed based on the improved target detection model to detect all suspected tree obstacle locations. This includes: sequentially performing image resizing, normalization, and image enhancement processing on the high-resolution remote sensing image to obtain a target resolution remote sensing image; inputting the target resolution remote sensing image into the improved target detection model, so that the Backbone module in the improved target detection model extracts image features from the target resolution remote sensing image through multi-layer convolution operations to generate the multi-scale feature maps; using the Neck module to fuse the multi-scale feature maps through a feature pyramid network and a path aggregation network to obtain a fused feature map; and using the Head module to detect and sequentially output all suspected tree obstacle locations from the fused feature map, thus obtaining all suspected tree obstacle locations.

[0010] Optionally, the network structure of the improved point cloud deep learning model includes a transformation network, a feature extraction network, and a global feature aggregation and classification output module. The global feature aggregation and classification output module includes at least a max pooling layer and the output layer. Based on the improved point cloud deep learning model, the airborne point cloud data is classified to extract power line results. This includes: sequentially performing removal and normalization operations on the airborne point cloud data to obtain valid airborne point cloud data; the removal operation involves removing point cloud data unrelated to power lines; inputting the valid airborne point cloud data into the improved point cloud deep learning model, so that the transformation network in the improved point cloud deep learning model aligns the valid airborne point cloud data to a standard transformation matrix through multiple convolutional layers and fully connected layers; the feature extraction network uses a multilayer perceptron to extract local point features from the transformation matrix; the max pooling layer in the global feature aggregation and classification module aggregates the local point features into a global feature vector; and the dedicated classifier in the output layer remaps the global feature vector to generate linear point cloud features for classification output, thus obtaining the power line results.

[0011] Optionally, the network structure of the improved point cloud object detection model includes a voxelization module, a feature extraction module, and a classification and regression module. The voxelization module is used to perform voxelization operations. The feature extraction module includes a voxel feature encoding module and a convolutional intermediate layer. The classification and regression module is used to classify and locate target objects through a region proposal network, retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtain the target point cloud data corresponding to the suspected tree obstacle location, and use the improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain tree classification results, including: using the improved target detection model to retrieve the suspected tree obstacle detection box corresponding to the suspected tree obstacle location; and mapping the suspected tree obstacle detection box to the image coordinate system. The target location is mapped to a three-dimensional spatial coordinate system, and a corresponding three-dimensional retrieval region is set; the airborne point cloud data falling within the three-dimensional retrieval region is filtered out to obtain the target point cloud data; the target point cloud data is input into the improved point cloud object detection model, so that the voxelization module in the improved point cloud object detection model divides the target point cloud data into multiple voxel grids of a set size; the voxel feature encoding module performs local feature extraction on the target point cloud data within the non-empty voxel grids to obtain voxel features; and the convolutional intermediate layer further extracts the spatial features between voxels from all the voxel features; the region proposal network classifies and locates trees within the three-dimensional retrieval region based on the spatial features to obtain the tree classification result.

[0012] Optionally, based on the tree classification results and the power line results, the distance between trees and power lines within the coverage area is calculated, and gradient partitioning is performed based on the distances to obtain partitioning results, including: calculating the distances according to a first formula, wherein the first formula is... d represents the distance between the tree and the power line, v represents the direction vector of the power line, p0 represents the coordinates of any point on the power line, and q represents the coordinates of any tree within the voxelized region; the distance is compared with the first tree barrier safety distance; if the distance is less than or equal to the first tree barrier safety distance, the division result is determined to be the tree barrier region; if the distance is greater than the first tree barrier safety distance, the division result is determined to be the non-tree barrier region.

[0013] Optionally, when the division result is the tree-blocked area, the tree-blocked hazard level of the tree-blocked area where the target suspected tree-blocked location is located is determined by analyzing the corresponding distance in conjunction with the power of the transmission line. This includes: determining the minimum safe distance between the tree and the transmission line based on the power of the transmission line; determining the tree-blocked hazard level of the tree-blocked area corresponding to the target suspected tree-blocked location as the low hazard level when the distance is less than or equal to the first safe distance and greater than the second safe distance, wherein the second safe distance is less than the first safe distance and greater than the minimum safe distance; determining the tree-blocked hazard level of the tree-blocked area corresponding to the target suspected tree-blocked location as the medium hazard level when the distance is less than or equal to the second safe distance and greater than the minimum safe distance; and determining the tree-blocked hazard level of the tree-blocked area corresponding to the target suspected tree-blocked location as the high hazard level when the distance is less than or equal to the minimum safe distance.

[0014] According to another aspect of this application, an automatic identification device for tree obstruction hazards along power transmission lines based on UAV multi-source remote sensing is provided. The device includes: an acquisition unit for acquiring multi-source remote sensing data, the multi-source remote sensing data including at least high-resolution remote sensing images and airborne point cloud data, the high-resolution remote sensing images being remote sensing images of the power lines of the power transmission line, and the airborne point cloud data including at least spatial positioning data of the power lines, trees, and transmission towers; and a feature processing unit for using the high-resolution remote sensing images to perform image feature extraction and feature fusion processing based on an improved target detection model, to detect all suspected tree obstruction locations, the improved target detection model being based on the YOLOv8 model. An improved model is obtained by introducing a channel attention mechanism based on the existing model; an extraction unit is used to classify the airborne point cloud data based on the improved point cloud deep learning model to extract power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is specifically used to separate linear point cloud data; a retrieval and classification unit is used to perform retrieval and classification steps, retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtain the target point cloud data corresponding to the suspected tree obstacle location, and perform voxel classification on the target point cloud data using the improved point cloud object detection model. The tree classification results are obtained, and the suspected tree obstacle location is any one of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function based on the VoxelNet model. A calculation unit is used to perform calculation steps, calculating the distance between trees and power lines within the coverage area based on the tree classification results and the power line results, and performing gradient partitioning based on the distances to obtain partitioning results. The partitioning results are either tree obstacle regions or non-tree obstacle regions. Tree obstacle regions are areas where the distance is less than a first tree obstacle safety distance, and non-tree obstacle regions are areas where the distance is greater than the first tree obstacle safety distance. The system includes: a distance-based area determination unit, which performs a determination step to analyze the distance and power of the transmission line when the division result is the tree-blocked area, and determines the tree-blocked hazard level of the suspected tree-blocked location in the tree-blocked area, wherein the tree-blocked hazard level is one of the following: high hazard level, medium hazard level, and low hazard level; and a repeating unit, which sequentially repeats the retrieval and classification step, the calculation step, and the determination step at least once until all the suspected tree-blocked locations are determined, thereby obtaining all the tree-blocked areas and their corresponding tree-blocked hazard levels, so as to inspect the corresponding tree-blocked areas according to the tree-blocked hazard levels.

[0015] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform any of the methods described.

[0016] According to another aspect of this application, an automatic identification system for tree obstruction hazards on power transmission lines is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for performing any one of the methods described.

[0017] Applying the technical solution of this application, in the automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing, firstly, multi-source remote sensing data is acquired. This multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of the power lines of the power transmission line, and the airborne point cloud data includes at least spatial positioning data of the power lines, trees, and transmission towers. Then, image feature extraction and feature fusion processing are performed using the high-resolution remote sensing images based on an improved target detection model to detect all suspected tree obstruction locations. The improved target detection model is based on the YOLOv8 model. An improved model was obtained by introducing a channel attention mechanism. Then, an improved point cloud deep learning model was used to classify airborne point cloud data and extract power line results. This improved model was obtained by adding a dedicated classifier to the output layer of the PointNet model; the dedicated classifier is specifically designed to separate linear point cloud data. Next, a retrieval and classification step was performed to retrieve airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtaining the target point cloud data corresponding to the suspected tree obstacle location. Finally, an improved point cloud object detection model was used to process the target point cloud data. The process begins with voxelization and classification to obtain tree classification results. The target suspected tree obstacle location is any one of the suspected tree obstacle locations. The point cloud object detection model is an improved model based on the VoxelNet model, using the GELU function instead of the ReLU function as the activation function. Next, a calculation step is performed. Based on the tree classification results and power line results, the distance between trees and power lines within the coverage area is calculated, and gradient partitioning is performed based on the distance to obtain partitioning results. The partitioning results are either tree obstacle areas or non-tree obstacle areas. Tree obstacle areas are areas with a distance less than the first tree obstacle safety distance, and non-tree obstacle areas are areas with a distance greater than the first tree obstacle safety distance. Then, a determination step is performed. If the partitioning result is a tree obstacle area, the corresponding distance and the power of the transmission line are analyzed to determine the tree obstacle hazard level of the tree obstacle area where the target suspected tree obstacle location is located. The tree obstacle hazard level is one of the following: high hazard level, medium hazard level, or low hazard level. Finally, the retrieval and classification steps, calculation steps, and determination steps are repeated at least once until all suspected tree obstacle locations are determined, resulting in all tree obstacle areas and their corresponding tree obstacle hazard levels. These tree obstacle areas are then inspected according to their hazard levels. This application fully leverages the advantages of image information and point cloud data acquired by UAVs and processes them directly. Based on an improved YOLO v8 model, image features are extracted and fused from the acquired high-resolution remote sensing images to detect and acquire the locations of suspected tree obstacles for further analysis of potential risk areas. Then, an improved PointNet model is used to classify the point cloud data, extract power lines, and retrieve the point cloud data within the detected suspected tree obstacle locations.Subsequently, an improved VoxelNet model is used to classify these point cloud data, thereby identifying and voxelizing tree regions to support subsequent accurate analysis. Finally, the distance between trees and power lines is calculated based on the classification results, and the distance calculation results are divided to exclude non-tree-obstructed areas, thus achieving accurate interpretation of the area. Combined with the power of the transmission line, the level of tree obstruction hazard is determined, and the final detection results are output and displayed. Compared with existing methods, the method of this application can obtain tree obstruction extraction results more quickly and accurately. This application solves the problem of low inspection efficiency for tree obstructions on transmission lines in existing technologies. Attached Figure Description

[0018] Figure 1 A hardware structure block diagram of a mobile terminal for performing an automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing is shown in an embodiment of this application.

[0019] Figure 2 A flowchart illustrating an automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing, according to an embodiment of this application, is shown.

[0020] Figure 3 A flowchart illustrating an automatic identification method for tree obstruction hazards on power transmission lines according to an embodiment of this application is shown.

[0021] Figure 4 A framework architecture diagram of an improved point cloud deep learning model provided according to embodiments of this application is shown;

[0022] Figure 5 A structural block diagram of an automatic identification device for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing, according to an embodiment of this application, is shown.

[0023] The above figures include the following reference numerals:

[0024] 102. Processor; 104. Memory; 106. Transmission device; 108. Input / output device. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0029] Tree obstruction of power transmission lines: As trees grow, the distance between them and the power lines decreases, causing distortion of the electric field on the surface of the power lines. The electric field strength between the power lines and the trees increases, eventually leading to flashover of the power transmission lines and causing power outages.

[0030] The backbone, also known as the core network, is the most fundamental part of a deep learning model, primarily tasked with extracting features from the input data. In image processing tasks, the backbone is typically a convolutional neural network (CNN) used to extract useful information from the raw image, such as edges, textures, and shapes. The performance of the backbone directly impacts the overall model performance; therefore, choosing a suitable backbone is crucial. Common backbone networks include ResNet, VGG, and MobileNet. These network architectures have undergone extensive training and optimization, proving their excellent performance across a variety of tasks.

[0031] The Neck module, located between the Backbone and Head, primarily integrates and processes the features extracted from the Backbone. In tasks such as object detection and face recognition, the Neck module typically includes specific layers, such as ROI Pooling and FPN (Feature Pyramid Network). These layers effectively integrate features from different levels, improving model performance. Furthermore, the Neck module can introduce attention mechanisms, allowing the model to focus more on important features.

[0032] The Head module is the last layer of the deep learning model, responsible for mapping the features processed by the Backbone and Neck to the final output space. The structure of the Head module varies depending on the task. For example, in image classification tasks, the Head module is typically a fully connected layer or a softmax classifier; in object detection tasks, the Head module may include bounding box regressors and classifiers. As described in the background section, current UAV inspection images and point cloud data often require manual analysis, relying on professionals for tree obstruction hazard identification and analysis. This approach cannot achieve fully automated detection and still suffers from low efficiency and subjective results. To address the low efficiency of power transmission line tree obstruction inspection, embodiments of this application provide an automatic identification method for power transmission line tree obstruction hazards based on UAV multi-source remote sensing, an automatic identification device for power transmission line tree obstruction hazards based on UAV multi-source remote sensing, a computer-readable storage medium, and an automatic identification system for power transmission line tree obstruction hazards.

[0033] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0034] The methods and embodiments provided in this application can be executed on a mobile terminal, computer terminal, or similar computing device. Taking running on a mobile terminal as an example, Figure 1 This is a hardware structure block diagram of a mobile terminal for an automatic identification method of tree obstruction hazards on power transmission lines based on multi-source remote sensing from unmanned aerial vehicles (UAVs) according to an embodiment of the present invention. Figure 1 As shown, a mobile terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the mobile terminal described above. For example, the mobile terminal may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0035] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the automatic identification method for power transmission line tree obstruction hazards based on UAV multi-source remote sensing in this embodiment of the invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the aforementioned networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] This embodiment provides an automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing, which runs on a mobile terminal, computer terminal, or similar computing device. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Also, although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than that shown here.

[0037] Figure 2 This is a flowchart of a method for automatic identification of tree obstruction hazards on power transmission lines based on multi-source remote sensing from unmanned aerial vehicles, according to an embodiment of this application. Figure 2 As shown, the method includes the following steps:

[0038] Step S201: Acquire multi-source remote sensing data. The multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines. The airborne point cloud data includes at least the spatial positioning data of the power lines, trees and transmission towers.

[0039] Specifically, when planning drone flight paths, it is necessary to comprehensively consider the direction and location of power lines to ensure the safety and efficiency of the flight path. Multi-source remote sensing refers to a technology that comprehensively analyzes and processes remote sensing data from different sensors, at different times, or with different spatial resolutions. In the context of drone platforms, this includes high-resolution imagery and LiDAR point cloud data. Drone flight path planning is based on the direction and location of power lines, employing a multi-sensor drone platform to acquire multi-source remote sensing data, including high-resolution remote sensing imagery and airborne LiDAR point cloud data.

[0040] Step S202: Using the above high-resolution remote sensing image, image feature extraction and feature fusion processing are performed based on the improved target detection model to detect all suspected tree obstacle locations. The above improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model.

[0041] Specifically, using the aforementioned high-resolution remote sensing imagery, image feature extraction and fusion were performed based on the improved YOLO v8 model to detect and acquire suspected tree barrier locations. During the feature fusion process in the YOLO v8 model, the SENet channel attention mechanism was introduced. By adaptively adjusting the weights of each channel, the expressive power of tree barrier features was enhanced, the impact of low-level vegetation on detection was reduced, and thus the model's detection performance was improved.

[0042] Step S203: Classify the airborne point cloud data based on the improved point cloud deep learning model and extract the power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data.

[0043] Specifically, based on the original PointNet model, a dedicated classifier, LineSegNet (Line Segment Network), is added to the output layer. This classifier is specifically designed for separating linear point cloud structures and is called the PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above). This model further optimizes the network's ability to separate linear point clouds, enabling it to perform fine-grained classification of power lines in the point cloud by simultaneously combining geometric features and contextual information. Furthermore, a multi-task learning framework is employed, using a weighted loss function to enhance the classifier's focus on power lines, ensuring the network's accuracy in identifying power lines. With this improvement, when the aforementioned airborne point cloud data is input into the improved point cloud deep learning model for classification and power line extraction, the power line features can be extracted more accurately, effectively improving the high-precision classification of power line point clouds.

[0044] Step S204, retrieval and classification step: retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location to obtain the target point cloud data corresponding to the suspected tree obstacle location, and use the improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain the tree classification result. The suspected tree obstacle location is any of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function based on the VoxelNet model.

[0045] Specifically, using the suspected tree obstacle detection boxes obtained from the YOLO v8 network, the positions of the detection boxes in the image coordinate system are mapped to a three-dimensional spatial coordinate system, defining a corresponding three-dimensional retrieval region. This allows for the selection of point clouds falling within this region from the original point cloud data, i.e., the aforementioned target point cloud data. In the VoxelNet model, the GELU activation function is used instead of the traditional ReLU function to increase the feature representation capability of the network structure, enabling it to obtain more precise tree voxels based on tree structural features. This model is called the VoxelNet-TR model (i.e., the aforementioned improved point cloud object detection model). The aforementioned target point cloud data is input into the VoxelNet-TR model for voxelization classification to obtain the tree classification results.

[0046] It should be noted that this invention does not perform calculations on all point cloud data, reducing computational load and improving model efficiency. The PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above) is used for power line extraction, and the VoxelNet-TR model (i.e., the improved point cloud object detection model mentioned above) performs voxelization classification on tree obstacle point clouds, achieving accurate separation and identification of tree obstacles and power lines. This avoids the computation of all point cloud data, reduces the model's dependence on computer performance, and improves computational efficiency.

[0047] Step S205, calculation step: Based on the above tree classification results and the above power line results, calculate the distance between the trees and power lines within the above coverage area, and perform gradient division based on the above distance to obtain the division result. The division result is a tree barrier area or a non-tree barrier area. The tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance.

[0048] Specifically, using the power lines and voxelized trees obtained from the classification, the distance between the power lines and the tree voxel grids is calculated, and the calculation results are divided to obtain the division results, so as to exclude non-tree barrier areas.

[0049] Step S206, Determination Step: In the case that the above division result is the above-mentioned tree barrier area, the tree barrier hazard level of the above-mentioned tree barrier area where the above-mentioned suspected tree barrier location is located is determined by analyzing the corresponding distance and the power of the above-mentioned transmission line. The tree barrier hazard level is one of the following: high hazard level, medium hazard level and low hazard level.

[0050] Specifically, based on the identification of the tree-blocked area, and in conjunction with the power rating of the transmission line, the potential hazards of the tree-blocked area and its impact on the operation of the transmission line are further assessed. The power rating of the transmission line determines its ability to transmit electrical energy and the magnitude of the current; higher-power lines typically have stricter requirements for safe distances from tree-blocked areas. For tree-blocked areas located within the line's protection zone, the hazard level is determined according to the distance outside the boundary line specified in the "Operation Regulations for Overhead Transmission Lines".

[0051] Step S207: Repeat the above-mentioned retrieval and classification steps, calculation steps, and determination steps at least once until all the above-mentioned suspected tree obstacle locations are determined, so as to obtain all the above-mentioned tree obstacle areas and the corresponding tree obstacle hazard levels, and to inspect the corresponding tree obstacle areas according to the above-mentioned tree obstacle hazard levels.

[0052] Specifically, the above-mentioned retrieval and classification steps, calculation steps, and determination steps are executed iteratively until all suspected tree obstacle locations are identified, all non-tree obstacle areas are gradually eliminated, and the actual tree obstacle areas and their corresponding hazard levels are displayed in a graded manner so that the corresponding tree obstacle areas can be inspected according to the tree obstacle hazard levels, ensuring the accuracy and comprehensiveness of the inspection results.

[0053] In this embodiment, firstly, multi-source remote sensing data is acquired. This multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines, and the airborne point cloud data includes at least spatial positioning data of power lines, trees, and transmission towers. Then, image feature extraction and feature fusion are performed using the high-resolution remote sensing images based on an improved target detection model to detect all suspected tree obstruction locations. The improved target detection model is an improved model obtained by introducing a channel attention mechanism on top of the YOLOv8 model. Next, the airborne point cloud data is classified based on an improved point cloud deep learning model to extract power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. This dedicated classifier is specifically used to separate linear point cloud data. Finally, a retrieval and classification step is performed to retrieve airborne point cloud data within the coverage area of ​​the suspected tree obstruction locations, obtaining the target point cloud data corresponding to the suspected tree obstruction locations. The improved point cloud object detection model is then used to perform voxel-based classification of the target point cloud data to obtain the tree classification results. The target suspected tree obstacle location is any suspected tree obstacle location. The point cloud object detection model is an improved model obtained by replacing the ReLU function with the GELU function as the activation function on the basis of the VoxelNet model. Then, the calculation step is performed. Based on the tree classification results and power line results, the distance between trees and power lines within the coverage area is calculated, and gradient division is performed based on the distance to obtain the division result. The division result is a tree obstacle area or a non-tree obstacle area. The tree obstacle area is the area where the distance is less than the first tree obstacle safety distance, and the non-tree obstacle area is the area where the distance is greater than the first tree obstacle safety distance. Then, the determination step is performed. If the division result is a tree obstacle area, the tree obstacle hazard level of the tree obstacle area where the target suspected tree obstacle location is located is determined by analyzing the corresponding distance and the power of the transmission line. The tree obstacle hazard level is one of the following: high hazard level, medium hazard level, and low hazard level. Finally, the retrieval classification step, calculation step, and determination step are repeated at least once until all suspected tree obstacle locations are judged, and all tree obstacle areas and their corresponding tree obstacle hazard levels are obtained, so that the corresponding tree obstacle areas can be inspected according to the tree obstacle hazard level. This application fully leverages the advantages of image information and point cloud data acquired by UAVs and processes them directly. Based on an improved YOLO v8 model, image features are extracted and fused from the acquired high-resolution remote sensing images to detect and acquire the locations of suspected tree obstacles for further analysis of potential risk areas. Then, an improved PointNet model is used to classify the point cloud data, extract power lines, and retrieve the point cloud data within the detected suspected tree obstacle locations.Subsequently, an improved VoxelNet model is used to classify these point cloud data, thereby identifying and voxelizing tree regions to support subsequent accurate analysis. Finally, the distance between trees and power lines is calculated based on the classification results, and the distance calculation results are divided to exclude non-tree-obstructed areas, thus achieving accurate interpretation of the area. Combined with the power of the transmission line, the level of tree obstruction hazard is determined, and the final detection results are output and displayed. Compared with existing methods, the method of this application can obtain tree obstruction extraction results more quickly and accurately. This application solves the problem of low inspection efficiency for tree obstructions on transmission lines in existing technologies.

[0054] To enable those skilled in the art to better understand the technical solution of this application, the implementation process of the automatic identification method for tree obstruction hazards of power transmission lines based on UAV multi-source remote sensing will be described in detail below with reference to specific embodiments.

[0055] To improve the accuracy and comprehensiveness of tree obstacle detection, in one optional implementation, step S201 includes:

[0056] Step S2011: Plan the flight trajectory of the UAV based on the direction and location distribution of the power lines to obtain the flight trajectory of the target UAV;

[0057] In step S2012, the medium-to-high resolution remote sensing equipment mounted on the UAV platform is controlled to acquire detailed image data of the power line according to the flight trajectory of the target UAV. At the same time, the airborne point cloud data is acquired using the lidar mounted on the UAV platform.

[0058] In the above embodiments, the flight trajectory of the UAV is manually planned according to the direction and location distribution of the power transmission line to ensure that the flight path can fully cover the key areas of the power transmission line and its surrounding environment. By setting appropriate UAV flight altitude, speed, and flight path, the high-resolution remote sensing equipment on the UAV platform acquires detailed image data of the power line, accurately capturing line details and changes in the surrounding environment. Multispectral sensor equipment acquires high-resolution image data to capture details of the power transmission line and changes in its surrounding environment. Simultaneously, the lidar system on the UAV platform acquires three-dimensional LiDAR point cloud data to achieve spatial positioning of different targets (such as power lines, trees, transmission towers, etc.) in the power transmission line, effectively covering power transmission lines in different environments and improving the accuracy and comprehensiveness of detection. In this example, high-resolution remote sensing images are acquired through ground observation using optical sensors; airborne lidar point cloud data is acquired through ground observation using airborne lidar sensors.

[0059] To improve the expressive power of tree barrier features in order to accurately obtain the location of suspected tree barriers, in an optional implementation, step S202 above includes:

[0060] Step S2021: The above high-resolution remote sensing image is subjected to image resizing, normalization and image enhancement processing in sequence to obtain the target resolution remote sensing image.

[0061] Step S2022: Input the target resolution remote sensing image into the improved target detection model, and use the Backbone module in the improved target detection model to extract the image features of the target resolution remote sensing image through multi-layer convolution operations to generate the multi-scale feature map. Use the Neck module to fuse the multi-scale feature map through a feature pyramid network and a path aggregation network to obtain a fused feature map. Use the Head module to detect the fused feature map and output all the suspected tree obstacle locations in sequence to obtain all the suspected tree obstacle locations.

[0062] In the above embodiments, data preprocessing is first performed, including normalization and data augmentation of the high-resolution remote sensing images to improve the robustness and generalization ability of the model, such as... Figure 3The upper part shows the input of the target high-resolution remote sensing image into the improved YOLO v8 model. The YOLO v8 model structure mainly consists of three parts: Backbone, Neck, and Head. Each part plays a key role in feature extraction, fusion, and target detection. The Backbone module is used for image feature extraction to generate multi-scale feature maps. The Neck module is used for feature fusion processing of the multi-scale feature maps to generate fused feature maps. The Head module is used to detect the fused feature maps and output them. The preprocessed image (the target resolution remote sensing image) is input into the Backbone module for feature extraction. The core of this stage is to extract low-level to high-level feature maps of the image through multi-layer convolutional operations. The Backbone uses a deep neural network to ensure that key visual information can be fully expressed. The Backbone module mainly extracts individual-scale features from the images generated by the C2f and SPPF modules. The C2f module replaces the original C3 module, reducing one convolutional layer and making the model more lightweight. SPPF reduces the number of network layers based on the original SPP, which can perform feature fusion more quickly and eliminate redundant operations. Next, the extracted multi-scale feature maps are passed to the Neck module. The Neck module effectively fuses feature maps of different resolutions and semantic levels through a combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN). FPN enhances the semantic expressiveness of higher-level feature maps through top-down feature fusion, ensuring the model can understand complex objects in the image. Meanwhile, PAN preserves the detailed information of lower-level feature maps through bottom-up path enhancement, making the model more accurate in detecting targets of varying sizes. The introduction of the SENet channel attention mechanism adaptively adjusts the weights of each channel, extracting global information for each channel using global average pooling, and then generating weights through a two-layer fully connected network. After weight allocation between channels, the network can focus on more meaningful channel features and suppress irrelevant or redundant information. This mechanism enhances the model's ability to capture important features and improves the accuracy of object detection, especially when dealing with complex scenes and multi-scale targets. After feature fusion, the output of the Neck module is passed to the Head module. The Head module is responsible for converting the fused feature map into specific detection results. Through a series of convolutional operations and a classifier, the model can output the specific location and category information of the tree barriers. In the feature fusion process of the YOLO v8 model, the SENet channel attention mechanism is introduced. By adaptively adjusting the weights of each channel, the expressive power of the tree barrier features is improved, the influence of low-level vegetation on detection is reduced, and thus the model's detection performance is improved.

[0063] To extract power line features more accurately and effectively improve the high-precision classification of power line point clouds, in an optional implementation, step S203 includes:

[0064] Step S2031: The airborne point cloud data is sequentially removed and normalized to obtain valid airborne point cloud data. The removal operation is to remove point cloud data that is not related to power lines.

[0065] Step S2032: The above-mentioned effective airborne point cloud data is input into the above-mentioned improved point cloud deep learning model, so that the above-mentioned transformation network in the above-mentioned improved point cloud deep learning model aligns the above-mentioned effective airborne point cloud data to the standard transformation matrix through multiple convolutional layers and fully connected layers. The above-mentioned feature extraction network uses a multilayer perceptron to extract local point features from the above-mentioned transformation matrix. The max pooling layer in the above-mentioned global feature aggregation and classification module aggregates the above-mentioned local point features into a global feature vector. The above-mentioned dedicated classifier in the above-mentioned output layer remaps the above-mentioned global feature vector to generate linear point cloud features for classification output, thereby obtaining the above-mentioned power line result.

[0066] In the above embodiments, the network structure of the improved point cloud deep learning model includes a transformation network, a feature extraction network, and a global feature aggregation and classification output module. The global feature aggregation and classification output module includes at least a max pooling layer and the aforementioned output layer. Irrelevant point cloud data, such as ground, building, and other noisy data, is removed from the acquired airborne point cloud data to reduce the impact of errors on classification. The point cloud data is then normalized so that the spatial location of all points is standardized to a uniform coordinate range, resulting in the aforementioned effective airborne point cloud data. Figure 4 As shown, firstly, the point cloud data (i.e., the effective airborne point cloud data) is input into the transformation network, which uses a T-Net structure. Through multiple 3×3 convolutional layers and fully connected layers, it learns a transformation matrix to align the input effective airborne point cloud data to a standard pose. The feature extraction network uses a pointwise MLP (Multilayer Perceptron) to extract point features, where each MLP layer is equivalent to a 1×1 convolutional layer, extracting local features of the points with ReLU activation. During this process, no spatial pooling is performed to preserve the spatial information of the points. The feature vector output by the feature extraction network is aggregated into a global feature vector through max pooling. This operation integrates the local information of the point cloud into a fixed-length global descriptor, exhibiting spatial transformation invariance. Finally, through several fully connected layers, the network maps the global features to the output space of a specific task (such as classification or segmentation), ultimately calculating class probabilities through a Softmax layer or achieving point cloud segmentation through the pointwise MLP.

[0067] In this embodiment, as Figure 4As shown, the PointNet model is improved as follows: Based on the original PointNet model, a dedicated classifier, LineSegNet (Line Segment Network), is added to the output layer specifically for separating linear point cloud structures. Furthermore, a multi-task learning framework is adopted, using a weighted loss function to enhance the classifier's focus on power lines, ensuring the network's accuracy in identifying power lines. This improved model is called the PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above). Based on the PointNet-PL model, power lines are extracted by classifying point cloud data. The LineSegNet classifier is added to the feature aggregation module of the original PointNet. This classifier uses a dedicated MLP layer to remap the aggregated global features, generating specific features for linear point clouds. In this process, the structure of the MLP layer remains consistent with the original PointNet, but an additional fully connected layer with specific weight initialization is added before the output layer to strengthen the recognition of linear structures. At the classifier's output layer, a weighted loss function is introduced, specifically assigning higher weights to the power line category, thereby enhancing the classifier's focus on power lines. With this improvement, the PointNet-PL model can extract electric field line features more accurately, effectively improving the high-precision classification of electric field line point clouds.

[0068] To ensure more accurate tree information detection in complex scenarios, in one optional implementation, step S204 includes:

[0069] Step S2041: Use the improved target detection model described above to retrieve the suspected tree obstacle detection box corresponding to the suspected tree obstacle location of the target.

[0070] Step S2042: Map the coordinate position of the suspected tree obstacle detection box in the image coordinate system to the three-dimensional spatial coordinate system, and set the corresponding three-dimensional search area;

[0071] Step S2043: Filter out the airborne point cloud data that falls within the above-mentioned three-dimensional search area to obtain the above-mentioned target point cloud data;

[0072] Step S2044: The target point cloud data is input into the improved point cloud object detection model, so that the voxelization module in the improved point cloud object detection model divides the target point cloud data into multiple voxel grids of a set size. The voxel feature encoding module extracts local features of the target point cloud data in the non-empty voxel grids to obtain voxel features. The convolutional intermediate layer further extracts spatial features between voxels from all the voxel features. The region proposal network classifies and locates trees in the three-dimensional retrieval area based on the spatial features to obtain the tree classification result.

[0073] In the above embodiments, the network structure of the improved point cloud object detection model includes a voxelization module, a feature extraction module, and a classification and regression module. The voxelization module is used to perform voxelization operations. The feature extraction module includes a voxel feature encoding module and a convolutional intermediate layer. The classification and regression module is used to classify and locate target objects through a region proposal network. First, the suspected tree obstacle detection boxes obtained by the improved YOLO v8 model are mapped from the image coordinate system to the three-dimensional spatial coordinate system, defining a corresponding three-dimensional retrieval region. This allows the point cloud data falling within this region (i.e., the target point cloud data) to be filtered from the original point cloud data. The target point cloud data is then divided into a fixed-size three-dimensional voxel grid using the improved VoxelNet model. The point cloud within each non-empty voxel is processed by the VFE module to extract local features. The VFE module uses a multilayer perceptron (MLP) with shared parameters to perform feature mapping on the coordinates of each point and cancels the traditional ReLU function, applying the GELU activation function. This process can be described as: f ij =GELU(W1p ij +b1), where p ij Voxel v j The coordinates of the i-th point in the matrix are given, and W1 and b1 are the weight matrix and bias term of the MLP, respectively. After convolution, the network downsamples through a max-pooling layer, gradually reducing the spatial size of the feature map. The second and third layers use 128 and 256 convolutional kernels for feature extraction, respectively, ultimately reducing the spatial size of the feature map to 1 / 8 of the original voxel grid, but increasing the number of channels to 256. On the feature map extracted by 3D convolution, a Region Proposal Network (RPN) is used for object classification and localization. The RPN contains a 3D convolutional layer, which is then divided into a classification branch and a regression branch. The former calculates the class probability using Softmax, while the latter outputs the bounding box regression parameters. The entire network structure utilizes the GELU activation function to enhance non-linear expressive power, enabling the improved VoxelNet model (also known as the VoxelNet-TR model) to detect tree information more accurately in complex scenes.

[0074] In order to accurately exclude non-tree-barrier areas, in an optional implementation, step S205 above includes:

[0075] Step S2051: Calculate the distance according to the first formula, whereby the first formula is... d represents the distance between the tree and the power line, v represents the direction vector of the power line, p0 represents the coordinates of any point on the power line, and q represents the coordinates of any tree within the voxelized region.

[0076] Step S2052: Compare the above distance with the above-mentioned first tree barrier safe distance;

[0077] Step S2053: If the distance is less than or equal to the first tree barrier safety distance, determine the division result as the tree barrier area.

[0078] Step S2054: If the distance is greater than the first tree barrier safety distance, the above division result is determined to be the above non-tree barrier area.

[0079] In the above embodiment, using the classified power lines and voxelized trees, the distance d between the power lines and the tree voxel mesh is calculated, and the calculation results are divided to exclude the non-tree obstacle areas obtained in the first step of fast detection. Specifically, assuming the power line vector v represents the direction vector of the power line, the distance d is calculated as follows: Based on the calculation results, the distance d between the power line and the tree voxel grid is divided into gradients for the elimination of suspected tree obstacle hazards. That is, the above distance is compared with the first tree obstacle safety distance. If the above distance is less than or equal to the first tree obstacle safety distance, the above division result is the tree obstacle area; if the above distance is greater than the first tree obstacle safety distance, the above division result is the non-tree obstacle area, and the non-tree obstacle area can be directly excluded and will not be used for subsequent tree obstacle hazard level judgment.

[0080] To improve the efficiency and accuracy of tree obstacle assessment, in one optional implementation, step S206 includes:

[0081] Step S2061: Determine the minimum safe distance between the trees and the transmission line based on the power of the transmission line.

[0082] Step S2062: When the distance is less than or equal to the first tree barrier safety distance and greater than the second tree barrier safety distance, the tree barrier hazard level of the tree barrier area corresponding to the suspected tree barrier location is determined to be the low hazard level, and the second tree barrier safety distance is less than the first tree barrier safety distance and greater than the minimum tree barrier safety distance.

[0083] Step S2063: If the distance is less than or equal to the second tree barrier safety distance and greater than the minimum tree barrier safety distance, determine the tree barrier hazard level of the tree barrier area corresponding to the suspected tree barrier location as the medium hazard level.

[0084] Step S2064: If the distance is less than or equal to the minimum safe distance for tree barriers, determine the tree barrier hazard level of the tree barrier area corresponding to the suspected tree barrier location as the high hazard level.

[0085] In the above embodiments, the power of the transmission line determines its ability to transmit electrical energy and the magnitude of the current. Higher-power lines typically have stricter requirements for the safe distance from tree obstructions. If a detected tree obstruction is within the transmission line's protection zone and the line has a high power rating, the tree obstruction is classified as a high-risk hazard; otherwise, it is classified as a low-risk or medium-risk hazard. Based on the assessment results, tree obstruction hazards are divided into three different levels: low risk, medium risk, and high risk. These levels are determined comprehensively based on factors such as the distance between the tree obstruction and the line, and the line's power rating. The minimum safe distance from tree obstructions corresponding to the power of the transmission line can be determined in conjunction with the "Operating Regulations for Overhead Transmission Lines." Different distances between trees and transmission lines of different power ratings are interpreted to determine the tree obstruction hazard level, and the detection results are output and displayed. By calculating the distance between trees and power lines and automatically assessing the hazard level in conjunction with the power of the transmission line, the assessment efficiency and accuracy are greatly improved, the uncertainty caused by human judgment is reduced, and the intelligence level of power line inspection is enhanced.

[0086] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0087] This application also provides an automatic identification device for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing. It should be noted that this device can be used to execute the automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing provided in this application. This device is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0088] The following describes the automatic identification device for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing provided in the embodiments of this application.

[0089] Figure 5 This is a structural block diagram of an automatic identification device for tree obstruction hazards on power transmission lines based on multi-source remote sensing from unmanned aerial vehicles, according to an embodiment of this application. Figure 5 The device includes:

[0090] The acquisition unit 10 is used to acquire multi-source remote sensing data, which includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of the power lines of the power transmission line, and the airborne point cloud data includes at least the spatial positioning data of the power lines, trees and power transmission towers.

[0091] Specifically, when planning drone flight paths, it is necessary to comprehensively consider the direction and location of power lines to ensure the safety and efficiency of the flight path. Multi-source remote sensing refers to a technology that comprehensively analyzes and processes remote sensing data from different sensors, at different times, or with different spatial resolutions. In the context of drone platforms, this includes high-resolution imagery and LiDAR point cloud data. Drone flight path planning is based on the direction and location of power lines, employing a multi-sensor drone platform to acquire multi-source remote sensing data, including high-resolution remote sensing imagery and airborne LiDAR point cloud data.

[0092] The feature processing unit 20 is used to perform image feature extraction and feature fusion processing based on the above-mentioned high-resolution remote sensing image and the improved target detection model to detect all suspected tree obstacle locations. The above-mentioned improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model.

[0093] Specifically, using the aforementioned high-resolution remote sensing imagery, image feature extraction and fusion were performed based on the improved YOLO v8 model to detect and acquire suspected tree barrier locations. During the feature fusion process in the YOLO v8 model, the SENet channel attention mechanism was introduced. By adaptively adjusting the weights of each channel, the expressive power of tree barrier features was enhanced, the impact of low-level vegetation on detection was reduced, and thus the model's detection performance was improved.

[0094] Extraction unit 30 is used to classify the above airborne point cloud data based on the improved point cloud deep learning model and extract the power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data.

[0095] Specifically, based on the original PointNet model, a dedicated classifier, LineSegNet (Line Segment Network), is added to the output layer. This classifier is specifically designed for separating linear point cloud structures and is called the PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above). This model further optimizes the network's ability to separate linear point clouds, enabling it to perform fine-grained classification of power lines in the point cloud by simultaneously combining geometric features and contextual information. Furthermore, a multi-task learning framework is employed, using a weighted loss function to enhance the classifier's focus on power lines, ensuring the network's accuracy in identifying power lines. With this improvement, when the aforementioned airborne point cloud data is input into the improved point cloud deep learning model for classification and power line extraction, the power line features can be extracted more accurately, effectively improving the high-precision classification of power line point clouds.

[0096] The retrieval and classification unit 40 is used to perform the retrieval and classification steps, retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtain the target point cloud data corresponding to the suspected tree obstacle location, and use an improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain the tree classification result. The suspected tree obstacle location is any of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function based on the VoxelNet model.

[0097] Specifically, using the suspected tree obstacle detection boxes obtained from the YOLO v8 network, the positions of the detection boxes in the image coordinate system are mapped to a three-dimensional spatial coordinate system, defining a corresponding three-dimensional retrieval region. This allows for the selection of point clouds falling within this region from the original point cloud data, i.e., the aforementioned target point cloud data. In the VoxelNet model, the GELU activation function is used instead of the traditional ReLU function to increase the feature representation capability of the network structure, enabling it to obtain more precise tree voxels based on tree structural features. This model is called the VoxelNet-TR model (i.e., the aforementioned improved point cloud object detection model). The aforementioned target point cloud data is input into the VoxelNet-TR model for voxelization classification to obtain the tree classification results.

[0098] It should be noted that this invention does not perform calculations on all point cloud data, reducing computational load and improving model efficiency. The PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above) is used for power line extraction, and the VoxelNet-TR model (i.e., the improved point cloud object detection model mentioned above) performs voxelization classification on tree obstacle point clouds, achieving accurate separation and identification of tree obstacles and power lines. This avoids the computation of all point cloud data, reduces the model's dependence on computer performance, and improves computational efficiency.

[0099] The calculation unit 50 is used to perform calculation steps, calculate the distance between trees and power lines within the coverage area based on the tree classification results and the power line results, and perform gradient division based on the distance to obtain the division result, which is a tree barrier area or a non-tree barrier area. The tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance.

[0100] Specifically, using the power lines and voxelized trees obtained from the classification, the distance between the power lines and the tree voxel grids is calculated, and the calculation results are divided to obtain the division results, so as to exclude non-tree barrier areas.

[0101] The determination unit 60 is used to perform the determination step. When the above division result is the above-mentioned tree barrier area, it analyzes the corresponding distance and the power of the above-mentioned transmission line to determine the tree barrier hazard level of the above-mentioned tree barrier area where the above-mentioned suspected tree barrier location is located. The tree barrier hazard level is one of the following: high hazard level, medium hazard level, and low hazard level.

[0102] Specifically, based on the identification of the tree-blocked area, and in conjunction with the power rating of the transmission line, the potential hazards of the tree-blocked area and its impact on the operation of the transmission line are further assessed. The power rating of the transmission line determines its ability to transmit electrical energy and the magnitude of the current; higher-power lines typically have stricter requirements for safe distances from tree-blocked areas. For tree-blocked areas located within the line's protection zone, the hazard level is determined according to the distance outside the boundary line specified in the "Operation Regulations for Overhead Transmission Lines".

[0103] The repeating unit 70 is used to execute the above-mentioned retrieval and classification steps, calculation steps and determination steps at least once in sequence until all the above-mentioned suspected tree obstacle locations are judged, so as to obtain all the above-mentioned tree obstacle areas and the corresponding tree obstacle hazard levels, so as to inspect the corresponding tree obstacle areas according to the above-mentioned tree obstacle hazard levels.

[0104] Specifically, the above-mentioned retrieval and classification steps, calculation steps, and determination steps are executed iteratively until all suspected tree obstacle locations are identified, all non-tree obstacle areas are gradually eliminated, and the actual tree obstacle areas and their corresponding hazard levels are displayed in a graded manner so that the corresponding tree obstacle areas can be inspected according to the tree obstacle hazard levels, ensuring the accuracy and comprehensiveness of the inspection results.

[0105] In this embodiment, the acquisition unit is used to acquire multi-source remote sensing data, which includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines, and the airborne point cloud data includes at least spatial positioning data of the power lines, trees, and transmission towers. The feature processing unit is used to perform image feature extraction and feature fusion processing based on the high-resolution remote sensing images and an improved target detection model to detect all suspected tree obstacle locations. The improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model. The extraction unit is used to extract features based on the improved model. An improved point cloud deep learning model is used to classify the aforementioned airborne point cloud data and extract power lines. This improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. This dedicated classifier is specifically designed to separate linear point cloud data. A retrieval classification unit is used to perform the retrieval classification step, retrieving the aforementioned airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtaining the target point cloud data corresponding to the suspected tree obstacle location, and using an improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain tree classification results. The suspected tree obstacle location is an arbitrary... The above-mentioned suspected tree barrier location is defined as follows: the point cloud object detection model is an improved model obtained by replacing the ReLU function with the GELU function as the activation function based on the VoxelNet model; a calculation unit is used to perform calculation steps, calculate the distance between trees and power lines within the coverage area based on the tree classification results and the power line results, and perform gradient partitioning based on the distances to obtain partitioning results, which are tree barrier areas or non-tree barrier areas. The tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance; a determination unit is used. The determination unit is used to perform the following steps: When the above-described classification result indicates a tree-blocked area, the unit analyzes the distance and power of the transmission line to determine the tree-blocked hazard level of the suspected tree-blocked location within that area. The tree-blocked hazard level is one of the following: high hazard level, medium hazard level, or low hazard level. The repeating unit is used to sequentially repeat the above-described retrieval and classification steps, the above-described calculation steps, and the above-described determination steps at least once until all suspected tree-blocked locations are identified, obtaining all tree-blocked areas and their corresponding tree-blocked hazard levels. This allows for the inspection of the corresponding tree-blocked areas based on the tree-blocked hazard levels. This application fully leverages the advantages of image information and point cloud data collected by UAVs and processes them directly.Based on an improved YOLO v8 model, image features are extracted and fused from acquired high-resolution remote sensing images to detect and identify suspected tree obstruction locations for further analysis of potential risk areas. Next, an improved PointNet model is used to classify point cloud data, extract transmission lines, and retrieve point cloud data within the detected suspected tree obstruction locations. Subsequently, an improved VoxelNet model is used to classify these point cloud data, thereby identifying and voxelizing tree areas to support subsequent accurate analysis. Finally, the distance between trees and power lines is calculated based on the classification results, and the distance calculation results are divided to exclude non-tree obstruction areas, thus achieving accurate interpretation of the area. The tree obstruction hazard level is determined by combining the power line power, and the final detection results are output and displayed. Compared with existing methods, the method in this application can obtain tree obstruction extraction results more quickly and accurately. This application solves the problem of low inspection efficiency for tree obstructions on transmission lines in existing technologies.

[0106] To improve the accuracy and comprehensiveness of tree obstacle detection, in one optional embodiment, the acquisition unit includes:

[0107] The planning module plans the flight trajectory of the UAV based on the direction and location distribution of the aforementioned power lines, thus obtaining the flight trajectory of the target UAV.

[0108] The acquisition module controls the medium-to-high resolution remote sensing equipment on the UAV platform to acquire detailed image data of the power lines based on the flight trajectory of the target UAV. At the same time, it uses the lidar on the UAV platform to acquire the airborne point cloud data.

[0109] In the above embodiments, the flight trajectory of the UAV is manually planned according to the direction and location distribution of the power transmission line to ensure that the flight path can fully cover the key areas of the power transmission line and its surrounding environment. By setting appropriate UAV flight altitude, speed, and flight path, the high-resolution remote sensing equipment on the UAV platform acquires detailed image data of the power line, accurately capturing line details and changes in the surrounding environment. Multispectral sensor equipment acquires high-resolution image data to capture details of the power transmission line and changes in its surrounding environment. Simultaneously, the lidar system on the UAV platform acquires three-dimensional LiDAR point cloud data to achieve spatial positioning of different targets (such as power lines, trees, transmission towers, etc.) in the power transmission line, effectively covering power transmission lines in different environments and improving the accuracy and comprehensiveness of detection. In this example, high-resolution remote sensing images are acquired through ground observation using optical sensors; airborne lidar point cloud data is acquired through ground observation using airborne lidar sensors.

[0110] To enhance the expressive power of tree barrier features and accurately locate suspected tree barriers, in one optional embodiment, the feature processing unit includes:

[0111] The first preprocessing module sequentially performs image resizing, normalization, and image enhancement on the aforementioned high-resolution remote sensing image to obtain the target resolution remote sensing image.

[0112] The first input module inputs the target resolution remote sensing image into the improved target detection model. The Backbone module in the improved target detection model extracts the image features of the target resolution remote sensing image through multi-layer convolution operations to generate the multi-scale feature map. The Neck module fuses the multi-scale feature map through a feature pyramid network and a path aggregation network to obtain a fused feature map. The Head module detects the fused feature map and outputs all the suspected tree obstacle locations in sequence to obtain all the suspected tree obstacle locations.

[0113] In the above embodiments, data preprocessing is first performed, including normalization and data augmentation of the high-resolution remote sensing images to improve the robustness and generalization ability of the model, such as... Figure 3The upper part shows the input of the target high-resolution remote sensing image into the improved YOLO v8 model. The YOLO v8 model structure mainly consists of three parts: Backbone, Neck, and Head. Each part plays a key role in feature extraction, fusion, and target detection. The Backbone module is used for image feature extraction to generate multi-scale feature maps. The Neck module is used for feature fusion processing of the multi-scale feature maps to generate fused feature maps. The Head module is used to detect the fused feature maps and output them. The preprocessed image (the target resolution remote sensing image) is input into the Backbone module for feature extraction. The core of this stage is to extract low-level to high-level feature maps of the image through multi-layer convolutional operations. The Backbone uses a deep neural network to ensure that key visual information can be fully expressed. The Backbone module mainly extracts individual-scale features from the images generated by the C2f and SPPF modules. The C2f module replaces the original C3 module, reducing one convolutional layer and making the model more lightweight. SPPF reduces the number of network layers based on the original SPP, which can perform feature fusion more quickly and eliminate redundant operations. Next, the extracted multi-scale feature maps are passed to the Neck module. The Neck module effectively fuses feature maps of different resolutions and semantic levels through a combination of Feature Pyramid Network (FPN) and Path Aggregation Network (PAN). FPN enhances the semantic expressiveness of higher-level feature maps through top-down feature fusion, ensuring the model can understand complex objects in the image. Meanwhile, PAN preserves the detailed information of lower-level feature maps through bottom-up path enhancement, making the model more accurate in detecting targets of varying sizes. The introduction of the SENet channel attention mechanism adaptively adjusts the weights of each channel, extracting global information for each channel using global average pooling, and then generating weights through a two-layer fully connected network. After weight allocation between channels, the network can focus on more meaningful channel features and suppress irrelevant or redundant information. This mechanism enhances the model's ability to capture important features and improves the accuracy of object detection, especially when dealing with complex scenes and multi-scale targets. After feature fusion, the output of the Neck module is passed to the Head module. The Head module is responsible for converting the fused feature map into specific detection results. Through a series of convolutional operations and a classifier, the model can output the specific location and category information of the tree barriers. In the feature fusion process of the YOLO v8 model, the SENet channel attention mechanism is introduced. By adaptively adjusting the weights of each channel, the expressive power of the tree barrier features is improved, the influence of low-level vegetation on detection is reduced, and thus the model's detection performance is improved.

[0114] To extract electric field line features more accurately and effectively improve the high-precision classification of electric field line point clouds, in one optional embodiment, the extraction unit includes:

[0115] The second preprocessing module performs removal and normalization operations on the airborne point cloud data in sequence to obtain valid airborne point cloud data. The removal operation is to remove point cloud data that is unrelated to power lines.

[0116] The second input module inputs the aforementioned effective airborne point cloud data into the aforementioned improved point cloud deep learning model, so that the aforementioned transformation network in the aforementioned improved point cloud deep learning model aligns the aforementioned effective airborne point cloud data to a standard transformation matrix through multiple convolutional layers and fully connected layers. The aforementioned feature extraction network uses a multilayer perceptron to extract local point features from the aforementioned transformation matrix. The aforementioned global feature aggregation and classification module uses a max pooling layer to aggregate the aforementioned local point features into a global feature vector, and uses the aforementioned dedicated classifier in the aforementioned output layer to remap the aforementioned global feature vector to generate linear point cloud features for classification output, thereby obtaining the aforementioned power line result.

[0117] In the above embodiments, the network structure of the improved point cloud deep learning model includes a transformation network, a feature extraction network, and a global feature aggregation and classification output module. The global feature aggregation and classification output module includes at least a max pooling layer and the aforementioned output layer. Irrelevant point cloud data, such as ground, building, and other noisy data, is removed from the acquired airborne point cloud data to reduce the impact of errors on classification. The point cloud data is then normalized so that the spatial location of all points is standardized to a uniform coordinate range, resulting in the aforementioned effective airborne point cloud data. Figure 4 As shown, firstly, the point cloud data (i.e., the effective airborne point cloud data) is input into the transformation network, which uses a T-Net structure. Through multiple 3×3 convolutional layers and fully connected layers, it learns a transformation matrix to align the input effective airborne point cloud data to a standard pose. The feature extraction network uses a pointwise MLP (Multilayer Perceptron) to extract point features, where each MLP layer is equivalent to a 1×1 convolutional layer, extracting local features of the points with ReLU activation. During this process, no spatial pooling is performed to preserve the spatial information of the points. The feature vector output by the feature extraction network is aggregated into a global feature vector through max pooling. This operation integrates the local information of the point cloud into a fixed-length global descriptor, exhibiting spatial transformation invariance. Finally, through several fully connected layers, the network maps the global features to the output space of a specific task (such as classification or segmentation), ultimately calculating class probabilities through a Softmax layer or achieving point cloud segmentation through the pointwise MLP.

[0118] In this embodiment, as Figure 4As shown, the PointNet model is improved as follows: Based on the original PointNet model, a dedicated classifier, LineSegNet (Line Segment Network), is added to the output layer specifically for separating linear point cloud structures. Furthermore, a multi-task learning framework is adopted, using a weighted loss function to enhance the classifier's focus on power lines, ensuring the network's accuracy in identifying power lines. This improved model is called the PointNet-PL model (i.e., the improved point cloud deep learning model mentioned above). Based on the PointNet-PL model, power lines are extracted by classifying point cloud data. The LineSegNet classifier is added to the feature aggregation module of the original PointNet. This classifier uses a dedicated MLP layer to remap the aggregated global features, generating specific features for linear point clouds. In this process, the structure of the MLP layer remains consistent with the original PointNet, but an additional fully connected layer with specific weight initialization is added before the output layer to strengthen the recognition of linear structures. At the classifier's output layer, a weighted loss function is introduced, specifically assigning higher weights to the power line category, thereby enhancing the classifier's focus on power lines. With this improvement, the PointNet-PL model can extract electric field line features more accurately, effectively improving the high-precision classification of electric field line point clouds.

[0119] To ensure more accurate tree information detection in complex scenarios, in one optional implementation, the retrieval classification unit includes:

[0120] The retrieval module uses the improved target detection model described above to retrieve the suspected tree obstacle detection boxes corresponding to the suspected tree obstacle locations of the above targets;

[0121] The mapping module maps the coordinates of the suspected tree obstacle detection box in the image coordinate system to the three-dimensional spatial coordinate system and sets the corresponding three-dimensional search area.

[0122] The filtering module filters out the airborne point cloud data that falls within the above-mentioned three-dimensional search area to obtain the above-mentioned target point cloud data.

[0123] The third input module inputs the target point cloud data into the improved point cloud object detection model, so that the voxelization module in the improved point cloud object detection model divides the target point cloud data into multiple voxel grids of a set size. The voxel feature encoding module extracts local features of the target point cloud data in the non-empty voxel grids to obtain voxel features. The convolutional intermediate layer further extracts spatial features between voxels from all the voxel features. The region proposal network classifies and locates trees in the three-dimensional retrieval area based on the spatial features to obtain the tree classification result.

[0124] In the above embodiments, the network structure of the improved point cloud object detection model includes a voxelization module, a feature extraction module, and a classification and regression module. The voxelization module is used to perform voxelization operations. The feature extraction module includes a voxel feature encoding module and a convolutional intermediate layer. The classification and regression module is used to classify and locate target objects through a region proposal network. First, the suspected tree obstacle detection boxes obtained by the improved YOLO v8 model are mapped from the image coordinate system to the three-dimensional spatial coordinate system, defining a corresponding three-dimensional retrieval region. This allows the point cloud data falling within this region (i.e., the target point cloud data) to be filtered from the original point cloud data. The target point cloud data is then divided into a fixed-size three-dimensional voxel grid using the improved VoxelNet model. The point cloud within each non-empty voxel is processed by the VFE module to extract local features. The VFE module uses a multilayer perceptron (MLP) with shared parameters to perform feature mapping on the coordinates of each point and cancels the traditional ReLU function, applying the GELU activation function. This process can be described as: f ij =GELU(W1p ij +b1), where p ij Voxel v j The coordinates of the i-th point in the matrix are given, and W1 and b1 are the weight matrix and bias term of the MLP, respectively. After convolution, the network downsamples through a max-pooling layer, gradually reducing the spatial size of the feature map. The second and third layers use 128 and 256 convolutional kernels for feature extraction, respectively, ultimately reducing the spatial size of the feature map to 1 / 8 of the original voxel grid, but increasing the number of channels to 256. On the feature map extracted by 3D convolution, a Region Proposal Network (RPN) is used for object classification and localization. The RPN contains a 3D convolutional layer, which is then divided into a classification branch and a regression branch. The former calculates the class probability using Softmax, while the latter outputs the bounding box regression parameters. The entire network structure utilizes the GELU activation function to enhance non-linear expressive power, enabling the improved VoxelNet model (also known as the VoxelNet-TR model) to detect tree information more accurately in complex scenes.

[0125] To accurately exclude non-tree-barrier areas, in one optional implementation, the above-mentioned calculation unit includes:

[0126] The calculation module calculates the distance according to the first formula, which is: d represents the distance between the tree and the power line, v represents the direction vector of the power line, p0 represents the coordinates of any point on the power line, and q represents the coordinates of any tree within the voxelized region.

[0127] The comparison module compares the above distance with the above-mentioned first tree barrier safety distance;

[0128] The first determining module determines the above division result as the above-mentioned tree barrier area when the above-mentioned distance is less than or equal to the above-mentioned first tree barrier safety distance;

[0129] The second determining module determines the above division result as the above non-tree barrier area when the above distance is greater than the above first tree barrier safety distance.

[0130] In the above embodiment, using the classified power lines and voxelized trees, the distance d between the power lines and the tree voxel mesh is calculated, and the calculation results are divided to exclude the non-tree obstacle areas obtained in the first step of fast detection. Specifically, assuming the power line vector v represents the direction vector of the power line, the distance d is calculated as follows: Based on the calculation results, the distance d between the power line and the tree voxel grid is divided into gradients for the elimination of suspected tree obstacle hazards. That is, the above distance is compared with the first tree obstacle safety distance. If the above distance is less than or equal to the first tree obstacle safety distance, the above division result is the tree obstacle area; if the above distance is greater than the first tree obstacle safety distance, the above division result is the non-tree obstacle area, and the non-tree obstacle area can be directly excluded and will not be used for subsequent tree obstacle hazard level judgment.

[0131] To improve the efficiency and accuracy of tree barrier assessment, in one optional implementation, the determining unit includes:

[0132] The third determining module determines the minimum safe distance between the trees and the transmission lines based on the power of the transmission lines.

[0133] The fourth determining module determines that, when the distance is less than or equal to the first tree barrier safety distance and greater than the second tree barrier safety distance, the tree barrier hazard level corresponding to the tree barrier area of ​​the suspected tree barrier location is the low hazard level, and the second tree barrier safety distance is less than the first tree barrier safety distance and greater than the minimum tree barrier safety distance.

[0134] The fifth determining module determines the tree obstacle hazard level of the tree obstacle area corresponding to the suspected tree obstacle location as the medium hazard level when the distance is less than or equal to the second tree obstacle safety distance and greater than the minimum tree obstacle safety distance.

[0135] The sixth determining module determines the tree hazard level of the tree area corresponding to the suspected tree obstacle location as a high hazard level when the distance is less than or equal to the minimum tree obstacle safety distance.

[0136] In the above embodiments, the power of the transmission line determines its ability to transmit electrical energy and the magnitude of the current. Higher-power lines typically have stricter requirements for the safe distance from tree obstructions. If a detected tree obstruction is within the transmission line's protection zone and the line has a high power rating, the tree obstruction is classified as a high-risk hazard; otherwise, it is classified as a low-risk or medium-risk hazard. Based on the assessment results, tree obstruction hazards are divided into three different levels: low risk, medium risk, and high risk. These levels are determined comprehensively based on factors such as the distance between the tree obstruction and the line, and the line's power rating. The minimum safe distance from tree obstructions corresponding to the power of the transmission line can be determined in conjunction with the "Operating Regulations for Overhead Transmission Lines." Different distances between trees and transmission lines of different power ratings are interpreted to determine the tree obstruction hazard level, and the detection results are output and displayed. By calculating the distance between trees and power lines and automatically assessing the hazard level in conjunction with the power of the transmission line, the assessment efficiency and accuracy are greatly improved, the uncertainty caused by human judgment is reduced, and the intelligence level of power line inspection is enhanced. The aforementioned automatic identification device for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing includes a processor and a memory. The acquisition unit, feature processing unit, and extraction unit are all stored as program units in the memory, and the processor executes these program units to achieve the corresponding functions. All of the above modules are located in the same processor; alternatively, the modules may be located in different processors in any combination.

[0137] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can address the problem of low efficiency in inspecting tree obstructions along power transmission lines in existing technologies.

[0138] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0139] This invention provides a computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device containing the computer-readable storage medium to perform the above-mentioned automatic identification method for power transmission line tree obstruction hazards based on UAV multi-source remote sensing.

[0140] This invention provides a processor for running a program, wherein the program executes the above-mentioned automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing.

[0141] This invention provides an automatic identification system for tree obstruction hazards on power transmission lines. The system includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs at least the following steps:

[0142] Step S201: Obtain multi-source remote sensing data. The multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines. The airborne point cloud data includes at least the spatial positioning data of the power lines, trees and transmission towers.

[0143] Step S202: Using the above high-resolution remote sensing image, image feature extraction and feature fusion processing are performed based on the improved target detection model to detect all suspected tree obstacle locations. The above improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model.

[0144] Step S203: Classify the airborne point cloud data based on the improved point cloud deep learning model and extract the power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data.

[0145] Step S204, retrieval and classification step: retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location to obtain the target point cloud data corresponding to the suspected tree obstacle location, and use the improved point cloud object detection model to perform voxel classification on the target point cloud data to obtain the tree classification result. The suspected tree obstacle location is any of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function on the basis of the VoxelNet model.

[0146] Step S205, calculation step: Based on the above tree classification results and the above power line results, calculate the distance between the trees and power lines within the above coverage area, and perform gradient division based on the above distance to obtain the division result. The above division result is a tree barrier area or a non-tree barrier area. The above tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the above non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance.

[0147] Step S206, Determination Step: In the case that the above division result is the above tree barrier area, the tree barrier hazard level of the above tree barrier area where the above suspected tree barrier location is located is determined by analyzing the corresponding distance and the power of the above transmission line. The tree barrier hazard level is one of the following: high hazard level, medium hazard level and low hazard level.

[0148] Step S207: Repeat the above-mentioned retrieval and classification steps, calculation steps, and determination steps at least once until all the above-mentioned suspected tree obstacle locations are determined, so as to obtain all the above-mentioned tree obstacle areas and the corresponding tree obstacle hazard levels, and to inspect the corresponding tree obstacle areas according to the above-mentioned tree obstacle hazard levels.

[0149] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having at least the following method steps:

[0150] Step S201: Obtain multi-source remote sensing data. The multi-source remote sensing data includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines. The airborne point cloud data includes at least the spatial positioning data of the power lines, trees and transmission towers.

[0151] Step S202: Using the above high-resolution remote sensing image, image feature extraction and feature fusion processing are performed based on the improved target detection model to detect all suspected tree obstacle locations. The above improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model.

[0152] Step S203: Classify the airborne point cloud data based on the improved point cloud deep learning model and extract the power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data.

[0153] Step S204, retrieval and classification step: retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location to obtain the target point cloud data corresponding to the suspected tree obstacle location, and use the improved point cloud object detection model to perform voxel classification on the target point cloud data to obtain the tree classification result. The suspected tree obstacle location is any of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function on the basis of the VoxelNet model.

[0154] Step S205, calculation step: Based on the above tree classification results and the above power line results, calculate the distance between the trees and power lines within the above coverage area, and perform gradient division based on the above distance to obtain the division result. The above division result is a tree barrier area or a non-tree barrier area. The above tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the above non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance.

[0155] Step S206, Determination Step: In the case that the above division result is the above tree barrier area, the tree barrier hazard level of the above tree barrier area where the above suspected tree barrier location is located is determined by analyzing the corresponding distance and the power of the above transmission line. The tree barrier hazard level is one of the following: high hazard level, medium hazard level and low hazard level.

[0156] Step S207: Repeat the above-mentioned retrieval and classification steps, calculation steps, and determination steps at least once until all the above-mentioned suspected tree obstacle locations are determined, so as to obtain all the above-mentioned tree obstacle areas and the corresponding tree obstacle hazard levels, and to inspect the corresponding tree obstacle areas according to the above-mentioned tree obstacle hazard levels.

[0157] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0158] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0164] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0166] As can be seen from the above description, the embodiments of this application achieve the following technical effects:

[0167] 1) This application presents an automatic identification method for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing. This method fully leverages the advantages of image information and point cloud data acquired by UAVs and processes them directly. Based on an improved YOLO v8 model, image features are extracted and fused from the acquired high-resolution remote sensing images to detect and identify the locations of suspected tree obstructions for further analysis of potential risk areas. Next, an improved PointNet model is used to classify the point cloud data, extract the power transmission lines, and retrieve the point cloud data within the detected suspected tree obstruction locations. Subsequently, an improved VoxelNet model is used to classify these point cloud data, thereby identifying and voxelizing tree areas to support subsequent accurate analysis. Finally, the distance between the trees and power lines is calculated based on the classification results, and the distance calculation results are divided to exclude non-tree obstruction areas, thus achieving accurate interpretation of the area. The tree obstruction hazard level is determined by combining the power of the power transmission line, and the final detection results are output and displayed. Compared with existing methods, this application's method can obtain tree obstruction extraction results more quickly and accurately. This application solves the problem of low inspection efficiency for tree obstructions on power transmission lines in existing technologies.

[0168] 2) This application's automatic identification device for tree obstruction hazards on power transmission lines based on UAV multi-source remote sensing fully leverages the advantages of image information and point cloud data acquired by UAVs and processes them directly. Based on an improved YOLO v8 model, image features are extracted and fused from the acquired high-resolution remote sensing images to detect and acquire the locations of suspected tree obstructions for further analysis of potential risk areas. Next, an improved PointNet model is used to classify the point cloud data, extract the power transmission lines, and retrieve the point cloud data within the detected suspected tree obstruction locations. Subsequently, an improved VoxelNet model is used to classify these point cloud data, thereby identifying and voxelizing tree areas to support subsequent accurate analysis. Finally, the distance between trees and power lines is calculated based on the classification results, and the distance calculation results are divided to exclude non-tree obstruction areas, thus achieving accurate interpretation of the area. Combined with the power of the power transmission line, the tree obstruction hazard level is determined, and the final detection results are output and displayed. Compared with existing methods, the method of this application can acquire tree obstruction extraction results more quickly and accurately. This application solves the problem of low inspection efficiency for tree obstructions on power transmission lines in existing technologies.

[0169] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for automatic identification of tree obstruction hazards on power transmission lines based on multi-source remote sensing from unmanned aerial vehicles (UAVs), characterized in that, include: Acquire multi-source remote sensing data, which includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of power lines of transmission lines, and the airborne point cloud data includes at least spatial positioning data of the power lines, trees, and transmission towers. Using the high-resolution remote sensing imagery, image feature extraction and feature fusion processing were performed based on an improved target detection model to detect all suspected tree obstacle locations. The improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model. The airborne point cloud data is classified based on an improved point cloud deep learning model to extract power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data. The retrieval and classification steps involve retrieving the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location to obtain the target point cloud data corresponding to the suspected tree obstacle location. An improved point cloud object detection model is then used to perform voxelization classification on the target point cloud data to obtain tree classification results. The suspected tree obstacle location is any one of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by replacing the ReLU function with the GELU function as the activation function based on the VoxelNet model. The calculation steps are as follows: based on the tree classification results and the power line results, the distance between the trees and power lines within the coverage area is calculated, and gradient division is performed based on the distance to obtain the division result. The division result is a tree barrier area or a non-tree barrier area. The tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance. The determination step involves analyzing the tree barrier area based on the corresponding distance and the power of the transmission line, in the case where the division result is the tree barrier area, to determine the tree barrier hazard level of the tree barrier area where the suspected tree barrier location is located. The tree barrier hazard level is one of the following: high hazard level, medium hazard level, and low hazard level. The retrieval and classification step, the calculation step, and the determination step are repeated at least once until all the suspected tree obstacle locations are determined, so as to obtain all the tree obstacle areas and the corresponding tree obstacle hazard levels, and to conduct inspections on the corresponding tree obstacle areas according to the tree obstacle hazard levels.

2. The method according to claim 1, characterized in that, Acquire multi-source remote sensing data, including: The flight trajectory of the UAV is planned based on the direction and location distribution of the power lines to obtain the flight trajectory of the target UAV; Based on the target UAV's flight trajectory, the UAV platform uses a medium-to-high resolution remote sensing device to acquire detailed image data of the power lines. Simultaneously, the UAV platform uses a lidar to acquire airborne point cloud data.

3. The method according to claim 1, characterized in that, The improved target detection model includes a Backbone module, a Neck module, and a Head module. The Backbone module is used for image feature extraction to generate multi-scale feature maps. The Neck module is used for feature fusion processing of the multi-scale feature maps to generate fused feature maps. The Head module is used to detect the fused feature maps and output them. Using the high-resolution remote sensing image, image feature extraction and feature fusion processing are performed based on the improved target detection model to detect all suspected tree obstacle locations, including: The high-resolution remote sensing image is sequentially subjected to image resizing, normalization, and image enhancement to obtain the target resolution remote sensing image. The target resolution remote sensing image is input into the improved target detection model. The Backbone module in the improved target detection model extracts the image features of the target resolution remote sensing image through multi-layer convolution operations to generate the multi-scale feature map. The Neck module fuses the multi-scale feature map through a feature pyramid network and a path aggregation network to obtain a fused feature map. The Head module detects the fused feature map and outputs all the suspected tree obstacle locations in sequence to obtain all the suspected tree obstacle locations.

4. The method according to claim 1, characterized in that, The improved point cloud deep learning model's network structure includes a transform network, a feature extraction network, and a global feature aggregation and classification output module. The global feature aggregation and classification output module includes at least a max pooling layer and the output layer. Based on the improved point cloud deep learning model, the model classifies the airborne point cloud data and extracts power line results, including: The airborne point cloud data is sequentially subjected to removal and normalization operations to obtain valid airborne point cloud data. The removal operation is the operation of removing point cloud data that is unrelated to power lines. The effective airborne point cloud data is input into the improved point cloud deep learning model, so that the transformation network in the improved point cloud deep learning model aligns the effective airborne point cloud data to a standard transformation matrix through multiple convolutional layers and fully connected layers. The feature extraction network uses a multilayer perceptron to extract local point features from the transformation matrix. The max pooling layer in the global feature aggregation and classification module aggregates the local point features into a global feature vector, and the dedicated classifier in the output layer remaps the global feature vector to generate linear point cloud features for classification output, thus obtaining the power line result.

5. The method according to claim 1, characterized in that, The improved point cloud object detection model's network structure includes a voxelization module, a feature extraction module, and a classification and regression module. The voxelization module performs voxelization operations. The feature extraction module includes a voxel feature encoding module and a convolutional intermediate layer. The classification and regression module classifies and locates target objects using a region proposal network, retrieves airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtains target point cloud data corresponding to the suspected tree obstacle location, and uses the improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain tree classification results, including: The improved target detection model is used to retrieve the suspected tree obstacle detection box corresponding to the suspected tree obstacle location of the target; The coordinates of the suspected tree obstacle detection box in the image coordinate system are mapped to the three-dimensional spatial coordinate system, and the corresponding three-dimensional search area is set. The airborne point cloud data that falls within the three-dimensional search area is filtered out to obtain the target point cloud data; The target point cloud data is input into the improved point cloud object detection model, so that the voxelization module in the improved point cloud object detection model divides the target point cloud data into multiple voxel grids of a set size. The voxel feature encoding module extracts local features of the target point cloud data in the non-empty voxel grids to obtain voxel features. The convolutional intermediate layer further extracts spatial features between voxels from all the voxel features. The region proposal network classifies and locates trees in the three-dimensional retrieval area based on the spatial features to obtain the tree classification result.

6. The method according to claim 1, characterized in that, Based on the tree classification results and the power line results, the distances between trees and power lines within the coverage area are calculated, and gradient partitioning is performed based on these distances to obtain partitioning results, including: The distance is calculated according to a first formula, which is: d represents the distance between the tree and the power line, v represents the direction vector of the power line, p0 represents the coordinates of any point on the power line, and q represents the coordinates of any tree within the voxelized region; Compare the distance with the first tree barrier safety distance; If the distance is less than or equal to the first tree barrier safety distance, the division result is determined as the tree barrier area; If the distance is greater than the first tree barrier safety distance, the division result is determined to be the non-tree barrier area.

7. The method according to claim 1, characterized in that, In the case where the classification result is the tree-blocked area, the tree-blocking hazard level of the suspected tree-blocked location in the tree-blocked area is determined by analyzing the corresponding distance and the power of the transmission line, including: The minimum safe distance between the trees and the power transmission line is determined based on the power of the power transmission line. If the distance is less than or equal to the first tree barrier safety distance and greater than the second tree barrier safety distance, the tree barrier hazard level corresponding to the tree barrier area at the target suspected tree barrier location is determined to be the low hazard level, and the second tree barrier safety distance is less than the first tree barrier safety distance and greater than the minimum tree barrier safety distance; If the distance is less than or equal to the second tree barrier safety distance and greater than the minimum tree barrier safety distance, the tree barrier hazard level corresponding to the target suspected tree barrier location in the tree barrier area is determined to be the medium hazard level; If the distance is less than or equal to the minimum safe distance for tree barriers, the tree barrier hazard level of the tree barrier area corresponding to the suspected tree barrier location is determined to be the high hazard level.

8. An automatic identification device for tree obstruction hazards on power transmission lines based on multi-source remote sensing from unmanned aerial vehicles, characterized in that, The device includes: The acquisition unit is used to acquire multi-source remote sensing data, which includes at least high-resolution remote sensing images and airborne point cloud data. The high-resolution remote sensing images are remote sensing images of the power lines of the power transmission line, and the airborne point cloud data includes at least the spatial positioning data of the power lines, trees and power transmission towers. The feature processing unit is used to perform image feature extraction and feature fusion processing based on the high-resolution remote sensing image and an improved target detection model to detect all suspected tree obstacle locations. The improved target detection model is an improved model obtained by introducing a channel attention mechanism on the basis of the YOLOv8 model. An extraction unit is used to classify the airborne point cloud data based on an improved point cloud deep learning model and extract power line results. The improved point cloud deep learning model is an improved model obtained by adding a dedicated classifier to the output layer of the PointNet model. The dedicated classifier is a classifier specifically used to separate linear point cloud data. The retrieval and classification unit is used to perform the retrieval and classification steps, retrieve the airborne point cloud data within the coverage area of ​​the suspected tree obstacle location, obtain the target point cloud data corresponding to the suspected tree obstacle location, and use an improved point cloud object detection model to perform voxelization classification on the target point cloud data to obtain tree classification results. The suspected tree obstacle location is any one of the suspected tree obstacle locations. The point cloud object detection model is an improved model obtained by using the GELU function instead of the RELU function as the activation function based on the VoxelNet model. The calculation unit is used to perform calculation steps, calculate the distance between trees and power lines within the coverage area based on the tree classification results and the power line results, and perform gradient division based on the distance to obtain the division result, which is a tree barrier area or a non-tree barrier area. The tree barrier area is the area where the distance is less than the first tree barrier safety distance, and the non-tree barrier area is the area where the distance is greater than the first tree barrier safety distance. The determining unit is used to perform the determining step. When the division result is the tree barrier area, it analyzes the corresponding distance and the power of the transmission line to determine the tree barrier hazard level of the tree barrier area where the target suspected tree barrier location is located. The tree barrier hazard level is one of the following: high hazard level, medium hazard level, and low hazard level. The repeating unit is used to sequentially repeat the retrieval and classification step, the calculation step, and the determination step at least once until all the suspected tree obstacle locations are determined, so as to obtain all the tree obstacle areas and the corresponding tree obstacle hazard levels, so as to inspect the corresponding tree obstacle areas according to the tree obstacle hazard levels.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1 to 7.

10. An automatic identification system for tree obstruction hazards on power transmission lines, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs comprising methods for performing any one of claims 1 to 7.

Images