Tree obstacle intelligent detection method and system based on multi-modal perception

Abstract

The invention relates to an intelligent tree barrier detection method and system based on multi-modal perception, and the method employs an unmanned plane to carry a sensor to collect data, including RGB images, LIDAR point clouds, multispectral data and the like. Firstly, data is preprocessed to form standardized input. Hough transformation is carried out on the LIDAR point cloud, the position of the power line is extracted, and a three-dimensional line model is generated. And then positioning a tree bounding box from the RGB image by using a deep learning algorithm, and screening a high-risk point cloud region in combination with a line model. And constructing a three-dimensional tree model, calculating the minimum safety clearance distance between the power line and the tree in real time through a collision detection algorithm, obtaining the three-dimensional tree model with a safety distance mark, and finally carrying out space risk assessment based on the model. According to the invention, the reliability and efficiency of power line inspection can be significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent recognition, and particularly to a tree obstacle intelligent detection method and system based on multi-modal perception. Background Art

[0002] With the rapid expansion of the power transmission line network, the safe operation of transmission lines faces more and more challenges. Among them, tree obstacles are one of the important factors leading to line failures. According to statistics, accidents caused by the rapid growth, toppling of trees or their intrusion into the safety clearance range of power lines under the action of adverse weather (such as strong winds, heavy rains), resulting in short circuits, power outages and even equipment damage, have a high proportion. Traditional tree obstacle detection methods mainly rely on manual inspections and ground surveys. However, this method not only has high costs and low efficiency, but also complex environments such as rain and strong winds may increase the detection difficulty and risks. At the same time, traditional methods also have significant limitations in scenarios with a wide area, complex terrain (such as mountainous areas, high altitude areas) or large line spans. Summary of the Invention

[0003] To achieve the above object, the present invention adopts the following technical solutions: A tree obstacle intelligent detection method based on multi-modal perception, comprising the following steps: S1: Collect data through sensors carried by an unmanned aerial vehicle (UAV), obtain sensor data, and preprocess the sensor data to form a standardized input; wherein, the sensor data includes RGB images, LIDAR point clouds, multi-spectral data and UAV data; S2: Perform Hough transform on the LIDAR point cloud in the standardized input, extract the primary position of the power line, and generate a three-dimensional line model of the wire; S3: Use a deep learning object detection algorithm to locate the tree target bounding box from the RGB image in the standardized input, and combine it with the three-dimensional line model of the wire to screen the high-risk point cloud regions in the space near the wire of the power line; S4: According to the high-risk point cloud regions in the space near the wire of the power line, construct a three-dimensional tree model, and through a collision detection algorithm, calculate the minimum safety clearance distance between the power line and the tree in real time from the three-dimensional tree model to obtain a three-dimensional tree model with safety distance annotation; S5: Based on the three-dimensional tree model with safety distance annotation, perform a spatial risk assessment.

[0004] Further, the preprocessing of the sensor data to form a standardized input in step S1 is specifically: Spatio-temporal synchronization processing, normalizing sensor data with different acquisition frequencies based on timestamps to ensure spatio-temporal correlation; Spatial registration and coordinate system I. Using a checkerboard calibration board, the rotation matrix R and the translation vector t are solved through joint calibration, and the satisfaction condition is expressed as: ; Among them, represents the point in the camera coordinate system, represents the point in the radar coordinate system; In this process, the Levenberg-Marquardt algorithm is used to minimize the reprojection error, which is expressed as: ; Among them, is the RGB camera projection model, is the coordinate of the th corner point in the RGB image, that is, the actual observation point in the camera coordinate system; is the number of point pairs of the matching points in the camera coordinate system and the radar coordinate system, represents the minimum evaluation function; represents the th three-dimensional point in the radar coordinate system; The Brown-Conrady model is used to process the RGB image to correct the lens distortion, which is expressed as: ; Among them, represents the radial distance from the point to the optical axis, is the radial distortion coefficient, is the tangential distortion coefficient; is the actual pixel position of the RGB image obtained after correction; is the pixel position of the RGB image obtained before correction; Voxel downsampling is performed on the LIDAR point cloud, and the LIDAR point cloud is divided into voxel grids with side length , and the centroid is retained for each voxel, which is expressed as: ; Among them, is the centroid of the voxel ; is the total number of points in the voxel v, is the voxel coordinates of each point inside; Feature matching is performed on the multispectral data based on the SIFT algorithm to achieve multispectral band alignment, and the aligned multispectral data is obtained; The extended Kalman filter is used to fuse the and of the UAV data to obtain the longitude, latitude, altitude and attitude data of the UAV; represents the real-time differential positioning data of the UAV; IMU represents the inertial measurement data of the UAV; Based on this, a standardized output is formed, formalized as a time-synchronized tuple , expressed as: ; where, is the corrected RGB image; is the downsampled LIDAR point cloud; is the aligned multispectral data; is the longitude, latitude, and elevation, is the attitude data.

[0005] Furthermore, the extended Kalman filter is used to fuse the and of the UAV data to obtain the longitude, latitude, elevation, and attitude data of the UAV; specifically: Based on the longitude, latitude, elevation, and attitude data of the UAV, the state vector of the UAV is obtained , where, is the position of the UAV; is the speed of the UAV; is the attitude quaternion of the UAV, and the attitude quaternion is used to represent spatial rotation; represents the transpose; Through drive the prediction step, expressed as: ; ; ; where, is the position information of the UAV at time ; is the speed of the UAV at time ; is the attitude quaternion of the UAV at time ; is the time increment; represents the rotation matrix from the coordinate system of the IMU to the global coordinate system, calculated from the attitude quaternion of the UAV at time ; is the acceleration of the UAV at time ; is the gravitational acceleration; is the quaternion multiplication; is the angular velocity of the UAV at time ; Update step based on GPS+RTK observations of UAV data, expressed as: ; where is the position information of the UAV measured in real time by GPS+RTK, used to correct the cumulative error of the IMU; is the observation matrix; is the state vector of the UAV at time .

[0006] Furthermore, the specific method for generating the three-dimensional line model of the wire in step S2 is as follows: The wire of the power line is represented as a straight-line parametric equation in 3D space , expressed as: ; where is an arbitrary point on the straight-line parametric equation , is the reference point on the straight-line parametric equation , is the direction vector of the straight-line parametric equation , is the straight-line parameter of the straight-line parametric equation ; The Hough parameter space is defined as: ; Satisfying the condition: ; where is the distance from the straight line to the origin; is the azimuth angle of the straight line in the XY plane; is the pitch angle of the straight line with the XY plane; represents the normal vector of the straight line; represents an arbitrary point on the straight line; For each point , calculate the voting value in the Hough parameter space, expressed as: ; For the roughly selected straight lines detected by Hough, minimize the residual of the distance from the point to the straight line through non-linear optimization, expressed as: ; where is the number of LIDAR point clouds of the wire; The wire presents a parabolic sag under its own weight, and the height h is distributed along the arc length s, expressed as: ; Among them, represents the height at the arc length s, is the starting height of the wire; w is the weight per unit length of the wire; is the horizontal tension of the wire; is the span, that is, the horizontal distance between the support points of the wire; Fitting the sag point cloud data, fitting w and by weighted least squares, expressed as: ; Among them, represents the predicted wire height, is the point The projected arc length along the line direction of the wire; M represents the total number of LIDAR point clouds of the wire; represents the minimum evaluation function; Finally, the three-dimensional wire route model is expressed as follows: .

[0007] Furthermore, S3 is specifically: Using the deep learning object detection algorithm to locate the tree objects in the RGB image in the standardized input, and obtaining the two-dimensional positions of each tree in the image by detecting the object bounding boxes; Combining the three-dimensional wire route model and the LIDAR point cloud, mapping the tree objects in the RGB image to the corresponding LIDAR point cloud; According to the three-dimensional wire route model Construct a spatial search area; specifically, taking the wire as the center, establish a cylindrical safety area, and the parameters of the cylindrical safety area are a cylinder with a radius r and a length The length is the corresponding span; The radius r is defined as follows: ; Make the following judgment on each point of the LIDAR point cloud: Query each point set near the wire in the LIDAR point cloud through the KD-Tree, retain the points within the cylindrical safety area, and further screen in combination with the tree point cloud clusters mapped from the RGB image corresponding to each bounding box; Calculate the minimum distance from the tree point cloud cluster to the wire through the shortest distance formula from a line to a point , that is, the minimum clearance distance; expressed as: ; Among them, represents the tree point cloud cluster; Perform risk determination, specifically as follows: Set a safety threshold , if , mark it as a high-risk tree point cloud cluster, that is, a high-risk point cloud area in the space near the wire.

[0008] Furthermore, S4 is specifically as follows: S41: Cluster the LIDAR point cloud in the high-risk point cloud area through the DBSCAN algorithm to segment out the point cloud clusters of individual trees; S42: Extract the trunk in the height direction from the point cloud clusters of individual trees, and use the RANSAC algorithm to fit the trunk into a cylindrical model; S43: Separate the trunk and branches and leaves according to the local geometric attributes of the LIDAR point cloud, and utilize spatial continuity to grow from the trunk and classify adjacent points as the crown part; S44: Generate a gradient field based on the normal vector of the LIDAR point cloud, construct an implicit function by solving the Poisson equation based on the gradient field, perform Poisson surface reconstruction, and fit to obtain the tree surface model; S45: Fuse the corrected RGB image into the aligned multi-spectral data. Specifically, calculate the vegetation index using the aligned multi-spectral data, and use the corrected RGB image to assign texture mapping to the grid surface of the tree surface model to obtain a three-dimensional tree model; S46: Use the constructed three-dimensional power line model and three-dimensional tree model, and calculate the safe clearance distance through the collision detection algorithm to obtain a three-dimensional tree model with safety distance markings.

[0009] Furthermore, S5 is specifically as follows: Extract tree features from the tree model with safety distance markings. The tree features include tree height and tree density ; and combine with the wind speed to calculate the dynamic minimum clearance distance; Based on this, construct a feature vector , expressed as: ; where, is the annual average growth rate of the tree; is the minimum clearance distance; is the health score, obtained by calculating the NDVI value through multi-spectral; is the tension factor; Construct a risk assessment model, and output a risk level R for each tree based on the LIDAR point cloud and the three-dimensional tree model, expressed as: ; where, is the risk score, obtained according to the risk assessment model; Provide maintenance suggestions for the risk level of each tree, specifically pruning or removing in the dangerous area; conducting regular monitoring in the warning area; no treatment is required in the safe area.

[0010] Furthermore, the risk assessment model is expressed as: ; ; ; ; ; wherein, is the clearance distance score; is the risk-free reference distance; is the health risk score; is the density risk score; are the minimum and maximum density values respectively; is the wind deflection risk score; is the wind speed, are all preset weight coefficients.

[0011] A tree obstacle intelligent detection system based on multi-modal perception, including a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, it specifically executes the steps in the above-mentioned tree obstacle intelligent detection method based on multi-modal perception.

[0012] The present invention has the following beneficial effects:

[0013] 1. The present invention integrates RGB images, LIDAR point clouds, multi-spectral data, and UAV data, and realizes real-time detection and safety assessment of tree obstacles through efficient edge computing and lightweight AI models. Through the intelligent collaboration of sensors, precise alignment of data preprocessing, multi-stage fusion detection network, and cloud-edge collaborative update mechanism, this solution can not only significantly improve the detection accuracy and real-time response ability, but also achieve long-term stable inspection tasks to ensure the safe operation of the power grid.

[0014] 2. The present invention quantifies the risk of trees to power lines through a tree model with safety distance annotations, combined with multi-dimensional features such as geometry, health, and density, and provides scientific support for power inspection through dynamic threshold adjustment and visualization. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is the flowchart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Referring to Figure 1 , an intelligent tree obstacle detection method based on multi-modal perception, comprising the following steps: S1: Collect data through sensors carried by the drone, obtain sensor data, and preprocess the sensor data to form a standardized input; wherein, the sensor data includes RGB images, LIDAR point clouds, multi-spectral data, and drone data; S2: Perform Hough transform on the LIDAR point cloud in the standardized input, extract the primary position of the power line, and generate a three-dimensional line model of the wire; S3: Use a deep learning object detection algorithm to locate the tree target bounding box from the RGB image in the standardized input, and combine it with the three-dimensional line model of the wire to screen the high-risk point cloud area in the space near the wire of the power line; S4: According to the high-risk point cloud area in the space near the wire of the power line, construct a three-dimensional tree model, and through a collision detection algorithm, calculate the minimum safety clearance distance between the power line and the tree in real time from the three-dimensional tree model to obtain a three-dimensional tree model with safety distance annotation; S5: Based on the three-dimensional tree model with safety distance annotation, perform spatial risk assessment.

[0018] Further, the preprocessing of the sensor data in step S1 to form a standardized input is specifically as follows: Spatio-temporal synchronization processing, normalizing sensor data with different acquisition frequencies based on timestamps to ensure spatio-temporal correlation; Spatial registration and coordinate system unification, using a checkerboard calibration board, and solving the rotation matrix R and translation vector t through joint calibration, and the condition is expressed as: ; Wherein, represents the point in the camera coordinate system, represents the point in the radar coordinate system; In this process, the Levenberg-Marquardt algorithm is used to minimize the reprojection error, which is expressed as: ; Wherein, is the RGB camera projection model, is the th corner coordinate in the RGB image, that is, the actual observation point in the camera coordinate system; is the number of point pairs of the matching points in the camera coordinate system and the radar coordinate system, represents the minimum evaluation function; Represents the th three-dimensional point in the radar coordinate system; The Brown-Conrady model is used to process the RGB image to correct lens distortion, expressed as: ; Where, Represents the radial distance from point to the optical axis, is the radial distortion coefficient, is the tangential distortion coefficient; Is the actual pixel position of the RGB image obtained after correction; Is the pixel position of the RGB image obtained before correction; Voxel downsampling is performed on the LIDAR point cloud, and the LIDAR point cloud is divided into voxel grids with side length , and the centroid of each voxel is retained, expressed as: ; Where, Is the centroid of voxel ; Is the total number of points in voxel v, Is the coordinate of each point in voxel ; Feature matching is performed on the multispectral data based on the SIFT algorithm to achieve multispectral band alignment, and the aligned multispectral data is obtained; The extended Kalman filter is used to fuse the of the UAV data with to obtain the latitude, longitude, altitude and attitude data of the UAV; Represents the real-time differential positioning data of the UAV data; IMU represents the inertial measurement data of the UAV data; Based on this, a standardized output is formed, formalized as a tuple synchronized in time, expressed as: ; Where, Is the RGB image after correction; Is the downsampled LIDAR point cloud; Is the aligned multispectral data; Is the latitude, longitude and altitude, Is the attitude data.

[0019] Furthermore, the extended Kalman filter is used to fuse the of the UAV data with to obtain the latitude, longitude, altitude and attitude data of the UAV; specifically: Obtain the state vector of the UAV based on the longitude, latitude, altitude and attitude data of the UAV , where is the position of the UAV; is the speed of the UAV; is the attitude quaternion of the UAV, and the attitude quaternion is used to represent spatial rotation; represents transpose; Through Drive the prediction step, expressed as: ; ; ; where is the position information of the UAV at time ; is the speed of the UAV at time ; is the attitude quaternion of the UAV at time ; is the time increment; represents the rotation matrix from the coordinate system of the IMU to the global coordinate system, calculated from the attitude quaternion of the UAV at time ; is the acceleration of the UAV at time ; is the gravitational acceleration; is quaternion multiplication; is the angular velocity of the UAV at time ; According to the GPS+RTK observation update step of the UAV data, expressed as: ; where is the position information of the UAV measured in real time by GPS+RTK, used to correct the cumulative error of the IMU; is the observation matrix; is the state vector of the UAV at time .

[0020] Furthermore, the specific generation of the three-dimensional line model of the wire in step S2 is as follows: Represent the wire of the power line as a straight-line parametric equation in 3D space , expressed as: ; where is any point on the straight-line parametric equation , is the straight-line parametric equation The reference point on is the parametric equation of a straight line is the direction vector of is the parametric equation of a straight line is the line parameter; The Hough parameter space is defined as: ; satisfies the condition: ; where is the distance from the straight line to the origin; is the azimuth angle of the straight line in the XY plane; is the pitch angle of the straight line with the XY plane; represents the normal vector of the straight line; represents any point on the straight line; For each point , calculate the voting value in the Hough parameter space, expressed as: ; For the roughly selected straight lines detected by Hough, minimize the residual of the distance from the point to the straight line through nonlinear optimization, expressed as: ; where is the number of LIDAR point clouds of the wire; The wire presents a parabolic sag under its own weight, and the height h is distributed along the arc length s, expressed as: ; where represents the height at the arc length s, is the starting height of the wire; w is the weight per unit length of the wire; is the horizontal tension of the wire; is the span, that is, the horizontal distance between the support points of the wire; Fit the sag point cloud data, and fit w and by weighted least squares, expressed as: ; where represents the predicted wire height, is the point is the projected arc length along the line direction of the wire; M represents the total number of LIDAR point clouds of the wire; represents the minimum evaluation function; Finally, the three-dimensional line model of the wire is expressed as follows: .

[0021] Further, S3 is specifically as follows: Use a deep learning object detection algorithm to locate tree targets in the RGB image of the standardized input, and obtain the two-dimensional positions of each tree in the image by detecting the target bounding boxes; Combine the three-dimensional line model and the LIDAR point cloud to map the tree targets in the RGB image to the corresponding LIDAR point cloud; According to the three-dimensional line model of the wire Construct a spatial search area; specifically, take the wire as the center and establish a cylindrical safety area. The parameters of the cylindrical safety area are the radius r and the length of the cylinder, and the length is the corresponding span; The radius r is defined as follows: ; Make the following judgment for each point in the LIDAR point cloud: Query each point set near the wire in the LIDAR point cloud through KD-Tree, retain the points within the cylindrical safety area, and further screen in combination with the tree point cloud clusters mapped from the RGB image corresponding to each bounding box; Calculate the minimum distance from the tree point cloud cluster to the wire through the shortest distance formula from a line to a point , that is, the minimum clearance distance; expressed as: ; Among them, represents the tree point cloud cluster; Perform risk determination, specifically as follows: Set a safety threshold , if , mark it as a high-risk tree point cloud cluster, that is, a high-risk point cloud area in the space near the wire.

[0022] Further, S4 is specifically as follows: S41: Cluster the LIDAR point cloud in the high-risk point cloud area through the DBSCAN algorithm to segment out single-tree point cloud clusters; S42: Extract the trunk in the height direction from the single-tree point cloud cluster, and use the RANSAC algorithm to fit the trunk into a cylindrical model; S43: Separate the trunk and branches and leaves according to the local geometric properties of the LIDAR point cloud, and use spatial continuity to grow from the trunk and classify adjacent points into the crown part; S44: Generate a gradient field based on the normal vector of the LIDAR point cloud, construct an implicit function by solving the Poisson equation based on the gradient field, perform Poisson surface reconstruction, and fit to obtain the tree surface model; S45: Integrate the corrected RGB image into the aligned multispectral data. Specifically, calculate the vegetation index using the aligned multispectral data, and assign texture mapping to the grid surface of the tree surface model using the corrected RGB image to obtain a three-dimensional tree model. S46: Use the constructed three-dimensional power line model and three-dimensional tree model, and calculate the safe clearance distance through a collision detection algorithm to obtain a three-dimensional tree model with safety distance markings.

[0023] Further, S5 is specifically as follows: Extract tree features from the tree model with safety distance markings. The tree features include tree height and tree density ; and combine with the wind speed to calculate the dynamic minimum clearance distance; Based on this, construct a feature vector , expressed as: ; where, is the average annual growth rate of the tree; is the minimum clearance distance; is the health score, obtained by calculating the NDVI value through multispectral data; is the tension factor; Construct a risk assessment model, and output a risk level R for each tree based on the LIDAR point cloud and the three-dimensional tree model, expressed as: ; where, is the risk score, obtained according to the risk assessment model; Provide maintenance suggestions for the risk level of each tree. Specifically, prune or remove in the dangerous area; conduct regular monitoring in the warning area; no treatment is required in the safe area.

[0024] Further, the risk assessment model is expressed as: ; ; ; ; ; where, is the clearance distance score; is the risk-free reference distance; is the health risk score; is the density risk score; are the minimum and maximum density values respectively; is the wind deflection risk score; is the wind speed, are all preset weight coefficients.

[0025] An intelligent tree obstacle detection system based on multi-modal perception, including a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, it specifically executes the steps in the above-mentioned intelligent tree obstacle detection method based on multi-modal perception.

[0026] In an example, the sensors include an RGB camera, a LIDAR, a multi-spectral camera, a GPS+RTK module, and an IMU module, specifically as follows: The RGB camera obtains high-definition visual information, that is, RGB images; the LIDAR collects three-dimensional point cloud data of tree obstacles and power grids, that is, LIDAR point clouds, for modeling the spatial distance relationship between trees and wires; the multi-spectral camera detects the health status of leaves and identifies diseased trees that may be toppled or dead; the GPS+RTK module realizes centimeter-level real-time position calibration and clearly records the position and spatial distribution of each tree; the IMU module enables stable flight in real time and compensates for errors caused by the movement and vibration of the sensors due to the UAV.

[0027] Specifically, a deep learning object detection algorithm (such as YOLOv8 or an improved Faster R-CNN) is specifically used to locate tree targets in the RGB image, and the two-dimensional position of each tree is obtained by detecting the bounding box.

[0028] As described above, it is only a preferred embodiment of the present invention and not a limitation of the present invention in other forms. Any person skilled in the art may use the disclosed technical content to make changes or modifications into equivalent embodiments with equivalent changes. However, any simple modification, equivalent change, and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution content of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. An intelligent tree obstacle detection method based on multi-modal perception, characterized in that, It includes the following steps: S1: Collect data through sensors carried by drones, obtain sensor data, and preprocess the sensor data to form a standardized input; among them, the sensor data includes RGB images, LIDAR point clouds, multispectral data, and drone data; S2: Perform Hough transform on the LIDAR point cloud in the standardized input, extract the primary position of the power line, and generate a three-dimensional line model of the wire; S3: Use a deep learning object detection algorithm to locate the tree target bounding box in the RGB image in the standardized input, and combine it with the three-dimensional line model of the wire to screen the high-risk point cloud area in the space near the wire of the power line; S4: According to the high-risk point cloud area in the space near the wire of the power line, construct a three-dimensional tree model, and through a collision detection algorithm, calculate the minimum safe clearance distance between the power line and the tree in real time from the three-dimensional tree model to obtain a three-dimensional tree model with safety distance annotation; S5: Based on the three-dimensional tree model with safety distance annotation, perform spatial risk assessment.

2. The intelligent tree obstacle detection method based on multi-modal perception according to claim 1, characterized in that The specific preprocessing of the sensor data in step S1 to form a standardized input is as follows: Spatio-temporal synchronization processing, based on timestamps, perform normalization processing on sensor data with different acquisition frequencies to ensure spatio-temporal correlation; Spatial registration and coordinate system unification, use a checkerboard calibration board, and solve the rotation matrix R and translation vector t through joint calibration. The condition is expressed as: ； Among them, represents a point in the camera coordinate system, represents a point in the radar coordinate system; During this process, the Levenberg-Marquardt algorithm is adopted to minimize the reprojection error, expressed as: ; Among them, is the RGB camera projection model, is the coordinate of the th corner point in the RGB image, that is, the actual observation point in the camera coordinate system; is the number of point pairs of the matching points in the camera coordinate system and the radar coordinate system, represents the minimum evaluation function; represents the th three-dimensional point in the radar coordinate system; Use the Brown-Conrady model to process the RGB image to correct lens distortion, expressed as: ； Among them, represents the radial distance from the point to the optical axis, is the radial distortion coefficient, and is the actual pixel position of the RGB image obtained after correction; is the pixel position of the RGB image obtained before correction; Voxel downsampling is performed on the LIDAR point cloud, and the LIDAR point cloud is divided into voxel grids with side lengths . The centroid is retained for each voxel, which is expressed as: ; Among them, is the centroid of the voxel ; is the total number of points in the voxel v, is the voxel coordinates of each point within; Based on the SIFT algorithm, perform feature matching on the multispectral data to achieve multispectral band alignment and obtain the aligned multispectral data; Using the extended Kalman filter, the and of the UAV data are fused to obtain the latitude, longitude, altitude and attitude data of the UAV; represents the real-time differential positioning data of the UAV data; IMU represents the inertial measurement data of the UAV data; Based on this, a standardized output is formed and formalized into a tuple for time synchronization , which is expressed as: ； Among them, is the corrected RGB image; is the downsampled LIDAR point cloud; is the aligned multispectral data; is the latitude, longitude and elevation, is the attitude data.

3. The intelligent tree obstacle detection method based on multi-modal perception according to claim 2, wherein Using the extended Kalman filter, fuse the and of the UAV data to obtain the latitude, longitude, altitude and attitude data of the UAV; specifically: Obtaining the state vector of a drone based on the longitude, latitude, altitude, and attitude data of the drone , where is the position of the drone; is the velocity of the drone; is the attitude quaternion of the drone, and the attitude quaternion is used to represent spatial rotation; represents the transpose; By driving the prediction step, expressed as: ； ； ； wherein, is the position information of the UAV at time ; is the speed of the UAV at time ; is the attitude quaternion of the UAV at time ; is the time increment; represents the rotation matrix for rotating from the coordinate system of the IMU to the global coordinate system, which is calculated from the attitude quaternion of the UAV at time ; is the acceleration of the UAV at time ; is the gravitational acceleration; is the quaternion multiplication; is the angular velocity of the UAV at time ; According to the GPS+RTK observation update step of the drone data, expressed as: ； Among them, is the position information of the UAV measured in real time by GPS+RTK, which is used to correct the cumulative error of the IMU; is the observation matrix; is the UAV at time state vector.

4. The intelligent tree obstacle detection method based on multi-modal perception according to claim 3, wherein The specific generation of the three-dimensional line model of the wire in step S2 is as follows: Represent the electric wire of the power line as a straight-line parametric equation in 3D space , expressed as: ； Among them, is the parametric equation of a straight line is any point on is the parametric equation of a straight line is the reference point on is the parametric equation of a straight line is the direction vector of is the parametric equation of a straight line is the straight-line parameter of The Hough parameter space is defined as: ; Meet the condition: ； wherein, is the distance from the straight line to the origin; is the azimuth angle of the straight line in the XY plane; is the pitch angle of the straight line with respect to the XY plane; represents the normal vector of the straight line; represents any point on the straight line; For each point , calculate the voting value in the Hough parameter space, expressed as: ； For the roughly selected straight lines detected by Hough, minimize the residual of the distance from the point to the straight line through non-linear optimization, expressed as: ； Among them, is the number of LIDAR point clouds of the wire; The wire presents a parabolic sag under its own weight, and the height h is distributed along the arc length s, expressed as: ； Among them, represents the height at the arc length s, is the starting height of the wire; w is the weight per unit length of the wire; is the horizontal tension of the wire; is the span, that is, the horizontal distance between the support points of the wire; Fitting the sag point cloud data, through weighted least squares fitting of w and , expressed as: ； Among them, represents the predicted wire height, is the point the projected arc length along the wire's line direction; M represents the total number of LIDAR points of the wire; represents the minimum evaluation function; Finally, the three-dimensional circuit model of the wire is represented as follows: 。 5. The intelligent tree obstacle detection method based on multi-modal perception according to claim 4, characterized in that, S3 is specifically as follows: Use a deep learning object detection algorithm to locate the tree target in the RGB image in the standardized input, and obtain the two-dimensional position of each tree in the image by detecting the target bounding box; Combine the three-dimensional line model and the LIDAR point cloud to map the tree target in the RGB image to the corresponding LIDAR point cloud; According to the three-dimensional line model of the wire Construct a spatial search area; specifically, with the wire as the center, establish a cylindrical safety area, and the parameters of the cylindrical safety area are the radius r and the length of the cylinder, and the length is the corresponding span; The radius r is defined as follows: ； Make the following judgment on each point of the LIDAR point cloud: Query each point set near the wire in the LIDAR point cloud through KD-Tree, retain the points within the cylindrical safety area, and further screen in combination with the tree point cloud clusters mapped from the RGB image corresponding to each bounding box; Calculate the minimum distance from the tree point cloud cluster to the wire by using the formula for the shortest distance from a point to a line, which is also the minimum clearance distance; expressed as: ； Among them, represents the tree point cloud cluster; Perform risk determination, specifically as: Set the safety threshold If , it is marked as a high-risk tree point cloud cluster, that is, a high-risk point cloud area in the space near the wire.

6. The intelligent tree obstacle detection method based on multi-modal perception according to claim 5, characterized in that S4 is specifically as follows: S41: Cluster the LIDAR point cloud in the high-risk point cloud area through the DBSCAN algorithm to segment out the single-tree point cloud clusters; S42: Extract the trunk in the height direction from the single-tree point cloud clusters, and use the RANSAC algorithm to fit the trunk into a cylindrical model; S43: Separate the tree trunk and branches / leaves according to the local geometric attributes of the LIDAR point cloud, and utilize spatial continuity to start growing from the tree trunk and classify adjacent points into the crown part; S44: Generate a gradient field based on the normal vectors of the LIDAR point cloud, construct an implicit function by solving the Poisson equation based on the gradient field, perform Poisson surface reconstruction, and fit to obtain the tree surface model; S45: Fuse the corrected RGB image into the aligned multi-spectral data. Specifically, calculate the vegetation index using the aligned multi-spectral data, and assign texture mapping to the mesh surface of the tree surface model using the corrected RGB image to obtain a three-dimensional tree model; S46: Use the constructed three-dimensional power line model and three-dimensional tree model, and calculate the safe clearance distance through a collision detection algorithm to obtain a three-dimensional tree model with safety distance markings.

7. A multi-modal perception-based intelligent tree obstacle detection method according to claim 6, characterized in that S5 specifically includes: Extract tree features from a tree model with safety distance markings, where the tree features include tree height and tree density ; and combine with wind speed to calculate the dynamic minimum clearance distance; Construct a feature vector based on this , which is expressed as: ； Among them, is the annual average growth rate of trees; is the minimum clearance distance; is the health score, obtained by calculating the NDVI value through multispectral; is the tension factor; Construct a risk assessment model, and output a risk level R for each tree based on the LIDAR point cloud and the three-dimensional tree model, expressed as: ； Among them, is the risk score, obtained according to the risk assessment model; Provide maintenance suggestions for the risk level of each tree. Specifically, perform pruning or removal in the dangerous area; conduct regular monitoring in the warning area; no treatment is required in the safe area.

8. The intelligent tree obstacle detection method based on multi-modal perception according to claim 7, characterized in that The risk assessment model is expressed as: ； ； ； ； ； Among them, is the clearance distance score; is the risk-free reference distance; is the health risk score; is the density risk score; are the minimum and maximum density values respectively; is the wind deviation risk score; is the wind speed, are all preset weight coefficients.

9. An intelligent tree obstacle detection system based on multimodal perception, characterized in that, It includes a processor, a memory, and a computer program stored on the memory. When the processor executes the computer program, it specifically executes the steps in a multi-modal perception-based intelligent tree obstacle detection method according to any one of claims 1-8.

Images