Forest patrol method and device based on unmanned aerial vehicle, electronic equipment and medium

Abstract

The invention discloses a forest patrol method and device based on an unmanned aerial vehicle, electronic equipment and a medium, and relates to the technical field of forest resource management. The unmanned aerial vehicle carrying a laser radar and image acquisition equipment is controlled to acquire initial laser radar data and initial image data according to a preset flight route; the three-dimensional perception of the complex three-dimensional environment of the forest is realized, and the concealment abnormity is effectively captured. Performing data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data; and carrying out feature fusion on the target laser radar data and the target image data. Through data preprocessing and multi-source data fusion analysis, the precision of capturing subtle anomalies is improved, and the limitation of a single sensing means is overcome. And finally, forest anomaly recognition is performed based on the fusion features, the geographic information system data and the multi-source sensor data, so that the comprehensive sensing capability of forest anomaly is comprehensively improved, and the forest patrol effect is optimized.

Description

Technical Field

[0001] The present invention relates to the technical field of forest resource management, and in particular to a forest inspection method, device, electronic equipment and medium based on an unmanned aerial vehicle (UAV). Background Art

[0002] Traditional artificial forest patrol methods are constrained by factors such as complex terrain, labor costs, and patrol scope, and have problems such as low efficiency and long monitoring cycles. They are unable to meet the needs of real-time and dynamic monitoring of large forest areas, and are unable to detect subtle or hidden abnormalities in a timely manner, resulting in poor patrol accuracy.

[0003] With the development of drone technology, drone-based remote sensing monitoring has gradually been applied to forest inspections. However, current mainstream drone forest inspection technology still faces the problem of insufficient accuracy. Inspection solutions that rely on a single sensor have obvious limitations. For example, relying solely on visible light / infrared cameras to obtain two-dimensional images is limited by the two-dimensional perspective and cannot accurately capture key spatial information such as forest canopy height and tree density. It is also easy to miss abnormal conditions such as pests and diseases and small fire sources hidden under dense vegetation. While the use of lidar can obtain three-dimensional data, due to its complex data processing and susceptibility to environmental interference when extracting forest features, it cannot accurately identify subtle structural changes or early abnormalities in the forest. Summary of the Invention

[0004] The present invention provides a forest patrol method, device, electronic equipment and medium based on drones, which solves the technical problem that existing forest patrol methods lack the comprehensive perception ability of hidden and subtle anomalies in the complex three-dimensional environment of the forest.

[0005] A first aspect of the present invention provides a forest inspection method based on an unmanned aerial vehicle, comprising: Control the UAV equipped with lidar and image acquisition equipment to follow the preset flight route to collect initial lidar data and initial image data; Performing data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data; Performing feature fusion on the target lidar data and the target image data to generate fusion features; Forest anomalies are identified based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data.

[0006] Optionally, the step of performing data preprocessing on the initial lidar data and the initial image data to generate target lidar data and target image data includes: Performing point cloud denoising on the initial lidar data using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency check to generate intermediate lidar data; Performing radiation correction on the initial image data using a preset atmospheric scattering model to generate intermediate image data; The intermediate lidar data and the intermediate image data are temporally and spatially aligned to generate target lidar data and target image data.

[0007] Optionally, the step of performing point cloud denoising on the initial lidar data using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency check to generate intermediate lidar data includes: Divide the initial lidar data into space-time cubes to generate multiple space-time cubes; Calculating point density in the space-time cube to generate point density distribution data; Calculating the low-density component probability value corresponding to each point density in the point density distribution data by using a Gaussian mixture model to generate noise probability mapping data; Performing dynamic threshold filtering on the noise probability map data to generate target denoised point cloud data; Extracting curvature features from the target denoised point cloud data to generate curvature feature data; The target denoised point cloud data is divided into regions according to the curvature feature data, and filtered according to the regional partitions to generate intermediate lidar data.

[0008] Optionally, the step of performing spatiotemporal registration on the intermediate lidar data and the intermediate image data to generate target lidar data and target image data includes: Performing multimodal feature extraction on the intermediate lidar data to generate point cloud geometric feature data; Performing semantic segmentation on the intermediate image data to generate multiple semantic features; Performing weighted fusion of the point cloud geometric feature data and the semantic features of the corresponding area to generate multiple semantic fusion features; Using the semantic fusion features to construct a matrix and generate an initial transformation matrix; Updating the initial transformation matrix using preset semantic constraints to generate an intermediate transformation matrix; Optimizing the intermediate transformation matrix using a point-to-surface iterative closest point algorithm to generate a target transformation matrix; The intermediate lidar data and intermediate image data corresponding to the target transformation matrix are used as target lidar data and target image data.

[0009] Optionally, the step of performing feature fusion on the target lidar data and the target image data to generate fusion features includes: Dividing the target lidar data into multi-resolution voxel grids to generate multi-scale point cloud feature matrices of different scales; Extracting local geometric features of the multi-scale point cloud feature matrix to generate point cloud multi-scale feature data; Projecting the multi-scale feature data of the point cloud to the image coordinate system through the camera intrinsic parameters and extrinsic parameters to generate a projected point cloud feature set; Perform bilinear interpolation on the projected point cloud feature set to generate a feature candidate set; According to the spatial coordinate weight of the region corresponding to the target image data, the original spectral features of the target image data are weightedly fused with the feature candidate set to generate a fused feature.

[0010] Optionally, the step of identifying forest anomalies based on the fusion features, geographic information system data, and multi-source sensor data to generate forest patrol data includes: Extracting features from the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features; Using the auxiliary features of the geographic information system to set dynamic weights and generate weight data; Performing weighted splicing on the standardized fusion features, the geographic information system auxiliary features, and the environmental perception features according to the weight data to generate a comprehensive feature; The comprehensive features are judged to be abnormal by a preset lightweight classification model to generate forest inspection data.

[0011] Optionally, the step of extracting features from the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features includes: Normalizing the spatial coordinates of the fused features to generate standardized fused features; Extract terrain factors and vegetation type data from GIS data to generate GIS auxiliary features; Perform spatiotemporal matching on multi-source sensor data to generate environmental perception features.

[0012] A second aspect of the present invention provides a forest inspection device based on an unmanned aerial vehicle, comprising: The data acquisition module is used to control the UAV equipped with the lidar and image acquisition equipment to collect initial lidar data and initial image data according to the preset flight route; A data preprocessing module, configured to perform data preprocessing on the initial lidar data and the initial image data to generate target lidar data and target image data; A fusion feature generation module, used to perform feature fusion on the target lidar data and the target image data to generate a fusion feature; The forest patrol data generation module is used to identify forest anomalies based on the fusion features, geographic information system data and multi-source sensor data, and generate forest patrol data.

[0013] A third aspect of the present invention provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the virtual impedance control parameter optimization method as described in any one of the above items.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the virtual impedance control parameter optimization method as described in any one of the above items.

[0015] It can be seen from the above technical solutions that the present invention has the following advantages: The present invention controls an unmanned aerial vehicle (UAV) equipped with a laser radar and an image acquisition device to collect initial laser radar data and initial image data according to a preset flight route. The initial laser radar data and initial image data are then preprocessed to generate target laser radar data and target image data. The target laser radar data and target image data are then feature-fused to generate fusion features. Finally, forest anomalies are identified based on the fusion features, geographic information system data, and multi-source sensor data to generate forest patrol data. The present invention collects data and fuses features using an unmanned aerial vehicle (UAV) equipped with a laser radar and an image acquisition device, thereby achieving stereoscopic perception of the complex three-dimensional environment of the forest and effectively capturing hidden anomalies. Combining data preprocessing with multi-source data fusion analysis improves the accuracy of capturing subtle anomalies and overcomes the limitations of a single perception method. By integrating three-dimensional features, geographic information, and multi-source sensor data for anomaly identification, the comprehensive perception capability of forest anomalies is comprehensively improved, and the forest patrol effect is optimized. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1A flowchart of a forest inspection method based on a drone provided in Example 1 of the present invention; Figure 2 A flowchart of a forest inspection method based on a drone provided in the second embodiment of the present invention; Figure 3 This is a structural block diagram of a forest inspection device based on a drone provided in Example 3 of the present invention; Figure 4 This is a structural block diagram of a computer device provided in Example 4 of the present invention. DETAILED DESCRIPTION

[0018] The embodiments of the present invention provide a forest patrol method, device, electronic equipment and medium based on drones, which are used to solve the technical problem that existing forest patrol methods lack the comprehensive perception ability of hidden and subtle anomalies in the complex three-dimensional environment of the forest.

[0019] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] See also Figure 1 , Figure 1 This is a flowchart of the steps of a forest inspection method based on a drone provided in Example 1 of the present invention.

[0021] The present invention provides a forest inspection method based on an unmanned aerial vehicle, comprising: Step 101: Control a drone equipped with a lidar and an image acquisition device to collect initial lidar data and initial image data according to a preset flight route.

[0022] The drone is an industrial-grade hexacopter with the following specifications: a maximum flight time of 40 minutes (when equipped with dual sensors); wind resistance of ≤12m / s (Force 6 wind), suitable for the complex airflow conditions of forests; and an integrated RTK-GNSS positioning system with centimeter-level positioning accuracy (±1cm horizontally, ±2cm vertically), ensuring data temporal and spatial coordinate consistency. A dual-sensor mounting bracket is used, with the lidar and image acquisition equipment rigidly connected to ensure a field of view overlap of ≥80% to prevent spatial data misalignment.

[0023] The LiDAR system uses a lightweight solid-state LiDAR (such as the Livox Mid-40) with the following parameters: a point cloud density of ≥ 200 points / m2 (at a flight altitude of 100m), which meets the requirements for analyzing the three-dimensional structure of the forest canopy; a ranging range of 5-200m, which can penetrate medium-density vegetation (leaf area index LAI ≤ 3) to obtain understory terrain data; and a wavelength of 905nm (near-infrared band) to reduce interference from direct sunlight and improve data quality on cloudy days.

[0024] The image acquisition device uses a 6-band camera with the following parameters: a resolution of 5 million pixels, bands covering visible light (red, green, and blue) and near-infrared (705nm and 840nm), and support for calculating NDVI (Normalized Difference Vegetation Index); a frame rate ≥10fps, matching the flight speed of the drone (ensuring heading overlap ≥80%).

[0025] A preset flight route refers to building a three-dimensional forest scene model based on the geographic information system (GIS), combining sensor performance parameters and monitoring accuracy requirements to automatically generate an optimal route with comprehensive coverage and low data redundancy.

[0026] In this embodiment of the present invention, a preset route is entered through ground station software, and the drone autonomously executes the "takeoff → cruise → data collection → return" process, supporting breakpoint-resumeable flight (collection can resume from the breakpoint after a signal interruption or low battery). The drone, equipped with a lidar and image acquisition equipment, follows the preset flight route to collect initial lidar data containing three-dimensional coordinates (X, Y, Z), reflection intensity values, and timestamps, as well as initial image data containing spectral reflectance values, temperature values ​​(thermal infrared), and GPS coordinates. This initial lidar data lays the foundation for subsequent extraction of tree characteristics such as height and diameter at breast height. The initial image data also provides a basis for analyzing vegetation health.

[0027] Step 102: pre-process the initial lidar data and the initial image data to generate target lidar data and target image data.

[0028] Target lidar data includes three-dimensional coordinates, reflection intensity values, timestamps and trajectory information, curvature and density characteristics, etc.

[0029] The target image data includes multispectral reflectance values, thermal infrared temperature values, geographic coordinates and time synchronization information, semantic segmentation preprocessing features, etc.

[0030] In this embodiment of the present invention, a hybrid filtering algorithm based on density clustering and spatiotemporal consistency verification is used to perform point cloud denoising on the initial LiDAR data, generating intermediate LiDAR data. Radiometric correction is performed on the initial image data using a preset atmospheric scattering model, generating intermediate image data. The intermediate LiDAR data and intermediate image data are then spatiotemporally aligned to generate target LiDAR data and target image data.

[0031] It is worth mentioning that the hybrid filtering based on density clustering and spatiotemporal consistency verification can accurately remove discrete noise from the initial point cloud while retaining effective information such as tree structure, thereby improving the signal-to-noise ratio of the intermediate lidar data. By performing radiation correction through a preset atmospheric scattering model, the interference of factors such as atmospheric scattering and uneven illumination on the initial image can be eliminated, the true spectral characteristics of vegetation can be restored, the spectral accuracy of the intermediate image data can be improved, and the accuracy of feature extraction such as vegetation health status can be ensured. The intermediate lidar data and image data are spatiotemporally aligned to unify their spatiotemporal benchmarks, solving the spatiotemporal misalignment problem of different sensor data, so that the generated target lidar data and target image data can accurately correspond to the same scene, providing spatiotemporally consistent multi-dimensional data support for subsequent feature fusion and forest anomaly recognition.

[0032] Step 103: Fusing the target lidar data and the target image data to generate fusion features.

[0033] In an embodiment of the present invention, target lidar data is divided into a multi-resolution voxel grid to generate a multi-scale point cloud feature matrix at different scales. This method captures local geometric features of the point cloud at different scales and effectively extracts multi-dimensional, subtle spatial structural information within complex forest environments. Local geometric features are extracted from the multi-scale point cloud feature matrix to generate multi-scale feature data for the point cloud. The multi-scale feature data is projected into the image coordinate system using camera intrinsic and extrinsic parameters to generate a projected point cloud feature set. The projected point cloud feature set is then supplemented with bilinear interpolation to generate a feature candidate set. This bilinear interpolation generates a dense and continuous set of feature candidates, resolving the mismatch between the feature distributions of the point cloud and image coordinate systems and providing rich and aligned basic data for feature fusion. The original spectral features of the target image data are weightedly fused with the feature candidate set according to the spatial coordinate weights of the corresponding regions of the target image data to generate fused features. By organically combining the target image spectral features with the multi-scale geometric features of the point cloud, the fused features incorporate both spectral and three-dimensional spatial structural information, significantly enhancing the comprehensive characterization of forest concealment and subtle anomalies.

[0034] Step 104: Identify forest anomalies based on the fused features, geographic information system data, and multi-source sensor data to generate forest inspection data.

[0035] Geographic information system data refers to the basic data set used to store and process spatial geographic information of forest areas (such as terrain, vegetation distribution, infrastructure, vegetation type) and environmental factors (such as slope, altitude, and canopy density), thereby providing spatial benchmarks and environmental constraints for forest anomaly identification.

[0036] Multi-source sensor data refers to real-time environmental parameters (such as temperature, humidity, and wind speed) and dynamic monitoring data (such as surface temperature and soil moisture content) collected by various sensors (such as meteorological sensors, thermal infrared cameras, and soil moisture sensors) deployed on drones, ground monitoring stations, and other equipment, thereby providing a comprehensive data set of real-time environmental status information for forest anomaly identification.

[0037] In an embodiment of the present invention, first, feature extraction is performed on the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features, thereby realizing structured processing and standardized expression of multi-source data. Next, dynamic weights are set using geographic information system auxiliary features to generate weight data, so that the importance of features in different regions can be adaptively adjusted according to differences in the geographical environment, thereby enhancing the spatial pertinence and environmental adaptability of feature fusion. Then, weighted splicing is performed on the standardized fusion features, geographic information system auxiliary features, and environmental perception features according to the weight data to generate comprehensive features. Finally, anomalies of the comprehensive features are determined using a preset lightweight classification model to generate forest patrol data. The combination of weighted splicing based on dynamic weights and the lightweight classification model reduces computational complexity while retaining the complementary advantages of multi-source features, achieves efficient and accurate determination of forest anomalies, and improves the reliability and processing efficiency of patrol data.

[0038] In an embodiment of the present invention, a drone equipped with a lidar and imaging device is controlled to operate along a preset flight path, achieving three-dimensional coverage of a forest scene. Initial lidar data containing the three-dimensional structure of trees and initial image data with spectral / radiometric information are obtained, overcoming the traditional monitoring blind spots in dense forests and steep slopes, providing multimodal raw data for hidden anomalies. Data preprocessing is performed on the initial lidar and image data to remove noise from the lidar data while preserving subtle structural features. Image spectral distortion is corrected and weak signal features are enhanced. While ensuring strict spatiotemporal alignment between the two types of data, this overcomes the problem of feature misinterpretation caused by noise interference and spatiotemporal misalignment in a single data set, providing a high-precision foundation for subsequent analysis. The target lidar and image data are deeply fused to generate fused features that combine structural and functional information. Based on these fused features, combined with the spatial environmental constraints provided by geographic information system data and the real-time status reflected by multi-source sensor data, collaborative analysis of multi-source data allows for the precise detection and location of subtle and hidden anomalies at the individual tree level, generating inspection data that includes anomaly type and location. This solves the technical problem that existing forest inspection methods lack the comprehensive perception ability of hidden and subtle anomalies in the complex three-dimensional environment of the forest.

[0039] See also Figure 2 , Figure 2 A flowchart of the steps of a forest inspection method based on a drone is provided in Example 2 of the present invention.

[0040] The present invention provides a forest inspection method based on an unmanned aerial vehicle, comprising: Step 201: Control a drone equipped with a lidar and an image acquisition device to collect initial lidar data and initial image data according to a preset flight route.

[0041] In this embodiment of the present invention, ground station software imports a flight route generated using a GIS (Geographic Information System) in KML (Keyhole Markup Language) format, including waypoint coordinates, flight altitude, flight speed, and flight mode. A drone equipped with a lidar and image acquisition device is controlled to take off vertically from a reference point, climb to a target altitude along a preset route, and enter autocruise mode. During flight, the drone receives real-time commands from the ground station and automatically adjusts its flight attitude based on real-time wind speed to ensure sensor stability, thereby consistently capturing initial lidar and image data along the flight route.

[0042] Step 202: Use a hybrid filtering algorithm based on density clustering and spatiotemporal consistency verification to perform point cloud denoising on the initial lidar data to generate intermediate lidar data.

[0043] Furthermore, step 202 includes the following sub-steps: S11. Divide the initial lidar data into space-time cubes to generate multiple space-time cubes.

[0044] In this embodiment of the present invention, initial lidar data is converted into structured space-time units through dynamic parameter configuration and regularized gridding. First, the cube side length L is set to 0.01H + 5 based on the flight altitude H. A space-time cube is constructed using a 10-second time window, with a spatial resolution of UTM (Universal Transverse Mercator) coordinate system 1. Edge computing units are used to sort the initial data by timestamp and filter outliers. The spatial region is then divided into variable-scale grids, with boundary points assigned using distance-weighted distribution to avoid information loss. The time dimension is segmented using a sliding window (5-second step size), and low-point count windows (<500 points) are merged. Finally, a space-time cube in HDF5 format is generated, containing 3D coordinates, reflection intensity, and time series information.

[0045] It's worth noting that the HDF5 format (Hierarchical Data Format Version 5) is a flexible, efficient, cross-platform data storage format designed specifically for managing and organizing large-scale, complex scientific data. Compared to existing technologies, the HDF5 format used in this paper can integrate multiple data types through a hierarchical structure, improve storage access efficiency through efficient compression and block-based reading and writing, and is compatible with the entire process tool chain and embeds metadata to ensure traceability, effectively addressing issues such as data fragmentation, inefficient processing, and poor cross-platform adaptability.

[0046] S12. Calculate the point density in the space-time cube and generate point density distribution data.

[0047] In an embodiment of the present invention, spatial voxelization and multi-scale statistical analysis are used to accurately quantify the point cloud density in the space-time cube: a voxelized grid with a resolution of 0.1m is used (each voxel is a 0.1m×0.1m×0.1m cube), and the space-time cube in HDF5 format is preprocessed based on the PCL point cloud library. After removing invalid points, the three-dimensional voxels are divided by hash index, and the number of point clouds in a single voxel and a 3×3×3 neighborhood is counted. The global average density (reflecting the overall sparsity of the region) and the local neighborhood density (suppressing dynamic noise) are calculated respectively. After smoothing with Gaussian filtering, the point density distribution data in HDF5 format containing the voxel-level density matrix and the global density value are finally generated, and low-density abnormal areas are simultaneously marked. Low-density abnormal areas refer to areas with a local density of less than 100 points / m 3 And the area with ≥5 consecutive voxels.

[0048] It is worth mentioning that the processing equipment uses an edge computing unit equipped with an Intel i7 processor. The Intel i7 processor is Intel's mid-to-high-end desktop / mobile CPU. It has multiple cores, high main frequency and hyperthreading technology, supports parallel computing and high-speed data processing, and is suitable for the real-time operation of complex algorithms.

[0049] S13. Calculate the low-density component probability value corresponding to each point density in the point density distribution data through a Gaussian mixture model to generate noise probability mapping data.

[0050] It should be noted that the Gaussian mixture model is a machine learning model based on probability statistics that can effectively describe the probability distribution patterns of different categories in complex data. Therefore, by using the Gaussian mixture model, the present invention can adaptively learn density distribution patterns based on historical data of different forest terrains and vegetation types (for example, distinguishing naturally sparse areas in mountainous areas from abnormally low density areas caused by pests and diseases), thus solving the problem that a single threshold cannot adapt to complex scenarios.

[0051] In the embodiment of the present invention, the probability analysis of the point density distribution data is performed by using a Gaussian mixture model. The specific analysis process is as follows: first, two Gaussian components are preset (corresponding to the "normal high-density area" and the "low-density noise area", respectively), and the mean is initialized based on historical data (about 300 points / m in the normal area). 3 、Noise area: about 80 points / m 3 ) and covariance parameters, the expectation maximization algorithm is used to iteratively train the standardized point density distribution data, and the posterior probability of each point density in the point density distribution data belonging to low-density noise is calculated, that is, the possibility that the voxel is noise. During the training process, the log-likelihood value change is less than 10 -4 Or reaching 100 iterations is the termination condition, and finally the noise probability mapping data with the same resolution as the space-time cube is generated. The data records the noise probability of each spatial unit, with a value range of 0-1, and marks the strong noise area with a probability exceeding 0.9.

[0052] S14. Perform dynamic threshold filtering on the noise probability map data to generate target denoised point cloud data.

[0053] It should be noted that the point cloud density change rate threshold refers to the density of the denoised point cloud (the number of points per unit volume) compared to the density before denoising, which cannot change by more than 15%. This threshold is set to avoid excessive removal of valid point cloud data during the denoising process, ensuring that sufficient critical information is retained and preventing the loss of excessive valid data from affecting subsequent analysis. For example, if the original point cloud density is 100 points / cubic meter, the density after denoising must be no less than 85 points / cubic meter (100 - 100 × 15% = 85); otherwise, it will be considered substandard.

[0054] The residual noise threshold specifies that the ratio of the number of noise points remaining after denoising must not exceed 5% of the total number of denoised point clouds. This threshold ensures thorough noise removal and prevents residual noise points from interfering with subsequent processing. For example, if the total number of denoised point clouds is 1000, the number of noise points must be limited to 50 (1000 x 5% = 50). If this number exceeds 50, reprocessing is required through secondary filtering.

[0055] In the embodiment of the present invention, the noise probability mapping data is adaptively screened by a dynamic threshold filtering algorithm, which is specifically implemented as follows: first, a three-dimensional noise probability matrix is ​​read from the HDF5 file corresponding to the noise probability mapping data, and the global noise probability mean is calculated. and standard deviation , based on the formula Dynamically generate the current space-time cube exclusive threshold, which ranges from 0.4 to 0.7, to avoid the adaptive deviation of the fixed threshold. Use the edge computing unit to use the point cloud library and the open source computer vision library to perform conditional judgment on each voxel: if the noise probability , then retain the corresponding point cloud data and attach the confidence label , otherwise it is marked as noise point removal. Finally, the initial denoised point cloud data in PLY format is generated. While generating the initial denoised point cloud data, the point cloud density change rate and noise residual rate are calculated in real time. The calculation formula corresponding to the point cloud density change rate is: ; in, is the point density after denoising, that is, the ratio of the number of retained point clouds to their volume; is the original point density before denoising (unit: points / m 3 ), which is obtained by counting the ratio of the number of all point clouds to the volume in the space-time cube.

[0056] Compare the calculated density change rate with the point cloud density change rate threshold: If the calculated density change rate is less than or equal to the point cloud density change rate threshold, it means that the point density after denoising remains above 85% of the original density, and sufficient valid point clouds are retained, meeting the "structural integrity" requirements.

[0057] If the calculated density change rate is greater than the point cloud density change rate threshold, it is determined that valid points are excessively lost, and adjustments need to be triggered to expand the retention range.

[0058] The calculation formula corresponding to the noise residual rate is: ; in, is the number of noise points remaining after denoising, that is, the noise probability But the points that are not eliminated are found by traversing all the retained point clouds. And count the number of points exceeded; is the total number of points after denoising. Compare the calculated noise residual rate with the noise residual rate threshold: If the calculated noise residual rate is less than or equal to the noise residual rate threshold, it means that the noise is completely removed, avoiding interference with subsequent feature extraction.

[0059] If the calculated noise residual rate is greater than the noise residual rate threshold, it is determined that the noise removal is insufficient and an adjustment is triggered to improve the removal standard.

[0060] If the point cloud density change rate is less than or equal to the density change rate, and the noise residual rate meets the standard and is less than or equal to the noise residual rate, the preliminary denoising data is directly confirmed as the final result; if only the density change rate exceeds the standard (>15%), it means that too many valid points are lost. By reducing the adjustment coefficient k (such as from 1.5 to 1.2), the threshold is lowered, the point cloud retention range is expanded and secondary filtering is performed until the standard is met; if only the noise residual rate exceeds the standard (>5%), it means that the noise removal is insufficient. By increasing k (such as to 1.8), the threshold is increased, the removal standard is strictly enforced and secondary filtering is performed; if both indicators do not meet the standard, the valid points are retained first and then the noise is removed. After each adjustment, recalibrate and retry up to 3 times to finally generate the target denoised point cloud data that meets the requirements that the point cloud density change rate is less than or equal to the density change rate and the noise residual rate meets the standard and is less than or equal to the noise residual rate, so as to ensure a dynamic balance between valid point retention and noise removal.

[0061] S15. Extract curvature features from the target denoised point cloud data to generate curvature feature data.

[0062] In an embodiment of the present invention, the three-dimensional coordinates of each point in the target denoised point cloud data are obtained and the confidence label is retained. The higher the value of the label, the greater the probability that the point is a valid signal. In order to eliminate the influence of absolute position, the coordinates of all points are subtracted from the center of mass of the scene, that is, the average value of the coordinates of all points, and converted into relative coordinates. When searching for the neighboring points of each point, higher weights are given to points with high confidence, that is, points with a probability of more than 80% of valid signals, to reduce the interference of noise points on the calculation of neighboring areas. The kd tree data structure is used to quickly find the k nearest neighbors of each point. Here, 20 neighboring points are preset. If the proportion of noise points in the neighboring points exceeds 30%, it is automatically increased to 30 neighboring points to ensure the reliability of the calculation of local geometric features.

[0063] Based on the neighboring point set of each point, the local covariance matrix is ​​calculated to describe the distribution characteristics of the point cloud, and three eigenvalues ​​are obtained by eigenvalue decomposition (the first eigenvalue is sorted from small to large). , the second eigenvalue , the third eigenvalue ), respectively represent the distribution discreteness of the local point cloud in different directions. The first eigenvalue is used , the second eigenvalue , the third eigenvalue Calculate the mean curvature and Gaussian curvature for each point in the target denoised point cloud data. The mean curvature reflects the average curvature of the local surface, while the Gaussian curvature describes the curvature type of the surface (e.g., flat, convex, or concave).

[0064] It is worth mentioning that the mean curvature H The calculation formula is: ; in, H ≈0 indicates a flat / cylindrical surface, such as a tree trunk; H >0.1 indicates severe curvature, such as the edge of a leaf.

[0065] Gaussian curvature K The calculation formula is: ; in, K >0 is convex, such as the top of a tree crown; K <0 is a saddle surface, such as the fork of a tree branch.

[0066] Count all points H The mean , standard deviation ,as well as K The mean , standard deviation , use these means and standard deviations to construct the screening range, so H Falling between, at the same time K Falling Finally, the points that meet the range are H 、 K , three-dimensional coordinates, confidence, as well as the mean value, standard deviation and other information just calculated, are packaged together into a binary file to obtain the curvature feature data.

[0067] S16. Divide the target denoised point cloud data into regions according to the curvature feature data, and perform filtering according to the regional partitions to generate intermediate lidar data.

[0068] In this embodiment of the present invention, curvature feature data is used to process the target denoised point cloud data. First, the DBSCAN clustering algorithm is used, with a neighborhood radius of 0.3 meters and a minimum number of cluster points of 15. The point cloud is then divided into different regions, such as trunks (low curvature), canopies (high curvature), and ground, based on the point cloud's 3D coordinates, mean curvature H, Gaussian curvature K, and retention confidence. Corresponding region labels are then generated. Subsequently, a filtering strategy is selected based on the curvature characteristics of each region. For low-curvature trunks and ground regions (with an absolute value of H less than 0.1), Gaussian filtering is used to smooth noise while preserving smooth structures such as cylindrical surfaces. For high-curvature leaves and branch bifurcations (with an absolute value of H greater than or equal to 0.1), bilateral filtering is used to remove noise while preserving edge details. Finally, intermediate lidar data is output, along with the region labels and filtered coordinates.

[0069] Step 203: Perform radiation correction on the initial image data using a preset atmospheric scattering model to generate intermediate image data.

[0070] It should be noted that the preset atmospheric scattering model is a mathematical model based on physical mechanisms, which is used to simulate the absorption and scattering process of light radiation by the atmosphere.

[0071] In an embodiment of the present invention, the process of radiometrically correcting initial image data using a preset atmospheric scattering model (such as MODTRAN) to generate intermediate image data is as follows: First, the initial image's band information and geometric parameters at the time of imaging are analyzed. These geometric parameters include the solar / observation zenith angle and relative azimuth. Atmospheric parameters, including aerosol optical depth, water vapor content, and ozone column concentration, are simultaneously acquired. The image DN value is converted to apparent radiance through radiometric calibration. The atmospheric scattering model is then used to calculate atmospheric upwelling radiation, transmittance, and spherical albedo. The surface reflectance is inverted using the radiative transfer equation to eliminate the effects of atmospheric scattering. A cosine correction model is introduced for areas with undulating terrain to compensate for shadow effects. Finally, each pixel is corrected row by row to generate intermediate image data containing the radiometrically corrected reflectance or radiance. The atmospheric model parameters, geometric parameters, and calibration coefficients are recorded in the header file to support subsequent quantitative analysis applications such as vegetation index calculation and land feature classification.

[0072] Step 204: perform spatiotemporal registration on the intermediate lidar data and the intermediate image data to generate target lidar data and target image data.

[0073] Furthermore, step 204 includes the following sub-steps: S21. Perform multimodal feature extraction on the intermediate lidar data to generate point cloud geometric feature data.

[0074] In an embodiment of the present invention, a method combining a deep neural network with geometric feature calculation is used to process intermediate lidar data to generate point cloud geometric feature data: the PointNet++ network architecture is used, the voxel grid size is set to 0.05 meters, the number of sampling points at the farthest point is 1024, and geometric features are extracted step by step through the hidden layer dimensions of the multi-layer perceptron [64, 128, 256]. At the same time, based on the voxel grid division results, features including normal vectors, curvature features, and local density are calculated in each voxel, and the initial feature vector is constructed in combination with the regional label. Then, after multi-scale aggregation, that is, using a spherical neighborhood radius of 0.1 meters and 0.3 meters and an attention mechanism for weighting, the local geometric features are spliced ​​with deep semantic features to generate point cloud geometric feature data. It should be noted that the normal vector is obtained by solving the plane fitting of the point set in the voxel. The curvature feature refers to the curvature calculated from the spatial distribution of the point set in the voxel neighborhood. The local density refers to the ratio of the number of points in the voxel to the voxel volume.

[0075] S22. Perform semantic segmentation on the intermediate image data to generate multiple semantic features.

[0076] In an embodiment of the present invention, the intermediate image data after radiation correction is first preprocessed, using median filtering to reduce noise and normalize the spectral reflectance; then, spectral features, texture features, and geometric features are extracted to form pixel feature vectors. A random forest classifier is trained with sufficient samples, and after classifying the feature vectors, morphological processing is performed to optimize the results by removing noise, connecting regions, and removing small patches. Finally, multiple semantic features with geographic coordinates are generated to record the category information of the ground objects. This method efficiently processes intermediate image data and improves classification accuracy through multi-feature fusion. The generated semantic features can be aligned with lidar data and are suitable for detailed analysis of scenes such as agriculture and forestry.

[0077] S23. Perform weighted fusion on the point cloud geometric feature data and the semantic features of the corresponding area to generate multiple semantic fusion features.

[0078] In an embodiment of the present invention, a universal transverse Mercator projection coordinate system is first used to perform spatial coordinate alignment on the geometric feature data of the point cloud and the semantic features of the corresponding area, so that the three-dimensional position of the point cloud accurately corresponds to the feature category label of the semantic map; a geometric feature vector is extracted for each point and the semantic category vector of the corresponding position is associated, which is weighted and merged according to a preset weight after standardization, and then integrated with the geometric mean and semantic mode within a 0.3-meter neighborhood to form a multidimensional feature vector that simultaneously contains geometric structure, feature category and spatial association, and generates multiple semantic fusion features, which have both point cloud geometric details and image semantic information.

[0079] It should be noted that the preset weight refers to the fusion ratio parameter assigned to the geometric features and semantic features during the weighted fusion process of the point cloud geometric feature data and the semantic features of the corresponding area. It is used to balance the contribution of the two types of features in the final semantic fusion feature. The default weight of geometric features is 50%-60%, that is, the weight is 0.5-0.6; the weight of semantic features is 40%-50%, that is, the weight is 0.4-0.5. This ratio can be adjusted according to specific task requirements.

[0080] S24. Use semantic fusion features to construct a matrix and generate an initial transformation matrix.

[0081] In an embodiment of the present invention, the point cloud coordinates and fused feature vectors in the semantic fusion features are first separated, and the feature vectors are standardized to eliminate dimensional differences; then the Euclidean distance is calculated for the numerical geometric features and neighborhood statistical features, and the cosine similarity is calculated for the semantic one-hot encoding vector, and the two are fused with weights of 0.6 and 0.4 to obtain a comprehensive similarity; then the similarity threshold is set to 0.3 to construct a sparse matrix, which is normalized to form a probability transfer matrix so that the sum of the elements in each row is 1; finally, the probability transfer matrix is ​​reduced in dimension using principal component analysis, retaining 90% of the variance to obtain the initial transformation matrix, and its effectiveness is ensured through verification to ensure its effective compression and transformation of the feature space.

[0082] S25. Update the initial transformation matrix using the preset semantic constraints to generate an intermediate transformation matrix.

[0083] It should be noted that preset semantic constraints refer to the use of pre-defined land feature category information, such as labels such as "forest", "farmland", and "building" as prior knowledge, to impose directional constraints on the feature transformation process.

[0084] In the embodiment of the present invention, firstly, a semantic consistency loss function is constructed based on the feature category label (in the form of one-hot encoding) in the semantic fusion feature, and a joint optimization target is formed by combining the geometric similarity preservation loss. (Default 0.5, range 0.2-0.8) Controls the weight of the influence of semantic information on matrix updates; uses the Adam optimizer to iteratively optimize the objective function (terminating at a maximum of 200 iterations), forcing similar points to cluster in the transformed feature space in each update, with points whose mean cosine similarity is greater than 0.9, and separating heterogeneous points to improve category discrimination; after gradient descent optimization, calculates the category separation index to verify the effectiveness of the constraints, requiring that the constraint effectiveness be improved by ≥ 20% compared to the initial matrix, and finally outputs the intermediate transformation matrix with additional constraint parameter metadata.

[0085] S26. Use the point-to-surface iterative closest point algorithm to optimize the intermediate transformation matrix and generate the target transformation matrix.

[0086] In the embodiment of the present invention, the intermediate transformation matrix, the semantic fusion features of the source point cloud and the target point cloud are used as input, the intermediate transformation matrix is ​​used to reduce the dimension of the semantic fusion features of the source point cloud, and the normal vector of the point is calculated; the maximum number of iterations is set to 50 times, and the root mean square error change rate is < As the iteration termination condition, with 0.5 meters as the corresponding point search radius, according to the geometric feature weight of 0.7 and the semantic constraint weight of 0.3, the nearest point set of the same category within the source point search radius is found in the target point cloud; the error function is defined based on the point-to-surface distance formula, and the optimal rotation matrix and translation vector are solved by singular value decomposition, and the transformation matrix is ​​iteratively updated until the preset termination condition is met; finally, the median of the plane and elevation errors is calculated to verify the registration accuracy, requiring the registration error of similar objects to be ≤0.05 meters (plane) / 0.1 meters (elevation), and the target transformation matrix containing information such as the rotation matrix, translation vector, and registration error is obtained.

[0087] It should be noted that the error function is a core metric for measuring point cloud registration accuracy. In this step, it is constructed using point-to-plane distance and semantic constraint weights to quantify the difference in spatial position between the source and target point clouds after transformation. Singular value decomposition is a matrix decomposition technique used to solve for the optimal rotation matrix and translation vector, and is the core mathematical tool of the iterative closest point (ICP) algorithm. The preset termination condition is a rule that controls the maximum number of iterations, the root mean square error rate of change, and the absolute error threshold to achieve efficient termination while maintaining registration accuracy.

[0088] S27. Use the intermediate lidar data and intermediate image data corresponding to the target transformation matrix as target lidar data and target image data.

[0089] In this embodiment of the present invention, coordinate transformation of the intermediate LiDAR data and geometric correction of the intermediate image data are performed based on the target transformation matrix, aligning both to the UTM coordinate system and meeting registration accuracy requirements. Ultimately, the target LiDAR data with transformed metadata and the target image data in the updated geographic reference system are output, providing highly aligned multimodal data for subsequent tasks.

[0090] Step 205: Fusing the target lidar data and the target image data to generate fusion features.

[0091] Furthermore, step 205 includes the following sub-steps: S31. Divide the target lidar data into multi-resolution voxel grids to generate a multi-scale point cloud feature matrix.

[0092] In an embodiment of the present invention, the multi-level voxel side length is first set, for example, covering three levels of resolution from centimeters to meters. The input is the target lidar data with unified spatial coordinates ( ). During execution, the data is first converted to a local coordinate system and a three-dimensional voxel grid is initialized, dividing the space into different resolution levels. Then, geometric and semantic features are calculated within each voxel. Each voxel feature is mapped to an internal point cloud through nearest neighbor interpolation. After normalization, the multi-level resolution features are fused to form a multi-scale feature vector for each point. Finally, the feature vectors of all points are arranged in rows to generate a multi-scale point cloud feature matrix with a dimension of N × D, where N is the number of points in the point cloud and D is the dimension of the fused features.

[0093] S32. Extract local geometric features of the multi-scale point cloud feature matrix to generate point cloud multi-scale feature data.

[0094] In an embodiment of the present invention, based on the three-dimensional coordinates in the matrix, a set of neighboring points is first searched for for each point within a preset three-level adaptive radius to construct a local coordinate system. Next, basic geometric features such as normal vectors, principal curvatures, and point density, as well as higher-order structural features such as linearity and flatness, are calculated. The geometric feature vectors generated by the three-level neighborhood are then concatenated with the original features by column to form an enhanced feature matrix, which is then normalized. Feature validity is then verified by calculating feature repeatability and discriminability metrics. Finally, the verified feature matrix is ​​stored as an indexed binary file, with metadata such as neighborhood parameters and feature dimensions appended to form multi-scale feature data for the point cloud. This data, while retaining the global semantic information of the original voxel features, supplements the geometric details from the microscopic to the mesoscopic level, directly supporting subsequent projection fusion steps.

[0095] S33. Project the multi-scale feature data of the point cloud to the image coordinate system through the camera intrinsic parameters and extrinsic parameters to generate a projected point cloud feature set.

[0096] In an embodiment of the present invention, the three-dimensional coordinates in the multi-scale feature data of the point cloud are first obtained, and the camera extrinsic parameters, including the rotation matrix and the translation vector, are used to transform the point cloud coordinates from the world coordinate system to the camera coordinate system to obtain the three-dimensional coordinates under the camera perspective; then, the camera intrinsic parameters, including the focal length, principal point coordinates and other camera inherent properties, are used to perform perspective projection, and the three-dimensional coordinates in the camera coordinate system are converted into pixel coordinates in the image coordinate system to form a projection point set corresponding to the image pixel position; the multi-scale features of the point cloud, such as three-dimensional geometric features, semantic features, multi-resolution voxel features, etc., are completely retained during the projection process, and finally a projected point cloud feature set is generated, wherein each projection point records its pixel coordinates in the image and the corresponding multi-dimensional feature vector, providing an accurate spatial correspondence for subsequent cross-modal feature fusion.

[0097] S34. Perform bilinear interpolation to supplement the projected point cloud feature set to generate a feature candidate set.

[0098] In an embodiment of the present invention, the sub-pixel coordinates of the projection point in the image coordinate system are first obtained, which may be a non-integer pixel position, and the four nearest integer pixels around it are determined. Then, the corresponding weights are calculated based on the horizontal and vertical distances between the sub-pixel position and the four neighboring pixels, with the closer the distance, the higher the weight. For the multi-scale feature vector carried by each projection point, the feature values ​​of the neighboring pixel positions are linearly interpolated according to the above weights to fill in the feature information of the non-projection point positions. Finally, a feature candidate set covering all pixels of the image is generated, so that each pixel corresponds to a vector containing the interpolated multi-scale point cloud features, effectively solving the problem of feature loss caused by data sparsity after point cloud projection.

[0099] S35 , performing weighted fusion on the original spectral features of the target image data and the feature candidate set according to the spatial coordinate weights of the region corresponding to the target image data, to generate fused features.

[0100] In an embodiment of the present invention, the Euclidean distance of each pixel in the feature candidate set to the adjacent point cloud projection point is first calculated based on its spatial coordinates, and a spatial coordinate weight is obtained, where the closer the distance, the higher the weight; then the original spectral features of the corresponding pixels are extracted and matched with the multi-scale point cloud features in the feature candidate set according to the dimensions; finally, the weighted summation is performed so that each pixel forms a fused feature vector containing spectral and point cloud features. All vectors are combined in pixel order to generate a fused feature covering the entire image, which has both spectral resolution and spatial structure information and can be directly used for subsequent analysis.

[0101] Step 206: Identify forest anomalies based on the fused features, geographic information system data, and multi-source sensor data to generate forest patrol data.

[0102] Furthermore, step 206 includes the following sub-steps: S41. Extract features from fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features.

[0103] Furthermore, step S41 includes the following sub-steps: S411. Standardize the spatial coordinates of the fused features to generate standardized fused features.

[0104] In an embodiment of the present invention, the original spatial coordinates carried in the fusion feature are first read, and based on a preset coordinate conversion relationship, the original coordinates are converted into a unified geodetic coordinate system; the accuracy of the converted coordinates is checked, and abnormal points beyond the valid geographic range are eliminated; the original spectral and point cloud feature information in the fusion feature is retained, and only its spatial coordinate attributes are updated to form a standardized fusion feature in which each pixel or feature point is associated with a geodetic coordinate, ensuring that the data format is fully compatible with the coordinate system of subsequent geographic information system data and multi-source sensor data.

[0105] It should be noted that the preset coordinate transformation relationship is used to transform the original image pixel coordinates or local three-dimensional coordinates of the fusion features into a unified geodetic coordinate system coordinate to ensure the consistency of the spatial reference of multi-source data.

[0106] S412: Extract terrain factors and vegetation type data from the geographic information system data to generate geographic information system auxiliary features.

[0107] In an embodiment of the present invention, a geographic information system database interface is first called to clip data according to the spatial range of the forest patrol area, and terrain factors and vegetation type data are screened out; the terrain factors are rasterized, that is, the resolution is matched with the fusion features, such as 1 meter × 1 meter, and the vegetation type data is converted into a one-hot encoded vector; finally, a geographic information system auxiliary feature containing terrain quantization values ​​and vegetation type codes is formed, and the feature vector of each spatial position corresponds one-to-one to the coordinates of the standardized fusion feature.

[0108] It should be noted that terrain factors include slope, aspect, and elevation, with an accuracy of ≤1 meter. Vegetation type data includes tree / shrub classifications and dominant tree species codes, using the national standard classification system.

[0109] S413: Perform spatiotemporal matching on multi-source sensor data to generate environmental perception features.

[0110] In an embodiment of the present invention, multi-source sensor data carried by a drone is first collected, and the timestamp and spatial coordinates of each data are extracted, where the spatial coordinates are based on the longitude and latitude obtained by the sensor positioning module; data interpolation is performed based on the timestamp to fill the sampling interval differences, and grid matching is performed based on the spatial coordinates to ensure that the resolution is consistent with the fusion features; key parameters related to forest anomalies, such as air humidity, carbon dioxide concentration, infrared radiation intensity, etc., are screened, and the spatiotemporally aligned data are combined into feature vectors according to parameter types to form environmental perception features covering the patrol area, ensuring that the features of each spatial position correspond one-to-one to the standardized fusion features and the auxiliary features of the geographic information system in the spatiotemporal dimensions.

[0111] S42. Use geographic information system auxiliary features to set dynamic weights and generate weight data.

[0112] In this embodiment, terrain factors and vegetation type data are extracted from auxiliary features of a geographic information system. Weights are assigned to different geographical conditions according to preset allocation rules. Weight coefficients for vegetation type features are pre-set based on the ecological importance of the vegetation type. The terrain grade at each spatial location is combined with the weight value corresponding to the vegetation type to generate weight data that corresponds one-to-one with the geographic coordinates. Dynamic weighting is achieved through direct rule configuration without complex calculations, ensuring that the weights reflect the impact of the geographic environment on the importance of the feature.

[0113] It should be noted that preset allocation rules refer to the weight allocation logic pre-set before a forest patrol mission begins, based on forestry knowledge and historical experience. These include terrain factor weighting rules and vegetation type weighting rules. Terrain factor weighting rules directly assign weight levels based on the physical meaning of terrain parameters such as slope and altitude. Vegetation type weighting rules pre-set basic weight coefficients based on the ecological conservation value or economic value of vegetation.

[0114] Ecological importance refers to the degree to which a vegetation type or geographic area is critical to a forest ecosystem.

[0115] Preset vegetation type features refer to standardized vegetation classification codes pre-defined in geographic information system data.

[0116] S43. Perform weighted splicing on the standardized fusion features, geographic information system auxiliary features and environmental perception features according to the weight data to generate comprehensive features.

[0117] In an embodiment of the present invention, the standardized fusion features, geographic information system auxiliary features, and environmental perception features are first aligned according to a unified spatial coordinate to ensure that the dimensions of the three types of feature vectors at each location match; then, for each spatial location, the three types of features are weighted and summed according to a preset weight ratio; finally, the weighted feature vectors are spliced ​​in spatial order to generate comprehensive features covering the entire patrol area. The comprehensive features of each location simultaneously contain spectral, structural, geographic, and environmental information.

[0118] It should be noted that the preset weight ratio refers to the numerical ratio pre-set based on the terrain factor level and ecological importance of vegetation type in the auxiliary features of the geographic information system, which is used to quantify the contribution of standardized fusion features, auxiliary features of the geographic information system, and environmental perception features in different geographical areas.

[0119] S44. Use a preset lightweight classification model to determine abnormalities in the comprehensive features and generate forest inspection data.

[0120] It should be noted that the preset lightweight classification model is a pre-trained, simplified model. This model, trained on historical forest anomaly samples, can accurately identify abnormal patterns within comprehensive features. Deployed on equipment such as drones, it can rapidly translate comprehensive features into anomaly results, directly supporting the generation of forest inspection data and serving as a core tool for anomaly determination.

[0121] In an embodiment of the present invention, comprehensive features including spectral, topographic, and environmental information are input into a pre-trained lightweight classification model. The model directly outputs the abnormality category of each spatial location, such as "normal", "suspected pests and diseases", "lodging risk", and the corresponding confidence level based on the input features; and synchronously associates the geographic coordinates of the abnormal results to generate forest patrol data in an industry standard format.

[0122] See also Figure 4 , Figure 4 This is a structural block diagram of a forest patrol device based on a drone provided in Example 3 of the present invention.

[0123] The present invention provides a forest inspection device based on an unmanned aerial vehicle, comprising: The data acquisition module 301 is used to control the UAV equipped with the laser radar and image acquisition equipment to collect initial laser radar data and initial image data according to a preset flight route; A data preprocessing module 302 is used to preprocess the initial lidar data and the initial image data to generate target lidar data and target image data; The fusion feature generation module 303 is used to perform feature fusion on the target lidar data and the target image data to generate fusion features; The forest patrol data generation module 304 is used to identify forest anomalies based on fusion features, geographic information system data and multi-source sensor data, and generate forest patrol data.

[0124] Furthermore, the data preprocessing module 302 includes: The intermediate lidar data generation module is used to perform point cloud denoising on the initial lidar data using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency verification to generate intermediate lidar data; An intermediate image data generation module is used to perform radiation correction on the initial image data using a preset atmospheric scattering model to generate intermediate image data; The target data generation module is used to perform spatiotemporal registration on the intermediate lidar data and the intermediate image data to generate target lidar data and target image data.

[0125] Furthermore, the intermediate lidar data generation module may perform the following steps: Divide the initial lidar data into space-time cubes to generate multiple space-time cubes; Calculate the point density in the space-time cube and generate point density distribution data; The Gaussian mixture model is used to calculate the probability value of the low-density component corresponding to each point density in the point density distribution data to generate noise probability mapping data; Perform dynamic threshold filtering on the noise probability map data to generate target denoised point cloud data; Extract curvature features from the target denoised point cloud data to generate curvature feature data; The target denoised point cloud data is divided into regions according to the curvature feature data, and filtered according to the regional partitions to generate intermediate lidar data.

[0126] Furthermore, the target data generation module may perform the following steps: Perform multimodal feature extraction on the intermediate lidar data to generate point cloud geometric feature data; Perform semantic segmentation on the intermediate image data to generate multiple semantic features; Perform weighted fusion of point cloud geometric feature data and semantic features of corresponding regions to generate multiple semantic fusion features; Semantic fusion features are used to construct the matrix and generate the initial transformation matrix; Update the initial transformation matrix using preset semantic constraints to generate an intermediate transformation matrix; The point-to-surface iterative closest point algorithm is used to optimize the intermediate transformation matrix and generate the target transformation matrix; The intermediate lidar data and intermediate image data corresponding to the target transformation matrix are used as target lidar data and target image data.

[0127] Furthermore, the fusion feature generation module 303 may perform the following steps: Divide the target lidar data into multi-resolution voxel grids to generate multi-scale point cloud feature matrices of different scales; Extract local geometric features of the multi-scale point cloud feature matrix to generate multi-scale feature data of the point cloud; Project the multi-scale feature data of the point cloud to the image coordinate system through the camera intrinsic and extrinsic parameters to generate a projected point cloud feature set; Perform bilinear interpolation to supplement the projected point cloud feature set to generate a feature candidate set; According to the spatial coordinate weights of the corresponding areas of the target image data, the original spectral features of the target image data are weightedly fused with the feature candidate set to generate fused features.

[0128] Furthermore, the forest patrol data generation module 304 may perform the following steps: Extract features from fusion features, GIS data and multi-source sensor data to generate standardized fusion features, GIS auxiliary features and environmental perception features; Use geographic information system auxiliary features to set dynamic weights and generate weight data; According to the weight data, the standardized fusion features, the GIS auxiliary features and the environmental perception features are weighted and spliced ​​to generate comprehensive features; Through the preset lightweight classification model, the comprehensive features are judged as abnormal and forest inspection data is generated.

[0129] Furthermore, the forest inspection data generation module 304 may further perform the following steps: Normalize the spatial coordinates of the fused features to generate standardized fused features; Extract terrain factors and vegetation type data from GIS data to generate GIS auxiliary features; Perform spatiotemporal matching on multi-source sensor data to generate environmental perception features.

[0130] See also Figure 4 , Figure 4 This is a structural block diagram of a computer device provided in Example 4 of the present invention.

[0131] An electronic device according to an embodiment of the present invention includes: a memory 401 and a processor 402, wherein the memory 401 stores a computer program; when the computer program is executed by the processor 402, the processor 402 executes the drone-based forest patrol method as in any of the above embodiments.

[0132] Memory 401 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Memory 401 has storage space 403 for program code 413 for executing any of the method steps described above. For example, storage space 403 for program code may include individual program codes 413 for implementing various steps in the method described above. These program codes may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. The program codes may be compressed, for example, in a suitable format. When executed by a processing device, these codes cause the processing device to execute the various steps in the method described above. These program codes may be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards, or floppy disks. The program codes may be compressed, for example, in a suitable format. When these codes are executed by a computing and processing device, they cause the computing and processing device to perform the various steps in the UAV-based forest patrol method described above.

[0133] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, a forest inspection method based on a drone as described in any embodiment of the present invention is implemented.

[0134] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0136] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0137] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0139] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A forest inspection method based on drones, characterized in that: include: Control the UAV equipped with lidar and image acquisition equipment to follow the preset flight route to collect initial lidar data and initial image data; Performing data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data; Performing feature fusion on the target lidar data and the target image data to generate fusion features; Forest anomalies are identified based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data.

2. The forest inspection method based on drone according to claim 1 is characterized in that: The step of preprocessing the initial laser radar data and the initial image data to generate target laser radar data and target image data includes: Performing point cloud denoising on the initial lidar data using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency check to generate intermediate lidar data; Performing radiation correction on the initial image data using a preset atmospheric scattering model to generate intermediate image data; The intermediate lidar data and the intermediate image data are temporally and spatially aligned to generate target lidar data and target image data.

3. The forest inspection method based on drone according to claim 2 is characterized in that: The step of performing point cloud denoising on the initial lidar data using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency check to generate intermediate lidar data includes: Divide the initial lidar data into space-time cubes to generate multiple space-time cubes; Calculating point density in the space-time cube to generate point density distribution data; Calculating the low-density component probability value corresponding to each point density in the point density distribution data by using a Gaussian mixture model to generate noise probability mapping data; Performing dynamic threshold filtering on the noise probability map data to generate target denoised point cloud data; Extracting curvature features from the target denoised point cloud data to generate curvature feature data; The target denoised point cloud data is divided into regions according to the curvature feature data, and filtered according to the regional partitions to generate intermediate lidar data.

4. The forest inspection method based on drone according to claim 2, characterized in that: The step of performing spatiotemporal registration of the intermediate lidar data and the intermediate image data to generate target lidar data and target image data comprises: Performing multimodal feature extraction on the intermediate lidar data to generate point cloud geometric feature data; Performing semantic segmentation on the intermediate image data to generate multiple semantic features; Performing weighted fusion of the point cloud geometric feature data and the semantic features of the corresponding area to generate multiple semantic fusion features; Using the semantic fusion features to construct a matrix and generate an initial transformation matrix; Updating the initial transformation matrix using preset semantic constraints to generate an intermediate transformation matrix; Optimizing the intermediate transformation matrix using a point-to-surface iterative closest point algorithm to generate a target transformation matrix; The intermediate lidar data and intermediate image data corresponding to the target transformation matrix are used as target lidar data and target image data.

5. The forest inspection method based on drone according to claim 1, characterized in that: The step of performing feature fusion on the target lidar data and the target image data to generate fusion features includes: Dividing the target lidar data into multi-resolution voxel grids to generate multi-scale point cloud feature matrices of different scales; Extracting local geometric features of the multi-scale point cloud feature matrix to generate point cloud multi-scale feature data; Projecting the multi-scale feature data of the point cloud to the image coordinate system through the camera intrinsic parameters and extrinsic parameters to generate a projected point cloud feature set; Perform bilinear interpolation on the projected point cloud feature set to generate a feature candidate set; According to the spatial coordinate weight of the region corresponding to the target image data, the original spectral features of the target image data are weightedly fused with the feature candidate set to generate a fused feature.

6. The forest inspection method based on drone according to claim 1, characterized in that: The step of identifying forest anomalies based on the fusion features, geographic information system data, and multi-source sensor data to generate forest patrol data includes: Extracting features from the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features; Using the auxiliary features of the geographic information system to set dynamic weights and generate weight data; Performing weighted splicing on the standardized fusion features, the geographic information system auxiliary features, and the environmental perception features according to the weight data to generate a comprehensive feature; The comprehensive features are judged to be abnormal by a preset lightweight classification model to generate forest inspection data.

7. The forest inspection method based on drone according to claim 6, characterized in that: The step of extracting features from the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features includes: Normalizing the spatial coordinates of the fused features to generate standardized fused features; Extract terrain factors and vegetation type data from GIS data to generate GIS auxiliary features; Perform spatiotemporal matching on multi-source sensor data to generate environmental perception features.

8. A forest inspection device based on a drone, characterized in that: include: The data acquisition module is used to control the UAV equipped with the lidar and image acquisition equipment to collect initial lidar data and initial image data according to the preset flight route; A data preprocessing module, configured to perform data preprocessing on the initial lidar data and the initial image data to generate target lidar data and target image data; A fusion feature generation module, used to perform feature fusion on the target lidar data and the target image data to generate a fusion feature; The forest patrol data generation module is used to identify forest anomalies based on the fusion features, geographic information system data and multi-source sensor data, and generate forest patrol data.

9. An electronic device, characterized in that: It includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the drone-based forest patrol method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the drone-based forest inspection method according to any one of claims 1 to 7 is implemented.

Images