A forest patrol method and device based on a UAV, an electronic device, and a medium

Abstract

The application discloses a forest patrol method and device based on a UAV, electronic equipment and medium, and relates to the technical field of forest resource management, which realizes the stereoscopic perception of the complex three-dimensional environment of the forest by controlling the UAV carrying a laser radar and an image acquisition device to collect initial laser radar data and initial image data, and effectively captures hidden abnormalities. Then, the initial laser radar data and the initial image data are subjected to data preprocessing to generate target laser radar data and target image data. The target laser radar data and the target image data are subjected to feature fusion. Through data preprocessing and multi-source data fusion analysis, the capture accuracy of subtle abnormalities is improved, and the limitations of single perception means are overcome. Finally, based on the fusion features, geographic information system data and multi-source sensor data, forest anomaly recognition is carried out, the comprehensive perception ability of the forest anomaly is improved, and the forest patrol effect is optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of forest resource management, and in particular to a forest patrol method and device based on a UAV, an electronic device, and a medium. BACKGROUND

[0002] Traditional manual forest patrol methods have low efficiency, long monitoring periods, and other problems due to constraints such as complex terrain, labor costs, and patrol range, and are difficult to meet the needs of real-time and dynamic monitoring of large forest areas, and cannot timely detect subtle or hidden abnormalities, resulting in poor patrol accuracy.

[0003] With the development of UAV technology, remote sensing monitoring based on UAVs has gradually been applied to the field of forest patrol. However, the current mainstream UAV forest patrol technology still faces the problem of insufficient accuracy. The patrol scheme relying on a single sensor has obvious limitations, for example, relying only on visible light / infrared cameras to obtain two-dimensional images is limited by two-dimensional perspective and is difficult to accurately capture key spatial information such as forest canopy height and tree density, and for abnormalities such as pests and diseases hidden in dense vegetation and small fires, it is easy to miss detection; while using laser radar can obtain three-dimensional data, but due to the complexity of data processing, and the environment interference in extracting forest features, it is difficult to accurately identify subtle structural changes or early abnormal phenomena in the forest. SUMMARY

[0004] The present application provides a forest patrol method and device based on a UAV, an electronic device, and a medium, which solves the technical problem of the lack of comprehensive perception ability for hidden and subtle abnormalities in the complex three-dimensional environment of the forest in the existing forest patrol method.

[0005] The present application provides a forest patrol method and device based on a UAV, an electronic device, and a medium, which solves the technical problem of the lack of comprehensive perception ability for hidden and subtle abnormalities in the complex three-dimensional environment of the forest in the existing forest patrol method.

[0006] The UAV carrying a laser radar and an image acquisition device is controlled to follow a preset flight route to collect initial laser radar data and initial image data;

[0007] The initial laser radar data and the initial image data are preprocessed to generate target laser radar data and target image data;

[0008] The target laser radar data and the target image data are fused to generate fusion features;

[0009] Based on the fusion features, geographic information system data, and multi-source sensor data, forest anomaly recognition is performed to generate forest patrol data.

[0010] Optionally, the step of data preprocessing the initial lidar data and the initial image data to generate target lidar data and target image data comprises:

[0011] performing point cloud denoising on the initial lidar data by using a hybrid filtering algorithm based on density clustering and spatio-temporal consistency verification to generate intermediate lidar data;

[0012] performing radiation correction on the initial image data by using a preset atmospheric scattering model to generate intermediate image data;

[0013] performing spatio-temporal registration on the intermediate lidar data and the intermediate image data to generate target lidar data and target image data.

[0014] Optionally, the step of performing point cloud denoising on the initial lidar data by using a hybrid filtering algorithm based on density clustering and spatio-temporal consistency verification to generate intermediate lidar data comprises:

[0015] performing spatio-temporal cubic division on the initial lidar data to generate a plurality of spatio-temporal cubes;

[0016] calculating point density in the spatio-temporal cubes to generate point density distribution data;

[0017] calculating a low-density component probability value corresponding to each point density in the point density distribution data by using a Gaussian mixture model to generate noise probability mapping data;

[0018] performing dynamic threshold filtering on the noise probability mapping data to generate target denoised point cloud data;

[0019] performing curvature feature extraction on the target denoised point cloud data to generate curvature feature data;

[0020] dividing the target denoised point cloud data according to the curvature feature data, and performing filtering according to the region division to generate intermediate lidar data.

[0021] Optionally, the step of performing spatio-temporal registration on the intermediate lidar data and the intermediate image data to generate target lidar data and target image data comprises:

[0022] performing multi-modal feature extraction on the intermediate lidar data to generate point cloud geometric feature data;

[0023] performing semantic segmentation on the intermediate image data to generate a plurality of semantic features;

[0024] performing weighted fusion on the point cloud geometric feature data and the semantic features of the corresponding region to generate a plurality of semantic fusion features;

[0025] perform matrix construction using the semantic fusion features to generate an initial transformation matrix;

[0026] perform preset semantic constraint update on the initial transformation matrix to generate an intermediate transformation matrix;

[0027] perform point-to-plane iterative closest point algorithm optimization on the intermediate transformation matrix to generate a target transformation matrix;

[0028] corresponding intermediate lidar data and intermediate image data of the target transformation matrix as target lidar data and target image data.

[0029] Optionally, the step of performing feature fusion on the target lidar data and the target image data to generate fusion features comprises:

[0030] perform multi-resolution voxel grid division on the target lidar data to generate multi-scale point cloud feature matrices of different scales;

[0031] extract local geometric features of the multi-scale point cloud feature matrices to generate point cloud multi-scale feature data;

[0032] project the point cloud multi-scale feature data to an image coordinate system through camera intrinsic and extrinsic parameters to generate a projected point cloud feature set;

[0033] perform bilinear interpolation supplementation on the projected point cloud feature set to generate a feature candidate set;

[0034] perform weighted fusion of original spectral features of the target image data and the feature candidate set according to spatial coordinate weights of a corresponding region of the target image data to generate fusion features.

[0035] Optionally, the step of performing forest anomaly recognition based on the fusion features, geographic information system data, and multi-source sensor data to generate forest patrol data comprises:

[0036] perform feature extraction on the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environmental perception features;

[0037] set dynamic weights using the geographic information system auxiliary features to generate weight data;

[0038] perform weighted splicing of the standardized fusion features, the geographic information system auxiliary features, and the environmental perception features according to the weight data to generate comprehensive features;

[0039] perform anomaly determination on the comprehensive features through a preset lightweight classification model to generate forest patrol data.

[0040] Optionally, the step of performing feature extraction on the fusion features, geographic information system data and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features and environment perception features comprises:

[0041] Performing spatial coordinate standardization on the fusion features to generate standardized fusion features;

[0042] Extracting terrain factors and vegetation type data in the geographic information system data to generate geographic information system auxiliary features;

[0043] Performing spatio-temporal matching on the multi-source sensor data to generate environment perception features.

[0044] The second aspect of the present application provides a forest patrol device based on a UAV, comprising:

[0045] A data acquisition module is configured to control the UAV carrying a laser radar and an image acquisition device to collect initial laser radar data and initial image data according to a preset flight route;

[0046] A data preprocessing module is configured to perform data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data;

[0047] A fusion feature generation module is configured to perform feature fusion on the target laser radar data and the target image data to generate fusion features;

[0048] A forest patrol data generation module is configured to perform forest anomaly recognition based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data.

[0049] The third aspect of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to make the processor execute the steps of the virtual impedance control parameter optimization method according to any one of the above aspects.

[0050] The fourth aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the virtual impedance control parameter optimization method according to any one of the above aspects.

[0051] From the above technical solutions, the present application has the following advantages:

[0052] The unmanned aerial vehicle carrying the laser radar and the image acquisition device collects initial laser radar data and initial image data according to the preset flight route. Then, the initial laser radar data and the initial image data are preprocessed to generate target laser radar data and target image data. Then, the target laser radar data and the target image data are fused to generate fusion features. Finally, based on the fusion features, geographic information system data and multi-source sensor data, forest anomaly recognition is performed to generate forest patrol data. The unmanned aerial vehicle carries the laser radar and the image device to collect data and fuse features, realizes the stereoscopic perception of the complex three-dimensional environment of the forest, and effectively captures the hidden anomalies. Combined with data preprocessing and multi-source data fusion analysis, the capture accuracy of subtle anomalies is improved, and the limitations of single sensing means are overcome. By integrating three-dimensional features, geographic information and multi-source sensing data for anomaly recognition, the comprehensive perception ability of forest anomalies is comprehensively improved, and the forest patrol effect is optimized. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0054] Figure 1 A step flow chart of a forest patrol method based on an unmanned aerial vehicle provided by the first embodiment of the present application is shown in the figure.

[0055] Figure 2 A step flow chart of a forest patrol method based on an unmanned aerial vehicle provided by the second embodiment of the present application is shown in the figure.

[0056] Figure 3 A structure block diagram of a forest patrol device based on an unmanned aerial vehicle provided by the third embodiment of the present application is shown in the figure.

[0057] Figure 4 A structure block diagram of a computer device provided by the fourth embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0058] The embodiments of the present application provide a forest patrol method, device, electronic device and medium based on an unmanned aerial vehicle, which are used to solve the technical problem that the existing forest patrol method lacks comprehensive perception ability of hidden and subtle anomalies in the complex three-dimensional environment of the forest.

[0059] In order to make the application purposes, features and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the following described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0060] Please refer to Figure 1 , Figure 1 A step flow chart of a forest patrol method based on a UAV is provided for the first embodiment of the present application.

[0061] The present application provides a forest patrol method based on a UAV, comprising:

[0062] Step 101, control the UAV carrying a laser radar and an image acquisition device to collect initial laser radar data and initial image data according to a preset flight route.

[0063] It should be noted that the UAV is an industrial-grade six-rotor UAV, which has the following parameters: maximum endurance time of 40 minutes (when carrying double sensors); wind resistance level ≤12m / s (6-grade wind), suitable for complex airflow environment in the forest; positioning system is integrated with RTK-GNSS module, with centimeter-level positioning accuracy (horizontal ±1cm, vertical ±2cm), to ensure the consistency of data space coordinates. A double-sensor mounting bracket is used, and the laser radar and the image acquisition device are rigidly connected to ensure that the field of view overlap rate of the two is ≥80%, avoiding spatial misplacement of data.

[0064] The laser radar is a lightweight solid-state laser radar (such as Livox Mid-40), which has the following parameters: point cloud density ≥200 points / ㎡ (when the flight height is 100m), meeting the demand for analyzing the three-dimensional structure of the forest canopy; the ranging range is 5-200m, which can penetrate medium-density vegetation (leaf area index LAI ≤3) to obtain the terrain data under the forest; the wavelength is 905nm (near-infrared band), which reduces the interference of direct sunlight and improves the data quality on cloudy days.

[0065] The image acquisition device is a 6-band camera, which has the following parameters: resolution of 5 million pixels, band coverage of visible light (red, green, blue) and near-infrared (705nm, 840nm), support for calculating NDVI (normalized difference vegetation index); frame rate ≥10fps, matching the flight speed of the UAV (ensuring that the heading overlap degree is ≥80%).

[0066] The preset flight route means that a three-dimensional scene model of the forest is constructed based on a geographic information system (GIS), combined with the performance parameters of the sensors and the monitoring accuracy requirements, to automatically generate an optimal route that covers comprehensively and has low data redundancy.

[0067] In the embodiment of the present application, the unmanned aerial vehicle autonomously performs the process of "take-off -> cruising -> data acquisition -> return" by presetting the route through the ground station software, and supports breakpoint continuation (after interruption due to signal interruption or low power interruption, it can continue to acquire from the breakpoint). The unmanned aerial vehicle carries a laser radar and an image acquisition device to acquire initial laser radar data containing three-dimensional coordinates (X, Y, Z), reflection intensity values and time stamps, and initial image data containing spectral reflection values, temperature values (thermal infrared) and GPS coordinates according to the preset flight route. Through the initial laser radar data, the foundation is laid for subsequent extraction of tree height, diameter at breast height and other features. Through the initial image data, the basis is provided for vegetation health state analysis.

[0068] Step 102, data preprocessing of the initial laser radar data and the initial image data to generate target laser radar data and target image data.

[0069] The target laser radar data includes three-dimensional coordinates, reflection intensity values, time stamps, trajectory information, curvature and density characteristics, etc.

[0070] The target image data includes multispectral reflection values, thermal infrared temperature values, geographic coordinates and time synchronization information, semantic segmentation preprocessing features, etc.

[0071] In the embodiment of the present application, a hybrid filtering algorithm based on density clustering and spatio-temporal consistency verification is used to denoise the initial laser radar data to generate intermediate laser radar data. A preset atmospheric scattering model is used to perform radiation correction on the initial image data to generate intermediate image data. The intermediate laser radar data and the intermediate image data are spatio-temporally registered to generate target laser radar data and target image data.

[0072] It is worth mentioning that the hybrid filtering based on density clustering and spatio-temporal consistency verification can accurately remove discrete noise in the initial point cloud while retaining effective information such as tree structure, so that the signal-to-noise ratio of the intermediate laser radar data is improved. Through the radiation correction by the preset atmospheric scattering model, the interference of atmospheric scattering, uneven illumination and other factors on the initial image can be eliminated, and the real spectral characteristics of the vegetation are restored, so that the spectral accuracy of the intermediate image data is improved, and the accuracy of the extraction of the vegetation health state and other features is ensured. The intermediate laser radar data and the image data are spatio-temporally registered to unify the spatio-temporal reference of the two, solving the spatio-temporal misalignment problem of different sensor data, so that the generated target laser radar data and target image data can accurately correspond to the same scene, providing spatio-temporally consistent multi-dimensional data support for subsequent feature fusion and forest anomaly recognition.

[0073] Step 103, feature fusion of the target laser radar data and the target image data to generate fusion features.

[0074] In the embodiment of the present application, the target laser radar data is divided into a multi-resolution voxel grid to generate multi-scale point cloud feature matrices of different scales, which can capture local geometric features of point clouds from different scales and effectively extract multi-dimensional subtle spatial structure information in complex forest environments. The local geometric features of the multi-scale point cloud feature matrices are extracted to generate point cloud multi-scale feature data. The point cloud multi-scale feature data is projected to the image coordinate system through the camera intrinsic and extrinsic parameters to generate a projected point cloud feature set. The projected point cloud feature set is supplemented by bilinear interpolation to generate a feature candidate set. After the projected point cloud feature set is supplemented by bilinear interpolation, a dense and continuous feature candidate set is generated, which solves the problem of mismatch between point cloud and image coordinate system feature distribution and provides rich and aligned basic data for feature fusion. The original spectral features of the target image data are weighted and fused with the feature candidate set according to the spatial coordinate weight of the corresponding region of the target image data to generate fusion features. By organically combining the target image spectral features with the point cloud multi-scale geometric features, the fusion features have both spectral and three-dimensional spatial structure information, which significantly improves the comprehensive representation ability of forest concealed and subtle anomalies.

[0075] Step 104, based on the fusion features, geographic information system data and multi-source sensor data, forest anomaly recognition is performed to generate forest patrol data.

[0076] Geographic information system data refers to a basic data set for storing and processing spatial geographic information (such as terrain, vegetation distribution, infrastructure, vegetation type) and environmental factors (such as slope, elevation, canopy density) of forest areas, thereby providing spatial reference and environmental constraint conditions for forest anomaly recognition.

[0077] Multi-source sensor data refers to real-time environmental parameters (such as temperature, humidity, wind speed) and dynamic monitoring data (such as land surface temperature, soil moisture content) collected by various sensors (such as weather sensors, thermal infrared cameras, soil moisture sensors) deployed on unmanned aerial vehicles, ground monitoring stations and other equipment, thereby providing comprehensive data sets of real-time environmental state information for forest anomaly recognition.

[0078] In the embodiment of the present application, firstly, the fusion features, geographic information system data and multi-source sensor data are feature extracted to generate standardized fusion features, geographic information system auxiliary features and environment perception features, realizing the structured processing and standardized expression of multi-source data. Then, the geographic information system auxiliary features are used to set dynamic weights to generate weight data, so that the feature importance of different regions can be adaptively adjusted according to the geographical environment difference, enhancing the spatial pertinence and environmental adaptability of feature fusion. Then, the standardized fusion features, geographic information system auxiliary features and environment perception features are weighted and spliced according to the weight data to generate comprehensive features. Finally, the comprehensive features are determined by a pre-set lightweight classification model to generate forest patrol data. The combination of weighted splicing based on dynamic weights and lightweight classification model reduces the computational complexity while retaining the complementary advantages of multi-source features, realizes efficient and accurate determination of forest anomalies, and improves the reliability and processing efficiency of patrol data.

[0079] In the embodiment of the present application, by controlling the unmanned aerial vehicle carrying the laser radar and the image device to operate according to the preset flight route, the stereoscopic coverage of the forest scene is realized, the initial laser radar data containing the three-dimensional structure of the trees and the initial image data containing the spectral / radiation information are obtained, the monitoring blind area of traditional methods in dense forest, steep slope and other areas is broken through, and multi-modal original data for hidden anomalies are provided. By data preprocessing of the initial laser radar data and the initial image data, the noise in the laser radar data is removed and the fine structure features are retained, the image spectral distortion is corrected and the weak signal features are enhanced, while ensuring the strict spatio-temporal alignment of the two types of data, solving the feature misreading problem caused by noise interference and spatio-temporal misalignment of single data, and providing a high-precision basis for subsequent analysis. The target laser radar data and the target image data are deeply fused to generate fusion features with structure information and function information, based on the fusion features, combined with the spatial environment constraints provided by the geographic information system data and the real-time state reflected by the multi-source sensor data, through multi-source data collaborative analysis, accurate detection and positioning of single-level fine anomalies and hidden anomalies are realized, and patrol data containing anomaly type and location are generated. The technical problem that the existing forest patrol method lacks comprehensive perception ability for hidden and fine anomalies in the complex three-dimensional environment of the forest is solved.

[0080] Please refer to Figure 2 , Figure 2 The step flow chart of a forest patrol method based on an unmanned aerial vehicle provided in Embodiment Two of the present application.

[0081] The forest patrol method based on an unmanned aerial vehicle provided by the present application comprises:

[0082] Step 201, control the unmanned aerial vehicle carrying the laser radar and the image acquisition device to collect initial laser radar data and initial image data according to the preset flight route.

[0083] In the embodiment of the present application, the KML (Keyhole Markup Language) format flight route generated based on GIS (Geographic Information System) is imported by the ground station software, including waypoint coordinates, flight height, flight speed and flight mode. The unmanned aerial vehicle carrying the laser radar and image acquisition device vertically takes off from the reference point, climbs to the target height according to the preset route, and enters the automatic cruising mode. Real-time ground station instructions are received during flight, and the flight attitude is automatically adjusted according to the real-time wind speed to ensure the stability of the sensor, so that the initial laser radar data and initial image data on the flight route are stably collected.

[0084] Step 202, using a hybrid filtering algorithm based on density clustering and spatio-temporal consistency checking to denoise the initial laser radar data to generate intermediate laser radar data.

[0085] Further, step 202 includes the following sub-steps:

[0086] S11, dividing the initial laser radar data into a plurality of spatio-temporal cubes.

[0087] In the embodiment of the present application, the initial laser radar data is converted into structured spatio-temporal units through dynamic parameter configuration and regularized grid division: first, based on the flight height H, the cube side length L = 0.01H + 5 is set, and a spatio-temporal cube is constructed in combination with a 10-second time window. The spatial resolution is determined by the UTM (Universal Transverse Mercator) coordinate system I. The edge computing unit is used to sort the initial data by timestamp, filter outliers, and then divide the spatial region according to the variable scale grid. The boundary points are distributed by distance weighting to avoid information loss; the time dimension is divided by a sliding window (step size 5s), and low point windows (<500 points) are merged, finally generating an HDF5 format spatio-temporal cube containing three-dimensional coordinates, reflectivity and time sequence information.

[0088] It is worth mentioning that HDF5 format (Hierarchical Data Format Version 5) is a flexible and efficient cross-platform data storage format designed for managing and organizing large-scale and complex scientific data. Compared with the prior art, the present application uses HDF5 format to integrate multiple types of data through hierarchical structure, efficiently compresses and block reads and writes to improve storage access efficiency, is compatible with the whole process tool chain and embeds metadata to guarantee traceability, effectively solving the problems of data dispersion, low processing efficiency and poor cross-platform adaptation.

[0089] S12, calculate the point density in the space-time cube, and generate point density distribution data.

[0090] In the embodiment of the application, the point cloud density in the space-time cube is accurately quantified through spatial voxelization and multi-scale statistical analysis: a voxelization grid with a resolution of 0.1 m (each voxel is a 0.1 m x 0.1 m x 0.1 m cube) is adopted, the space-time cube in HDF5 format is preprocessed based on the PCL point cloud library, the invalid points are removed, and the three-dimensional voxels are divided by hash index, the number of point clouds in a single voxel and a 3x3x3 neighborhood is counted, the global average density (reflecting the overall sparsity of the region) and the local neighborhood density (suppressing dynamic noise) are calculated respectively, and the final point density distribution data in HDF5 format containing the voxel-level density matrix and the global density value is generated after Gaussian filtering smoothing processing, and the low-density abnormal region is marked synchronously, the low-density abnormal region refers to the region with local density < 100 points / m 3 and continuous voxels ≥ 5.

[0091] It is worth mentioning that the processing device adopts an edge computing unit equipped with an Intel i7 processor, the Intel i7 processor is a mid-to-high-end desktop / mobile central processor launched by Intel, which has multi-core, high main frequency and super-thread technology, supports parallel computing and high-speed data processing, and is suitable for real-time running of complex algorithms.

[0092] S13, calculate the low-density component probability value corresponding to each point density in the point density distribution data by using a Gaussian mixture model, and generate noise probability mapping data.

[0093] It should be noted that the Gaussian mixture model is a machine learning model based on probability statistics, which can effectively describe the probability distribution law of different categories in complex data. Therefore, by using the Gaussian mixture model, the application can adaptively learn the density distribution pattern according to the historical data of different forest terrains and vegetation types (such as distinguishing the natural sparse area in the mountainous area from the abnormal low-density area caused by pests and diseases), and solve the problem that a single threshold cannot adapt to complex scenes.

[0094] In the embodiment of the application, the point density distribution data is analyzed by using the Gaussian mixture model, and the specific analysis process is as follows: first, two Gaussian components (corresponding to "normal high-density area" and "low-density noise area" respectively) are preset, the mean value (about 300 points / m 3 for the normal area and about 80 points / m 3 for the noise area) and the covariance parameter are initialized based on the historical data, the expectation maximization algorithm is used to iteratively train the standardized point density distribution data, and the posterior probability that each point density in the point density distribution data belongs to the low-density noise, i.e. the possibility that the voxel is noise, is calculated. During the training process, the log-likelihood value is less than 10 -4or reaching 100 iterations as the termination condition, finally generating noise probability mapping data with the same resolution as the spatiotemporal cube, which records the noise probability of each spatial unit, with a value range of 0-1, and marks the strong noise area with a probability exceeding 0.9.

[0095] S14, performing dynamic threshold filtering on the noise probability mapping data to generate target denoised point cloud data.

[0096] It should be noted that the point cloud density change rate threshold refers to the change amplitude of the density (the number of points in a unit volume) of the denoised point cloud compared with the density before denoising, which cannot exceed 15%. This threshold is set to avoid excessive removal of effective point cloud data during denoising, to ensure that sufficient key information is retained, and to prevent excessive loss of effective data from affecting subsequent analysis. For example, if the original point cloud density is 100 points per cubic meter, the density after denoising should not be less than 85 points per cubic meter (100-100x15%=85), otherwise it is considered to be substandard.

[0097] The noise residual rate threshold refers to the proportion of the number of noise points remaining after denoising to the total number of point clouds after denoising, which cannot exceed 5%. This threshold is used to ensure the thoroughness of noise removal and avoid interference from noise points that have not been removed in subsequent processing. For example, when the total number of point clouds after denoising is 1000 points, the number of noise points should be controlled within 50 (1000x5%=50), and if it exceeds, it needs to be reprocessed through secondary filtering.

[0098] In the embodiment of the present application, the noise probability mapping data is adaptively screened through a dynamic threshold filtering algorithm, which is implemented as follows: first, read the three-dimensional noise probability matrix from the HDF5 file corresponding to the noise probability mapping data, calculate the global noise probability mean and standard deviation , generate a current spatiotemporal cube-specific threshold based on the formula The value range of the spatiotemporal cube-specific threshold is 0.4-0.7, to avoid the adaptability deviation of the fixed threshold. The edge computing unit uses the point cloud library and the open computer vision library to perform conditional judgment on each voxel: if the noise probability , the corresponding point cloud data is retained and a retention confidence label is added , otherwise it is marked as a noise point for removal. Finally, the initial denoised point cloud data in PLY format is generated, and the point cloud density change rate and the noise residual rate are calculated in real time while generating the initial denoised point cloud data. The calculation formula of the point cloud density change rate is:

[0099] wherein, is the point density after denoising, i.e. the ratio of the number of retained point clouds to the volume; is the original point density before denoising (unit: points / m 3 ​), the ratio of the number of all points in the spatiotemporal cube to the volume is obtained.

[0100] The calculated density change rate is compared with the point cloud density change rate threshold value:

[0101] If the calculated density change rate is less than or equal to the point cloud density change rate threshold value, it indicates that the point density after denoising is maintained above 85% of the original density, the effective point cloud is retained sufficiently, and the requirement of "structural integrity" is met.

[0102] If the calculated density change rate is greater than the point cloud density change rate threshold value, it is determined that the effective points are excessively lost, and adjustment needs to be triggered to expand the retention range.

[0103] The corresponding calculation formula of the noise residual rate is: ;

[0104] wherein, is the number of noise points remaining after denoising, i.e., the noise probability But the points that are not eliminated, by traversing all the retained point clouds and counting the number of points that exceed the standard, are obtained; is the total number of points after denoising. The calculated noise residual rate is compared with the noise residual rate threshold value:

[0105] If the calculated noise residual rate is less than or equal to the noise residual rate threshold value, it indicates that the noise elimination is complete, and interference on subsequent feature extraction is avoided.

[0106] If the calculated noise residual rate is greater than the noise residual rate threshold value, it is determined that the noise elimination is insufficient, and adjustment needs to be triggered to improve the elimination standard.

[0107] If the point cloud density change rate is less than or equal to the density change rate, and the noise residual rate is less than or equal to the noise residual rate, respectively, the preliminary denoising data is directly confirmed as the final result; if only the density change rate exceeds the standard (>15%), it indicates that too many effective points are lost, the adjustment coefficient k is reduced (such as from 1.5 to 1.2) to reduce the threshold value, the point cloud retention range is expanded, and secondary filtering is performed until the standard is met; if only the noise residual rate exceeds the standard (>5%), it indicates that the noise elimination is insufficient, k is increased (such as to 1.8) to increase the threshold value, the elimination standard is strictly eliminated, and secondary filtering is performed; if both indicators do not meet the standard, the effective points are preferentially retained and then the noise is removed. After each adjustment, recheck, a maximum of 3 retries, finally generate the target denoising point cloud data that meets the point cloud density change rate less than or equal to the density change rate, and the noise residual rate less than or equal to the noise residual rate, respectively, to ensure the dynamic balance of effective point retention and noise elimination.

[0108] S15, curvature feature extraction is performed on the target denoising point cloud data to generate curvature feature data.

[0109] In this embodiment of the invention, the three-dimensional coordinates and confidence labels of each point in the target denoised point cloud data are obtained. A higher label value indicates a greater probability that the point is a valid signal. To eliminate the influence of absolute position, the scene centroid is subtracted from the coordinates of all points, i.e., the average of all point coordinates, to convert them into relative coordinates. When searching for neighboring points of each point, points with high confidence, i.e., those with a valid signal probability exceeding 80%, are given higher weights to reduce the interference of noise points on the calculation of the neighboring area. A kd-tree data structure is used to quickly find the k nearest neighbors of each point. Here, 20 neighboring points are selected by default. If the proportion of noise points among the neighboring points exceeds 30%, the number is automatically increased to 30 neighboring points to ensure the reliability of local geometric feature calculation.

[0110] Based on the set of neighboring points for each point, the local covariance matrix is ​​calculated to describe the distribution characteristics of the point cloud. Three eigenvalues ​​are obtained through eigenvalue decomposition (the first eigenvalue is ordered from smallest to largest). Second eigenvalue Third eigenvalue ), representing the degree of dispersion of the local point cloud distribution in different directions, respectively. The first eigenvalue is used. Second eigenvalue Third eigenvalue Calculate the average curvature and Gaussian curvature for each point cloud in the target denoised point cloud data. The average curvature reflects the average degree of curvature of the local surface. The Gaussian curvature describes the type of curvature of the surface (e.g., planar, convex, or concave).

[0111] It is worth mentioning that the mean curvature H The calculation formula is:

[0112] ;

[0113] in, H ≈0 represents a plane / cylindrical surface, such as a tree trunk; H A value greater than 0.1 indicates severe bending, such as at the edge of a blade.

[0114] Gaussian curvature K The calculation formula is:

[0115] ;

[0116] in, K >0 indicates a convex surface, such as the top of a tree canopy; K <0 represents the saddle surface, such as the fork of a tree branch.

[0117] Count all points H mean Standard deviation ,as well as K mean , standard deviation , the screening range is constructed with these mean and standard deviation, let H fall between , while K fall between ; finally, the points that meet the range are packed into binary files with H , K , three-dimensional coordinates, confidence, and the mean and standard deviation just calculated, so as to obtain the curvature feature data.

[0118] S16, according to the curvature feature data, the target denoising point cloud data is regionally divided, and filtering is performed according to the regional partition to generate intermediate lidar data.

[0119] In the embodiment of the application, the target denoising point cloud data is processed by using the curvature feature data: first, the DBSCAN clustering algorithm is used, the neighborhood radius is set to 0.3 meters, the minimum cluster point number is set to 15, the three-dimensional coordinates, the average curvature H, the Gaussian curvature K and the reserved confidence of the point cloud are combined, the point cloud is divided into different regions such as the trunk area (low curvature), the crown layer area (high curvature), the ground area, and corresponding regional labels are generated. Then, according to the curvature features of each region, a filtering strategy is selected: for the low-curvature trunk and ground area (absolute value of H less than 0.1), Gaussian filtering is used to smooth the noise and retain the smooth structure of the cylindrical surface; for the high-curvature leaf and branch bifurcation area (absolute value of H greater than or equal to 0.1), bilateral filtering is used to filter out noise while retaining edge details. Finally, the intermediate lidar data with additional regional labels and filtered coordinates is output.

[0120] Step 203, perform radiation correction on the initial image data by a preset atmospheric scattering model to generate intermediate image data.

[0121] It should be noted that the preset atmospheric scattering model is a mathematical model based on physical mechanism, which is used to simulate the absorption and scattering process of light radiation by the atmosphere.

[0122] In the embodiment of the present application, the process of generating intermediate image data by pre-setting atmospheric scattering model (such as MODTRAN) to perform radiation correction on the initial image data is as follows: first, analyze the band information of the initial image and the geometric parameters at the imaging time, wherein the geometric parameters at the imaging time include the sun / observation zenith angle and the relative azimuth angle. And synchronously obtain the atmospheric parameters, including the aerosol optical thickness, the water vapor content, and the ozone column concentration. Convert the image DN value to apparent radiance through radiation calibration, then call the atmospheric scattering model to calculate the atmospheric uplink radiation, transmittance and spherical albedo, and through the radiation transfer equation, the surface reflectivity is inverted to eliminate the influence of atmospheric scattering, and the cosine correction model is introduced for the terrain fluctuation area to compensate the shadow effect. Finally, each pixel is corrected row by row to generate intermediate image data containing the reflectivity or radiance after radiation correction, and the atmospheric model parameters, geometric parameters and calibration coefficients are recorded in the header file to support subsequent quantitative analysis applications such as vegetation index calculation and ground object classification.

[0123] Step 204, spatiotemporally registering the intermediate lidar data and the intermediate image data to generate target lidar data and target image data.

[0124] Further, step 204 includes the following sub-steps:

[0125] S21, performing multi-modal feature extraction on the intermediate lidar data to generate point cloud geometric feature data.

[0126] In the embodiment of the present application, a method combining deep neural network and geometric feature calculation is used to process the intermediate lidar data to generate point cloud geometric feature data: the PointNet++ network architecture is used, the voxel grid size is set to 0.05 meters, and the number of farthest point samples is set to 1024, and the geometric features are extracted through the multi-layer perception hidden layer dimensions [64, 128, 256] step by step; at the same time, based on the voxel grid division result, the features including normal vector, curvature feature and local density are calculated in each voxel, the initial feature vector is constructed combining the region label, and then through multi-scale aggregation, i.e. using the spherical neighborhood radius of 0.1 meters and 0.3 meters and the attention mechanism weighting, the local geometric features and deep semantic features are spliced to generate the point cloud geometric feature data. It should be noted that the normal vector is obtained by fitting the plane of the point set in the voxel. The curvature feature refers to the spatial distribution curvature calculation result of the point set in the voxel neighborhood. The local density refers to the ratio of the number of points in the voxel to the volume of the voxel.

[0127] S22, performing semantic segmentation on the intermediate image data to generate a plurality of semantic features.

[0128] In the embodiment of the present application, the intermediate image data corrected by radiation is preprocessed, the noise is reduced by median filtering, and the spectral reflectance is normalized; then the spectral features, texture features and geometric features are extracted to form pixel feature vectors. A random forest classifier is trained with sufficient samples, and after the feature vectors are classified, morphological processing is performed to optimize the results, such as denoising, region connection and removing fine spots, to finally generate multiple semantic features with geographic coordinates and record the information of the feature class. This method efficiently processes intermediate image data, improves classification accuracy through multi-feature fusion, and generates semantic features that can be registered with lidar data, suitable for fine analysis in agricultural, forestry and other scenarios.

[0129] S23, weight fusion of the point cloud geometric feature data and the semantic feature of the corresponding region is performed to generate multiple semantic fusion features.

[0130] In the embodiment of the present application, first, the point cloud geometric feature data and the semantic feature of the corresponding region are aligned in space coordinates by using the general transverse Mercator projection coordinate system, so that the three-dimensional position of the point cloud is accurately corresponding to the feature class label of the semantic map; a geometric feature vector is extracted for each point and associated with the semantic class vector of the corresponding position, and after standardization, the weighted combination is performed according to the preset weight, and then the geometric mean and semantic mode in the 0.3-meter neighborhood are fused to form a multi-dimensional feature vector containing geometric structure, feature class and spatial correlation, and multiple semantic fusion features are generated, which have both point cloud geometric details and image semantic information.

[0131] It should be noted that the preset weight refers to the fusion proportion parameter allocated to the geometric feature and the semantic feature in the process of weight fusion of the point cloud geometric feature data and the semantic feature of the corresponding region, which is used to balance the contribution of the two types of features in the final semantic fusion feature. Among them, the geometric feature accounts for 50%-60%, i.e. the weight is 0.5-0.6; the semantic feature accounts for 40%-50%, i.e. the weight is 0.4-0.5, and the proportion can be adjusted according to the specific task requirements.

[0132] S24, matrix construction is performed using the semantic fusion feature to generate an initial transformation matrix.

[0133] In the embodiment of the present application, first, the point cloud coordinates and the fusion feature vectors in the semantic fusion feature are separated, the feature vectors are standardized to eliminate the dimensional differences; then the Euclidean distance is calculated for the numerical geometric features and the neighborhood statistical features, the cosine similarity is calculated for the semantic one-hot encoding vectors, and the comprehensive similarity is obtained by fusing the two with weights of 0.6 and 0.4; then a similarity threshold of 0.3 is set to construct a sparse matrix, which is normalized to form a probability transition matrix, so that the sum of each row elements is 1; finally, the probability transition matrix is reduced by principal component analysis, 90% of the variance is retained, and the initial transformation matrix is obtained, and its effectiveness is verified to ensure its effective compression and transformation of the feature space.

[0134] S25, updating the initial transformation matrix with the preset semantic constraint to generate an intermediate transformation matrix.

[0135] It should be noted that the preset semantic constraint refers to using pre-defined ground object category information, such as using labels such as "forest", "farmland", "building", etc. as prior knowledge to impose directional constraints on the feature transformation process.

[0136] In the embodiment of the application, first, a semantic consistency loss function is constructed based on the ground object category label (one-hot encoding form) in the semantic fusion feature, a geometric similarity preservation loss is combined to form a joint optimization target, and a constraint strength coefficient (default 0.5, range 0.2-0.8) is used to control the influence weight of semantic information on matrix update; an Adam optimizer is used to iteratively optimize the objective function (terminated when the maximum iteration is 200 times), and in each update, the same class points are forced to gather in the transformed feature space, the cosine similarity average of points > 0.9, and the different class points are separated, to improve the class distinction; after gradient descent optimization, the class separation degree index is calculated to verify the effectiveness of the constraint, which requires that compared with the initial matrix, the constraint effectiveness is improved by ≥20%, and finally the intermediate transformation matrix with additional constraint parameter metadata is output.

[0137] S26, using the point-to-plane iterative closest point algorithm to optimize the intermediate transformation matrix to generate a target transformation matrix.

[0138] In the embodiment of the application, the intermediate transformation matrix, the semantic fusion features of the source point cloud and the target point cloud are input, the intermediate transformation matrix is used to reduce the dimension of the semantic fusion features of the source point cloud, and the normal vector of the point is calculated; the maximum iteration number is set to 50 times, the root mean square error change rate is set to <0.001, and the singular value decomposition is used to solve the optimal rotation matrix and translation vector, and the transformation matrix is iteratively updated until the preset termination condition is met; finally, the plane and elevation error median are calculated to verify the registration accuracy, and the registration error of the same class ground object is required to be ≤0.05 meters (plane) / 0.1 meters (elevation), and the target transformation matrix containing the rotation matrix, translation vector, registration error, etc. is obtained.

[0139] ​It should be noted that the error function is a core index for measuring the accuracy of point cloud registration, which is constructed by the point-to-plane distance and the semantic constraint weight in this step, and is used to quantify the spatial position difference of the source point cloud and the target point cloud after transformation. Singular value decomposition is a matrix decomposition technique used to solve the optimal rotation matrix and translation vector, which is the core mathematical tool of the iterative closest point algorithm (ICP). The preset termination condition refers to the rules for efficiently terminating iteration while ensuring registration accuracy by controlling the maximum number of iterations, the root mean square error change rate and the absolute error threshold.

[0140] S27, the intermediate lidar data corresponding to the target transformation matrix and the intermediate image data are taken as target lidar data and target image data.

[0141] In the embodiment of the application, the intermediate lidar data is subjected to coordinate conversion and the intermediate image data is subjected to geometric correction based on the target transformation matrix, so that both are unified to the UTM coordinate system and meet the registration accuracy requirement. The target lidar data with transformation metadata and the target image data with updated geographic reference system are finally output, providing high-precision aligned multi-modal data for subsequent tasks.

[0142] Step 205, feature fusion is performed on the target lidar data and the target image data to generate fused features.

[0143] Further, step 205 includes the following sub-steps:

[0144] S31, the target lidar data is subjected to multi-resolution voxel grid division to generate a multi-scale point cloud feature matrix.

[0145] In the embodiment of the application, first, a multi-level voxel side length is set, such as 3 levels of resolution covering centimeter to meter, and the input is the target lidar data unified in the spatial coordinate system. When executed, the data is first converted to a local coordinate system and a three-dimensional voxel grid is initialized, and the space is divided according to different resolution levels; then the geometric features and semantic features are calculated in each voxel, the voxel features are mapped to the internal point cloud by nearest neighbor interpolation, and after standardization, the multi-level resolution features are fused to form a multi-scale feature vector for each point; finally, the feature vectors of all points are arranged in rows to generate a multi-scale point cloud feature matrix with dimensions NxD, where N is the number of point cloud points and D is the feature dimension after fusion.

[0146] S32, local geometric features of the multi-scale point cloud feature matrix are extracted to generate point cloud multi-scale feature data.

[0147] In the embodiment of the present application, firstly, based on the three-dimensional coordinates in the matrix, each point is searched for a set of adjacent points according to a preset 3-level adaptive radius and a local coordinate system is constructed; then, basic geometric features such as normal vector, principal curvature, point density, and high-order structure features such as linearity and flatness are calculated; then, the geometric feature vectors generated by the 3-level neighborhood are spliced with the original features by column to form an enhanced feature matrix and standardized processing is performed; then, the feature effectiveness is verified by calculating the feature repeatability and distinguishability indexes; finally, the verified feature matrix is stored as an indexed binary file, with metadata such as neighborhood parameters and feature dimensions, to form point cloud multi-scale feature data. The data supplements the geometric details from the micro to the macro on the basis of retaining the global semantic information of the original voxel features, and can directly support the subsequent projection fusion step.

[0148] S33, project the point cloud multi-scale feature data to the image coordinate system through the camera intrinsic and extrinsic parameters to generate a projected point cloud feature set.

[0149] In the embodiment of the present application, firstly, the three-dimensional coordinates in the point cloud multi-scale feature data are acquired, and the point cloud coordinates are converted from the world coordinate system to the camera coordinate system by using the camera extrinsic parameters including the rotation matrix and the translation vector to obtain the three-dimensional coordinates under the camera view angle; then, the three-dimensional coordinates in the camera coordinate system are converted into pixel coordinates in the image coordinate system by perspective projection through the camera intrinsic parameters including the focal length, the principal point coordinates and other camera inherent properties to form a projected point set corresponding to the image pixel positions; the multi-scale features of the point cloud such as the three-dimensional geometric features, the semantic features and the multi-resolution voxel features are completely retained during the projection process, and finally the projected point cloud feature set is generated, wherein each projected point records its pixel coordinates in the image and the corresponding multi-dimensional feature vector, thereby providing accurate spatial correspondence for subsequent cross-modal feature fusion.

[0150] S34, supplement the projected point cloud feature set by bilinear interpolation to generate a feature candidate set.

[0151] In the embodiment of the present application, firstly, the sub-pixel coordinates of the projected point in the image coordinate system are acquired, which may be non-integer pixel positions, and the four nearest integer pixel points around the sub-pixel coordinates are determined; then, the corresponding weights are calculated according to the horizontal and vertical distances between the sub-pixel position and the four adjacent pixels, and the closer the distance, the higher the weight; for the multi-scale feature vector carried by each projected point, the feature values of the adjacent pixel positions are linearly interpolated according to the above-mentioned weights, and the feature information of the non-projected point positions is filled; finally, the feature candidate set covering all the pixels of the image is generated, so that each pixel corresponds to a vector containing the multi-scale point cloud features after interpolation, thereby effectively solving the feature missing problem caused by the sparse data after the point cloud projection.

[0152] S35, according to the spatial coordinate weight of the target image data corresponding region, the original spectral characteristics of the target image data are weighted and fused with the feature candidate set to generate a fused feature.

[0153] In the embodiment of the application, first, the spatial coordinate weight is calculated according to the spatial coordinates of each pixel in the feature candidate set and the Euclidean distance from the adjacent point cloud projection point, so that the closer the distance, the higher the weight of the spatial coordinate weight; then the original spectral characteristics of the corresponding pixel are extracted, and the multi-scale point cloud features in the feature candidate set are matched in dimension; finally, the weighted sum is calculated according to the weight, so that each pixel forms a fused feature vector containing spectral and point cloud features, and all vectors are combined in pixel order to generate a fused feature covering the whole image, which has spectral resolution and spatial structure information and can be directly used for subsequent analysis.

[0154] Step 206, based on the fused feature, geographic information system data and multi-source sensor data, forest anomaly recognition is performed to generate forest patrol data.

[0155] Further, step 206 includes the following sub-steps:

[0156] S41, feature extraction is performed on the fused feature, geographic information system data and multi-source sensor data to generate standardized fused features, geographic information system auxiliary features and environment perception features.

[0157] Further, step S41 includes the following sub-steps:

[0158] S411, the spatial coordinate of the fused feature is standardized to generate a standardized fused feature.

[0159] In the embodiment of the application, first, the original spatial coordinates carried in the fused feature are read, and the original coordinates are converted to a unified geodetic coordinate system based on a preset coordinate conversion relationship; the converted coordinates are subjected to precision verification, and abnormal points exceeding the effective geographic range are removed; the original spectral and point cloud feature information in the fused feature is retained, and only the spatial coordinate attribute is updated to form a standardized fused feature in which each pixel or feature point is associated with a geodetic coordinate, ensuring that the data format is completely compatible with the coordinate system of subsequent geographic information system data and multi-source sensor data.

[0160] It should be noted that the preset coordinate conversion relationship is used to convert the original image pixel coordinates or local three-dimensional coordinates of the fused feature into a unified geodetic coordinate system, so that the spatial reference of the multi-source data is consistent.

[0161] S412, terrain factors and vegetation type data in the geographic information system data are extracted to generate geographic information system auxiliary features.

[0162] In the embodiment of the present application, first, the geographic information system database interface is called, the data is cut according to the spatial range of the forest patrol area, and the terrain factor and vegetation type data are screened out; the terrain factor is rasterized, that is, the resolution is matched with the fusion feature, such as 1m*1m, and the vegetation type data is converted into a one-hot encoding vector; finally, the geographic information system auxiliary features containing the terrain quantitative value and the vegetation type code are formed, and the feature vector of each spatial position corresponds to the coordinates of the standardized fusion feature one by one.

[0163] It should be noted that the terrain factor includes slope, aspect, elevation, etc., and the precision is less than or equal to 1 meter. The vegetation type data includes tree / shrub classification, dominant tree species code, etc., and the national standard classification system is adopted.

[0164] S413, spatiotemporal matching is performed on the multi-source sensor data to generate environment perception features.

[0165] In the embodiment of the present application, first, the multi-source sensor data carried by the unmanned aerial vehicle is collected, the timestamp and spatial coordinates of each data are extracted, and the spatial coordinates are the longitude and latitude obtained based on the sensor positioning module; data interpolation is performed based on the timestamp to fill in the sampling interval difference, and grid matching is performed based on the spatial coordinates to ensure that the resolution is consistent with the fusion feature; key parameters related to forest anomalies, such as air humidity, carbon dioxide concentration, and infrared radiation intensity, are screened out; the spatiotemporally aligned data is combined into a feature vector according to the parameter type to form environment perception features covering the patrol area, and it is ensured that the features of each spatial position correspond to the standardized fusion features and the geographic information system auxiliary features in the spatiotemporal dimension one by one.

[0166] S42, setting dynamic weights using geographic information system auxiliary features to generate weight data.

[0167] In the embodiment of the present application, the terrain factor and vegetation type data are extracted from the geographic information system auxiliary features, and weights are assigned to different geographic conditions according to a preset assignment rule; the weight coefficient of the pre-vegetation type feature is determined according to the ecological importance of the vegetation type. The weight values corresponding to the terrain grade and vegetation type of each spatial position are combined to generate weight data corresponding to geographic coordinates one by one. Without complex calculation, dynamic weight setting is realized directly through rule configuration, and it is ensured that the weight reflects the influence of the geographic environment on the importance of the feature.

[0168] It should be noted that the preset assignment rule refers to the weight assignment logic preset based on forestry field knowledge and historical experience before the start of the forest patrol task. It includes terrain factor weight rules and vegetation type weight rules. The terrain factor weight rule refers to a terrain factor weight rule that directly divides weight levels according to the physical meaning of terrain parameters such as slope and elevation. The vegetation type weight rule refers to a rule for presetting a basic weight coefficient according to the ecological protection value or economic value of vegetation.

[0169] Ecological importance refers to the critical degree of vegetation type or geographical area to the forest ecosystem.

[0170] The preset vegetation type feature refers to a standardized vegetation classification code predefined in geographical information system data.

[0171] S43, the standardized fusion feature, the geographical information system auxiliary feature and the environment perception feature are spliced by weight data, and the comprehensive feature is generated.

[0172] In the embodiment of the application, first, the standardized fusion feature, the geographical information system auxiliary feature and the environment perception feature are aligned according to a unified spatial coordinate, and the dimension of the three types of feature vectors at each position is matched; then, for each spatial position, the three types of features are weighted and summed according to a preset weight ratio; finally, the weighted feature vectors are spliced in spatial order to generate comprehensive features covering the entire patrol area, and the comprehensive features at each position contain spectral, structural, geographical and environmental information.

[0173] It should be noted that the preset weight ratio refers to a numerical ratio predefined based on the terrain factor level in the geographical information system auxiliary feature and the ecological importance of the vegetation type, which is used to quantify the contribution of the standardized fusion feature, the geographical information system auxiliary feature and the environment perception feature in different geographical areas.

[0174] S44, the comprehensive feature is determined by a preset lightweight classification model, and the forest patrol data is generated.

[0175] It should be noted that the preset lightweight classification model is a simple model trained in advance, which can accurately identify the abnormal patterns contained in the comprehensive feature through historical forest abnormal samples. After being deployed on devices such as unmanned aerial vehicles, it can quickly convert from the comprehensive feature to the abnormal result, directly support the generation of forest patrol data, and is the core tool for realizing abnormal judgment.

[0176] In the embodiment of the application, the comprehensive feature containing spectral, terrain and environmental information is input into the pre-trained lightweight classification model, which directly outputs the abnormal category of each spatial position, such as "normal", "suspected disease and insect pests" and "lodging risk", and the corresponding confidence; and synchronously associates the geographical coordinates of the abnormal result, generates the forest patrol data according to the industry standard format.

[0177] Please refer to Figure 4 , Figure 4 A structure block diagram of a forest patrol device based on an unmanned aerial vehicle provided in the third embodiment of the application.

[0178] The forest patrol device based on an unmanned aerial vehicle provided by the application comprises:

[0179] The data acquisition module 301 is configured to control the unmanned aerial vehicle carrying the laser radar and the image acquisition device to collect initial laser radar data and initial image data according to a preset flight route.

[0180] The data preprocessing module 302 is configured to perform data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data.

[0181] The fusion feature generation module 303 is configured to perform feature fusion on the target laser radar data and the target image data to generate fusion features.

[0182] The forest patrol data generation module 304 is configured to perform forest anomaly recognition based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data.

[0183] Further, the data preprocessing module 302 comprises:

[0184] The intermediate laser radar data generation module is configured to perform point cloud denoising on the initial laser radar data by using a hybrid filtering algorithm based on density clustering and spatiotemporal consistency verification to generate intermediate laser radar data.

[0185] The intermediate image data generation module is configured to perform radiation correction on the initial image data by using a preset atmospheric scattering model to generate intermediate image data.

[0186] The target data generation module is configured to perform spatiotemporal registration on the intermediate laser radar data and the intermediate image data to generate target laser radar data and target image data.

[0187] Further, the intermediate laser radar data generation module can perform the following steps:

[0188] The initial laser radar data is divided into a plurality of spatiotemporal cubes.

[0189] The point density in the spatiotemporal cube is calculated to generate point density distribution data.

[0190] The low-density component probability value corresponding to each point density in the point density distribution data is calculated by using a Gaussian mixture model to generate noise probability mapping data.

[0191] The noise probability mapping data is dynamically threshold filtered to generate target denoising point cloud data.

[0192] The target denoising point cloud data is subjected to curvature feature extraction to generate curvature feature data.

[0193] The target denoising point cloud data is divided into regions according to the curvature feature data, and is filtered according to the region division to generate intermediate laser radar data.

[0194] Further, the target data generation module can perform the following steps:

[0195] Performing multi-modal feature extraction on the intermediate lidar data to generate point cloud geometry feature data;

[0196] Performing semantic segmentation on the intermediate image data to generate a plurality of semantic features;

[0197] Performing weighted fusion of the point cloud geometry feature data and the semantic features of the corresponding region to generate a plurality of semantic fusion features;

[0198] Using the semantic fusion features to construct a matrix to generate an initial transformation matrix;

[0199] Updating the initial transformation matrix using a preset semantic constraint to generate an intermediate transformation matrix;

[0200] Optimizing the intermediate transformation matrix using a point-to-plane iterative closest point algorithm to generate a target transformation matrix;

[0201] Taking the intermediate lidar data and the intermediate image data corresponding to the target transformation matrix as target lidar data and target image data.

[0202] Further, the fusion feature generation module 303 can perform the following steps:

[0203] Performing multi-resolution voxel grid division on the target lidar data to generate a plurality of multi-scale point cloud feature matrices of different scales;

[0204] Extracting local geometry features of the multi-scale point cloud feature matrices to generate point cloud multi-scale feature data;

[0205] Projecting the point cloud multi-scale feature data to the image coordinate system through the camera intrinsic and extrinsic parameters to generate a projected point cloud feature set;

[0206] Performing bilinear interpolation on the projected point cloud feature set to generate a feature candidate set;

[0207] According to the spatial coordinate weight of the corresponding region of the target image data, performing weighted fusion of the original spectral features of the target image data and the feature candidate set to generate fusion features.

[0208] Further, the forest patrol data generation module 304 can perform the following steps:

[0209] Performing feature extraction on the fusion features, geographic information system data, and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features, and environment perception features;

[0210] Using the geographic information system auxiliary features to set a dynamic weight to generate weight data;

[0211] The standardized fusion features, the geographic information system auxiliary features and the environment perception features are spliced according to the weight data to generate comprehensive features;

[0212] The comprehensive features are determined for abnormality by a preset lightweight classification model to generate forest patrol data.

[0213] Further, the forest patrol data generation module 304 can further perform the following steps:

[0214] The fusion features are standardized in space coordinates to generate standardized fusion features;

[0215] The terrain factors and vegetation type data in the geographic information system data are extracted to generate geographic information system auxiliary features;

[0216] The multi-source sensor data are matched in time and space to generate environment perception features.

[0217] Please refer to Figure 4 , Figure 4 A structural block diagram of a computer device provided for the fourth embodiment of the present application.

[0218] An electronic device of an embodiment of the present application, the electronic device comprising: a memory 401 and a processor 402, the memory 401 storing a computer program; the computer program being executed by the processor 402 to cause the processor 402 to perform the forest patrol method based on a UAV as in any of the above embodiments.

[0219] The memory 401 can be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM, a hard disk or a ROM. The memory 401 has a storage space 403 for program codes 413 for performing any of the method steps described above. For example, the storage space 403 for program codes can comprise individual program codes 413 for implementing the various steps in the above methods, respectively. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as a hard disk, a compact disk (CD), a memory card or a floppy disk. The program codes can be compressed, for example, in a suitable form. These codes, when run by a computing processing device, cause the computing processing device to perform the individual steps in the above described methods. These program codes can be read from or written to one or more computer program products. These computer program products include program code carriers such as a hard disk, a compact disk (CD), a memory card or a floppy disk. The program codes can be compressed, for example, in a suitable form. These codes, when run by a computing processing device, cause the computing processing device to perform the individual steps in the above described methods based on unmanned aerial vehicle forest patrol.

[0220] The embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed to implement the unmanned aerial vehicle forest patrol method according to any of the embodiments of the present application.

[0221] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0222] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0223] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e., may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0224] In addition, each functional unit in each embodiment of the application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0225] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application or the whole or part of the technical solutions that contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0226] The above, the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the application.

Claims

1. A forest patrol method based on a UAV, characterized by, The method comprises the steps of: controlling a UAV carrying a laser radar and an image acquisition device to collect initial laser radar data and initial image data according to a preset flight route; performing data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data; performing feature fusion on the target laser radar data and the target image data to generate fusion features; performing forest anomaly recognition based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data; the step of performing forest anomaly recognition based on the fusion features, geographic information system data and multi-source sensor data to generate forest patrol data comprises: extracting features from the fusion features, geographic information system data and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features and environmental perception features; setting dynamic weights using the geographic information system auxiliary features to generate weight data; performing weighted splicing on the standardized fusion features, the geographic information system auxiliary features and the environmental perception features according to the weight data to generate comprehensive features; performing anomaly determination on the comprehensive features through a preset lightweight classification model to generate forest patrol data; the step of extracting features from the fusion features, geographic information system data and multi-source sensor data to generate standardized fusion features, geographic information system auxiliary features and environmental perception features comprises: performing spatial coordinate standardization on the fusion features to generate standardized fusion features; extracting terrain factors and vegetation type data in the geographic information system data to generate geographic information system auxiliary features; performing spatio-temporal matching on multi-source sensor data to generate environmental perception features. 2.The UAV-based forest patrol method of claim 1, wherein, the step of performing data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data comprises: performing point cloud denoising on the initial laser radar data using a hybrid filtering algorithm based on density clustering and spatio-temporal consistency checking to generate intermediate laser radar data; performing radiation correction on the initial image data through a preset atmospheric scattering model to generate intermediate image data; performing spatio-temporal registration on the intermediate laser radar data and the intermediate image data to generate target laser radar data and target image data. 3.The UAV-based forest patrol method of claim 2, wherein, the step of performing point cloud denoising on the initial laser radar data using a hybrid filtering algorithm based on density clustering and spatio-temporal consistency checking to generate intermediate laser radar data comprises: performing spatio-temporal cubic division on the initial laser radar data to generate a plurality of spatio-temporal cubes; calculating the point density in the spatio-temporal cubes to generate point density distribution data; calculating the low-density component probability value corresponding to each point density in the point density distribution data through a Gaussian mixture model to generate noise probability mapping data; performing dynamic threshold filtering on the noise probability mapping data to generate target denoising point cloud data; performing curvature feature extraction on the target denoising point cloud data to generate curvature feature data; According to the curvature feature data, the target denoising point cloud data is regionally divided, and filtering is performed according to the region division, to generate intermediate laser radar data. 4.The UAV-based forest patrol method of claim 2, wherein, The step of performing spatio-temporal registration on the intermediate laser radar data and the intermediate image data to generate target laser radar data and target image data comprises: performing multi-modal feature extraction on the intermediate laser radar data to generate point cloud geometric feature data; performing semantic segmentation on the intermediate image data to generate a plurality of semantic features; performing weighted fusion on the point cloud geometric feature data and the semantic features of the corresponding region to generate a plurality of semantic fusion features; performing matrix construction using the semantic fusion features to generate an initial transformation matrix; updating the initial transformation matrix using a preset semantic constraint to generate an intermediate transformation matrix; optimizing the intermediate transformation matrix using a point-to-plane iterative closest point algorithm to generate a target transformation matrix; using the target transformation matrix to correspond to the intermediate laser radar data and the intermediate image data as the target laser radar data and the target image data. 5.The UAV-based forest patrol method of claim 1, wherein, The step of performing feature fusion on the target laser radar data and the target image data to generate fusion features comprises: performing multi-resolution voxel grid division on the target laser radar data to generate a multi-scale point cloud feature matrix of different scales; extracting local geometric features of the multi-scale point cloud feature matrix to generate point cloud multi-scale feature data; projecting the point cloud multi-scale feature data to an image coordinate system through camera intrinsic and extrinsic parameters to generate a projected point cloud feature set; performing bilinear interpolation supplement on the projected point cloud feature set to generate a feature candidate set; performing weighted fusion on the original spectral features of the target image data and the feature candidate set according to the spatial coordinate weight of the corresponding region of the target image data to generate fusion features. 6.A forest patrol device based on a UAV, characterized in that, The forest patrol device based on a UAV is used to implement the forest patrol method based on a UAV as claimed in any one of claims 1-5, and comprises: a data acquisition module configured to control a UAV carrying a laser radar and an image acquisition device to collect initial laser radar data and initial image data according to a preset flight route; a data preprocessing module configured to perform data preprocessing on the initial laser radar data and the initial image data to generate target laser radar data and target image data; a fusion feature generation module configured to perform feature fusion on the target laser radar data and the target image data to generate fusion features; a forest patrol data generation module configured to perform forest anomaly recognition based on the fusion features, geographic information system data, and multi-source sensor data to generate forest patrol data.

7. An electronic device, comprising: The computer program is executed to implement the forest patrol method based on a UAV as claimed in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed to implement the forest patrol method based on a UAV as claimed in any one of claims 1-5.

Images