Adverse geological detection method, system and device based on multi-source aerial remote sensing data fusion

Abstract

The present invention relates to the field of remote sensing technology, and discloses a method, system and device for detecting bad geology by fusing multi-source aerial remote sensing data, which obtains laser radar data, hyperspectral image data, CCD image data, infrared image data and combined navigation data of a target area and performs preprocessing; performs spatial matching processing on the preprocessed laser radar data, hyperspectral image data, CCD image data and infrared image data; fuses and applies the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing to perform geological detection, and obtains different geological detection data; and obtains the bad geological detection result of the target area by combining different geological detection data. The present invention can improve the processing accuracy of multi-source data fusion, effectively integrate multi-source heterogeneous information, realize comprehensive and efficient detection of various types of bad geology, reduce the risk of misjudgment and missed judgment, and improve the credibility of the bad geological detection result.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and in particular to a method, system and device for detecting adverse geology by fusing multi-source aerial remote sensing data. Background Art

[0002] Among the means of geological disaster monitoring, the use of remote sensing payloads of aerospace platforms for environmental monitoring and geological disaster identification and early warning has the advantages of large operating area, strong synchronization and high dynamics, and has been widely used.

[0003] In the application of geological disaster risk detection, it is usually necessary to simultaneously obtain the spatial three-dimensional information, spectral radiation information, texture information and surface temperature information of the ground object. In recent years, there have been ground observation methods that use laser radar scanners, hyperspectral cameras, CCD cameras and infrared cameras on the same platform. Based on the data obtained by each, the corresponding feature information is extracted for fusion processing. However, due to the completely different imaging mechanisms of active laser radar and passive hyperspectral / CCD / infrared cameras, the difference between the discreteness of laser point cloud data and the spatial continuity of spectral images, and the fusion method between the target distance information reflected by the laser radar and the target electromagnetic spectrum reflection information attributes reflected by the spectral image, the multi-source data fusion processing accuracy and information extraction are greatly restricted, and fail to meet the technical requirements of comprehensive detection of adverse geology.

[0004] Therefore, there is an urgent need for unfavorable geological detection methods, systems and devices that integrate multi-source aerial remote sensing data, which can improve the accuracy of multi-source data fusion processing, effectively integrate multi-source heterogeneous information from all aspects, realize comprehensive and efficient detection of various types of unfavorable geological conditions, reduce the risk of misjudgment and missed judgment, and improve the credibility of unfavorable geological detection results. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a method, system and device for detecting adverse geology by fusing multi-source aerial remote sensing data, which can improve the accuracy of multi-source data fusion processing, effectively integrate multi-source heterogeneous information from all aspects, realize comprehensive and efficient detection of various types of adverse geology, reduce the risk of misjudgment and missed judgment, and improve the credibility of adverse geological detection results.

[0006] The present invention provides a method for detecting adverse geology by fusing multi-source aerial remote sensing data, comprising the following steps:

[0007] S1. Obtain laser radar data, hyperspectral image data, CCD image data, infrared image data and integrated navigation data of the target area through aerial remote sensing devices;

[0008] S2. Preprocessing the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data according to the combined navigation data; wherein the preprocessing includes time synchronization processing and coordinate conversion processing;

[0009] S3, performing spatial matching processing on the pre-processed laser radar data, hyperspectral image data, CCD image data and infrared image data;

[0010] S4, fusing the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing to perform geological exploration and obtain different geological exploration data;

[0011] S5. Combine different geological exploration data to obtain adverse geological exploration results for the target area.

[0012] Furthermore, in S3, the pre-processed lidar data, hyperspectral image data, CCD image data and infrared image data are spatially matched, including:

[0013] S31, extracting feature points respectively according to the pre-processed laser radar data, hyperspectral image data, CCD image data and infrared image data;

[0014] S32, constructing feature descriptors of the laser radar data, feature descriptors of the hyperspectral image data, feature descriptors of the CCD image data, and feature descriptors of the infrared image data, respectively, according to feature points of the laser radar data, feature points of the hyperspectral image data, feature points of the CCD image data, and feature points of the infrared image data;

[0015] S33, performing feature point matching according to feature descriptors of laser radar data, feature descriptors of hyperspectral image data, feature descriptors of CCD image data, and feature descriptors of infrared image data, to obtain feature descriptor matching pairs between various types of data;

[0016] S34, performing thin plate spline transformation according to the feature descriptor matching pairs between various types of data, and obtaining the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing.

[0017] Further, in S33, feature point matching is performed based on the feature descriptors of the laser radar data, the feature descriptors of the hyperspectral image data, the feature descriptors of the CCD image data, and the feature descriptors of the infrared image data, and the feature descriptor matching pairs between the various types of data are obtained, including:

[0018] S331, taking each feature descriptor in each type of data as a target feature descriptor, and calculating the Euclidean distance between it and each feature descriptor in the other three types of data;

[0019] S332, respectively taking the feature descriptors with the smallest Euclidean distances in the other three types of data as candidate feature descriptors of the target feature descriptor;

[0020] S333, determining whether the target feature descriptor is also the feature descriptor with the smallest Euclidean distance among the candidate feature descriptors; if so, retaining the candidate feature descriptor, otherwise, eliminating the candidate feature descriptor;

[0021] S334: construct a feature descriptor matching pair based on the retained candidate feature descriptors and the target feature descriptor.

[0022] Further, in S34, thin plate spline transformation is performed according to the feature descriptor matching pairs between various types of data, and the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing are obtained, including:

[0023] S341, constructing a thin plate spline transformation model;

[0024] S342, selecting a set of feature descriptor matching pairs from the feature descriptor matching pairs between the various types of data, defining a reference data set, and solving the parameters of the thin plate spline transformation model;

[0025] S343. According to the parameters of the thin plate spline transformation model and the thin plate spline transformation model, thin plate spline transformation is performed on the feature descriptor matching pairs between various types of data to obtain the lidar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing.

[0026] Furthermore, in S4, the laser radar data, hyperspectral image data, CCD image data, and infrared image data after spatial matching processing are integrated for geological detection, and different geological detection data are obtained, including:

[0027] S41, performing terrain detection according to the laser radar data after spatial matching processing to obtain three-dimensional high-precision terrain detection data;

[0028] S42, performing geological lithology and structural interpretation based on the hyperspectral image data after spatial matching processing and combining it with the three-dimensional high-precision terrain detection data to obtain geological lithology and structural interpretation data;

[0029] S43, performing dangerous rockfall detection and water body information detection based on the CCD image data after spatial matching processing and in combination with the three-dimensional high-precision terrain detection data, to obtain dangerous rockfall detection data and water body information detection data;

[0030] S44. Based on the infrared image data after spatial matching processing, combined with the three-dimensional high-precision terrain detection data, high ground temperature identification is performed to obtain high ground temperature identification data.

[0031] Further, in S43, based on the CCD image data after the spatial matching processing, combined with the three-dimensional high-precision terrain detection data, dangerous rockfall detection is performed, and the obtained dangerous rockfall detection data includes:

[0032] Construct a dangerous rockfall detection model;

[0033] The dangerous rockfall detection model includes:

[0034] Based on the YOLOv4 network model, CSPDarknet53 is used as the backbone network. After the backbone network CSPDarknet53, the FPN structure is added, and the SENet module is embedded in each residual block of the backbone network CSPDarknet53.

[0035] Overlay the historical CCD image data onto the historical 3D high-precision terrain detection data, perform remote sensing interpretation of dangerous rockfalls, create a dangerous rockfall label construction data set, train the dangerous rockfall detection model, and obtain a trained dangerous rockfall detection model;

[0036] The CCD image data after spatial matching processing is input into the trained dangerous rockfall detection model to obtain dangerous rockfall detection data.

[0037] Further, in S43, water body information detection is performed based on the CCD image data after spatial matching processing and combined with the three-dimensional high-precision terrain detection data, and the water body information detection data obtained includes:

[0038] Construct a water body information extraction model;

[0039] The water body information extraction model includes:

[0040] Based on the U-Net network model, a SAM2 module is added after each upsampling layer of the U-Net network model, and an ECA module is added after each convolution layer of the U-Net network model;

[0041] The historical CCD image data is superimposed on the historical three-dimensional high-precision terrain detection data, and water body remote sensing interpretation is performed to produce a water body label construction data set, and the water body information extraction model is trained to obtain a trained water body information extraction model;

[0042] The CCD image data after spatial matching processing is input into the trained water body information extraction model to obtain water body information detection data.

[0043] The present invention also provides a system for detecting unfavorable geology by fusion of multi-source aerial remote sensing data, which is used to execute any of the above-mentioned methods for detecting unfavorable geology by fusion of multi-source aerial remote sensing data. The system includes the following modules:

[0044] A data acquisition module is used to acquire laser radar data, hyperspectral image data, CCD image data, infrared image data and integrated navigation data of the target area through an aerial remote sensing device;

[0045] A preprocessing module, connected to the data acquisition module, is used to preprocess the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data according to the combined navigation data; wherein the preprocessing includes time synchronization processing and coordinate conversion processing;

[0046] The geo-referencing module is connected with the pre-processing module and is used to perform spatial matching processing on the pre-processed lidar data, hyperspectral image data, CCD image data and infrared image data;

[0047] The geological detection module is connected with the geographic registration module and is used to fuse the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching for geological detection to obtain different geological detection data;

[0048] The output module is connected to the geological detection module and is used to combine different geological detection data to obtain adverse geological detection results of the target area.

[0049] The present invention also provides a device for detecting unfavorable geology by fusion of multi-source aerial remote sensing data, which is used to execute any of the above-mentioned methods for detecting unfavorable geology by fusion of multi-source aerial remote sensing data, and the device comprises:

[0050] An integrated optical observation group installed in a sunken manner on a manned platform with an optical window;

[0051] The integrated optical observation group includes a substrate, a lidar scanner, a hyperspectral camera, a CCD camera, an infrared camera, and an integrated navigator;

[0052] The laser radar scanner, CCD camera and infrared camera are arranged on the lower layer of the substrate;

[0053] The hyperspectral camera and the integrated navigator are arranged on the upper layer of the substrate.

[0054] The embodiments of the present invention have the following technical effects:

[0055] The present invention obtains lidar data, hyperspectral image data, CCD image data, infrared image data and combined navigation data through an aerial remote sensing device, thereby ensuring the diversity of data sources, enhancing the comprehensiveness and complementarity of information, and using combined navigation data for time synchronization and coordinate conversion, thereby ensuring the precise alignment of different sensor data in time and space, and improving the accuracy of spatial positioning of subsequent analysis results; by spatially matching the pre-processed multi-source data, it can ensure that the information obtained from each sensor can be correctly superimposed together, more accurately identify and associate target objects captured by different sensors, reduce the risk of misjudgment, reduce computing costs, and reduce redundant information, which provides a reliable data basis for subsequent geological exploration and integrated analysis of geology based on multi-source data; this solution can improve the accuracy of multi-source data fusion processing, effectively integrate multi-source heterogeneous information, realize comprehensive and efficient detection of various types of adverse geology, reduce the risk of misjudgment and missed judgment, and improve the credibility of adverse geological detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 It is a flow chart of a method for detecting unfavorable geology by fusing multi-source aerial remote sensing data provided by an embodiment of the present invention;

[0058] Figure 2 This is a dangerous rock and rockfall intelligent identification result diagram provided by an embodiment of the present invention;

[0059] Figure 3 This is a water body information intelligent extraction result diagram provided by an embodiment of the present invention;

[0060] Figure 4 It is a structural schematic diagram of a bad geological detection system for multi-source aerial remote sensing data fusion provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0062] The present invention proposes a method for detecting adverse geology by fusing multi-source aerial remote sensing data. Figure 1 is a flowchart of a method for detecting adverse geology by fusion of multi-source aerial remote sensing data provided by an embodiment of the present invention, see Figure 1 , specifically including:

[0063] S1. Obtain the laser radar data, hyperspectral image data, CCD image data, infrared image data and combined navigation data of the target area through aerial remote sensing devices.

[0064] In some embodiments, the lidar data includes lidar point clouds, the hyperspectral imaging data includes hyperspectral images, the CCD imaging data includes high-resolution optical images, the infrared imaging data includes thermal infrared images, and the combined navigation data includes position and attitude information obtained through devices such as GPS / IMU.

[0065] S2. Preprocess the lidar data, hyperspectral image data, CCD image data and infrared image data according to the combined navigation data.

[0066] In some embodiments, preprocessing includes denoising, filtering, radiation correction, atmospheric correction, and geometric correction.

[0067] In some embodiments, the preprocessing also includes time synchronization processing and coordinate conversion processing. The timestamp of the combined navigation data is used to synchronize the timestamps of the laser radar data, the hyperspectral image data, the CCD image data, and the infrared image data. The position and attitude information provided by the combined navigation data is used to convert all sensor data to a unified coordinate system (such as WGS84, UTM, etc.). For example, for the laser radar point cloud data, the attitude matrix R and position vector T of the combined navigation data are used for conversion:

[0068] P global =R·P local +T;

[0069] Among them, P local Represents the position of the laser radar point cloud in its own coordinate system, P global Indicates the position of the lidar point cloud after conversion to a unified coordinate system.

[0070] S3, performing spatial matching processing on the pre-processed lidar data, hyperspectral image data, CCD image data and infrared image data.

[0071] Specifically include:

[0072] S31, extracting feature points respectively according to the pre-processed laser radar data, hyperspectral image data, CCD image data and infrared image data.

[0073] In some embodiments, the point cloud can be clustered to find cluster centers representing different types of objects as feature points, or the radar point cloud can be converted into a digital surface model (DSM) or a digital terrain model (DTM), and then an edge detection algorithm can be applied thereto to find feature points.

[0074] In some embodiments, the hyperspectral image data and the CCD image data can use the SURF feature point detection algorithm to extract feature points. Furthermore, for hyperspectral images, because there are a large number of bands, the principal component analysis (PCA) technology can be first performed to reduce the number of bands, and then the above-mentioned feature point detection algorithm can be applied to extract feature points.

[0075] In some embodiments, the infrared image data may use an ORB feature point detection algorithm to extract feature points or detect the temperature gradient based on an edge detection algorithm to extract areas with drastic changes in the temperature field as feature points.

[0076] S32. Based on the feature points of the lidar data, the feature points of the hyperspectral image data, the feature points of the CCD image data and the feature points of the infrared image data, a feature descriptor of the lidar data, a feature descriptor of the hyperspectral image data, a feature descriptor of the CCD image data and a feature descriptor of the infrared image data are constructed respectively.

[0077] In some embodiments, the FPFH algorithm can be used to construct a feature descriptor of radar data based on radar point cloud data. The FPFH algorithm constructs a histogram by calculating the relative position relationship (such as angle, distance, etc.) between each point and its neighboring points. The FPFH descriptor not only considers the position information of the point, but also includes the change in the normal direction.

[0078] In some embodiments, the SURF algorithm can be used to construct feature descriptors of hyperspectral image data and CCD image data, define a square neighborhood around the feature points, divide multiple sub-regions, calculate Haar wavelet responses, and summarize them into a histogram.

[0079] In some embodiments, for each feature point of the infrared image data, the direction and intensity of the temperature gradient in its neighborhood may be calculated, and this information may be encoded into a histogram to construct a feature descriptor of the infrared image data.

[0080] S33, performing feature point matching according to feature descriptors of the laser radar data, feature descriptors of the hyperspectral image data, feature descriptors of the CCD image data, and feature descriptors of the infrared image data, to obtain feature descriptor matching pairs between various types of data.

[0081] S331, taking each feature descriptor in each type of data as a target feature descriptor, and calculating the Euclidean distance between it and each feature descriptor in the other three types of data.

[0082] The calculation formula of Euclidean distance d is as follows:

[0083] ;

[0084] Among them, q j and h j They represent the j-th element in the two feature descriptor vectors respectively, and m represents the dimension of the feature descriptor (i.e. the total number of elements in the feature descriptor vector).

[0085] S332. The feature descriptors with the smallest Euclidean distances in the other three types of data are respectively used as candidate feature descriptors for the target feature descriptor.

[0086] S333, determining whether the target feature descriptor is also the feature descriptor with the smallest Euclidean distance to the candidate feature descriptor; if so, retaining the candidate feature descriptor, otherwise, eliminating the candidate feature descriptor.

[0087] That is, not only is the nearest neighbor of a feature point required to be another feature point, but the nearest neighbor of the other feature point is also required to be the first feature point. This method can reduce the number of false matches.

[0088] S334: construct a feature descriptor matching pair based on the retained candidate feature descriptors and the target feature descriptor.

[0089] S34, performing thin plate spline transformation according to the feature descriptor matching pairs between various types of data, and obtaining the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing.

[0090] S341. Construct a thin plate spline transformation model.

[0091] The thin plate spline transformation model is as follows:

[0092] ;

[0093] ;

[0094] Among them, u(x,y) represents the new coordinates after transformation in the x direction, v(x,y) represents the new coordinates after transformation in the y direction, (x,y) represents the original coordinates of the feature points of a certain type of data, a0, a1, a2 represent the affine transformation parameters in the x direction, b0, b1, b2 represent the affine transformation parameters in the y direction, among which a0 and b0 control the global translation, a1 and b1 control the scaling along the x-axis and y-axis, a2 and b2 control the shear deformation, wi represents the non-rigid transformation weight coefficient in the x direction, i represents the i-th feature point, n represents the total number of feature points, k i represents the non-rigid transformation weight coefficient in the y direction, Indicates the original coordinates (x, y) of the feature point to any other feature point (x i ,y i )’s original coordinates, U(r i )=r i 2 logr i represents radial basis function.

[0095] S342. Select a set of feature descriptor matching pairs from the feature descriptor matching pairs between the various types of data, define a reference data set, and solve the parameters of the thin plate spline transformation model.

[0096] In some embodiments, a set of feature descriptor matching pairs is selected from the feature descriptor matching pairs between various types of data. Preferably, feature points distributed in different regions are selected to cover as large a range as possible.

[0097] In some embodiments, data with higher precision may be defined as a reference data set. For example, laser radar data is defined as a reference data set. In the feature descriptor matching pair, the position of the feature point of the laser radar data is used as the target position, and {(u i , v i )}, and the positions of the feature points of other data are represented by {(x i ,y i )} means, (u i , v i ) represents the feature points (x i ,y i ) feature points of the matched lidar data.

[0098] In some embodiments, solving the parameters of the thin plate spline transformation model is to minimize the following objective function E to obtain the optimal affine transformation parameters and non-rigid transformation weight coefficients:

[0099] ;

[0100] Among them, λ represents the regularization parameter, which is used to balance the fitting accuracy and smoothness, and ∇²f represents the Laplace operator of function f.

[0101] S343. According to the parameters of the thin plate spline transformation model and the thin plate spline transformation model, thin plate spline transformation is performed on the feature descriptor matching pairs between various types of data to obtain the lidar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing.

[0102] S4. The laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing are integrated for geological exploration to obtain different geological exploration data.

[0103] Specifically include:

[0104] S41. Perform terrain detection based on the laser radar data after spatial matching processing to obtain three-dimensional high-precision terrain detection data.

[0105] In some embodiments, three-dimensional high-precision terrain detection data can be obtained by generating a digital elevation model based on the lidar data after spatial matching processing, extracting terrain features (such as slope calculation, aspect calculation, curvature calculation, etc.) and terrain attribute analysis (such as terrain moisture index, terrain roughness, etc.).

[0106] S42. Based on the hyperspectral image data after spatial matching processing, combined with the three-dimensional high-precision terrain detection data, lithology and structure interpretation are performed to obtain lithology and structure interpretation data.

[0107] In some embodiments, various algorithms (such as N-Finder Algorithm, N-FINDR; Vertex Component Analysis, VCA) are used to determine the pure substances (i.e., end members) in the image. These end members represent the materials that may exist in the image, such as different types of rocks or minerals. For each pixel point, a linear or nonlinear unmixing method (such as a linear mixed model, LMM; SVM-based unmixing) is applied to estimate the proportion of its components, thereby identifying the lithology information contained in the pixel. Lithology classification is performed using machine learning or deep learning algorithms. Combined with geological background knowledge and other auxiliary data (such as seismic data and drilling data), the lithology and structural characteristics are fully interpreted, and finally a lithology and structural interpretation map is produced based on the information obtained in the above steps.

[0108] S43. Based on the CCD image data after spatial matching processing, combined with the three-dimensional high-precision terrain detection data, dangerous rockfall detection and water body information detection are performed to obtain dangerous rockfall detection data and water body information detection data.

[0109] Figure 2 This is a diagram of intelligent identification results of dangerous rocks and falling rocks provided by an embodiment of the present invention, see Figure 2 In some embodiments, the CCD image data after spatial matching is combined with the three-dimensional high-precision terrain detection data to detect dangerous rocks and rockfalls, and the dangerous rock and rockfall detection data obtained includes:

[0110] Construct a dangerous rockfall detection model;

[0111] The dangerous rockfall detection model includes:

[0112] Based on the YOLOv4 network model, CSPDarknet53 is used as the backbone network. After the backbone network CSPDarknet53, the FPN structure is added, and the SENet module is embedded in each residual block of the backbone network CSPDarknet53.

[0113] Among them, using CSPDarknet53 as the backbone network of the YOLOv4 network model can reduce the amount of calculation and improve the computing performance through the cross-stage partial connections (CSP) structure; the feature pyramid network (FPN) can effectively fuse feature maps of different scales and enhance the model's detection ability for multi-scale targets, which is very important for identifying dangerous rock falls of different sizes; embedding the SENet (Squeeze-and-Excitation Networks) module in each residual block of CSPDarknet53 can further improve the expressiveness of the model by adaptively recalibrating the importance of channel features.

[0114] Overlay the historical CCD image data onto the historical 3D high-precision terrain detection data, perform remote sensing interpretation of dangerous rockfalls, create a dangerous rockfall label construction data set, train the dangerous rockfall detection model, and obtain a trained dangerous rockfall detection model;

[0115] Collect historical CCD image data containing known dangerous rockfall events, and ensure that these data cover various environmental conditions (such as different seasons, weather conditions, etc.), superimpose the historical CCD image data on the historical three-dimensional high-precision terrain detection data, and manually perform remote sensing interpretation of dangerous rockfalls based on experience, mark the dangerous rockfall area in each image, and generate a corresponding label file. Due to the great variation in the shape of exposed rocks on the mountain, and the existence of slender rock bodies, when a single bounding box is used to mark them, the background area covered is too large, and the background accounts for more than the rock body, which seriously interferes with the extraction of rock body features. In some embodiments, the actual characteristics of the rock body can be analyzed to reduce the proportion of the background area in the bounding box. A segmented labeling method is used to label the entire rock body in segments to obtain a larger target feature area.

[0116] In some embodiments, the original image may be processed by cropping, scaling, enhancing (rotating, flipping, brightness adjustment, etc.) to increase data diversity and improve the robustness of the model. The dangerous rockfall detection model is trained using the constructed data set until the model converges to obtain a trained dangerous rockfall detection model.

[0117] The CCD image data after spatial matching processing is input into the trained dangerous rockfall detection model to obtain dangerous rockfall detection data.

[0118] The CCD image data after spatial matching is used as input and sent to the trained dangerous rockfall detection model, which outputs the predicted bounding box and its confidence score, identifies the possible dangerous rockfall location, and in some embodiments, provides classification information to help distinguish different types of rocks or rockfalls. The model output is post-processed, such as non-maximum suppression (NMS) to remove overlapping detection boxes.

[0119] Figure 3 This is a water body information intelligent extraction result diagram provided by an embodiment of the present invention, see Figure 3 In some embodiments, water body information detection is performed based on the CCD image data after spatial matching processing and combined with the three-dimensional high-precision terrain detection data, and the water body information detection data obtained includes:

[0120] Construct a water body information extraction model;

[0121] The water body information extraction model includes:

[0122] Based on the U-Net network model, a SAM2 module is added after each upsampling layer of the U-Net network model, and an ECA module is added after each convolution layer of the U-Net network model;

[0123] Among them, the SAM2 (Spatial Attention Module) module is used to enhance the spatial attention mechanism and help the model focus on important areas. It emphasizes key positions and suppresses irrelevant information by calculating the weighted average in the spatial dimension. Adding the SAM2 module after each upsampling layer of the U-Net network model can enhance the attention to the water area while restoring the image resolution; the ECA (Efficient Channel Attention) module is a lightweight channel attention mechanism. It calculates the dependencies between channels through one-dimensional convolution operations to enhance important channels. Adding the ECA module after each convolution layer of the U-Net network model can better capture the information between channels and improve the utilization of channel information. This model is improved on the basis of the classic U-Net. By introducing the ECA module and SAM2 auxiliary learning, its feature extraction ability in the water body segmentation task is enhanced, and the boundary of the segmentation result is clearly defined and the integrity of the object area is effectively ensured. The model has been significantly improved in feature extraction and boundary recognition. This combination not only improves the segmentation accuracy of the model for water bodies, but also enhances the robustness of the model in complex scenes. By Figure 3From the extraction results shown, it can be seen that the information of larger water bodies in the image has been basically extracted, the river information is more continuous, and the information of small lakes and ponds is obviously highlighted.

[0124] The historical CCD image data is superimposed on the historical three-dimensional high-precision terrain detection data, and water body remote sensing interpretation is performed to produce a label-constructed data set, and the water body information extraction model is trained to obtain a trained water body information extraction model;

[0125] Collect historical CCD image data containing known water body boundaries and ensure that these data cover different environmental conditions. Overlay the historical CCD image data on the historical three-dimensional high-precision terrain detection data. Based on experience, artificial water body remote sensing interpretation can be performed to mark the water area in each image and generate the corresponding binary mask.

[0126] In some embodiments, the original image may be processed by cropping, scaling, enhancing (rotating, flipping, brightness adjustment, etc.) to increase data diversity and improve the robustness of the model. The water body information extraction model is trained using the constructed data set until the model converges to obtain a trained water body information extraction model.

[0127] The CCD image data after spatial matching processing is input into the trained water body information extraction model to obtain water body information detection data.

[0128] The CCD image data after spatial matching is used as input and sent to the trained water body information extraction model to output a predicted water body mask map to identify the water body area in the image. At the same time, in some embodiments, a probability map can also be provided to show the possibility that each pixel belongs to the water body. In some embodiments, the results output by the model need to be post-processed, such as morphological operations (dilation, erosion), removal of small connected domains, etc., to improve the segmentation effect. In addition, other auxiliary information (such as terrain data) can be combined to further confirm the water body boundary.

[0129] S44. Based on the infrared image data after spatial matching processing, combined with the three-dimensional high-precision terrain detection data, high ground temperature identification is performed to obtain high ground temperature identification data.

[0130] In some embodiments, the temperature data obtained from the infrared image data is normalized, for example, the temperature range is mapped to the interval [0, 1]. A temperature threshold is set to distinguish between high geothermal temperature and normal surface temperature. Pixels above and equal to the temperature threshold are marked as "high geothermal temperature", and pixels below the temperature threshold are marked as "non-high geothermal temperature". Small isolated points or disconnected areas are removed through morphological operations (such as expansion and corrosion) to improve the segmentation results. For high geothermal areas with holes inside, a filling algorithm can be used to complete these areas. Identify and mark all connected high geothermal areas, assign a unique identifier to each area, calculate the area, perimeter, center coordinates and other geometric features of each connected domain, and generate the final high geothermal distribution map, including the location, shape, size and other information of the high geothermal areas.

[0131] S5. Combine different geological exploration data to obtain adverse geological exploration results for the target area.

[0132] In some embodiments, the unfavorable geological analysis can be performed according to the actual situation in combination with the terrain, lithology, structure, Wei Yan, water body and temperature information; the specific unfavorable geological classification can also be set according to the actual situation. For example, areas with a slope greater than 30 degrees and cracks or loose materials are prone to landslides, which are determined to be unfavorable landslide geology; areas with dense and intersecting structural lines are prone to earthquakes and landslides, which are determined to be unfavorable fault zone geology; unstable rock blocks exist on steep slopes, which are determined to be unfavorable dangerous rockfall geology; high temperature areas are concentrated in geothermally active areas, which may cause surface deformation and groundwater level changes, which are determined to be unfavorable geothermal activity geology, etc.

[0133] The present invention obtains lidar data, hyperspectral image data, CCD image data, infrared image data and combined navigation data through an aerial remote sensing device, thereby ensuring the diversity of data sources, enhancing the comprehensiveness and complementarity of information, and using combined navigation data for time synchronization and coordinate conversion, thereby ensuring the precise alignment of different sensor data in time and space, and improving the accuracy of spatial positioning of subsequent analysis results; by spatially matching the pre-processed multi-source data, it can ensure that the information obtained from each sensor can be correctly superimposed together, more accurately identify and associate target objects captured by different sensors, reduce the risk of misjudgment, reduce computing costs, and reduce redundant information, which provides a reliable data basis for subsequent geological exploration and integrated analysis of geology based on multi-source data; this solution can improve the accuracy of multi-source data fusion processing, effectively integrate multi-source heterogeneous information, realize comprehensive and efficient detection of various types of adverse geology, reduce the risk of misjudgment and missed judgment, and improve the credibility of adverse geological detection results.

[0134] Figure 4is a schematic diagram of the structure of a system for detecting unfavorable geology by fusion of multi-source aerial remote sensing data provided by an embodiment of the present invention. The system is used to execute the method for detecting unfavorable geology by fusion of multi-source aerial remote sensing data described in the above embodiment. Figure 4 As shown, the system includes the following modules:

[0135] A data acquisition module is used to acquire laser radar data, hyperspectral image data, CCD image data, infrared image data and integrated navigation data of the target area through an aerial remote sensing device;

[0136] A preprocessing module, connected to the data acquisition module, is used to preprocess the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data according to the combined navigation data; wherein the preprocessing includes time synchronization processing and coordinate conversion processing;

[0137] The geo-referencing module is connected with the pre-processing module and is used to perform spatial matching processing on the pre-processed lidar data, hyperspectral image data, CCD image data and infrared image data;

[0138] The geological detection module is connected with the geographic registration module and is used to fuse the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching for geological detection to obtain different geological detection data;

[0139] The output module is connected to the geological detection module and is used to combine different geological detection data to obtain adverse geological detection results of the target area.

[0140] The present invention also proposes a device for detecting unfavorable geology by fusion of multi-source aerial remote sensing data, which is used to execute the method for detecting unfavorable geology by fusion of multi-source aerial remote sensing data described in the above embodiment, and the device comprises:

[0141] An integrated optical observation group installed in a sunken manner on a manned platform with an optical window;

[0142] The integrated optical observation group includes a substrate, a lidar scanner, a hyperspectral camera, a CCD camera, an infrared camera, and an integrated navigator;

[0143] The laser radar scanner, CCD camera and infrared camera are arranged on the lower layer of the substrate;

[0144] The hyperspectral camera and the integrated navigator are arranged on the upper layer of the substrate.

[0145] Among them, the manned aircraft may be an airplane.

[0146] In some embodiments, the observation axes of each load device are matched and stable, and the overall size of the device is reduced as much as possible. This arrangement can minimize the size requirements of the optical window of the flight platform of the entire system and increase the platform adaptability of the system.

[0147] In some embodiments, communication connection with the lidar scanner, hyperspectral camera, CCD camera, infrared camera and combined navigator is achieved through the serial port and port of the airborne main control circuit board. The main control circuit board realizes comprehensive control of each imaging payload and the combined navigator by sending communication protocol instructions corresponding to each imaging payload.

[0148] In some embodiments, the timing information of the signal sent by the combined navigator is used as the basis for timing communication of each imaging payload, and the time of each imaging payload and the onboard main control circuit board is unified as the same benchmark. According to the working principle of the sensor, the laser radar scanner can adopt PPS (Pulse Per Second) + GPRMC (Global Positioning System Fix Data - Recommended Minimum Specific GPS / Transit Data) synchronous control; the combined navigator, hyperspectral camera, CCD camera and infrared camera adopt electrical signal synchronous trigger control (event synchronization), and the MARK2POS information of the combined navigator is recorded while controlling the triggering of the hyperspectral camera, CCD camera and infrared camera to take pictures.

[0149] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0150] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting adverse geology by fusion of multi-source aerial remote sensing data, characterized in that: The steps include: S1. Obtain laser radar data, hyperspectral image data, CCD image data, infrared image data and integrated navigation data of the target area through aerial remote sensing devices; S2, preprocessing the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data according to the combined navigation data; wherein the preprocessing includes time synchronization processing and coordinate conversion processing; S3, performing spatial matching processing on the pre-processed laser radar data, the hyperspectral image data, the CCD image data and the infrared image data; specifically comprising: S31, extracting feature points respectively according to the preprocessed laser radar data, the hyperspectral image data, the CCD image data and the infrared image data; S32, constructing a feature descriptor for the laser radar data, a feature descriptor for the hyperspectral image data, a feature descriptor for the CCD image data, and a feature descriptor for the infrared image data, respectively, according to the feature points of the laser radar data, the feature points of the hyperspectral image data, the feature points of the CCD image data, and the feature points of the infrared image data; S33, performing feature point matching according to the feature descriptor of the laser radar data, the feature descriptor of the hyperspectral image data, the feature descriptor of the CCD image data, and the feature descriptor of the infrared image data, to obtain feature descriptor matching pairs between various types of data; S34, performing thin plate spline transformation according to the feature descriptor matching pairs between various types of data, to obtain laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing; S4, fusing the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data after the spatial matching process to perform geological detection to obtain different geological detection data; S5. Combine different geological exploration data to obtain adverse geological exploration results for the target area.

2. The method for detecting adverse geological conditions by fusion of multi-source aerial remote sensing data according to claim 1, characterized in that: In S33, feature point matching is performed according to the feature descriptor of the laser radar data, the feature descriptor of the hyperspectral image data, the feature descriptor of the CCD image data, and the feature descriptor of the infrared image data, and feature descriptor matching pairs between various types of data are obtained, including: S331, taking each feature descriptor in each type of data as a target feature descriptor, and calculating the Euclidean distance between it and each feature descriptor in the other three types of data; S332, respectively taking the feature descriptors with the smallest Euclidean distances in the other three types of data as candidate feature descriptors for the target feature descriptor; S333, determining whether the target feature descriptor is also the feature descriptor with the smallest Euclidean distance among the candidate feature descriptors; if so, retaining the candidate feature descriptor; otherwise, eliminating the candidate feature descriptor; S334: construct a feature descriptor matching pair according to the retained candidate feature descriptors and the target feature descriptors.

3. The method for detecting adverse geology by fusion of multi-source aerial remote sensing data according to claim 1, characterized in that: In the above S34, thin plate spline transformation is performed according to the feature descriptor matching pairs between various types of data to obtain the laser radar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing, including: S341, constructing a thin plate spline transformation model; S342, selecting a set of feature descriptor matching pairs from the feature descriptor matching pairs between the various types of data, defining a reference data set, and solving the parameters of the thin plate spline transformation model; S343. According to the parameters of the thin plate spline transformation model and the thin plate spline transformation model, thin plate spline transformation is performed on the feature descriptor matching pairs between various types of data to obtain the lidar data, hyperspectral image data, CCD image data and infrared image data after spatial matching processing.

4. The method for detecting adverse geological conditions by fusion of multi-source aerial remote sensing data according to claim 1, characterized in that: In S4, the laser radar data, the hyperspectral image data, the CCD image data, and the infrared image data after spatial matching are integrated to perform geological detection, and different geological detection data are obtained, including: S41, performing terrain detection according to the laser radar data after spatial matching processing to obtain three-dimensional high-precision terrain detection data; S42, performing lithology and structural interpretation based on the hyperspectral image data after spatial matching processing and combining it with the three-dimensional high-precision terrain detection data to obtain lithology and structural interpretation data; S43, performing dangerous rockfall detection and water body information detection based on the CCD image data after spatial matching processing and in combination with the three-dimensional high-precision terrain detection data, to obtain dangerous rockfall detection data and water body information detection data; S44, performing high ground temperature identification based on the infrared image data after spatial matching processing and combining it with the three-dimensional high-precision terrain detection data to obtain high ground temperature identification data.

5. The method for detecting adverse geological conditions by fusion of multi-source aerial remote sensing data according to claim 4, characterized in that: In S43, dangerous rockfall detection is performed based on the CCD image data after spatial matching processing and combined with the three-dimensional high-precision terrain detection data, and the dangerous rockfall detection data obtained includes: Construct a dangerous rockfall detection model; The dangerous rockfall detection model includes: Based on the YOLOv4 network model, CSPDarknet53 is used as the backbone network. After the backbone network CSPDarknet53, the FPN structure is added, and the SENet module is embedded in each residual block of the backbone network CSPDarknet53. Overlaying historical CCD image data onto historical three-dimensional high-precision terrain detection data, performing remote sensing interpretation of dangerous rockfalls, producing dangerous rockfall label construction data sets, training the dangerous rockfall detection model, and obtaining a trained dangerous rockfall detection model; The CCD image data after the spatial matching process is input into the trained dangerous rockfall detection model to obtain dangerous rockfall detection data.

6. The method for detecting adverse geological conditions by fusion of multi-source aerial remote sensing data according to claim 4, characterized in that: In S43, water body information detection is performed based on the CCD image data after spatial matching processing and combined with the three-dimensional high-precision terrain detection data, and the water body information detection data obtained includes: Construct a water body information extraction model; The water body information extraction model includes: Based on the U-Net network model, a SAM2 module is added after each upsampling layer of the U-Net network model, and an ECA module is added after each convolution layer of the U-Net network model; Overlaying historical CCD image data onto historical three-dimensional high-precision terrain detection data, performing water body remote sensing interpretation, making a water body label construction data set, training the water body information extraction model, and obtaining a trained water body information extraction model; The CCD image data after spatial matching processing is input into the trained water body information extraction model to obtain water body information detection data.

7. A system for detecting unfavorable geology by fusion of multi-source aerial remote sensing data, used to implement the method for detecting unfavorable geology by fusion of multi-source aerial remote sensing data as described in any one of claims 1 to 6, characterized in that: The system includes the following modules: A data acquisition module is used to acquire laser radar data, hyperspectral image data, CCD image data, infrared image data and integrated navigation data of the target area through an aerial remote sensing device; A preprocessing module, connected to the data acquisition module, for preprocessing the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data according to the combined navigation data; wherein the preprocessing includes time synchronization processing and coordinate conversion processing; A geo-registration module, connected to the pre-processing module, for performing spatial matching processing on the pre-processed laser radar data, the hyperspectral image data, the CCD image data and the infrared image data; A geological detection module, connected to the geographic registration module, is used to fuse the laser radar data, the hyperspectral image data, the CCD image data and the infrared image data after spatial matching processing to perform geological detection and obtain different geological detection data; The output module is connected to the geological detection module and is used to combine different geological detection data to obtain adverse geological detection results of the target area.

8. An unfavorable geological detection device based on multi-source aerial remote sensing data fusion, used to implement the unfavorable geological detection method based on multi-source aerial remote sensing data fusion as described in any one of claims 1 to 6, characterized in that: The device includes: An integrated optical observation group installed in a sunken manner on a manned platform with an optical window; The integrated optical observation group includes a substrate, a laser radar scanner, a hyperspectral camera, a CCD camera, an infrared camera and an integrated navigator; The laser radar scanner, the CCD camera and the infrared camera are arranged on the lower layer of the substrate; The hyperspectral camera and the integrated navigator are arranged on the upper layer of the substrate.

Images