Landslide disaster identification method and system based on multi-modal remote sensing data fusion

Abstract

The invention discloses a landslide disaster identification method and system based on multi-modal remote sensing data fusion, and relates to the technical field of landslide disaster identification, and the method comprises the steps: obtaining multi-modal historical landslide remote sensing data of different time scales; extracting landslide risk characterization indexes from the historical landslide remote sensing data of each mode; the landslide risk characterization indexes of each modal at different time scales are subjected to noise reduction, smoothing and fusion by using extended Kalman filtering, the accuracy and robustness of data are improved, and a landslide disaster identification model is trained by using a multi-modal landslide data set. And carrying out landslide disaster identification on the multimodal remote sensing data at the current moment by adopting the trained landslide disaster identification model, thereby realizing intelligent identification of the landslide disaster and improving landslide identification precision.

Description

Technical Field

[0001] The present invention relates to the technical field of landslide hazard identification, and in particular to a landslide hazard identification method and system based on multi-modal remote sensing data fusion. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] For the identification of landslide hazards, traditional landslide monitoring methods mainly include ground surveys, geological mapping and on-site monitoring. These methods have problems such as being time-consuming, costly and difficult to achieve large-scale continuous monitoring.

[0004] In recent years, remote sensing technology has gradually become the primary method for landslide monitoring. However, a single type of remote sensing data often fails to comprehensively and accurately identify landslide hazards. For example, optical remote sensing data is significantly affected by weather conditions, surface image data acquired using synthetic aperture radar (SAR) technology (SAR data) has limited penetration in vegetated areas, and laser radar (LiDAR) data is costly for large-scale monitoring.

[0005] Currently, there are patents that fuse multi-source data to identify landslides. For example, the technical solutions disclosed in patents CN116030353B, CN113887515A, CN115952410B, and CN115661681B use deep learning methods to fuse multi-source data, but do not specify whether data of different time scales are used. Their data preprocessing mainly uses software such as ENVI for preliminary noise reduction, which is inefficient and has relatively poor data quality. In addition, all data are simply stacked and fused during fusion. Summary of the Invention

[0006] In order to solve the above problems, the present invention proposes a landslide hazard identification method and system based on multimodal remote sensing data fusion. It combines multimodal remote sensing data of different time scales, and extracts bands, temperature, deformation, elevation and slope as landslide risk characterization indicators. The extended Kalman filter is used to reduce noise, smooth and fuse the remote sensing data of different time scales to improve the accuracy and robustness of the data. The convolutional neural network is used to perform data-level fusion on the processed multimodal remote sensing data to realize intelligent identification of landslide hazards and improve the accuracy of landslide identification.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a landslide hazard identification method based on multimodal remote sensing data fusion, comprising:

[0009] Acquire multimodal historical landslide remote sensing data at different time scales;

[0010] Extracting landslide risk characterization indicators from historical landslide remote sensing data of each modality;

[0011] The landslide risk characterization indicators of each mode at different time scales are smoothed and fused to obtain a multi-modal landslide dataset.

[0012] The multimodal landslide dataset is used to train the landslide hazard identification model, and the trained landslide hazard identification model is used to identify landslide hazards on the multimodal remote sensing data at the current moment.

[0013] As an optional embodiment, the remote sensing data includes optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data.

[0014] As an optional implementation method, the process of extracting landslide risk characterization indicators includes:

[0015] Extracting the reflectance of visible light band and near infrared band from optical remote sensing data as the landslide risk characterization index;

[0016] Extracting surface temperature from thermal infrared remote sensing data as a landslide risk indicator;

[0017] Extracting surface deformation from ground-based synthetic aperture radar data and interferometric synthetic aperture radar data as indicators of landslide risk;

[0018] For lidar data, elevation and slope are extracted as indicators to characterize landslide risk.

[0019] As an optional implementation method, the landslide risk characterization indicators of each mode at different time scales are smoothed and fused using extended Kalman filtering, specifically:

[0020] According to the landslide state at the current moment, the landslide state and state error covariance matrix at the next moment are predicted; the Kalman gain is calculated, and the predicted landslide state and state error covariance matrix at the next moment are updated in combination with the observed value of the landslide risk characterization indicator at the next moment; the process is circulated sequentially along the time scale, and finally the landslide risk characterization indicators of each mode at different time scales are smoothed and fused.

[0021] As an optional implementation method, landslide samples are annotated on multimodal historical landslide remote sensing data to obtain landslide label data;

[0022] The landslide risk characterization index of the thermal infrared remote sensing data, the landslide risk characterization index of the ground-based synthetic aperture radar data, the landslide risk characterization index of the interferometric synthetic aperture radar data, and the landslide risk characterization index of the lidar data after fusion are normalized to the interval [0, 1]. The landslide risk characterization index of the optical remote sensing data after fusion is not normalized.

[0023] The multimodal landslide dataset consists of landslide label data, landslide risk characterization indicators of fused optical remote sensing data, normalized thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and landslide risk characterization indicators of lidar data.

[0024] As an optional implementation, the landslide hazard identification model is obtained by training a convolutional neural network using a multimodal landslide dataset. The convolutional neural network is equipped with 5 convolution modules, wherein the convolution kernel is 3×3, the maximum pooling layer is 2×2, the first convolution layer uses 128 neurons, the second and third layers contain 256 neurons, the fourth and fifth layers contain 512 neurons, and a normalization layer and a rectified linear unit are added after each convolution layer.

[0025] In a second aspect, the present invention provides a landslide hazard identification system based on multimodal remote sensing data fusion, comprising:

[0026] an acquisition module configured to acquire multimodal historical landslide remote sensing data at different time scales;

[0027] an extraction module configured to extract landslide risk characterization indicators from historical landslide remote sensing data of each modality;

[0028] The fusion module is configured to smooth and fuse the landslide risk characterization indicators of each modality at different time scales, thereby obtaining a multimodal landslide dataset;

[0029] The identification module is configured to use the multimodal landslide data set to train a landslide hazard identification model, and use the trained landslide hazard identification model to identify landslide hazards on the multimodal remote sensing data at the current moment.

[0030] In a third aspect, the present invention provides an electronic device comprising a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein the computer instructions, when executed by the processor, perform the method described in the first aspect.

[0031] In a fourth aspect, the present invention provides a computer-readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method described in the first aspect is performed.

[0032] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which implements the method described in the first aspect when executed by a processor.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention proposes a landslide hazard identification method and system based on multimodal remote sensing data fusion. The method combines multimodal remote sensing data of optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data at different time scales, and extracts band, temperature, deformation, elevation and slope as landslide risk characterization indicators. Then, an extended Kalman filter is used to achieve denoising, smoothing and fusion processing of remote sensing data at different time scales, realize dynamic updating of multimodal data, and improve data accuracy and robustness. Finally, a convolutional neural network is used to perform data-level fusion on the data after the extended Kalman filter, thereby completing the intelligent identification of landslide hazards and improving identification accuracy.

[0035] Because single-type remote sensing data often cannot comprehensively and accurately identify landslide hazards, the multimodal remote sensing data of the present invention can improve the accuracy of landslide hazard monitoring. The multimodal remote sensing data used in the present invention includes optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data, and lidar data. These data types each have unique advantages and can provide different aspects of surface information. Specifically: optical remote sensing data mainly provides visual information of the surface through changes in reflectivity in different bands, which is suitable for surface feature identification and classification; thermal infrared remote sensing data provides surface temperature distribution information by measuring infrared radiation emitted by the surface, which helps to identify potential landslide precursors; ground-based synthetic aperture radar data provides surface deformation and high-precision deformation monitoring by using microwave technology, which is suitable for all-weather and all-day monitoring and can monitor tiny surface displacements in real time; interferometric synthetic aperture radar data extracts surface deformation information through interferometric technology, providing long-term high-precision deformation data, which is suitable for large-scale landslide risk assessment; lidar data uses laser pulses to measure surface distance, providing high-resolution three-dimensional terrain data, which is suitable for terrain modeling and structural analysis.

[0036] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 Flowchart of a landslide hazard identification method based on multimodal remote sensing data fusion provided in Example 1 of the present invention;

[0039] Figure 2 A flow chart of landslide dataset processing provided in Example 1 of the present invention;

[0040] Figure 3 This is a flow chart of the extended Kalman filter-convolutional neural network model provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0041] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0042] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "include" and "comprise" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0044] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0045] Example 1

[0046] like Figure 1-2 As shown, this embodiment provides a method for intelligent identification of landslide hazards based on multimodal remote sensing data fusion, comprising the following steps:

[0047] Step 101: Acquire multimodal historical landslide remote sensing data at different time scales.

[0048] Specifically, the remote sensing data includes optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data.

[0049] Specifically, optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data of different time scales are obtained through satellites, airborne radars and ground-based radars. There is no restriction on the specific acquisition method.

[0050] Step 102: preprocessing the acquired multi-modal historical landslide remote sensing data to obtain preprocessed multi-modal historical landslide remote sensing data.

[0051] Specifically, since multimodal remote sensing data acquired in different ways may use different coordinate systems and are subject to factors such as atmospheric scattering, absorption, errors caused by atmospheric effects, and noise, this embodiment first uses software such as ENVI to perform atmospheric correction, radiation correction, geometric correction, image registration, and feature registration on the acquired multimodal remote sensing data.

[0052] Step 103: extracting landslide risk characterization indicators from the pre-processed multi-modal historical landslide remote sensing data.

[0053] Specifically, because remote sensing data contains a variety of information, data fusion will result in data redundancy. Therefore, this embodiment extracts indicators representing landslide areas from multimodal remote sensing data, which helps improve the efficiency of subsequent data fusion and reduces the impact of other data on landslide identification accuracy.

[0054] Specifically, optical remote sensing data mainly uses the reflectivity of visible light band and near-infrared band to characterize the probability of whether it is a landslide area; thus, the reflectivity of visible light band and near-infrared band is extracted from optical remote sensing data as a landslide risk characterization indicator;

[0055] Thermal infrared remote sensing data mainly detects surface temperature, which reflects whether landslides have occurred in the current area. Therefore, surface temperature is extracted from thermal infrared remote sensing data as a landslide risk indicator.

[0056] For ground-based synthetic aperture radar data, surface deformation is extracted as a landslide risk indicator;

[0057] Extracting surface deformation from interferometric synthetic aperture radar data as a landslide risk indicator;

[0058] For lidar data, elevation and slope are extracted as landslide risk characterization indicators; the specific indicators are shown in Table 1.

[0059] Table 1. Characterization indicators of landslide risk using different remote sensing data;

[0060]

[0061] Step 104: The landslide risk characterization indicators of each modality at different time scales are subjected to noise reduction, smoothing and fusion processing using an extended Kalman filter, thereby obtaining a multimodal fusion data set, enhancing data accuracy and robustness and ensuring data consistency.

[0062] Specifically, the extended Kalman filter can track the dynamic changes of the system in real time through recursive updates and provide more accurate state estimation;

[0063] Taking the surface temperature index of thermal infrared remote sensing data as an example, the basic principle is:

[0064] Define state variables and observation variables: The state variable represents the state of the system at time k. Taking the surface temperature data as an example, it can be defined as: The observed variable is the surface temperature data, which can be defined as: k =T k .

[0065] Among them, T k is the surface temperature at time k, ΔT k is the rate of change of surface temperature at time k (first-order difference).

[0066] Establishment of nonlinear state transition model: The state transition model describes the change of state variables over time. Assume that the surface temperature change is nonlinear and is affected by the ambient temperature T env , soil moisture θ and seasonal factors S k (seasonal temperature changes), the state transition model is expressed as:

[0067] x k =f(x k-1 ,u k-1 )+w k ;

[0068]

[0069] Where f(·) represents the nonlinear state transfer function, u k-1 is the control input (such as ambient temperature, soil moisture, seasonal factors, etc.), w k is the process noise, and α, β, γ, η are model parameters.

[0070] Establishment of nonlinear observation model: The observation model describes the relationship between the observation variables and the state variables. For surface temperature data, the observation model is expressed as:

[0071] z k =h(xk )+v k ;

[0072] Among them, h(·) is a nonlinear observation function. Since the observation value is directly related to the surface temperature, the nonlinear observation function is defined as: h(x k )=T k , v k is the observation noise.

[0073] Initialization: Given the initial state vector x0 and the initial error covariance matrix P0, the purpose is to provide the system with an initial state estimate and its initial value of uncertainty.

[0074] Prediction: Based on the current landslide state, predict the landslide state and state error covariance matrix at the next moment. Its expression is:

[0075] Landslide status prediction: x k|k-1 =f(x k-1 ,u k-1 );

[0076] Calculate the Jacobian matrix F of the state transfer matrix k :

[0077] Error covariance prediction:

[0078] Among them, x k|k-1 is the prior state estimate (predicted value) at time k, F k is the state transition matrix, x k-1|k-1 is the posterior state estimate (updated value) at time k-1, P k|k-1 is the prior error covariance matrix (predicted value) at time k, P k-1|k-1 is the posterior state estimate (updated value) at time k-1, Q k is the process noise covariance matrix.

[0079] Update: Based on the surface temperature observation value at the next moment, the predicted landslide state and state error covariance matrix at the next moment are corrected to improve the accuracy of state estimation and reduce errors.

[0080] Specifically:

[0081] The Kalman gain is introduced to weigh the uncertainty of the predicted value and the observed value, and the optimal weight is found to make the state estimation more accurate. By properly adjusting the Kalman gain, the data error is reduced. The Kalman gain is:

[0082]

[0083] Among them, K kis the Kalman gain, H k is the partial derivative matrix of the observation function h(·) with respect to the state variable x, i.e. the Jacobian matrix of the surface temperature observation matrix, R k is the observation noise covariance matrix.

[0084] The update of the landslide state and state error covariance matrix at the next moment is:

[0085] Landslide status update: x k|k =x k|k-1 +K k (z k -H k x k|k-1 );

[0086] State error covariance matrix update: P k|k =(IK k H k )P k|k-1 ;

[0087] Among them, x k|k is the posterior state estimate (updated value) at time k, z k is the observed surface temperature at time k, P k|k is the posterior error covariance matrix (updated value) at time k, and I is the identity matrix.

[0088] By Kalman gain K k Correcting the predicted value with the observed value can effectively reduce noise and achieve data smoothing. When the observation noise is small, the observed value is more trusted; when the observation noise is large, the predicted value is more trusted.

[0089] For data at different time scales (such as high-frequency data, low-frequency data, and data from different time periods), the extended Kalman filter can dynamically adjust the frequency of predictions and updates to ensure data fusion. Through recursive prediction and update steps, the extended Kalman filter gradually corrects the state estimate to generate continuous, smooth time series data. The fused data not only removes noise but also retains the dynamic characteristics of the data, providing high-quality data input for subsequent convolutional neural network models and improving the accuracy of prediction results. The fused time series data is converted into a convolutional neural network input format (such as a two-dimensional or three-dimensional matrix) through segmentation and reshaping methods, and then used for model training and prediction.

[0090] like Figure 2As shown in the figure, since the multimodal fusion dataset contains multimodal heterogeneous data, it is not uniform in dimensional scale and cannot be directly input into the convolutional neural network model. Therefore, the landslide risk characterization indicators of thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data in the multimodal fusion dataset obtained after extended Kalman filtering are normalized to the interval [0, 1], and the minimum-maximum normalization method is used for calculation. The normalized data are then saved in raster form. In order to retain the original spectral characteristics of the landslide, the landslide risk characterization indicators of the optical remote sensing data obtained after extended Kalman filtering are not normalized.

[0091] The expression of the minimum-maximum normalization method is:

[0092]

[0093] Among them, z represents the normalized value, m represents the original value of the data, min(X) represents the minimum value in the overall data, and max(X) represents the maximum value in the overall data.

[0094] In this embodiment, landslide samples are labeled on multimodal historical landslide remote sensing data to obtain landslide label data, and 0 and 1 are used to distinguish landslides from background, with landslides labeled as 1 and other labels as 0, but the present invention is not limited thereto.

[0095] Finally, a multimodal landslide dataset is constructed by landslide label data, landslide risk characterization indicators of optical remote sensing data obtained after extended Kalman filter processing, normalized thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data, and landslide risk characterization indicators of lidar data.

[0096] Step 105: Divide the multimodal landslide dataset into a training validation set and a test set in proportion, train the convolutional neural network model and test its accuracy, thereby obtaining a trained landslide hazard identification model.

[0097] Specifically, the multimodal landslide dataset is divided into a training validation set and a test set with a ratio of 8:2. The training validation set contains a training set (80%) and a validation set (20%). The training set is used for data expansion to train the convolutional neural network, the validation set is used to adjust the hyperparameters after each training is completed, and the test set is used to test the accuracy of the trained convolutional neural network.

[0098] The convolutional neural network is trained using the training validation set. The main steps are:

[0099] Due to the limited number of samples in the dataset, data enhancement operations such as cropping, rotation, and adding white noise were performed on the data before model training to improve the generalization ability of the model.

[0100] ResNet50 is used as the network feature extraction part to extract features from the multimodal landslide dataset; the decoder of the U-Net network is used to restore the image size;

[0101] Because the network depth of the convolutional neural network has a great influence on the accuracy of the results. The feature abstraction level of the shallow network is not high, while the deep network gradually extracts from the original features to the deep abstract features, and the extracted features are richer, but too high a network depth will cause network degradation. Therefore, this embodiment sets 5 convolution modules for processing, where the convolution kernel is 3×3, the maximum pooling layer is 2×2, the first convolution layer uses 128 neurons, the second and third layers contain 256 neurons, and the fourth and fifth layers contain 512 neurons, and a normalization layer and a rectified linear unit (ReLU) layer are added after each convolution layer to prevent overfitting.

[0102] After convolution and pooling operations, the convolutional neural network maps the data to output categories through a fully connected layer. Finally, a dropout layer and a regression layer are added. The dropout layer minimizes overfitting when the model has too many parameters but few training samples, achieving a certain degree of regularization. The regression layer is used for regression. This completes the training of the convolutional neural network model.

[0103] like Figure 3 As shown in the figure, the test set is used to test the accuracy of the trained convolutional neural network model. If the test accuracy is greater than the selected accuracy threshold, landslide hazard identification can be performed; otherwise, the data set needs to be updated and the convolutional neural network needs to be retrained, and the hyperparameters in the network are automatically adjusted according to the validation set until the test accuracy is greater than the selected accuracy threshold.

[0104] Step 106: Using the trained landslide hazard identification model to identify landslide hazards on the multimodal remote sensing data at the current moment.

[0105] Example 2

[0106] This embodiment provides a landslide hazard identification system based on multimodal remote sensing data fusion, including:

[0107] an acquisition module configured to acquire multimodal historical landslide remote sensing data at different time scales;

[0108] an extraction module configured to extract landslide risk characterization indicators from historical landslide remote sensing data of each modality;

[0109] The fusion module is configured to smooth and fuse the landslide risk characterization indicators of each modality at different time scales, thereby obtaining a multimodal landslide dataset;

[0110] The identification module is configured to use the multimodal landslide data set to train a landslide hazard identification model, and use the trained landslide hazard identification model to identify landslide hazards on the multimodal remote sensing data at the current moment.

[0111] It should be noted that the above modules correspond to the steps described in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0112] In further embodiments, there is also provided:

[0113] An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed by the processor, wherein when the computer instructions are executed by the processor, the method described in Example 1 is performed. For the sake of brevity, no further details are given here.

[0114] It should be understood that in this embodiment, the processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0115] The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0116] A computer-readable storage medium is used to store computer instructions, and when the computer instructions are executed by a processor, the method described in Example 1 is performed.

[0117] The method in Example 1 can be directly implemented as a hardware processor, or can be implemented using a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, it will not be described in detail here.

[0118] A computer program product includes a computer program, which implements the method described in embodiment 1 when executed by a processor.

[0119] The present invention also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions contained in program modules, which are executed in a device on a real or virtual processor of a target to perform the process / method described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functionality of program modules can be combined or divided between program modules as needed. The machine-executable instructions for the program modules can be executed in local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0120] The computer program code for implementing the method of the present invention can be written in one or more programming languages. These computer program codes can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the computer or other programmable data processing device, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on a computer, partially on a computer, as an independent software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server.

[0121] In the context of the present invention, computer program code or related data can be carried by any appropriate carrier to enable a device, apparatus, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer-readable media, and the like. Examples of signals include electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0122] Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0123] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A landslide hazard identification method based on multimodal remote sensing data fusion, characterized in that: include: Acquire multimodal historical landslide remote sensing data at different time scales; Extracting landslide risk characterization indicators from historical landslide remote sensing data of each modality; The landslide risk characterization indicators of each mode at different time scales are smoothed and fused to obtain a multi-modal landslide dataset. The multimodal landslide dataset is used to train the landslide hazard identification model, and the trained landslide hazard identification model is used to identify landslide hazards on the multimodal remote sensing data at the current moment.

2. The landslide hazard identification method based on multimodal remote sensing data fusion according to claim 1, characterized in that: Remote sensing data includes optical remote sensing data, thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and lidar data.

3. The landslide hazard identification method based on multimodal remote sensing data fusion according to claim 2, characterized in that: The process of extracting landslide risk characterization indicators includes: Extracting the reflectance of visible light band and near infrared band from optical remote sensing data as the landslide risk characterization index; Extracting surface temperature from thermal infrared remote sensing data as a landslide risk indicator; Extracting surface deformation from ground-based synthetic aperture radar data and interferometric synthetic aperture radar data as indicators of landslide risk; For lidar data, elevation and slope are extracted as indicators to characterize landslide risk.

4. The landslide hazard identification method based on multimodal remote sensing data fusion according to claim 1, characterized in that: The extended Kalman filter is used to smooth and fuse the landslide risk characterization indicators of each mode at different time scales, specifically: Based on the landslide state at the current moment, the landslide state and state error covariance matrix at the next moment are predicted; the Kalman gain is calculated, and combined with the observed value of the landslide risk characterization indicator at the next moment, the predicted landslide state and state error covariance matrix at the next moment are updated; The landslide risk characterization indicators of each mode at different time scales are smoothed and fused by cycling sequentially along the time scale.

5. The landslide hazard identification method based on multimodal remote sensing data fusion according to claim 1, characterized in that: Landslide samples are annotated on multimodal historical landslide remote sensing data to obtain landslide label data; The landslide risk characterization index of the thermal infrared remote sensing data, the landslide risk characterization index of the ground-based synthetic aperture radar data, the landslide risk characterization index of the interferometric synthetic aperture radar data, and the landslide risk characterization index of the lidar data after fusion are normalized to the interval [0, 1]. The landslide risk characterization index of the optical remote sensing data after fusion is not normalized. The multimodal landslide dataset consists of landslide label data, landslide risk characterization indicators of fused optical remote sensing data, normalized thermal infrared remote sensing data, ground-based synthetic aperture radar data, interferometric synthetic aperture radar data and landslide risk characterization indicators of lidar data.

6. The landslide hazard identification method based on multimodal remote sensing data fusion according to claim 1, characterized in that: The landslide hazard identification model is obtained by training a convolutional neural network using a multimodal landslide dataset. The convolutional neural network is equipped with five convolution modules, wherein the convolution kernel is 3×3, the maximum pooling layer is 2×2, the first convolution layer uses 128 neurons, the second and third layers contain 256 neurons, the fourth and fifth layers contain 512 neurons, and a normalization layer and a rectified linear unit are added after each convolution layer.

7. A landslide hazard identification system based on multimodal remote sensing data fusion, characterized in that: include: an acquisition module configured to acquire multimodal historical landslide remote sensing data at different time scales; an extraction module configured to extract landslide risk characterization indicators from historical landslide remote sensing data of each modality; The fusion module is configured to smooth and fuse the landslide risk characterization indicators of each modality at different time scales, thereby obtaining a multimodal landslide dataset; The identification module is configured to use the multimodal landslide data set to train a landslide hazard identification model, and use the trained landslide hazard identification model to identify landslide hazards on the multimodal remote sensing data at the current moment.

8. An electronic device, characterized in that: The method comprises a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method according to any one of claims 1 to 6 is completed.

9. A computer-readable storage medium, characterized in that Used to store computer instructions, which, when executed by a processor, complete the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program, which is used to implement the method according to any one of claims 1 to 6 when the computer program is executed by a processor.

Images