Landslide hidden danger multi-mode identification fusion method and system

Through the application of multimodal data fusion and machine learning models, the problems of low identification efficiency and easy human-influence in existing landslide hidden danger recognition technology are solved, and high-precision and efficient landslide hidden danger recognition are achieved.

CN118965270AActive Publication Date: 2024-11-15CHINA AERO GEOPHYSICAL SURVEY & REMOTE SENSING CENT FOR LAND & RESOURCES +2

Patent Information

Application Number
CN202411142406.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-20
Publication Date
2024-11-15
Estimated Expiration
2044-08-20

AI Technical Summary

Technical Problem

The existing landslide potential identification technology relies on limited InSAR images and manual discrimination, and is easily affected by timing intervals, surface coverage, sensor noise and atmospheric conditions. It has low recognition efficiency and the results are easily affected by human subjective experience.

Method used

采用滑坡隐患多模态识别融合方法,通过获取多源遥感数据,构建多光谱变化特征参数集和地形特征参数集,进行多模态数据融合,利用机器学习模型进行地表状态判定,提高识别精度和效率。

Benefits of technology

High-precision identification of landslide hidden dangers is achieved, the cost of manpower and material resources is reduced, the recognition efficiency is improved, the dependence of manual judgment is reduced, and the objectivity and accuracy of the identification results are enhanced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118965270A_ABST
    Figure CN118965270A_ABST
Patent Text Reader

Abstract

The invention discloses a landslide hidden danger multi-mode identification fusion method and system, and the method comprises the steps: introducing spectral features, terrain attributes and long-time-sequence change trend variables of hydrothermal factors to form a multi-spectral change feature parameter set based on optical, thermal infrared, microwave, terrain and InSAR multi-source remote sensing data; according to the method, the landform index is combined with the InSAR deformation data to form a landform characteristic parameter set, multi-modal data fusion is carried out, potential change characteristics of landslide disasters are comprehensively represented, more comprehensive background information and evolution trend can be provided, and the problem that the landslide hidden danger recognition precision is affected due to the fact that the InSAR deformation result is singly relied on is solved. Moreover, after the landslide hidden danger recognition model is trained, the recognition result can be quickly obtained only by respectively inputting the multispectral change feature set and the topographic feature parameter set into the model, so that the landslide recognition efficiency is remarkably improved, and the manpower and material resource costs are greatly reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method and a system, and in particular to a multi-modal identification and fusion method and system for landslide hidden dangers. Background Art

[0002] Landslide hazard identification monitors and analyzes the geological structure and geomorphic features on and below the surface to identify potential landslide risks and predict the occurrence of disaster events. With the continuous advancement of science and technology and the deepening of its application, landslide hazard identification technology has gradually become one of the important means of disaster prevention and mitigation. Traditional landslide identification methods mainly rely on manual observation and ground surveys. This method is limited by human resources and geographical scope and cannot achieve comprehensive monitoring and identification of large-scale and complex landforms. The development of modern remote sensing technology has provided a new solution for landslide identification. Remote sensing technology can obtain a large amount of data on and below the surface, including image data in multiple bands such as optical, thermal infrared, and radar, as well as various geographic information data such as terrain, landform, and vegetation coverage. These data can provide a rich information basis for landslide identification. In recent years, with the development of machine learning and artificial intelligence technology, landslide identification has entered a new stage. Machine learning algorithms can extract features from massive remote sensing data, identify the potential characteristics and laws of landslides, and realize automatic identification and prediction of landslides. Traditional machine learning algorithms have been widely used in the field of landslide identification and have achieved remarkable results. However, the characteristics of landslides are complex and diverse. How to extract and select effective features from a large amount of data is a challenge, which may directly affect the performance and final accuracy of the identification model.

[0003] The existing landslide hazard identification technology has the following main technical problems, which are reflected in the following two aspects: First, it only relies on a limited number of InSAR images to obtain surface deformation, which is easily affected by comprehensive factors such as time interval, surface coverage, sensor noise and atmospheric conditions; Second, the current main method requires the surface deformation obtained by InSAR images combined with expert judgment, and then manual judgment and hazard range delineation are carried out area by area, which requires a lot of visual judgment and manual operation time, has low recognition efficiency and the results are easily affected by human subjective experience. Summary of the invention

[0004] In order to solve the deficiencies of the above technologies, the present invention provides a multi-modal identification and fusion method and system for landslide hazards.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: a multi-modal identification and fusion method for landslide hazards, comprising the following steps:

[0006] S1. Acquisition and preprocessing of long-term multi-source data;

[0007] S2. Construction of multi-spectral change characteristic parameter set: According to the closeness between long-term multi-source data and landslide, the multi-spectral characteristic parameter set is screened and constructed, and the long-term series change characteristics of the corresponding parameters are estimated one by one to construct the multi-spectral change characteristic parameter set;

[0008] S3. Construction of terrain feature parameter set:

[0009] The terrain characteristic parameter set is constructed by combining the deformation rate and terrain index obtained from InSAR data;

[0010] S4, training sample selection and model training: according to the characteristics of the surface change in the landslide potential area, the surface state is divided and the model training samples are selected for different surface state types, and the model training is performed on the multi-spectral change characteristic parameter set and the terrain characteristic parameter set respectively, until a model based on the multi-spectral change characteristic parameter and a model based on the terrain characteristic parameter are obtained;

[0011] S5. Multimodal data fusion: Multimodal fusion is performed on the model prediction results of multispectral variation characteristic parameters and the model prediction results based on terrain characteristic parameters;

[0012] S6. Surface state determination: Determine the surface state based on the multimodal fusion model obtained in the above steps.

[0013] Preferably, in S1, the acquired data includes multi-source optical, thermal infrared, microwave, terrain and InSAR remote sensing data, specifically including:

[0014] Long-term Landsat and MODIS surface reflectance datasets of different scales; digital elevation models; Landsat and MODIS surface temperature products; microwave soil moisture products, such as SMAP, including global surface and root layer soil moisture data; corresponding time series InSAR data.

[0015] Preferably, in S2, the process of constructing the multi-spectral variation characteristic parameter set specifically includes:

[0016] 2.1) Construction of multi-spectral feature parameter set;

[0017] 2.2) Construction of multi-spectral variation characteristic parameter set;

[0018] 2.1), the construction of the multispectral feature parameter set includes two steps: the first is to construct the spectral index parameter set, and the second is to construct the hydrothermal factor spatial gradient parameter set;

[0019] The spectral index parameter set is constructed by combining the spectral indices closely related to landslides, including: Normalized Difference Vegetation Index NDVI, Fraction of Vegetation Coverage FVC and Soil Adjusted Vegetation Index SAVI;

[0020] The constructed spatial gradient parameter set of water and heat factors includes: land surface temperature gradient index NLSTG and soil moisture gradient index NSMG;

[0021] In 2.2), based on the spectral indexes NDVI, FVC, SAVI and the spatial gradient parameter indexes of hydrothermal factors NLSTG and NSMG obtained in the previous step, the surface state feature set is formed, and the corresponding multi-spectral change feature parameter set is further estimated. The change feature parameters include: main slope m, slope change rate r, intercept b, parameter free correlation ρ, and coefficient of variation CV. The dynamics of parameters related to landslides are reflected through the dynamics of long-term changes;

[0022] Therefore, each of the five surface state parameters has five variation characteristic parameters, and a set V consisting of 25 multispectral variation characteristic parameters is obtained. SUR .

[0023] Preferably, in 2.1), the specific calculation formulas of the spectral index including the normalized difference vegetation index NDVI, the vegetation coverage FVC and the soil adjusted vegetation index SAVI are as follows:

[0024]

[0025]

[0026] Among them, R nir is the reflectivity in the near-infrared band; R red is the reflectance of the red light band; L is the soil brightness correction factor, which ranges from (0,1) and is adjusted according to the specific vegetation type and soil characteristics; NDVI M and NDVI m are the maximum and minimum values ​​of NDVI, corresponding to the NDVI values ​​of bare soil and dense vegetation, respectively.

[0027] Preferably, in 2.1), the construction process of the hydrothermal factor spatial gradient parameter set is as follows:

[0028] The first step is to downscale to a spatial resolution consistent with the spectral index parameters to obtain high-resolution surface temperature and soil moisture data;

[0029] In the second step, combined with the high-resolution surface temperature and soil moisture data obtained in the previous step, the spatial gradients of water and heat parameters are estimated according to the following steps: First, the eight neighboring pixels around each pixel are selected to estimate the gradient; second, the pixel value difference between it and the surrounding neighboring pixels is calculated pixel by pixel; then, the distance between the central pixel and each neighboring pixel is calculated pixel by pixel; finally, the spatial gradient of each pixel is calculated by dividing the difference by the distance pixel by pixel;

[0030] For each pixel, the spatial gradients of land surface temperature and soil moisture are calculated by the following formulas respectively:

[0031]

[0032] where ΔX q is the average of the parameter differences between the central pixel and all neighboring pixels; Δd q is the distance between the central pixel and the q-th neighboring pixel; XG is the calculated spatial gradient of the parameter for each pixel;

[0033] In the third step, normalization of the hydrothermal parameter spatial gradients; The linear normalization method is selected to scale the spatial gradients of land surface temperature and soil moisture to [-1, 1] so that their values are within the same scale. The specific normalization formulas are as follows:

[0034]

[0035] where XG is the calculated spatial gradient of the parameter for each pixel; XG M and XG m are the maximum and minimum values corresponding to the spatial gradient of the parameter of the central pixel respectively; According to formulas (4) and (5), the normalized land surface temperature gradient index NLSTG and soil moisture gradient index NSMG can be calculated pixel by pixel.

[0036] Preferably, in 2.1) and 2.2), the specific calculation methods of the change characteristic parameters are as follows:

[0037] (2.21) The calculation formula for the main slope m is:

[0038]

[0039] where m n represents the slope of the n-th point pair, n represents the n-th point pair, i and j respectively represent different period numbers, satisfying i < j, and i, j ∈ [1, k]; y i and y j are the numerical values of the multi-spectral characteristic parameters corresponding to periods i and j respectively; x i and x j are the times corresponding to periods i and j respectively; k is the total number of long-time series data;

[0040] According to formula (6), two periods i and j are sequentially selected from the long-time series data, and finally N main slopes can be calculated, where the N m values are sorted from small to large in sequence to obtain an ordered point pair slope sequence {m1, m2... m N}, if N is odd, the median slope is the middle value of the ordered sequence, that is, the The slope corresponding to the point pair as the principal slope; if N is an even number, then the median slope is the and The average of the slopes of the points as the principal slope;

[0041] (2.22) The slope change rate r is calculated as:

[0042]

[0043] Among them, r represents the average slope change rate, k represents the total number of long time series data, and m ij represents the slope change rate corresponding to the i-th period and the j-th period, Δt ij represents the time interval between the i-th period and the j-th period;

[0044] (2.23) The formula for calculating the main intercept b is:

[0045]

[0046] Among them, p represents the pth pixel, s represents the sth point pair, m represents the main slope, and N is the total number of all point pairs, which can be expressed as k is the total number of long time series data, P p,s Represents the value of the multispectral characteristic parameter corresponding to the s-th point pair of the p-th pixel, t s Indicates the time corresponding to the sth point pair;

[0047] Sort the intercept values ​​of all the point pairs from small to large to get an ordered point pair intercept sequence {b1, b2...b N}, if N is an odd number, then the median slope is the middle value of the ordered sequence, that is, The intercept corresponding to the point pair as the principal intercept; if N is even, then the median slope is the and The average of the slopes of the points as the principal intercept;

[0048] (2.24) The calculation formula of parameter autocorrelation ρ is:

[0049]

[0050] in, It indicates the average value of the parameter P corresponding to the long time series, and ρ(z) indicates the degree of correlation between two parameter values ​​separated by z time units in the long time series. k represents the total number of long time series data;

[0051] (2.25) The coefficient of variation CV is calculated as:

[0052]

[0053] Where σ represents the standard deviation of the surface state parameters, represents the average value of the parameter P corresponding to the long time series, k represents the total number of long time series data, P i represents the i-th sample value in the surface state parameter data set, i∈[1,k];

[0054] After the above steps, the five change characteristic parameters {m, r, b, ρ, CV} corresponding to the spectral indexes NDVI, FVC, SAVI and the hydrothermal parameter spatial gradient indexes NLSTG and NSMG are estimated. When the calculation of the five change characteristic parameters corresponding to the five surface state parameters is completed, a set V consisting of 25 multi-spectral change characteristic parameters will be obtained. SUR , the specific expression is as follows:

[0055]

[0056] Variation characteristic parameter set V SUR It will subsequently be used as an independent variable for machine learning, thus completing the input variable for machine learning algorithm training.

[0057] Preferably, in S3, the process of constructing the terrain feature parameter set includes:

[0058] (3.1) Constructing terrain characteristic parameter set: using the preprocessed digital elevation model to obtain landslide-related terrain indices, including terrain curvature index TCI and terrain turbulence index TPI;

[0059] First, the second-order derivative of the surface is calculated based on the surface elevation data, as shown below:

[0060]

[0061] Among them, C represents the surface curvature, E is the surface elevation, represents partial derivative, x and y are the horizontal positions of the surface coordinates, both of which can be obtained from the digital elevation model;

[0062] Then, the calculated curvature is normalized, and the normalization calculation is as follows:

[0063]

[0064] Among them, C M and C mThey are the minimum and maximum values of the surface curvature respectively; TCI is used to judge the degree of terrain concavity and convexity. TCI > 0.7 indicates that the terrain is very flat, 0.3 < TCI ≤ 0.7 indicates that the terrain has certain undulations, and TCI ≤ 0.3 indicates that the terrain is very rugged;

[0065] The terrain turbulence index TPI can be obtained by statistically analyzing the elevation values around the pixel. The formula is as follows:

[0066]

[0067] Among them, E0 represents the elevation value of the central pixel, represents the average elevation of the surrounding neighborhood, both of which can be obtained from the digital elevation model; a positive TPI value indicates that the central pixel is located in a relatively high place, a negative value indicates that the central pixel is located in a relatively low place, and 0 indicates that the elevation value of the central pixel is equal to the average value of the surrounding neighborhood;

[0068] (3.2) Estimation of deformation rate based on InSAR:

[0069] An interferogram is generated from the registered multi-temporal InSAR images, and the deformation rate v of each pixel is extracted; the deformation rate v, terrain curvature index TCI, and terrain turbulence index TPI data are combined into a dataset. Among them, the deformation rate data v is downscaled; the combined dataset is normalized to obtain the terrain feature parameter set V DEF , V DEF Subsequently, it will be used as an independent variable in machine learning, thus completing the input variables for machine learning algorithm training.

[0070] Preferably, in S4, the method for selecting model training samples is as follows:

[0071] According to the characteristics of surface changes in the landslide hazard area, the surface is divided into three states, including stable, slightly changed, and unstable; a combination of randomly selected samples and stratified manually selected samples is used to select training samples for different surface state types;

[0072] In S4, the process of model training is as follows:

[0073] The multi-spectral change feature parameters form a set V SUR and the terrain feature parameter set V DEF are used as independent variables in machine learning respectively, and the surface state sample set S is used as the dependent variable in machine learning;

[0074] The input variables and surface state samples are both divided into 70% and 30%. Among them, 70% of V SUR_train and V DEF_train data will be used for training the surface state change in machine learning. The model mapping training process is represented by the following formula:

[0075] f SUR =f train (V SUR_train ,S train ) (15)

[0076] f DEF =f train (V DEF_train ,S train ) (16)

[0077] Among them, S strain is the 70% surface state sample set used for training; V SUR_train 、V DEF_train are the 70% multi-spectral variation feature parameter set and terrain feature parameters used for training, respectively. SUR 、f DEF They are nonlinear models trained by machine learning, which are trained by the selected 70% input parameters and surface state sample sets;

[0078] The remaining 30% of the data set is used to evaluate the performance of the machine learning algorithm to determine whether it has met the predetermined performance standards; if the algorithm performance fails to meet the established accuracy requirements, iterative training is performed until its recognition accuracy reaches or exceeds the expected threshold, and ultimately the above two types of models are obtained, namely, models based on multi-spectral change characteristic parameters and models based on terrain characteristic parameters.

[0079] Preferably, in S5, a probability averaging method is used to perform multimodal fusion and output the probability prediction of each category. For the two types of models, the probability prediction of each category is first estimated:

[0080] P sur (Y|V SUR_pre ) = f SUR (V SUR_pre ) (17)

[0081] P def (Y|V DEF_pre ) = f DEF (V DEF_pre ) (18)

[0082] Among them, f SUR 、f DEF Respectively represent the prediction models trained with multi-spectral variation characteristic parameter set and terrain characteristic parameter; V SUR_pre and V DEF_pre They represent the multi-spectral variation characteristic parameter set and terrain characteristic parameters involved in the prediction respectively; P sur represents f SUR For V SUR_preThe predicted probability for each category Y, P def represents f DEF For V DEF_pre The predicted probability for each category Y respectively;

[0083] Then, the probabilities of each class are weighted averaged to obtain the combined probability of each class, and finally the class with the highest combined probability is selected as the final prediction:

[0084] P(Y)=ω sur ×P sur (Y|V SUR_pre )+ω sar ×P sar (Y|V SAR_pre ) (19)

[0085] Where P(Y) represents the final category probability obtained by weighted averaging the probability predictions of different models; ω sur And ω def Represent the probability weight coefficient of each model respectively, and satisfy ω sur +ω def =1;

[0086] After obtaining the weighted average probability P(Y) of each category, the category with the highest comprehensive probability is finally selected as the final prediction result.

[0087] A multi-modal identification and fusion system for landslide hazards includes the following modules:

[0088] Long time series multi-source data preprocessing module, used for preprocessing the collected long time series multi-source data;

[0089] The module for estimating the characteristic parameter set of multi-spectral changes uses 30m optical data to estimate the normalized vegetation index NDVI, vegetation coverage FVC, and soil adjusted vegetation index SAVI. It uses a variety of machine learning algorithms to train the downscaling model to obtain high-resolution surface temperature and soil moisture, and estimates the spatial gradient index of water and heat parameters: surface temperature gradient index NLSTG, soil moisture gradient index NSMG; for NDVI, FVC, SAVI, NLSTG, and NSMG, the main slope, slope change rate, intercept, parameter autocorrelation, and coefficient of variation are calculated to obtain the characteristic parameter set of multi-spectral changes.

[0090] The terrain characteristic parameter set estimation module uses the digital elevation model to obtain the corresponding elevation, slope and slope direction, estimates the terrain curvature index TCI and terrain turbulence index TPII, and obtains the deformation rate of each pixel through multi-temporal InSAR images; the deformation rate data v is statistically downscaled to a spatial resolution of 30 meters, and merged with the TCI and TPI data into a data set, and the terrain characteristic parameter set is obtained after normalization;

[0091] The training sample selection and model training module selects samples from stable, slightly changing and unstable areas respectively. The multispectral change characteristic parameter set and terrain characteristic parameter are used as independent variables of machine learning respectively, and the surface state sample set is used as the dependent variable of machine learning. The model is iteratively trained and evaluated until a model based on multispectral change characteristic parameters and a model based on terrain characteristic parameters are obtained.

[0092] The multimodal fusion module fuses the two models, and the category with the highest comprehensive probability will be used as the final prediction model.

[0093] The surface state determination module calls the final prediction model to determine the surface state and output the drawn regional surface state spatial distribution map.

[0094] In view of the limitations of the existing landslide hazard identification technology, the present invention proposes a landslide hazard multimodal identification fusion method and system, aiming to achieve high-precision landslide hazard identification. On the one hand, based on optical, thermal infrared, microwave, terrain and InSAR multi-source remote sensing data, the long-term change trend variables of spectral features, terrain attributes and hydrothermal factors are introduced to form a multi-spectral change feature parameter set, and the terrain index is combined with InSAR deformation data to form a terrain feature parameter set. Multimodal data fusion is performed to comprehensively characterize the potential change characteristics of landslide hazards. Since the additional introduction of multi-source long-term multi-spectral change characteristics can provide more comprehensive background information and evolution trends, the coupling of the two can complement each other and overcome the problem of affecting the accuracy of landslide hazard identification due to single reliance on InSAR deformation results. On the other hand, after the landslide hazard identification model is trained, it only needs to input the multi-spectral change feature set and the terrain feature parameter set into the model respectively to quickly obtain the identification result, significantly improve the efficiency of landslide identification, and greatly reduce the cost of manpower and material resources. Compared with the traditional method that requires manual identification, landslide prediction will be more intelligent and convenient. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] Figure 1 It is the overall technical flow chart of the present invention.

[0096] Figure 2 This is a system module connection diagram of the present invention. DETAILED DESCRIPTION

[0097] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0098] The present invention fully considers the limitations of the existing landslide hazard identification technology, and proposes a landslide hazard multimodal identification fusion method and system. First, multi-source optical, thermal infrared, microwave, terrain and InSAR remote sensing data are obtained, and the corresponding spectral characteristics and hydrothermal factor multispectral feature parameter sets are screened and constructed according to the closeness to the landslide, and the long-term series change characteristics of the corresponding parameters are estimated one by one to construct a multispectral change feature parameter set. Then, the terrain feature parameter set is constructed in combination with the deformation rate of InSAR data and the terrain index. Model training is performed on the multispectral change feature parameter set and the terrain feature parameter set respectively, and finally, multimodal fusion is performed based on the model prediction results of the multispectral change characteristics and the model prediction results based on the terrain feature parameters to obtain a comprehensive judgment result of the surface state, thereby improving the accuracy of landslide hazard identification.

[0099] The overall technical flow chart of the present invention is as follows: Figure 1 As shown in the figure, it is mainly divided into the following six steps: S1, acquisition and preprocessing of long-term multi-source data; S2, construction of multi-spectral change characteristic parameter set; S3, construction of terrain characteristic parameter set; S4, training sample selection and model training; S5, multimodal data fusion; S6, surface state determination.

[0100] Combination Figure 1 The technical flow chart of the invention is as follows. The specific implementation methods of the multi-modal identification and fusion method for landslide hazards proposed in the present invention are as follows:

[0101] S1. Acquisition and preprocessing of long-term multi-source data

[0102] The acquired data include multi-source optical, thermal infrared, microwave, terrain and InSAR remote sensing data, including:

[0103] Obtain Landsat and MODIS surface reflectance datasets (30m and 500m spatial resolution) of different scales for long time series, with a time series of >=10 years. Select months with less interference from rain, ice and snow weather according to the actual area, with cloud cover less than 70%, for estimating spectral index; digital elevation models (such as SRTM and ASTER GDEM) with a spatial resolution of 30m and global coverage can be used to extract elevation and slope, which will be used to estimate terrain index later; download Landsat and MODIS surface temperature products (90m and 1km spatial resolution), with a time series of >=10 years; download microwave soil moisture products since 2015, such as SMAP, including global surface and root layer soil moisture data, with a time resolution of 3 days, a spatial resolution of 1-3 kilometers, and a data format of HDF5; obtain InSAR data of the corresponding time series (such as Sentinel 1, ALOS-2, etc.), with a time series of >=5 years, and perform preprocessing including denoising, terrain correction, and atmospheric correction.

[0104] The multi-source products downloaded above were geometrically corrected and reprojected, and the surface temperature products and SMAP data were resampled to a resolution consistent with other data sources. Outliers were removed to ensure the accuracy and reliability of the data, and the products were uniformly cropped into small areas of 30×30km (including 1000×1000 pixels). The preprocessing process ensured that the long-term remote sensing data obtained had high quality and consistency.

[0105] S2. Construction of multi-spectral variation characteristic parameter set

[0106] According to the closeness between long-term multi-source data and landslides, a multispectral feature parameter set is screened and constructed, and the long-term series change characteristics of the corresponding parameters are estimated one by one to construct a multispectral change feature parameter set. The specific process is as follows:

[0107] 2.1) Construction of multispectral feature parameter set

[0108] The construction of multi-spectral feature parameter set mainly includes two steps. The first is to construct the spectral index parameter set, and the second is to construct the hydrothermal factor spatial gradient parameter set.

[0109] (2.11) Construct spectral index parameter set

[0110] The spectral indices closely related to landslides will be combined into a spectral index parameter set, including the Normalized Difference Vegetation Index (NDVI), the Fractional Vegetation Cover (FVC) and the Soil-Adjusted Vegetation Index (SAVI). Among them, NDVI is an important indicator for measuring vegetation conditions and can be used to identify areas with sparse or damaged vegetation and provide early warning of landslide risks; FVC can reflect the impact of changes in surface cover types on vegetation-related indices; SAVI takes into account the regulatory effect of soil on vegetation reflectance and can be used to monitor areas with exposed soil.

[0111] The above three spectral indices are obtained by using the preprocessed surface reflectance product through specific band calculations. The specific calculation formula is as follows:

[0112]

[0113] Among them, NDVI is the normalized difference vegetation index; FVC is the vegetation coverage; SAVI is the soil adjusted vegetation index; R nir is the reflectivity in the near-infrared band; R red is the reflectance of the red light band; L is the soil brightness correction factor, which ranges from (0,1) and is usually set to 0.5. It can be adjusted according to the specific vegetation type and soil characteristics to obtain more accurate results; NDVI M and NDVI m are the maximum and minimum values ​​of NDVI, corresponding to the NDVI values ​​of bare soil and dense vegetation, respectively.

[0114] The above steps complete the construction of three types of spectral index parameter sets.

[0115] (2.12) Constructing the spatial gradient parameter set of hydrothermal factors

[0116] The first step is to downscale to a spatial resolution consistent with the spectral index parameters.

[0117] In order to keep the spatial resolution of surface temperature and soil moisture consistent with that of spectrum and terrain index, spectral index (such as NDVI) with a resolution of 30 meters can be used as auxiliary data, and a variety of machine learning algorithms can be used to construct the regression relationship between surface temperature and auxiliary variables, and between soil moisture and auxiliary variables, such as random forest method, gradient boosting regression tree method, neural network method, etc. At the same time, for each regression model, the optimal parameter combination is screened, and the optimal machine learning algorithm is selected to train the downscaling model to obtain high-resolution surface temperature and soil moisture.

[0118] The second step is to estimate the spatial gradients of hydrothermal parameters.

[0119] Combined with the high-resolution surface temperature and soil moisture data obtained in the previous step, the spatial gradient is estimated according to the following steps: First, eight neighboring pixels (i.e., a 3×3 neighborhood) around each pixel are selected to estimate the gradient; second, the pixel value difference between each pixel and the surrounding neighboring pixels is calculated pixel by pixel, where the average value of the pixel difference between the central pixel and each neighboring pixel can be used as the difference of the pixel; then, the distance between the central pixel and each neighboring pixel is calculated pixel by pixel, where the Euclidean distance between pixels can be selected to calculate the distance; finally, the difference is divided by the distance pixel by pixel to calculate the spatial gradient of each pixel, that is, for each pixel, the spatial gradient of the surface temperature and soil moisture can be calculated by the following formulas:

[0120]

[0121] Where ΔX q is the average value of the parameter difference between the central pixel and all neighboring pixels; Δd q is the distance between the central pixel and the qth neighboring pixel; XG is the parameter space gradient calculated for each pixel.

[0122] The third step is to normalize the spatial gradient of hydrothermal parameters.

[0123] Here we can choose the linear normalization method to scale the spatial gradients of surface temperature and soil moisture to [-1,1] so that their values ​​are within a unified scale. The specific normalization formula is as follows:

[0124]

[0125] Where XG is the calculated parameter spatial gradient of each pixel; XG M With XG m are the maximum and minimum values ​​of the parameter space gradient corresponding to the central pixel. According to formula (4) and (5), the normalized surface temperature gradient index NLSTG and soil moisture gradient index NSMG can be calculated pixel by pixel.

[0126] The above has completed the acquisition of two types of hydrothermal parameter spatial gradient indices NLSTG and NSMG.

[0127] 2.2) Construction of multi-spectral variation feature parameter set

[0128] Based on the surface state feature set consisting of the spectral index (NDVI, FVC, SAVI) and hydrothermal parameter spatial gradient index (NLSTG, NSMG) obtained in the previous step, the corresponding multi-spectral change feature parameter set is further estimated, and the change feature parameters will be used as input for model training.

[0129] For the 5 parameters in the multi-spectral feature parameter set of the long time series, the following 5 change feature parameters are calculated respectively, including: main slope, slope change rate, intercept, parameter free correlation, coefficient of variation, and the parameter dynamics related to landslides are reflected through the change dynamics of the long time series. Thus, each of the 5 surface state parameters has 5 change feature parameters, that is, each pixel corresponds to 25 different categories of information.

[0130] The following are the specific calculation methods for the 5 change feature parameters:

[0131] (2.21) Main slope m

[0132] For the long time series data of the given parameter, the main slope is calculated using the paired slope, that is, for each data point pair in the time series, the slope between two adjacent data points is calculated in turn, and the median of these slopes is used as the main slope on the time series. Assuming there are k periods of data in total, a total of k - 1 slopes need to be calculated, and the specific calculation formula for the slope is as follows:

[0133]

[0134] Among them, m n represents the slope of the nth point pair, n represents the nth point pair, i and j represent different period numbers respectively, satisfying i < j, and i, j ∈ [1, k]; y i and y j are the multi-spectral feature parameter values corresponding to periods i and j respectively; x i and x j are the times corresponding to periods i and j respectively; k is the total number of long time series data;

[0135] According to formula (6), two periods i and j are selected from the long time series data in turn, and finally N main slopes can be calculated, where The N m values are sorted in ascending order to obtain an ordered point pair slope sequence {m1, m2... m N}, if N is odd, the median slope is the middle value of the ordered sequence, that is, the slope corresponding to the th point pair is used as the main slope; if N is even, the median slope is the th and th point pair slopes of the ordered sequence average value of

[0136] (2.22) Slope change rate r

[0137] The rate of change of the slope can help identify the acceleration or deceleration of the parameter change trend. By calculating the average rate of change of the slope of all adjacent data points, the overall average slope change rate is obtained. The calculation formula is:

[0138]

[0139] Among them, r represents the average slope change rate, k represents the total number of long time series data, and m ij represents the slope change rate corresponding to the i-th period and the j-th period, Δt ij Represents the time interval corresponding to the i-th period and the j-th period.

[0140] (2.23) Main intercept b

[0141] After obtaining the main slope corresponding to the long time series parameter, calculate the difference between the multispectral characteristic parameter value of each point pair and the fitting trend line (the product of the main slope and time), average all the differences, and get the main intercept b of the pth pixel p , which can be expressed as follows:

[0142]

[0143] Among them, p represents the pth pixel, s represents the sth point pair, m represents the main slope, and N is the total number of all point pairs, which can be expressed as k is the total number of long time series data, P p,s Represents the value of the multispectral characteristic parameter corresponding to the s-th point pair of the p-th pixel, t s Represents the time corresponding to the sth point pair.

[0144] Sort the intercept values ​​of all the point pairs from small to large to get an ordered point pair intercept sequence {b1, b2...b N}, if N is an odd number, then the median slope is the middle value of the ordered sequence, that is, The intercept corresponding to the point pair as the principal intercept; if N is even, then the median slope is the and The average of the slopes of the points as the principal intercept.

[0145] (2.24) Parameter autocorrelation ρ

[0146] The autocorrelation of the surface state parameters is the correlation between the parameter value at a certain moment in the long time series and the parameter value at a certain moment before. Here, the autocorrelation function can be used to calculate the autocorrelation. Assume that the long time series surface state parameters are expressed as P = {P1, P2...P k}, where P kRepresents the parameter value of the kth period. The autocorrelation calculation formula of the parameter is as follows:

[0147]

[0148] in, It indicates the average value of the parameter P corresponding to the long time series, and ρ(z) indicates the degree of correlation between two parameter values ​​separated by z time units in the long time series. k represents the total number of long time series data. Here, the time interval z will be set according to the natural cycle of different surface state parameters.

[0149] (2.25) Coefficient of variation CV

[0150] The coefficient of variation is selected to measure the relative variability or dispersion of the data, that is, the volatility of the data relative to the average value. A high CV value indicates that the data has a larger fluctuation than the average value, while a low CV value indicates that the data is relatively stable. It is calculated by the ratio of the standard deviation of the surface state parameter to the average value. The specific formula is as follows:

[0151]

[0152] Where σ represents the standard deviation of the surface state parameters, represents the average value of the parameter P corresponding to the long time series, k represents the total number of long time series data, P i Represents the i-th sample value in the surface state parameter data set, i∈[1,k].

[0153] After the above steps, the five variation characteristic parameters {m, r, b, ρ, CV} corresponding to the spectral index (NDVI, FVC, SAVI) and the hydrothermal parameter spatial gradient index (NLSTG, NSMG) are estimated. When the calculation of the five variation characteristic parameters corresponding to the five surface state parameters is completed, a set V consisting of 25 multi-spectral variation characteristic parameters will be obtained. SUR , the specific expression is as follows:

[0154]

[0155] Here, V SUR It is a set of change characteristic parameters consisting of 25 surface parameter variables corresponding to the pixel; NDVI is the normalized difference vegetation index, FVC is the vegetation coverage, SAVI is the soil regulated vegetation index, NLSTG is the surface temperature spatial gradient index, NSMG is the soil moisture spatial gradient index, m represents the main slope, r represents the slope change rate, b represents the intercept, ρ represents the parameter autocorrelation, and CV represents the coefficient of variation. SUR It will subsequently be used as an independent variable for machine learning, thus completing the input variable for machine learning algorithm training.

[0156] S3. Construction of Terrain Feature Parameter Set

[0157] Construct a terrain feature parameter set by combining the deformation rate obtained from InSAR data and terrain indices (terrain curvature index TCI, topographic position index TPI). The specific process is as follows:

[0158] (3.1) Construction of Terrain Feature Parameter Set

[0159] Use the preprocessed digital elevation model to obtain the corresponding elevation, slope, and aspect, and obtain terrain indices related to landslides through corresponding operations: terrain curvature index (TCI, Terrain Curvature Index) and topographic position index (TPI, Topographic Position Index).

[0160] First, calculate the second derivative of the ground surface based on the surface elevation data. The specific formula is as follows:

[0161]

[0162] Among them, C represents the surface curvature, E is the surface elevation, represents the partial derivative, x and y are the horizontal positions of the surface coordinates, both of which can be obtained from the digital elevation model.

[0163] Then, normalize the calculated curvature to ensure that the values are within a unified range. The normalization calculation is as follows:

[0164]

[0165] Among them, C M and C m are the minimum and maximum values of the surface curvature respectively; TCI is the terrain curvature index, which can be used to judge the concavity and convexity of the terrain: TCI > 0.7: indicates that the terrain is very flat, usually appearing in areas such as plains and gentle slopes, 0.3 < TCI ≤ 0.7: indicates that the terrain has certain undulations, usually appearing in hilly or relatively gentle mountainous areas, TCI ≤ 0.3: indicates that the terrain is very rugged, usually appearing in areas such as ridges and steep slopes.

[0166] The topographic position index TPI can be obtained by statistically analyzing the elevation values around the pixel. The formula is as follows:

[0167]

[0168] Here, the value of TPI represents the relative position of the elevation value of the central pixel relative to the elevation values of the surrounding neighborhood. Eight adjacent pixels (i.e., a 3×3 neighborhood) around each pixel can be selected for estimation. E0 represents the elevation value of the central pixel, It indicates the average elevation of the surrounding neighborhood, which can be obtained from the digital elevation model. A positive TPI value indicates that the central pixel is located in a relatively high place, a negative value indicates that the central pixel is located in a relatively low place, and 0 indicates that the elevation value of the central pixel is equal to the average value of the surrounding neighborhood.

[0169] (3.2) Deformation rate estimation based on InSAR

[0170] Interference patterns are generated by the registered multi-phase InSAR images. DInSAR technology or other deformation monitoring technologies are used to analyze the multi-phase deformation phase data through phase unwrapping and time series analysis using the time series interference patterns to extract the deformation rate of each pixel.

[0171] The deformation rate data v, TCI and TPI data are combined into one data set, and the deformation rate data v is downscaled to a spatial resolution of 30 meters using statistical downscaling methods such as regression models or interpolation techniques. The combined data set is normalized to avoid deviations caused by differences in numerical ranges in the analysis, thereby obtaining the terrain characteristic parameter set V DEF , which will be used as the independent variable for machine learning in the future, thus completing the input variable for machine learning algorithm training.

[0172] S4. Training sample selection and model training

[0173] According to the characteristics of surface changes in landslide hazard areas, the surface state is divided and model training samples are selected for different surface state types. Model training is performed on the multi-spectral change characteristic parameter set and the terrain characteristic parameter set respectively until a model based on multi-spectral change characteristic parameters and a model based on terrain characteristic parameters are obtained. The specific process is as follows:

[0174] The first step is to select samples for model training.

[0175] According to the characteristics of surface changes in landslide-prone areas, the surface is divided into three states, including stable, slightly changed, and unstable. The stable state means that the surface in this state has no or only slight signs of deformation, showing long-term stability and no obvious abnormal changes; the slightly changed state means that the surface begins to show slight but recognizable changes, which may be due to seasonal factors, slight human interference or initial natural changes; the unstable state means that the surface shows obvious deformation or significant changes in other surface state parameters, which may indicate potential landslide risks.

[0176] For training samples of different state types, a combination of random sample selection and stratified manual sample selection can be used:

[0177] For the "stable" state samples, we use historical geological records to select areas where geological disasters (such as landslides, mudslides, etc.) have not occurred for a long time. Through years of optical and SAR images, we observe the historical records of surface deformation and geological disasters to determine the areas where no significant changes have occurred. Through the random point generation tool in the GIS software, we randomly select sample points in the above areas that are considered stable, and conduct field surveys or expert verification on the generated sample points to ensure that they are indeed located in the stable area. According to the verification results, the location of the sample points is adjusted to ensure the accuracy and representativeness of the samples, and the geographical distribution of the samples is random. Considering that the samples in the "stable" state may be far more than the other two states, the samples can be balanced through oversampling or undersampling techniques to ensure that the model does not favor any one state.

[0178] The samples of "slight change" and "unstable" states intend to use the existing historical landslide event records to mark the surface state of known disaster areas. In addition, through the knowledge of geologists and remote sensing experts, those areas that may not have experienced recorded disaster events but have potential risks are marked, and combined with the existing InSAR deformation results, they are marked as "slight change" and "unstable" states respectively. Expert review and possible on-site verification can be adopted to ensure the accuracy of sample labeling, thereby completing the construction of the surface state sample set S.

[0179] The second step is model training.

[0180] The multi-spectral variation characteristic parameter composition set V SUR And the terrain characteristic parameter set V DEF They are used as independent variables for machine learning, and the surface state sample set S is used as the dependent variable for machine learning. The surface state includes three types: stable, slightly changing, and unstable. The supporting independent variables and dependent variables are used as the input and output of the machine learning algorithm. In this process, appropriate machine learning or deep learning models, such as random forests, support vector machines, neural networks, etc., can be selected to simulate nonlinear complex relationships.

[0181] In order to facilitate the training of machine algorithms and the error estimation of subsequent machine learning algorithms, the input variables and surface state samples are divided into 70% and 30%, of which 70% of V SUR_train and V DEF_train The data will be used to train machine learning on changes in surface conditions. The model mapping training process can be expressed as follows:

[0182] f SUR =f train (V SUR_train ,S train ) (15)

[0183] f DEF =f train(V DEF_train ,S train ) (16)

[0184] Among them, S strain is the surface state sample set; V SUR_train 、V DEF_train are the 70% multi-spectral variation feature parameter set and terrain feature parameters used for training, respectively. SUR 、f DEF They are nonlinear models trained for machine learning, which are trained by the selected 70% input parameters and surface state sample sets.

[0185] The remaining 30% of the data set will be used to evaluate the performance of the machine learning algorithm to determine whether it has met the predetermined performance standards. If the algorithm performance fails to meet the established accuracy requirements, the algorithm must be iteratively trained until its recognition accuracy reaches or exceeds the expected threshold, and finally the above two types of models are obtained, namely the model based on multi-spectral change characteristic parameters and the model based on terrain characteristic parameters.

[0186] S5. Multimodal data fusion

[0187] Here, for the prediction results of the model of multi-spectral change characteristic parameters and the model based on terrain characteristic parameters, the probability average method is used for multi-modal fusion, which can output the probability prediction of each category. For the two types of models, the probability prediction of each category is first estimated:

[0188] P sur (Y|V SUR_pre ) = f SUR (V SUR_pre ) (17)

[0189] P def (Y|V DEF_pre ) = f DEF (V DEF_pre ) (18)

[0190] Among them, f SUR 、f DEF Respectively represent the prediction models trained with multi-spectral variation characteristic parameter set and terrain characteristic parameter; V SUR_pre and V DEF_pre They represent the multi-spectral variation characteristic parameter set and terrain characteristic parameters involved in the prediction respectively; P sur and P def Respectively represent f SUR and f DEF The conditional probability prediction for each possible class Y.

[0191] Then, the probabilities of each class are weighted averaged to obtain the combined probability of each class, and finally the class with the highest combined probability is selected as the final prediction.

[0192] P(Y)=ω sur ×P sur (Y|V SUR_pre )+ω sar ×P sar (Y|V SAR_pre ) (19)

[0193] Where P(Y) represents the final category probability obtained by weighted averaging the probability predictions of different models; V SUR_pre and V DEF_pre They represent the multi-spectral variation characteristic parameter set and terrain characteristic parameters involved in the prediction respectively; P sur represents f SUR For V SUR_pre The predicted probability for each category Y, P def represents f DEF For V DEF_pre The predicted probability for each category Y respectively; ω sur And ω def Represent the probability weight coefficient of each model respectively, and satisfy ω sur +ω def =1;

[0194] After obtaining the weighted average probability P(Y) of each category, the category with the highest comprehensive probability is finally selected as the final prediction result.

[0195] S6. Surface status determination

[0196] According to the multimodal fusion model obtained in the above steps, the aforementioned multispectral change characteristic parameter set and terrain characteristic parameters are used as independent variables of the recognition model, and the corresponding prediction probabilities are obtained respectively. Through weighted averaging, the final surface state can be obtained pixel by pixel. The results are mainly divided into three states: one is to maintain a stable state; the second is to show a slight change state; and the third is to show an unstable state. Finally, combined with the multimodal fusion prediction results, a regional surface state spatial distribution map can be drawn.

[0197] like Figure 2 As shown, this embodiment also provides a multi-modal fusion landslide hazard identification system, which includes six modules: a long-term multi-source data preprocessing module, a multi-spectral change characteristic parameter set estimation module, a terrain characteristic parameter set estimation module, a training sample selection and model training module, a multi-modal fusion module, and a surface state determination module. The functions of each module are as follows:

[0198] (1) Long-term multi-source data preprocessing module. It performs preprocessing such as denoising, terrain correction, and atmospheric correction of original remote sensing images, as well as geometric correction, reprojection, resampling, outlier removal, and region cropping of remote sensing products.

[0199] (2) Multispectral change characteristic parameter set estimation module. NDVI, FVC, and SAVI are estimated using 30m optical data, and a variety of machine learning algorithms are used to train the downscaling model to obtain high-resolution surface temperature and soil moisture, and to estimate the hydrothermal parameter spatial gradient index (NLSTG, NSMG). For NDVI, FVC, SAVI, NLSTG, and NSMG, the main slope, slope change rate, intercept, parameter autocorrelation, and coefficient of variation are calculated to obtain the multispectral change characteristic parameter set.

[0200] (3) Estimation model of terrain characteristic parameter set. The corresponding elevation, slope and aspect are obtained using the digital elevation model, and TCI and TPI are estimated. The deformation rate of each pixel is obtained through multi-temporal InSAR images. The deformation rate data v is statistically downscaled to a spatial resolution of 30 meters, and then merged with the TCI and TPI data into one data set. After normalization, the terrain characteristic parameter set is obtained.

[0201] (3) Training sample selection and model training module. Samples are selected from stable, slightly changing and unstable areas respectively. The multispectral change characteristic parameter set and terrain characteristic parameter set are used as independent variables for machine learning, and the surface state sample set is used as the dependent variable for machine learning. The input variables and surface state samples are divided into 70% and 30%, of which 70% and data will be used for machine learning surface state change training, and the remaining 30% of the data set will be used to evaluate the performance of the machine learning algorithm.

[0202] (4) Multimodal fusion module. The two models estimate the probability prediction for each category respectively, and the probability of each category is weighted averaged to obtain the comprehensive probability of each category. The category with the highest comprehensive probability will be used as the final prediction.

[0203] (5) Surface state determination module. The surface state of each pixel is divided into three states according to the category with the highest comprehensive probability: one is to maintain a stable state; the second is to show a slight change state; the third is to show an unstable state, and a spatial distribution map of the regional surface state is drawn.

[0204] In summary, the multimodal fusion landslide hazard identification method and system proposed in the present invention, in view of the limitations of the current landslide hazard identification technology and the high labor cost and low efficiency of the existing technology, is based on multi-source remote sensing data, and the long-term multispectral change feature set of parameters potentially related to landslides and terrain feature parameters are used to train the model respectively, and the results are multimodally fused, which can more comprehensively characterize the multispectral change characteristics related to landslides, capture dynamic change details, and efficiently and quickly carry out landslide hazard identification, effectively improve the accuracy of identification, and have the following technical advantages compared with the existing technology:

[0205] The present invention simultaneously considers the characteristic differences of the surface in optical, thermal infrared, microwave and InSAR signals, introduces the spectral, topographic and hydrothermal characteristic parameters that are closely related to landslide hazards, and performs multimodal fusion of the long-term multi-spectral change trend variables and the results based on the topographic characteristic parameters, so as to fully characterize the potential change characteristics of the surface state.

[0206] The present invention only needs to apply the multi-spectral change feature set and terrain feature parameters to the machine learning model to quickly obtain the recognition results of each pixel, greatly reducing the manpower and material costs, and solving the problem of inefficiency caused by the traditional method that requires a lot of time and manual processes.

[0207] The above implementation modes are not limitations of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the scope of the technical solution of the present invention also belong to the protection scope of the present invention.

Claims

1. A multi-modal identification and fusion method for landslide hazards, characterized by: The following steps are involved: S1. Acquisition and preprocessing of long-term multi-source data; S2. Construction of multi-spectral change characteristic parameter set: According to the closeness between long-term multi-source data and landslide, the multi-spectral characteristic parameter set is screened and constructed, and the long-term series change characteristics of the corresponding parameters are estimated one by one to construct the multi-spectral change characteristic parameter set; S3. Construction of terrain feature parameter set: The terrain characteristic parameter set is constructed by combining the deformation rate and terrain index obtained from InSAR data; S4, training sample selection and model training: according to the characteristics of the surface change in the landslide potential area, the surface state is divided and the model training samples are selected for different surface state types, and the model training is performed on the multi-spectral change characteristic parameter set and the terrain characteristic parameter set respectively, until a model based on the multi-spectral change characteristic parameter and a model based on the terrain characteristic parameter are obtained; S5. Multimodal data fusion: Multimodal fusion is performed on the model prediction results of multispectral variation characteristic parameters and the model prediction results based on terrain characteristic parameters; S6. Surface state determination: Determine the surface state based on the multimodal fusion model obtained in the above steps.

2. The multi-modal identification and fusion method for landslide hazard according to claim 1 is characterized by: In S1, the acquired data include multi-source optical, thermal infrared, microwave, terrain and InSAR remote sensing data, including: Long-term Landsat and MODIS surface reflectance datasets of different scales; digital elevation models; Landsat and MODIS surface temperature products; microwave soil moisture products, such as SMAP, including global surface and root layer soil moisture data; corresponding time series InSAR data.

3. The multi-modal identification and fusion method for landslide hazard according to claim 1 is characterized by: In S2, the construction process of the multi-spectral change characteristic parameter set is as follows: 2.1) Construction of multi-spectral feature parameter set; 2.2) Construction of multi-spectral variation characteristic parameter set; 2.1), the construction of the multispectral feature parameter set includes two steps: the first is to construct the spectral index parameter set, and the second is to construct the hydrothermal factor spatial gradient parameter set; The spectral index parameter set is constructed by combining the spectral indices closely related to landslides, including: Normalized Difference Vegetation Index NDVI, Fraction of Vegetation Coverage FVC and Soil Adjusted Vegetation Index SAVI; The constructed spatial gradient parameter set of water and heat factors includes: land surface temperature gradient index NLSTG and soil moisture gradient index NSMG; In 2.2), based on the spectral indexes NDVI, FVC, SAVI and the spatial gradient parameter indexes of hydrothermal factors NLSTG and NSMG obtained in the previous step, the surface state feature set is formed, and the corresponding multi-spectral change feature parameter set is further estimated. The change feature parameters include: main slope m, slope change rate r, intercept b, parameter free correlation ρ, and coefficient of variation CV. The dynamics of parameters related to landslides are reflected through the dynamics of long-term changes; Therefore, each of the five surface state parameters has five variation characteristic parameters, and a set V consisting of 25 multispectral variation characteristic parameters is obtained. SUR .

4. The multi-modal identification and fusion method for landslide hazard according to claim 3 is characterized by: 2.1), the specific calculation formulas of spectral index including normalized difference vegetation index NDVI, vegetation coverage fraction FVC and soil adjusted vegetation index SAVI are as follows: Among them, R nir is the reflectivity in the near-infrared band; R red is the reflectance of the red light band; L is the soil brightness correction factor, which ranges from (0,1) and is adjusted according to the specific vegetation type and soil characteristics; NDVI M and NDVI m are the maximum and minimum values ​​of NDVI, corresponding to the NDVI values ​​of bare soil and dense vegetation, respectively.

5. The multi-modal identification and fusion method for landslide hazard according to claim 4 is characterized by: 2.1), the construction process of the hydrothermal factor spatial gradient parameter set is: The first step is to downscale to a spatial resolution consistent with the spectral index parameters to obtain high-resolution surface temperature and soil moisture data; In the second step, combined with the high-resolution surface temperature and soil moisture data obtained in the previous step, the spatial gradients of water and heat parameters are estimated according to the following steps: First, the eight neighboring pixels around each pixel are selected to estimate the gradient; second, the pixel value difference between it and the surrounding neighboring pixels is calculated pixel by pixel; then, the distance between the central pixel and each neighboring pixel is calculated pixel by pixel; finally, the spatial gradient of each pixel is calculated by dividing the difference by the distance pixel by pixel; For each pixel, the spatial gradients of surface temperature and soil moisture are calculated using the following formulas: Where ΔX q is the average value of the parameter difference between the central pixel and all neighboring pixels; Δd q is the distance between the central pixel and the qth neighboring pixel; XG is the parameter space gradient calculated for each pixel; The third step is to normalize the spatial gradient of hydrothermal parameters. The linear normalization method is used to scale the spatial gradients of surface temperature and soil moisture to [-1, 1] so that their values ​​are within a unified scale. The specific normalization formula is as follows: Where XG is the calculated parameter spatial gradient of each pixel; XG M With XG m are the maximum and minimum values ​​of the parameter spatial gradient corresponding to the central pixel respectively; according to formula (4) (5), the normalized surface temperature gradient index NLSTG and soil moisture gradient index NSMG can be calculated pixel by pixel.

6. The multi-modal identification and fusion method for landslide hazard according to claim 5 is characterized by: 2.1) and 2.2), the specific calculation method of the change characteristic parameters is as follows: (2.21)The calculation formula of the main slope m is: where m n represents the slope of the nth point pair, n represents the nth point pair, i and j respectively represent different period numbers, satisfying i < j, and i, j ∈ [1, k]; y i and y j are respectively the multi-spectral feature parameter values corresponding to periods i and j; x i and x j are respectively the times corresponding to periods i and j; k is the total number of long time-series data; According to formula (6), two periods i and j are selected from the long time series data in turn, and finally N main slopes can be calculated, where Sort the N m values ​​from small to large to get an ordered point pair slope sequence {m1,m2...m N }, if N is an odd number, then the median slope is the middle value of the ordered sequence, that is, The slope corresponding to the point pair as the principal slope; if N is an even number, then the median slope is the and The average of the slopes of the points as the principal slope; (2.22) The slope change rate r is calculated as: Among them, r represents the average slope change rate, k represents the total number of long time series data, and m ij represents the slope change rate corresponding to the i-th period and the j-th period, Δt ij represents the time interval between the i-th period and the j-th period; (2.23) The formula for calculating the main intercept b is: Among them, p represents the pth pixel, s represents the sth point pair, m represents the main slope, and N is the total number of all point pairs, which can be expressed as k is the total number of long time series data, P p,s Represents the value of the multispectral characteristic parameter corresponding to the s-th point pair of the p-th pixel, t s Indicates the time corresponding to the sth point pair; Sort the intercept values ​​of all the point pairs from small to large to get an ordered point pair intercept sequence {b1, b2...b N }, if N is an odd number, then the median slope is the middle value of the ordered sequence, that is, The intercept corresponding to the point pair as the principal intercept; if N is even, then the median slope is the and The average of the slopes of the points as the principal intercept; (2.24) The calculation formula of parameter autocorrelation ρ is: in, It indicates the average value of the parameter P corresponding to the long time series, and ρ(z) indicates the degree of correlation between two parameter values ​​separated by z time units in the long time series. k represents the total number of long time series data; (2.25) The coefficient of variation CV is calculated as: Where σ represents the standard deviation of the surface state parameters, represents the average value of the parameter P corresponding to the long time series, k represents the total number of long time series data, P i represents the i-th sample value in the surface state parameter data set, i∈[1,k]; After the above steps, the five change characteristic parameters {m, r, b, ρ, CV} corresponding to the spectral indexes NDVI, FVC, SAVI and the hydrothermal parameter spatial gradient indexes NLSTG and NSMG are estimated. When the calculation of the five change characteristic parameters corresponding to the five surface state parameters is completed, a set V consisting of 25 multi-spectral change characteristic parameters will be obtained. SUR , the specific expression is as follows: Variation characteristic parameter set V SUR It will subsequently be used as an independent variable for machine learning, thus completing the input variable for machine learning algorithm training.

7. The multi-modal identification and fusion method for landslide hazard according to claim 1 is characterized by: In S3, the construction process of terrain feature parameter set is as follows: (3.1) Constructing terrain characteristic parameter set: using the preprocessed digital elevation model to obtain landslide-related terrain indices, including terrain curvature index TCI and terrain turbulence index TPI; First, the second-order derivative of the surface is calculated based on the surface elevation data, as shown below: Among them, C represents the surface curvature, E is the surface elevation, represents partial derivative, x and y are the horizontal positions of the surface coordinates, both of which can be obtained from the digital elevation model; Then, the calculated curvature is normalized, and the normalization calculation is as follows: Among them, C M and C m are the minimum and maximum values of the surface curvature respectively; TCI is used to judge the degree of terrain concavity and convexity. TCI > 0.7 indicates that the terrain is very flat, 0.3 < TCI ≤ 0.7 indicates that the terrain has a certain undulation, and TCI ≤ 0.3 indicates that the terrain is very rugged; The terrain turbulence index TPI can be obtained by counting the elevation values ​​around the pixel. The formula is as follows: Among them, E0 represents the elevation value of the central pixel, It indicates the average elevation of the surrounding neighborhood, which can be obtained from the digital elevation model. A positive TPI value indicates that the central pixel is located at a relatively high place, a negative value indicates that the central pixel is located at a relatively low place, and 0 indicates that the elevation value of the central pixel is equal to the average value of the surrounding neighborhood. (3.2) Deformation rate estimation based on InSAR: The interference pattern is generated by the registered multi-temporal InSAR images, and the deformation rate v of each pixel is extracted; the deformation rate v, terrain curvature index TCI and terrain turbulence index TPI data are merged into a data set, in which the deformation rate data v is downscaled; the merged data set is normalized, thereby obtaining the terrain characteristic parameter set V DEF , V DEF It will subsequently be used as an independent variable for machine learning, thus completing the input variable for machine learning algorithm training.

8. The multi-modal identification and fusion method for landslide hazard according to claim 1 is characterized in that: In S4, the method for selecting model training samples is: According to the characteristics of surface changes in landslide-prone areas, the surface is divided into three states, including stable, slightly changing, and unstable. Random selection samples and stratified manual selection samples are combined to select training samples for different surface state types. In S4, the model training process is as follows: The multi-spectral variation characteristic parameter composition set V SUR And the terrain characteristic parameter set V DEF They are used as independent variables for machine learning, and the surface state sample set S is used as the dependent variable for machine learning; The input variables and surface state samples are divided into 70% and 30%, of which 70% of V SUR_train and V DEF_train The data will be used to train machine learning on changes in surface conditions. The model mapping training process is represented by the following formula: f SUR =f train (V SUR_train ,S train ) (15) f DEF =f train (V DEF_train ,S train ) (16) Among them, S strain is the 70% surface state sample set used for training; V SUR_train 、V DEF_train are the 70% multi-spectral variation feature parameter set and terrain feature parameters used for training, respectively. SUR 、f DEF They are nonlinear models trained by machine learning, which are trained by the selected 70% input parameters and surface state sample sets; The remaining 30% of the data set is used to evaluate the performance of the machine learning algorithm to determine whether it has met the predetermined performance standards; if the algorithm performance fails to meet the established accuracy requirements, iterative training is performed until its recognition accuracy reaches or exceeds the expected threshold, and ultimately the above two types of models are obtained, namely, models based on multi-spectral change characteristic parameters and models based on terrain characteristic parameters.

9. The multi-modal identification and fusion method for landslide hazard according to claim 1 is characterized in that: In S5, the probability average method is used for multimodal fusion to output the probability prediction of each category. For the two types of models, the probability prediction of each category is first estimated: P sur (Y|V SUR_pre )=f SUR (V SUR_pre ) (17) P def (Y|V DEF_pre )=f DEF (V DEF_pre ) (18) Among them, f SUR 、f DEF Respectively represent the prediction models trained with multi-spectral variation characteristic parameter set and terrain characteristic parameter; V SUR_pre and V DEF_pre They represent the multi-spectral variation characteristic parameter set and terrain characteristic parameters involved in the prediction respectively; P sur represents f SUR For V SUR_pre The predicted probability for each category Y, P def represents f DEF For V DEF_pre The predicted probability for each category Y respectively; Then, the probabilities of each class are weighted averaged to obtain the combined probability of each class, and finally the class with the highest combined probability is selected as the final prediction: P(Y)=ω sur ×P sur (Y|V SUR_pre )+ω sar ×P sar (Y|V SAR_pre ) (19) Where P(Y) represents the final category probability obtained by weighted averaging the probability predictions of different models; ω sur And ω def Represent the probability weight coefficient of each model respectively, and satisfy ω sur +ω def =1; After obtaining the weighted average probability P(Y) of each category, the category with the highest comprehensive probability is finally selected as the final prediction result.

10. A multi-modal identification and fusion system for landslide hazards, characterized by: Includes the following modules: Long time series multi-source data preprocessing module, used for preprocessing the collected long time series multi-source data; The module for estimating the characteristic parameter set of multi-spectral changes uses 30m optical data to estimate the normalized vegetation index NDVI, vegetation coverage FVC, and soil adjusted vegetation index SAVI. It uses a variety of machine learning algorithms to train the downscaling model to obtain high-resolution surface temperature and soil moisture, and estimates the spatial gradient index of water and heat parameters: surface temperature gradient index NLSTG, soil moisture gradient index NSMG; for NDVI, FVC, SAVI, NLSTG, and NSMG, the main slope, slope change rate, intercept, parameter autocorrelation, and coefficient of variation are calculated to obtain the characteristic parameter set of multi-spectral changes. The terrain characteristic parameter set estimation module uses the digital elevation model to obtain the corresponding elevation, slope and slope direction, estimates the terrain curvature index TCI and terrain turbulence index TPII, and obtains the deformation rate of each pixel through multi-temporal InSAR images; the deformation rate data v is statistically downscaled to a spatial resolution of 30 meters, and merged with the TCI and TPI data into a data set, and the terrain characteristic parameter set is obtained after normalization; The training sample selection and model training module selects samples from stable, slightly changing and unstable areas respectively. The multispectral change characteristic parameter set and terrain characteristic parameter are used as independent variables of machine learning respectively, and the surface state sample set is used as the dependent variable of machine learning. The model is iteratively trained and evaluated until a model based on multispectral change characteristic parameters and a model based on terrain characteristic parameters are obtained. The multimodal fusion module fuses the two models, and the category with the highest comprehensive probability will be used as the final prediction model. The surface state determination module calls the final prediction model to determine the surface state and output the drawn regional surface state spatial distribution map.

Citation Information

Patent Citations

  • Hot melt pond state identification method combining long time sequence data and machine learning algorithm

    CN112801007A

  • Terrain recognition method based on multi-feature fusion

    CN114758246A

  • Landslide and surrounding ground feature identification method and system based on multi-modal feature fusion

    CN115830469A

  • Intelligent identification method for high-position collapse and slip geological disasters

    CN117216641A

  • Method, system, device and medium for landslide identification based on full polarimetric SAR

    US11747498B1

Cited By

  • Power grid emergency disposal method for risk assessment

    CN119962963A

  • Landslide disaster hidden danger point intelligent identification method based on unmanned aerial vehicle multispectral remote sensing

    CN121259650A

  • A landslide disaster hidden point intelligent identification method based on unmanned aerial vehicle multispectral remote sensing

    CN121259650B

  • Landslide deformation capturing method and system based on InSAR

    CN121686267A

  • A mountain landslide hidden danger identification method based on multi-source remote sensing fusion

    CN122712358A