A semantic correlation-based multi-source remote sensing image landslide extraction method

Abstract

The application provides a kind of multi-source remote sensing image landslide extraction method based on semantic association, specific steps are as follows: constructing multi-source remote sensing image landslide data set;Establish FMEformer neural network model;According to the obtained data set, the established model is trained, and a trained landslide extraction model is obtained;The multi-source remote sensing image data to be extracted is input into the trained landslide extraction model, and multi-source remote sensing image landslide extraction is realized.The application builds FMEformer model based on CNN-Transformer hybrid architecture, combines local feature coding with remote context modeling, introduces global-local frequency domain modulation module at the same time, carries out frequency domain modeling and local space pattern enhancement;While enhancing the high frequency details of landslide area, effectively suppresses background noise, solves the problem that landslide and surrounding environment are difficult to distinguish in complex background and small spectral difference remote sensing image, significantly improves the accuracy and robustness of landslide feature expression under different regional and multi-source image conditions.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method for extracting landslides from multi-source remote sensing images based on semantic association. Background Technology

[0002] Landslides are a significant type of natural disaster, particularly in mountainous areas and regions with frequent tectonic activity, causing widespread damage to human settlements, infrastructure, and ecosystems. With climate change exacerbating extreme rainfall events and freeze-thaw cycles, the frequency and severity of landslides globally are on the rise. Therefore, rapid identification of landslide events is crucial for post-disaster emergency response, infrastructure safety, and long-term disaster risk management.

[0003] The development of remote sensing technology has provided new avenues for landslide identification, especially the application of multi-source satellite data, such as optical, multispectral, and synthetic aperture radar imagery, which has made large-scale landslide monitoring possible. Traditional methods are widely used, but still rely on human expertise and cannot meet the needs of rapid mapping of diverse and rugged terrain. Therefore, deep learning techniques, especially convolutional neural networks and Transformer models, are gradually becoming key to improving the accuracy and generalization of landslide extraction.

[0004] However, existing methods mainly focus on spatial domain modeling. Although they improve extraction efficiency, their adaptability to complex terrain and variable environments is limited. Different regions have significant differences in spectral characteristics, land cover types, terrain complexity, vegetation density, and landslide morphology and distribution. When spatial domain methods are applied to scenes with different spectral characteristics, texture patterns, and morphological complexities, their performance degrades significantly, limiting the practical application of landslide extraction models.

[0005] Therefore, frequency domain learning, as an effective supplementary approach, has attracted attention in landslide extraction in recent years. Frequency domain methods can better capture high-frequency features related to landslide boundaries, thereby improving the accuracy of landslide extraction. However, frequency domain methods still face limitations such as high noise levels and a lack of explicit semantic correspondence. Without adaptive modulation, they may amplify redundant or irrelevant signals, limiting their effectiveness in complex environments. When existing schemes are applied to landslide detection tasks with complex terrain, they often struggle to balance global context with local fine-grained features, easily introducing background noise and blurring high-frequency edge details. In summary, the lack of adaptive modulation in frequency domain methods limits their effectiveness in handling complex and heterogeneous environments. Summary of the Invention

[0006] The main objective of this invention is to provide a method for extracting landslides from multi-source remote sensing images based on semantic association in complex terrain and variable environments.

[0007] The landslide extraction method based on semantic association from multi-source remote sensing images provided by this invention comprises the following steps:

[0008] S1. Construct a multi-source remote sensing imagery dataset of landslides;

[0009] S2. Establish the FMEformer neural network model;

[0010] S3. Based on the dataset obtained in step S1, train the model established in step S2 to obtain the trained landslide extraction model.

[0011] S4. Input the multi-source remote sensing image data to be extracted into the landslide extraction model trained in step S3 to realize landslide extraction from multi-source remote sensing images.

[0012] Step S1 is as follows:

[0013] Five earthquake-induced landslide datasets were constructed, and the region, occurrence time, data source, and spatial resolution of each dataset were obtained.

[0014] The above multi-source remote sensing images were preprocessed as follows: based on the actual location of the landslide, each image was randomly cropped into a 256×256 pixel patch; the image channels were uniformly selected as green, blue and near-infrared channels.

[0015] Two datasets were used as training sets with 80% of the samples, and the remaining 20% ​​were used as regional evaluation test sets. The other three datasets were used as cross-regional test sets, and no retraining or fine-tuning was performed during the evaluation process.

[0016] Step S2 is as follows:

[0017] The overall model architecture is constructed, and the FMEformer model includes the encoding path, boundary enhancement module, decoding path, and output path;

[0018] The encoding path is used for feature extraction and downsampling of the input image. It includes a convolutional front-end structure, a global-local frequency domain modulation module, a block merging layer, and a hybrid CNN-Transformer module, forming a sequentially connected input module and four downsampling stages. Each stage generates feature maps at different scales. The encoding path structure is as follows:

[0019] The input image is first processed by a convolutional front-end structure to extract features, resulting in an initial feature map. The initial feature map is then input into a global-local frequency domain modulation module for frequency domain structure readjustment, resulting in a readjusted feature map.

[0020] The readjusted feature map is downsampled through a block merging layer and input into a hybrid CNN-Transformer module to capture fine-grained spatial details and global semantic context. Then, it is input into a global-local frequency domain modulation module for frequency domain structure readjustment to obtain a first-stage feature map with a resolution of {1 / 4}.

[0021] The first-stage feature map is split into two paths. One path generates the first-level skip connection features through the boundary enhancement module, ultimately generating the first-stage enhanced feature map. The other path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the second-stage feature map with a resolution of {1 / 8}.

[0022] The second-stage feature map is split into two paths. One path generates the second-level skip connection features through the boundary enhancement module, ultimately generating the second-stage enhanced feature map. The other path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the third-stage feature map with a resolution of {1 / 16}.

[0023] The third-stage feature map is split into two paths. One path generates the third-level skip connection features through the boundary enhancement module, ultimately producing the third-stage enhanced feature map. The other path is transmitted to the next encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the fourth-stage feature map with a resolution of {1 / 32}. This fourth-stage feature map is used as the starting input for the decoding path.

[0024] The feature enhancement module is specifically a multi-directional stripe boundary enhancement module, located in the skip connection between the encoding path and the decoder path;

[0025] The decoding path includes multiple upsampling stages. Each upsampling stage fuses the output features of the previous level decoder with the enhanced features output by the same level feature enhancement module. The decoding path structure is as follows:

[0026] The decoder first receives the fourth-stage feature map, and after upsampling, it concatenates the feature map with the third-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the third-stage decoded feature map.

[0027] The third-stage decoded feature map is then upsampled and concatenated with the second-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the second-stage decoded feature map.

[0028] The second-stage decoded feature map is then upsampled and concatenated with the first-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the first-stage decoded feature map, which is used as the starting input for the output path.

[0029] The output path maps the final output of the decoding path to the target result; the first-stage decoded feature map is input into the classification head to obtain the final target image.

[0030] The specific structure of the global-local frequency domain modulation module is as follows:

[0031] The global-local frequency domain modulation module includes a Fourier global frequency domain modulation unit, a wavelet multi-scale decomposition unit, a multi-level local variance adaptive modulation unit, and an inverse wavelet reconstruction unit;

[0032] The specific structure of the Fourier global frequency domain modulation unit is as follows:

[0033] Given an input feature map First, the complex frequency domain spectrum is obtained by applying a two-dimensional fast Fourier transform. And decompose it into amplitude terms and normalized phase term The calculation formulas are as follows:

[0034]

[0035]

[0036]

[0037] In the formula, FFT represents Fast Fourier Transform, and "mag" is the complex frequency domain spectrum F. G The magnitude, i.e., the modulus; Re(F G ) represents the complex frequency domain spectrum F G The real part, Im(F) G ) represents the complex frequency domain spectrum F G The imaginary part of the complex number, sqrt() represents the square root operation; "phasor" represents the unit phase vector of the complex frequency domain spectrum; "epsilon" is a minimal constant to prevent numerical instability.

[0038] Amplitude Term The updated amplitude term is obtained by adaptively modulating a channel blending structure consisting of two 1×1 convolutional layers and a GELU activation function. Then with the phase term Recombining and returning to the spatial domain yields global modulation features. The calculation formula is:

[0039]

[0040] In the formula, iFFT represents the inverse fast Fourier transform; subsequently, depthwise separable 5×5 convolutions are used to further extract features, and then... iMultiply and then add the weighted sums to obtain the global modulation features. .

[0041] The specific structure of the wavelet multi-scale decomposition unit is as follows:

[0042] global modulation features Input wavelet multiscale decomposition unit and perform two-level wavelet decomposition: the first-level decomposition yields the low-frequency approximation component LL1 and three high-frequency detail components LH1, HL1 and HH1; the second-level wavelet decomposition of LL1 yields LL2, LH2, HL2 and HH2.

[0043] The specific structure of the multi-level local variance adaptive modulation unit is as follows:

[0044] The components obtained from wavelet decomposition, i.e., wavelet features The input is a multi-level local variance adaptive modulation unit, whose local variance is calculated using a 3×3 sliding window, and then encoded into a stable variance representation through a lightweight convolutional network. The calculation formula is:

[0045]

[0046]

[0047] In the formula, This represents the mean within a 3×3 local window, where x represents the pixel value within the local window that participates in the variance calculation. This represents a nonlinear mapping of three convolutional layers plus the GELU activation function; This represents the local variance feature calculated from the second moment of the local window and the squared difference of the mean; σ² represents the variance normalization performed on the local variance feature.

[0048] Global average pooling and a multilayer perceptron are used to generate the weights for each layer. Furthermore, direction-sensitive high-frequency weights are learned through additional transformations. Finally, the modulation weight of each high-frequency component is obtained by combining the hierarchical weight and the directional weight. The calculation formula is:

[0049]

[0050]

[0051]

[0052] In the formula, W1 represents the expression used to analyze the local variance feature F. v The first learnable mapping parameter for channel dimensionality reduction; W2 represents the layer weights w used to map the dimensionality-reduced features. levelThe second learnable mapping parameter; W3 represents the third learnable mapping parameter used to generate the direction-sensitive high-frequency weight intermediate features; W4 represents the parameter used to map the intermediate features to high-frequency weights W. HF The fourth learnable mapping parameter; δ represents the nonlinear activation function, preferably the GELU activation function; w level W represents the hierarchical weights of different wavelet levels. HF W represents direction-sensitive high-frequency weights. i This represents the final modulation weight corresponding to the i-th high-frequency component; express Features obtained through two convolutional layers and activation functions; Softmax() represents the normalization exponential function, used to normalize the layer weight scores; Sigmoid() represents the sigmoid activation function, used to limit the directional weight scores to between 0 and 1;

[0053] Introducing learnable scaling parameters To adjust variance features With modulation weights The response intensity between the two is used to generate a modulated signal. It is used to enhance two high-frequency wavelet subbands, and the calculation formula is as follows:

[0054]

[0055]

[0056] In the formula, h0 represents the high-frequency subband splicing feature obtained from the first-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH1, HL1 and HH1; h1 represents the high-frequency subband splicing feature obtained from the second-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH2, HL2 and HH2; Tanh() represents the hyperbolic tangent activation function.

[0057] The specific structure of the inverse wavelet reconstruction unit is as follows:

[0058] The enhanced high-frequency wavelet subbands are reconstructed step by step through inverse wavelet transform, and the second-level wavelet components are fused to obtain the final product. Then, it is reconstructed from the first-level wavelet components to generate... Finally, a gated residual learning mechanism is used to fuse the reconstructed features with the original input features to obtain the output features of the global-local frequency domain modulation module. The calculation formula is as follows:

[0059]

[0060]

[0061]

[0062] In the formula, iWT represents the inverse wavelet transform; F L This represents the second-level reconstructed feature obtained by inverse wavelet transform of the second-level wavelet components LL2, LH2', HL2', and HH2'; F L 'Indicates by F L The first-level reconstructed features are obtained by inverse wavelet transform of the first-level wavelet high-frequency components LH1', HL1', and HH1'; F i This represents the original characteristics of the input global-local frequency domain modulation module; F o This represents the fusion characteristics of the output of the global-local frequency domain modulation module; Conv 1×1 () denotes a 1×1 convolution operation, used to perform F... i Perform channel mapping and residual compensation; ⊙ indicates element-wise multiplication;

[0063] The specific structure of the multi-directional stripe boundary enhancement module is as follows:

[0064] The multi-directional stripe boundary enhancement module includes a directional pooling unit, a spatial recalibration unit, an edge perception enhancement unit, and an output unit;

[0065] The specific structure of the directional pooling unit is as follows:

[0066] Given input feature tensor Where C, H, and W represent the number of channels, height, and width of the feature map, respectively;

[0067] Horizontal pooling is used to highlight the lateral extension of landslides in the spatial domain, thereby obtaining an axial feature representation. Vertical pooling is used to highlight the longitudinal extension of landslides in the spatial domain, thereby obtaining an axial feature representation. Furthermore, diagonal pooling is defined to extract landslides with oblique features, producing a diagonal feature representation. The calculation formula is:

[0068]

[0069]

[0070]

[0071] In the formula, F p denoted by b; b represents the batch index; c represents the channel index; h represents the spatial position index along the height direction of the feature map; w represents the spatial position index along the width direction of the feature map; H represents the feature map height; W represents the feature map width; K represents the length of the diagonal pooling window; r represents the diagonal pooling radius, and K = 2r + 1; t represents the position offset within the diagonal pooling window.

[0072] The specific structure of the space recalibration unit is as follows:

[0073] Use depthwise separable convolutions to extract input features Local features are extracted, and then weighted and combined with diagonal pooling features to construct a direction-aware representation. Furthermore, rectangular attention weights are generated based on horizontal and vertical pooling features. Spatial recalibration is performed through interactive multiplication between diagonal and axial features to obtain refined output features. ;

[0074] The specific structure of the edge-aware enhancement unit is as follows:

[0075] Given input features The Scharr operator is introduced to calculate the horizontal gradient. and vertical gradient Based on this, the edge amplitude is obtained, and the calculation formula is:

[0076]

[0077]

[0078]

[0079] In the formula, x represents the feature tensor of the input edge-aware enhancement unit; i and j represent the spatial position indices in the height and width directions of the feature map, respectively; c represents the channel index; u and v represent the offsets of the Scharr convolution kernel in the height and width directions, respectively; K x Represents the horizontal Scharr convolution kernel; K y Indicates a vertical Scharr convolution kernel; G x G represents the horizontal gradient response; y ε represents the vertical gradient response; edge represents the edge magnitude map; ε is a small constant introduced for numerical stability.

[0080] The specific structure of the output unit is as follows:

[0081] Edge map After being adjusted by a learnable scaling factor α, it is added to the refined output features. In the middle, ReLU activation is performed to obtain the output features of the multi-directional stripe boundary enhancement module. .

[0082] Step S3 is as follows:

[0083] a. Training data preparation and augmentation; specific steps are as follows:

[0084] The training set constructed in step S1 is used as the original training samples, each sample being a 256×256 pixel three-channel remote sensing image patch; data augmentation strategies are applied to each original patch, including random cropping, rotation, flipping, contrast enhancement, and Gaussian blur; a total of 15048 training patches are generated.

[0085] b. Loss function design; the specific steps are as follows:

[0086] For the pixel-level binary classification task of landslide extraction, a composite loss function is used to supervise the training of the network.

[0087]

[0088] In the formula, L total L represents the total loss function; CE L represents the binary cross-entropy loss, used to constrain pixel-level classification results; Dic e represents the Dice loss, used to improve the overlap between the predicted landslide area and the actual labeled area; L Foca l represents the Focal loss, used to reduce the weights of easily classified samples and enhance the learning of difficult-to-classify samples; L Lovász denoted as Lovász loss, used to directly optimize the intersection-union ratio (IU) in the segmentation task; α, β, γ, and δ represent the weight coefficients of the above loss terms, with values ​​of 1.0, 0.6, 0.5, and 0.5, respectively, and are optimized based on the performance of the validation set.

[0089]

[0090] In the formula, N represents the number of samples or pixels; y i represents the true label of the i-th sample or pixel; p represents the probability that the model predicts the sample or pixel belongs to the landslide category;

[0091]

[0092] In the formula, A represents the target region predicted by the model; B represents the ground truth region; |A ∩ B| represents the intersection size of the predicted region and the ground truth region; |A| represents the area or number of pixels of the predicted region; |B| represents the area or number of pixels of the ground truth region.

[0093]

[0094] In the formula, This represents the model's predicted probability of the true class. It is the category balance factor, used to adjust the importance of different categories; γ is the focusing parameter;

[0095]

[0096] In the formula, C represents the set of categories; c represents a specific category; and IoU(c) represents the intersection-union ratio of category c.

[0097] c. Optimizer and learning rate scheduling configuration; the specific steps are as follows:

[0098] The AdamW optimizer was used to update the network parameters, with momentum parameters set to β1=0.9 and β2=0.999, and weight decay coefficient set to 1×10⁻. 4 The initial learning rate is set to 2 × 10⁻ 4 Furthermore, cosine annealing scheduling is introduced to dynamically adjust the learning rate and enhance convergence stability; considering hardware memory limitations, the batch size is fixed at 18;

[0099] d. Iterative training; the specific steps are as follows:

[0100] The 15048 generated training patches are randomly shuffled and fed into the FMEformer neural network model established in step S2 in batches of 18 for forward propagation to obtain the landslide prediction probability map for each pixel; the loss value is calculated according to the joint loss function and backpropagation is performed, and the AdamW optimizer updates the network weights according to the learning rate scheduling strategy.

[0101] During training, the change of the loss function value with each round is recorded. When the model reaches convergence after a certain round, that is, when the loss value tends to stabilize and no longer decreases significantly, the current model parameters and the corresponding network structure are saved to obtain the trained landslide extraction model.

[0102] Step S4 is as follows:

[0103] The multi-source remote sensing images of the landslides to be extracted are preprocessed to the same size and channels as the training samples; the images are cropped into 256×256 pixel patches according to the possible distribution of the landslides, and the three channels of green, blue and near-infrared are uniformly selected, and the pixel values ​​are normalized.

[0104] Input the preprocessed pixel patch into the trained FMEformer landslide extraction model obtained in step S3; obtain the landslide extraction result for the patch.

[0105] The beneficial effects of this invention are as follows:

[0106] 1. By constructing an FMEformer model based on a CNN-Transformer hybrid architecture, local feature encoding is combined with long-range context modeling. At the same time, a global-local frequency domain modulation module is introduced, and Fourier transform and wavelet decomposition are used to jointly perform frequency domain modeling and local spatial pattern enhancement. This feature expression mechanism that combines the frequency domain and spatial domain can adaptively adjust the global and local frequency domain structures, effectively suppressing background noise while enhancing high-frequency details in the landslide area. It solves the problem of difficulty in distinguishing landslides from the surrounding environment in remote sensing images with complex backgrounds and small spectral differences, and significantly improves the accuracy and robustness of landslide feature expression under different regional and multi-source image conditions.

[0107] 2. By setting up a multi-directional strip boundary enhancement module, directional pooling and edge awareness enhancement mechanisms are introduced to pool strip features in the horizontal, vertical and diagonal directions, and the Scharr operator is used to explicitly enhance the boundary gradient response. This can effectively capture the anisotropic features and complex boundary morphology of landslide areas. In particular, for landslide areas with irregular boundaries and fragmented morphology, the boundary localization accuracy and extraction completeness are significantly improved, and the model's ability to extract landslides under complex terrain and heterogeneous surface cover conditions is enhanced. Attached Figure Description

[0108] Figure 1 This is a flowchart of the landslide extraction method based on semantic association from multi-source remote sensing images according to the present invention.

[0109] Figure 2 This is a schematic diagram of the overall structure of the FMEformer neural network model.

[0110] Figure 3 This is a schematic diagram of the global-local frequency domain modulation module structure.

[0111] Figure 4 This is a schematic diagram of a multi-level local variance adaptive modulation unit structure.

[0112] Figure 5 This is a schematic diagram of the multi-directional stripe boundary reinforcement module structure. Detailed Implementation

[0113] The relevant technical solutions will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. The described embodiments are only a part of the embodiments, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0114] like Figure 1As shown, this embodiment discloses a landslide extraction method based on semantic association from multi-source remote sensing images. By constructing a multi-source cross-regional dataset and combining a frequency domain modeling and spatial context coding FMEformer neural network model, high-precision, cross-regional landslide extraction is achieved under complex terrain and variable remote sensing image conditions. Specifically, the method includes the following steps:

[0115] S1. Construct a multi-source remote sensing imagery dataset of landslides; the steps are as follows:

[0116] Based on two large-scale multi-sensor remote sensing datasets, CAS and GDCLD, five earthquake-induced landslide datasets were constructed, and the region, occurrence time, data source and spatial resolution of each dataset were obtained.

[0117] The above multi-source remote sensing images were preprocessed as follows: based on the actual location of the landslide, each image was randomly cropped into a 256×256 pixel patch; the image channels were uniformly selected as green, blue and near-infrared channels.

[0118] Two datasets were used as training sets with 80% of the samples, and the remaining 20% ​​were used as regional evaluation test sets. The other three datasets were used as cross-regional test sets to directly evaluate the transferability of the model, without retraining or fine-tuning during the evaluation process.

[0119] S2. Establish the FMEformer neural network model; the steps are as follows:

[0120] An FMEformer model based on a CNN-Transformer hybrid architecture was established, which combines local feature encoding with remote context modeling to solve common problems in remote sensing images such as weak landslide signals, large landslide scale variations, and fragmented boundary structures.

[0121] A. Construct the overall framework of the model; the specific steps are as follows:

[0122] like Figure 2 As shown, the FMEformer model includes an encoding path, a boundary enhancement module, a decoding path, and an output path;

[0123] The encoding path is used for feature extraction and downsampling of the input image. It includes a convolutional front-end structure, a global-local frequency domain modulation module, a block merging layer, and a hybrid CNN-Transformer module, forming a sequentially connected input module and four downsampling stages. Each stage generates feature maps at different scales. The encoding path structure is as follows:

[0124] The input image is first processed by a convolutional front-end structure to extract features, resulting in an initial feature map. The initial feature map is then input into a global-local frequency domain modulation module for frequency domain structure readjustment, resulting in a readjusted feature map.

[0125] The readjusted feature map is downsampled through a block merging layer and input into a hybrid CNN-Transformer module to capture fine-grained spatial details and global semantic context. Then, it is input into a global-local frequency domain modulation module for frequency domain structure readjustment to obtain a first-stage feature map with a resolution of {1 / 4}.

[0126] The first-stage feature map is split into two paths. One path generates the first-level skip connection features through the boundary enhancement module, ultimately generating the first-stage enhanced feature map. The other path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the second-stage feature map with a resolution of {1 / 8}.

[0127] The second-stage feature map is split into two paths. One path generates the second-level skip connection features through the boundary enhancement module, ultimately generating the second-stage enhanced feature map. The other path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the third-stage feature map with a resolution of 1 / 16.

[0128] The third-stage feature map is split into two paths. One path generates the third-level skip connection features through the boundary enhancement module, ultimately producing the third-stage enhanced feature map. The other path is transmitted to the next encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the fourth-stage feature map with a resolution of {1 / 32}. This fourth-stage feature map is used as the starting input for the decoding path.

[0129] The feature enhancement module is specifically a multi-directional stripe boundary enhancement module, located in the skip connection between the encoding path and the decoder path;

[0130] The decoding path includes multiple upsampling stages. Each upsampling stage fuses the output features of the previous level decoder with the enhanced features output by the same level feature enhancement module. The decoding path structure is as follows:

[0131] The decoder first receives the fourth-stage feature map, and after upsampling, it concatenates the feature map with the third-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the third-stage decoded feature map.

[0132] The third-stage decoded feature map is then upsampled and concatenated with the second-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the second-stage decoded feature map.

[0133] The second-stage decoded feature map is then upsampled and concatenated with the first-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the first-stage decoded feature map, which is used as the starting input for the output path.

[0134] The output path maps the final output of the decoding path to the target result; the first-stage decoded feature map is input into the classification head to obtain the final target image;

[0135] B. Construct a global-local frequency domain modulation module; the specific steps are as follows:

[0136] The aforementioned global-local frequency domain modulation module addresses common issues in landslide extraction, such as spectral inconsistency, suppression of local details, and background interference. Landslides typically exhibit weak spectral separation from the surrounding surface, are irregularly shaped and fragmented, and suffer from significant texture variations due to slope discontinuities, surface roughness, and the interaction between shadows and vegetation. To better capture these features, the global-local frequency domain modulation module introduces a novel dual-domain representation mechanism, jointly reshaping the global frequency domain structure and enhancing local spatial patterns, thereby providing more stable and discriminative feature representations for subsequent stages of the network.

[0137] like Figure 3 As shown, the global-local frequency domain modulation module includes a Fourier global frequency domain modulation unit, a wavelet multi-scale decomposition unit, a multi-level local variance adaptive modulation unit, and an inverse wavelet reconstruction unit.

[0138] The specific structure of the Fourier global frequency domain modulation unit is as follows:

[0139] Given an input feature map First, the complex frequency domain spectrum is obtained by applying a two-dimensional fast Fourier transform. And decompose it into amplitude terms and normalized phase term The calculation formulas are as follows:

[0140]

[0141]

[0142]

[0143] In the formula, FFT represents Fast Fourier Transform, and "mag" is the complex frequency domain spectrum F. G The magnitude, i.e., the modulus; Re(F G ) represents the complex frequency domain spectrum F G The real part, Im(F) G ) represents the complex frequency domain spectrum F GThe imaginary part of the complex number, sqrt() represents the square root operation; "phasor" represents the unit phase vector of the complex frequency domain spectrum; "epsilon" is a minimal constant to prevent numerical instability.

[0144] Amplitude Term The updated amplitude term is obtained by adaptively modulating a channel blending structure consisting of two 1×1 convolutional layers and a GELU activation function. Then with the phase term Recombining and returning to the spatial domain yields global modulation features. The calculation formula is:

[0145]

[0146] In the formula, iFFT represents the inverse fast Fourier transform; subsequently, depthwise separable 5×5 convolutions are used to further extract features, and then... i Multiply and then add the weighted sums to obtain the global modulation features. .

[0147] This adjustment reshapes the global frequency domain pattern while preserving geometric information, enabling the representation to better respond to coherent landslide morphologies distributed in complex mountainous terrain; at the same time, it obtains fine features through residual gating. ;

[0148] Although Fourier-based modulation provides powerful global modeling capabilities, its globally supported basis functions limit its ability to capture local and small-scale landslide structures, which typically manifest as narrow strips, fragmented deposits, or isolated patches. To address this limitation, wavelet transform is needed to obtain hierarchical multi-scale decomposition.

[0149] The specific structure of the wavelet multi-scale decomposition unit is as follows:

[0150] global modulation features Input wavelet multi-scale decomposition units and perform two-level wavelet decomposition: the first-level decomposition yields the low-frequency approximation component LL1 and three high-frequency detail components LH1, HL1 and HH1; the second-level wavelet decomposition of LL1 yields LL2, LH2, HL2 and HH2; thus enabling the network to jointly represent the fine-scale discontinuities and large-scale structural patterns of landslides.

[0151] Because landslides exhibit significant local variance characteristics, their boundaries and internal textures often show more abrupt changes than the surrounding terrain. To utilize this discriminative property, a multi-level local variance adaptive mechanism was integrated to guide wavelet subband modulation;

[0152] like Figure 4 As shown, the specific structure of the multi-level local variance adaptive modulation unit is as follows:

[0153] The components obtained from wavelet decomposition, i.e., wavelet features The input is a multi-level local variance adaptive modulation unit, whose local variance is calculated using a 3×3 sliding window, and then encoded into a stable variance representation through a lightweight convolutional network. The calculation formula is:

[0154]

[0155]

[0156] In the formula, This represents the mean within a 3×3 local window, where x represents the pixel value within the local window that participates in the variance calculation. This represents a nonlinear mapping of three convolutional layers plus the GELU activation function; This represents the local variance feature calculated from the second moment of the local window and the squared difference of the mean; σ² represents the variance normalization performed on the local variance feature.

[0157] To dynamically allocate importance across different wavelet layers, global average pooling and a multilayer perceptron are used to generate the weights for each layer. Furthermore, considering that the background region may also have high variance, orientation-sensitive high-frequency weights are learned through additional transformations. Finally, the modulation weight of each high-frequency component is obtained by combining the hierarchical weight and the directional weight. The calculation formula is:

[0158]

[0159]

[0160]

[0161] In the formula, W1 represents the expression used to analyze the local variance feature F. v The first learnable mapping parameter for channel dimensionality reduction; W2 represents the layer weights w used to map the dimensionality-reduced features. level The second learnable mapping parameter; W3 represents the third learnable mapping parameter used to generate the direction-sensitive high-frequency weight intermediate features; W4 represents the parameter used to map the intermediate features to high-frequency weights W. HF The fourth learnable mapping parameter; δ represents the nonlinear activation function, preferably the GELU activation function; w level W represents the hierarchical weights of different wavelet levels. HF W represents direction-sensitive high-frequency weights. i This represents the final modulation weight corresponding to the i-th high-frequency component; express Features obtained through two convolutional layers and activation functions; Softmax() represents the normalization exponential function, used to normalize the layer weight scores; Sigmoid() represents the sigmoid activation function, used to limit the directional weight scores to between 0 and 1;

[0162] Based on the obtained modulation weights, the size is adaptively adjusted to match the dimension of the corresponding high-frequency features and applied to modulate the two high-frequency components, thereby emphasizing the scene-dependent high-frequency subband; specifically, a learnable scaling parameter is introduced. To adjust variance features With modulation weights The response intensity between the two is used to generate a modulated signal. It is used to enhance two high-frequency wavelet subbands, and the calculation formula is as follows:

[0163]

[0164]

[0165] In the formula, h0 represents the high-frequency subband splicing feature obtained from the first-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH1, HL1 and HH1; h1 represents the high-frequency subband splicing feature obtained from the second-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH2, HL2 and HH2; Tanh() represents the hyperbolic tangent activation function.

[0166] The specific structure of the inverse wavelet reconstruction unit is as follows:

[0167] The enhanced high-frequency wavelet subbands are reconstructed step by step through inverse wavelet transform, and the second-level wavelet components are fused to obtain the final product. Then, it is reconstructed from the first-level wavelet components to generate... Finally, to ensure stability in the early stages of training, a gated residual learning mechanism is adopted to fuse the reconstructed features with the original input features, obtaining the output features of the global-local frequency domain modulation module. The calculation formula is as follows:

[0168]

[0169]

[0170]

[0171] In the formula, iWT represents the inverse wavelet transform; F L This represents the second-level reconstructed feature obtained by inverse wavelet transform of the second-level wavelet components LL2, LH2', HL2', and HH2'; F L 'Indicates by F LThe first-level reconstructed features are obtained by inverse wavelet transform of the first-level wavelet high-frequency components LH1', HL1', and HH1'; F i This represents the original characteristics of the input global-local frequency domain modulation module; F o This represents the fusion characteristics of the output of the global-local frequency domain modulation module; Conv 1×1 () denotes a 1×1 convolution operation, used to perform F... i Perform channel mapping and residual compensation; ⊙ indicates element-wise multiplication;

[0172] Thus, by combining Fourier-based global frequency domain modulation, wavelet-based local multi-scale decomposition, and variance-driven adaptive enhancement, the global-local frequency domain modulation module establishes a unified frequency domain-space modeling paradigm, which significantly enhances the representation of landslide-related textures and structural patterns at multiple scales.

[0173] C. Construct a multi-directional stripe boundary enhancement module; the specific steps are as follows:

[0174] In remote sensing imagery, landslide areas typically exhibit unique directional boundaries, fragmented textures, and diagonal structures—features that serve as highly discriminative visual cues. However, due to the complex spatial characteristics of landslide edges, such as striped geometry, slope transitions, and structural discontinuities, traditional convolution operations often struggle to effectively capture these features. To address this issue, a multi-directional striped boundary enhancement module is proposed to improve boundary localization and enhance the representation of landslide-specific structural patterns.

[0175] like Figure 5 As shown, the multi-directional stripe boundary enhancement module includes a directional pooling unit, a spatial recalibration unit, an edge perception enhancement unit, and an output unit;

[0176] The specific structure of the directional pooling unit is as follows:

[0177] Given input feature tensor Where C, H and W represent the number of channels, height and width of the feature map, respectively, and are used to introduce strip-based pooling operations to capture directional semantic information;

[0178] To characterize the landslide boundary and strip-like structural pattern, horizontal pooling is used to highlight the lateral expansion of the landslide in the spatial domain, thereby obtaining an axial feature representation. Vertical pooling is used to highlight the longitudinal extension of landslides in the spatial domain, thereby obtaining an axial feature representation. Furthermore, diagonal pooling is defined to extract landslides with oblique features, such as sloping slopes, producing a diagonal feature representation. The calculation formula is:

[0179]

[0180]

[0181]

[0182] In the formula, F p denoted by b; b represents the batch index; c represents the channel index; h represents the spatial position index along the height direction of the feature map; w represents the spatial position index along the width direction of the feature map; H represents the feature map height; W represents the feature map width; K represents the length of the diagonal pooling window; r represents the diagonal pooling radius, and K = 2r + 1; t represents the position offset within the diagonal pooling window.

[0183] Through the above operations, the module explicitly models the region of interest and adaptively adjusts the receptive field to match the slender and oblique structural pattern of the landslide.

[0184] The specific structure of the space recalibration unit is as follows:

[0185] Use depthwise separable convolution (DWConv) to extract input features Local features are extracted, and then weighted and combined with diagonal pooling features to construct a direction-aware representation. This enhances the representation of the sloping landslide structure; furthermore, rectangular attention weights are generated based on horizontal and vertical pooling features. Spatial recalibration is performed through interactive multiplication between diagonal and axial features to obtain refined output features. This process enhances the model's focus on major landslide areas while preserving spatial structure.

[0186] The specific structure of the edge-aware enhancement unit is as follows:

[0187] To further enhance the model's sensitivity to landslide boundaries and explicitly strengthen edge information, the Scharr operator is introduced from the features. Extracting edge responses; specifically, first given input features... The Scharr operator is introduced to calculate the horizontal gradient. and vertical gradient Based on this, the edge amplitude is obtained, and the calculation formula is:

[0188]

[0189]

[0190]

[0191] In the formula, x represents the feature tensor of the input edge-aware enhancement unit; i and j represent the spatial position indices in the height and width directions of the feature map, respectively; c represents the channel index; u and v represent the offsets of the Scharr convolution kernel in the height and width directions, respectively; K x Represents the horizontal Scharr convolution kernel; K y Indicates a vertical Scharr convolution kernel; G x G represents the horizontal gradient response; y ε represents the vertical gradient response; edge represents the edge magnitude map; ε is a small constant introduced for numerical stability.

[0192] The specific structure of the output unit is as follows:

[0193] Edge map After being adjusted by a learnable scaling factor α, it is added to the refined output features. In the middle, ReLU activation is performed to obtain the output features of the multi-directional stripe boundary enhancement module. ;

[0194] This mechanism explicitly enhances boundary gradients and sharpens edge responses, which is particularly helpful in extracting small or fragmented landslide areas. By combining global directional pooling and edge-aware enhancement, the multi-directional strip boundary enhancement module enables the network to better capture multi-directional textures, structural discontinuities, and oblique sliding patterns. These refined features provide a structurally rich and spatially adaptive representation for the subsequent decoder, significantly improving the accuracy of landslide segmentation, especially in complex terrain, heterogeneous surface coverage, or when landslide signals are weak.

[0195] D. Construct a hybrid CNN-Transformer module; the specific steps are as follows:

[0196] The hybrid CNN-Transformer module is a prior art module, which is implemented using a network structure that combines CNN and Transformer, which is known in the art. It is used to extract local features and model global features from input features. Specifically, the convolutional neural network part is used to extract local spatial features, and the Transformer part is used to obtain global context information.

[0197] S3. Based on the dataset obtained in step S1, train the model established in step S2 to obtain the trained landslide extraction model; the steps are as follows:

[0198] a. Training data preparation and augmentation; specific steps are as follows:

[0199] The training set constructed in step S1 is used as the original training samples, and each sample is a three-channel remote sensing image patch of 256×256 pixels.

[0200] To enhance the diversity of training samples and improve the generalization ability of the model, data augmentation strategies were applied to each original tile, including random cropping, rotation, flipping, contrast enhancement, and Gaussian blur. Through the above augmentation operations, a total of 15,048 training tiles were generated, and each tile generated by augmentation retained the landslide ground truth label corresponding to the original sample.

[0201] b. Loss function design; the specific steps are as follows:

[0202] For the pixel-level binary classification task of landslide extraction, a composite loss function is used to supervise the training of the network.

[0203]

[0204] In the formula, L total L represents the total loss function; CE L represents the binary cross-entropy loss, used to constrain pixel-level classification results; Dic e represents the Dice loss, used to improve the overlap between the predicted landslide area and the actual labeled area; L Foca l represents the Focal loss, used to reduce the weights of easily classified samples and enhance the learning of difficult-to-classify samples; L Lovász denoted as Lovász loss, used to directly optimize the intersection-union ratio (IU) in the segmentation task; α, β, γ, and δ represent the weight coefficients of the above loss terms, with values ​​of 1.0, 0.6, 0.5, and 0.5, respectively, and are optimized based on the performance of the validation set.

[0205]

[0206] In the formula, N represents the number of samples or pixels; y i represents the true label of the i-th sample or pixel; p represents the probability that the model predicts the sample or pixel belongs to the landslide category;

[0207]

[0208] In the formula, A represents the target region predicted by the model; B represents the ground truth region; |A ∩ B| represents the intersection size of the predicted region and the ground truth region; |A| represents the area or number of pixels of the predicted region; |B| represents the area or number of pixels of the ground truth region.

[0209]

[0210] In the formula, This represents the model's predicted probability of the true class. It is the category balance factor, used to adjust the importance of different categories; γ is the focusing parameter;

[0211]

[0212] In the formula, C represents the set of categories; c represents a specific category; and IoU(c) represents the intersection-union ratio of category c.

[0213] c. Optimizer and learning rate scheduling configuration; the specific steps are as follows:

[0214] The AdamW optimizer was used to update the network parameters, with momentum parameters set to β1=0.9 and β2=0.999, and weight decay coefficient set to 1×10⁻. 4 The initial learning rate is set to 2 × 10⁻ 4 Furthermore, cosine annealing scheduling is introduced to dynamically adjust the learning rate and enhance convergence stability; considering hardware memory limitations, the batch size is fixed at 18. These hyperparameters were determined through preliminary tuning experiments, aiming to achieve a balance between training efficiency and model performance.

[0215] d. Iterative training; the specific steps are as follows:

[0216] The 15048 generated training patches are randomly shuffled and fed into the FMEformer neural network model established in step S2 in batches of 18 for forward propagation to obtain the landslide prediction probability map for each pixel; the loss value is calculated according to the joint loss function and backpropagation is performed, and the AdamW optimizer updates the network weights according to the learning rate scheduling strategy.

[0217] During training, the change of the loss function value with each round is recorded. When the model reaches convergence after a certain round, that is, when the loss value tends to stabilize and no longer decreases significantly, the current model parameters and the corresponding network structure are saved to obtain the trained landslide extraction model.

[0218] S4. Input the multi-source remote sensing image data to be extracted into the landslide extraction model trained in step S3 to achieve landslide extraction from multi-source remote sensing images; the steps are as follows:

[0219] The multi-source remote sensing images of the landslides to be extracted are preprocessed to the same size and channels as the training samples; the images are cropped into 256×256 pixel patches according to the possible distribution of the landslides, and the three channels of green, blue and near-infrared are uniformly selected, and the pixel values ​​are normalized.

[0220] Input the preprocessed pixel patch into the trained FMEformer landslide extraction model obtained in step S3; obtain the landslide extraction result for the patch.

[0221] The technical solution of the present invention will be further described below with reference to specific embodiments.

[0222] This embodiment uses five earthquake-induced landslide datasets—Mesetas, Palu, Lushan, Jiuzhaigou, and Moxi Town—as data sources. It applies the multi-source remote sensing image landslide extraction method based on semantic association described in this invention to extract landslides, and then tests and evaluates the extraction results.

[0223] First, a multi-source remote sensing image landslide dataset was constructed according to step S1. Based on the two large-scale multi-sensor remote sensing datasets CAS and GDCLD, five datasets were obtained for Mesetas, Palu, Lushan, Jiuzhaigou, and Moxi Town. The occurrence time, data source, and spatial resolution of each earthquake-induced landslide are shown in Table 1.

[0224] Table 1 Summary of Landslide Dataset

[0225] The images from each dataset were preprocessed, and 80% of the samples from the Mesetas and Palu datasets were used as the training set; the remaining 20% ​​of the samples from these two datasets were used as regional evaluation test sets. The Lushan, Jiuzhaigou, and Moxi Town datasets were used as cross-regional test sets to directly evaluate the transferability of the model without retraining or fine-tuning during the evaluation process.

[0226] Then, an FMEformer neural network model is established according to step S2, and the model is trained according to step S3. The model converges in the 160th training round, resulting in a trained landslide extraction model.

[0227] Finally, based on the trained model deployed in step S4, landslide extraction was performed on the regional test set (the remaining 20% ​​of samples in Mesetas and Palu) and three cross-regional test sets (Lushan, Jiuzhaigou and Moxi Town) to obtain pixel-level landslide extraction results for each dataset.

[0228] To quantitatively evaluate extraction accuracy, four metrics were used to assess the extraction results: precision, recall, F1 score, and intersection-over-union ratio (IoU).

[0229] Precision reflects the proportion of pixels correctly identified as landslides out of all pixels predicted as such. Conversely, Recall captures the proportion of actual landslide pixels successfully identified, reflecting the completeness of the extraction. The F1 score, as the harmonic mean of precision and recall, balances these two metrics. IoU measures the spatial overlap between predicted landslides and ground truth annotations, and is a core metric in semantic segmentation tasks.

[0230] The calculation formulas for the four indicators are as follows:

[0231]

[0232]

[0233]

[0234]

[0235] In the formula, TP is the number of pixels correctly classified as landslides, FP is the number of non-landslide pixels misclassified as landslides, and FN is the number of landslide pixels that the model failed to identify. By using these metrics in combination, a comprehensive and robust evaluation of landslide extraction performance can be achieved.

[0236] To comprehensively evaluate the performance of FMEformer in landslide extraction, the FMEformer model of this invention is compared with 11 recently proposed deep learning models, including UNext, UNetFormer, PSPNet, CSUNet, UHRNet, SwinUNet, SegFormer, TransUNet (2021), TransUNet (2024), SKENet, and LSRFormer.

[0237] The quantitative results of each model on the intra-regional test set and the cross-regional test set are summarized in Tables 2 to 5.

[0238] Table 2. Accuracy Statistics for Mesetas and Palu Test Sets

[0239] The quantitative results on the Mesetas and Palu test sets are summarized in Table 2. FMEformer achieved the highest overall performance across all metrics, with an IoU of 78.43%, 1.56% higher than the second-ranked CSUNet. The model also demonstrated a strong balance between precision and recall, indicating effective control over false alarms and missed detections. In contrast, SwinUNet and UNeXt primarily rely on local window attention or local feature modeling, which limits their ability to capture the global distribution of landslides in complex terrain. Consequently, their IoU scores consistently remained below 71.5%. These methods primarily focus on spatial domain representations and do not explicitly model long-range dependencies or frequency domain features, thus limiting their ability to identify complex landslide morphologies. Conversely, FMEformer establishes long-range dependencies through frequency domain modeling and enhances multi-scale structural awareness. The adaptive wavelet mechanism further reinforces high-frequency responses associated with landslide boundaries, which are often difficult to recover using purely spatial methods. Through these design choices, FMEformer achieves more accurate and robust landslide extraction under in-domain conditions.

[0240] Table 3. Accuracy Statistics of the Jiuzhaigou Test Set

[0241] The performance metrics in Table 3 highlight the superiority of FMEformer. It achieves the best results on the Jiuzhaigou dataset, with an F1 score of 71.06% and an IoU of 55.12%. These results reflect FMEformer's strong generalization ability across resolution and regional settings. All benchmark models show a decline in performance, highlighting the difficulty of cross-domain generalization. CSUNet shows the most significant decline, with an IoU dropping to 30.06%, indicating poor adaptability to new spatial and spectral features. A clear trade-off between precision and recall is observed among the models. LSRFormer achieves the highest precision of 79.81%, but its low recall indicates that real landslide areas are frequently missed. TransUNet (2021) exhibits the opposite pattern, with high recall but poor precision, leading to a large number of false positives. In contrast, FMEformer maintains a good balance, achieving both high precision and high recall. This advantage stems from FMEformer's dual focus on global structural understanding and selective enhancement of local features, enabling accurate extraction in complex high-resolution landscapes.

[0242] Table 4. Accuracy Statistics of the Moxi Town Test Set

[0243] Table 4 presents a quantitative comparison of all methods on the Moxi Town dataset. Overall, the models performed poorly in this test due to significant domain disparities. The TransUNet series benefits from global modeling via Transformer, but its dependence on spatial domain context limits its ability to handle fine boundary structures. For example, TransUNet (2021) misclassifies many background regions near landslide boundaries, resulting in an accuracy of only 32.64%. TransUNet (2024) takes a more conservative approach, resulting in missed detections in some landslide boundary regions, with a recall of only 59.50%. FMEformer outperforms all other methods, achieving an F1 score of 53.18% and an IoU of 36.22%. This improvement is attributed to the multi-directional stripe boundary enhancement module, which shifts the modeling focus from global semantics to more stable morphological edges. Through multi-angle attention, this module enhances the model's ability to capture subtle terrain transitions and irregular edges, especially in mountainous areas with blurred or fragmented boundaries. FMEformer maintains edge consistency and achieves higher segmentation accuracy.

[0244] Table 5. Precision Statistics of the Lushan Test Set

[0245] Table 5 summarizes the performance of all models on the Lushan dataset. Despite LSRFormer's long and short range attention and multi-scale feedforward design, its recall is only 34.15%, indicating a weak ability to identify true positive samples under cross-regional variations, with an IoU of only 24.95%. In contrast, FMEformer achieves a recall of 50.79% and an IoU of 36.75%. This improvement is attributed to global-local frequency domain modulation, which enhances the model's ability to integrate contextual semantics and multi-scale features across domains.

[0246] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although detailed descriptions have been provided with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A landslide extraction method based on semantic association from multi-source remote sensing images, characterized in that, Includes the following steps: S1. Construct a multi-source remote sensing imagery dataset of landslides; S2. Establish the FMEformer neural network model; S3. Based on the dataset obtained in step S1, train the model established in step S2 to obtain the trained landslide extraction model. S4. Input the multi-source remote sensing image data to be extracted into the landslide extraction model trained in step S3 to realize landslide extraction from multi-source remote sensing images.

2. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 1, characterized in that, The specific steps of S1 are as follows: Five earthquake-induced landslide datasets were constructed, and the region, occurrence time, data source, and spatial resolution of each dataset were obtained. The above multi-source remote sensing images were preprocessed as follows: each image was randomly cropped into 256×256 pixel patches according to the actual location of the landslide; the image channels were uniformly selected as green, blue and near-infrared channels; 80% of the samples from two of the datasets were used as training sets for model training, and the remaining 20% ​​of the samples were used as regional evaluation test sets; the other three datasets were used as cross-regional test sets, and no retraining or fine-tuning was performed during the evaluation process.

3. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 1, characterized in that, Step S2 is as follows: The overall model architecture is constructed, and the FMEformer model includes the encoding path, boundary enhancement module, decoding path, and output path; The encoding path is used for feature extraction and downsampling of the input image. It includes a convolutional front-end structure, a global-local frequency domain modulation module, a block merging layer, and a hybrid CNN-Transformer module, forming a sequentially connected input module and four downsampling stages. Each stage generates feature maps at different scales. The encoding path structure is as follows: The input image is first processed by a convolutional front-end structure to extract features, resulting in an initial feature map. The initial feature map is then input into a global-local frequency domain modulation module for frequency domain structure readjustment, resulting in a readjusted feature map. The readjusted feature map is downsampled through a block merging layer and input into a hybrid CNN-Transformer module to capture fine-grained spatial details and global semantic context. Then, it is input into a global-local frequency domain modulation module for frequency domain structure readjustment to obtain a first-stage feature map with a resolution of {1 / 4}. The first-stage feature map is split into two paths: one path generates the first-level skip connection features through the boundary enhancement module, and finally generates the first-stage enhanced feature map. Another path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain a second-stage feature map with a resolution of {1 / 8}. The second-stage feature map is split into two paths. One path generates the second-level skip connection features through the boundary enhancement module, and finally generates the second-stage enhanced feature map. Another path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain the third-stage feature map with a resolution of {1 / 16}. The third-stage feature map is split into two paths: one path generates the third-level skip connection feature through the boundary enhancement module, and finally generates the third-stage enhanced feature map. Another path is transmitted to the next level encoder, downsampled through the block merging layer, and input into the hybrid CNN-Transformer module to obtain a fourth-stage feature map with a resolution of {1 / 32}; this is used as the starting input for the decoding path. The feature enhancement module is specifically a multi-directional stripe boundary enhancement module, located in the skip connection between the encoding path and the decoder path; The decoding path includes multiple upsampling stages. Each upsampling stage fuses the output features of the previous level decoder with the enhanced features output by the same level feature enhancement module. The decoding path structure is as follows: The decoder first receives the fourth-stage feature map, and after upsampling, it concatenates the feature map with the third-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the third-stage decoded feature map. The third-stage decoded feature map is then upsampled and concatenated with the second-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the second-stage decoded feature map. The second-stage decoded feature map is then upsampled and concatenated with the first-stage enhanced feature map. The concatenated features are then fused through a convolutional layer to generate the first-stage decoded feature map, which is used as the starting input for the output path. The output path maps the final output of the decoding path to the target result; The first-stage decoded feature map is input into the classification head to obtain the final target image.

4. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 3, characterized in that, The specific structure of the global-local frequency domain modulation module is as follows: The global-local frequency domain modulation module includes a Fourier global frequency domain modulation unit, a wavelet multi-scale decomposition unit, a multi-level local variance adaptive modulation unit, and an inverse wavelet reconstruction unit; The specific structure of the Fourier global frequency domain modulation unit is as follows: Given an input feature map First, the complex frequency domain spectrum is obtained by applying a two-dimensional fast Fourier transform. And decompose it into amplitude terms and normalized phase term The calculation formulas are as follows: In the formula, FFT represents Fast Fourier Transform, and "mag" is the complex frequency domain spectrum F. G The magnitude, i.e., the modulus; Re(F G ) represents the complex frequency domain spectrum F G The real part, Im(F) G ) represents the complex frequency domain spectrum F G The imaginary part of the complex frequency spectrum; sqrt() represents the square root operation; "phasor" represents the unit phase vector of the complex frequency spectrum; "epsilon" is a minimal constant to prevent numerical instability. Amplitude Term The updated amplitude term is obtained by adaptively modulating a channel blending structure consisting of two 1×1 convolutional layers and a GELU activation function. Then with the phase term Recombining and returning to the spatial domain yields global modulation features. The calculation formula is: In the formula, iFFT represents the inverse fast Fourier transform; subsequently, depthwise separable 5×5 convolutions are used to further extract features, and then... i Multiply and then add the weighted sums to obtain the global modulation features. .

5. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 4, characterized in that, The specific structure of the wavelet multi-scale decomposition unit is as follows: global modulation features Input wavelet multiscale decomposition unit and perform two-level wavelet decomposition: the first-level decomposition yields the low-frequency approximation component LL1 and three high-frequency detail components LH1, HL1 and HH1; the second-level wavelet decomposition of LL1 yields LL2, LH2, HL2 and HH2.

6. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 4, characterized in that, The specific structure of the multi-level local variance adaptive modulation unit is as follows: The components obtained from wavelet decomposition, i.e., wavelet features The input is a multi-level local variance adaptive modulation unit, whose local variance is calculated using a 3×3 sliding window, and then encoded into a stable variance representation through a lightweight convolutional network. The calculation formula is: In the formula, This represents the mean within a 3×3 local window, where x represents the pixel value within the local window that participates in the variance calculation. This represents a nonlinear mapping of three convolutional layers plus the GELU activation function; This represents the local variance feature calculated from the second moment of the local window and the squared difference of the mean; σ² represents the variance normalization performed on the local variance feature. Global average pooling and a multilayer perceptron are used to generate the weights for each layer. Furthermore, direction-sensitive high-frequency weights are learned through additional transformations. Finally, the modulation weight of each high-frequency component is obtained by combining the hierarchical weight and the directional weight. The calculation formula is: In the formula, W1 represents the expression used to analyze the local variance feature F. v The first learnable mapping parameter for channel dimensionality reduction; W2 represents the layer weights w used to map the dimensionality-reduced features. level The second learnable mapping parameter; W3 represents the third learnable mapping parameter used to generate the direction-sensitive high-frequency weight intermediate features; W4 represents the parameter used to map the intermediate features to high-frequency weights W. HF The fourth learnable mapping parameter; δ represents the nonlinear activation function, preferably the GELU activation function; w level This represents the hierarchical weights of different wavelet levels; W HF Indicates direction-sensitive high-frequency weights; W i This represents the final modulation weight corresponding to the i-th high-frequency component; express Features obtained after two convolutional layers and activation functions; Softmax() represents the normalization exponential function, used to normalize the hierarchical weight scores; Sigmoid() represents the sigmoid activation function, used to limit the directional weight scores to between 0 and 1. Introducing learnable scaling parameters To adjust variance features With modulation weights The response intensity between the two is used to generate a modulated signal. It is used to enhance two high-frequency wavelet subbands, and the calculation formula is as follows: In the formula, h0 represents the high-frequency subband splicing feature obtained from the first-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH1, HL1 and HH1; h1 represents the high-frequency subband splicing feature obtained from the second-level wavelet decomposition, that is, the high-frequency feature formed by splicing LH2, HL2 and HH2. Tanh() represents the hyperbolic tangent activation function.

7. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 4, characterized in that, The specific structure of the inverse wavelet reconstruction unit is as follows: The enhanced high-frequency wavelet subbands are reconstructed step by step through inverse wavelet transform, and the second-level wavelet components are fused to obtain the final product. Then, it is reconstructed from the first-level wavelet components to generate... Finally, a gated residual learning mechanism is used to fuse the reconstructed features with the original input features to obtain the output features of the global-local frequency domain modulation module. The calculation formula is as follows: In the formula, iWT represents the inverse wavelet transform; F L This represents the second-level reconstructed feature obtained by inverse wavelet transform of the second-level wavelet components LL2, LH2', HL2', and HH2'; F L 'Indicates by F L The first-level reconstructed features are obtained by inverse wavelet transform of the first-level wavelet high-frequency components LH1', HL1', and HH1'; Fi represents the original features input to the global-local frequency domain modulation module; F o This represents the fusion characteristics of the output of the global-local frequency domain modulation module; Conv 1×1 () denotes a 1×1 convolution operation, used to perform F... i Perform channel mapping and residual compensation; ⊙ indicates element-wise multiplication.

8. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 1, characterized in that, The specific structure of the multi-directional stripe boundary enhancement module is as follows: The multi-directional stripe boundary enhancement module includes a directional pooling unit, a spatial recalibration unit, an edge perception enhancement unit, and an output unit; The specific structure of the directional pooling unit is as follows: Given input feature tensor Where C, H, and W represent the number of channels, height, and width of the feature map, respectively; Horizontal pooling is used to highlight the lateral extension of landslides in the spatial domain, thereby obtaining an axial feature representation. Vertical pooling is used to highlight the longitudinal extension of landslides in the spatial domain, thereby obtaining an axial feature representation. Furthermore, diagonal pooling is defined to extract landslides with oblique features, producing a diagonal feature representation. The calculation formula is: In the formula, F p denoted by b; b represents the batch index; c represents the channel index; h represents the spatial position index along the height direction of the feature map; w represents the spatial position index along the width direction of the feature map; H represents the feature map height; W represents the feature map width; K represents the length of the diagonal pooling window; r represents the diagonal pooling radius, and K = 2r + 1; t represents the position offset within the diagonal pooling window. The specific structure of the space recalibration unit is as follows: Use depthwise separable convolutions to extract input features Local features are extracted, and then weighted and combined with diagonal pooling features to construct a direction-aware representation. Furthermore, rectangular attention weights are generated based on horizontal and vertical pooling features. Spatial recalibration is performed through interactive multiplication between diagonal and axial features to obtain refined output features. ; The specific structure of the edge-aware enhancement unit is as follows: Given input features The Scharr operator is introduced to calculate the horizontal gradient. and vertical gradient Based on this, the edge amplitude is obtained, and the calculation formula is: In the formula, x represents the feature tensor of the input edge-aware enhancement unit; i and j represent the spatial position indices in the height and width directions of the feature map, respectively; c represents the channel index; u and v represent the offsets of the Scharr convolution kernel in the height and width directions, respectively; K x Represents the horizontal Scharr convolution kernel; K y Indicates a vertical Scharr convolution kernel; G x G represents the horizontal gradient response; y ε represents the vertical gradient response; edge represents the edge magnitude map; ε is a small constant introduced for numerical stability. The specific structure of the output unit is as follows: Edge map After being adjusted by a learnable scaling factor α, it is added to the refined output features. In the middle, ReLU activation is performed to obtain the output features of the multi-directional stripe boundary enhancement module. .

9. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 1, characterized in that, Step S3 is as follows: a. Training data preparation and augmentation; specific steps are as follows: The training set constructed in step S1 is used as the original training samples, each sample being a 256×256 pixel three-channel remote sensing image patch; data augmentation strategies are applied to each original patch, including random cropping, rotation, flipping, contrast enhancement, and Gaussian blur; a total of 15048 training patches are generated. b. Loss function design; the specific steps are as follows: For the pixel-level binary classification task of landslide extraction, a composite loss function is used to supervise the training of the network. In the formula, L total L represents the total loss function; CE L represents the binary cross-entropy loss, used to constrain pixel-level classification results; Dic e represents the Dice loss, used to improve the overlap between the predicted landslide area and the actual labeled area; L Foca l represents the Focal loss, used to reduce the weights of easily classified samples and enhance the learning of difficult-to-classify samples; L Lovász denoted as Lovász loss, used to directly optimize the intersection-union ratio (IU) in the segmentation task; α, β, γ, and δ represent the weight coefficients of the above loss terms, with values ​​of 1.0, 0.6, 0.5, and 0.5, respectively, and are optimized based on the performance of the validation set. In the formula, N represents the number of samples or pixels; y i This represents the true label of the i-th sample or pixel; p represents the probability that the model predicts a sample or pixel belongs to the landslide category; In the formula, A represents the target region predicted by the model; B represents the ground truth region; |A ∩ B| represents the intersection size of the predicted region and the ground truth region; |A| represents the area or number of pixels of the predicted region; |B| represents the area or number of pixels of the ground truth region. In the formula, This represents the model's predicted probability of the true class. It is the category balance factor, used to adjust the importance of different categories; γ is the focusing parameter; In the formula, C represents the set of categories; c represents a specific category; and IoU(c) represents the intersection-union ratio of category c. c. Optimizer and learning rate scheduling configuration; the specific steps are as follows: The AdamW optimizer was used to update the network parameters, with momentum parameters set to β1=0.9 and β2=0.999, and weight decay coefficient set to 1×10⁻. 4 The initial learning rate is set to 2 × 10⁻ 4 Furthermore, cosine annealing scheduling is introduced to dynamically adjust the learning rate and enhance convergence stability; considering hardware memory limitations, the batch size is fixed at 18; d. Iterative training; the specific steps are as follows: The 15048 generated training patches are randomly shuffled and fed into the FMEformer neural network model established in step S2 in batches of 18 for forward propagation to obtain the landslide prediction probability map for each pixel; the loss value is calculated according to the joint loss function and backpropagation is performed, and the AdamW optimizer updates the network weights according to the learning rate scheduling strategy. During training, the change of the loss function value with each round is recorded. When the model reaches convergence after a certain round, that is, when the loss value tends to stabilize and no longer decreases significantly, the current model parameters and the corresponding network structure are saved to obtain the trained landslide extraction model.

10. The landslide extraction method based on semantic association from multi-source remote sensing images as described in claim 1, characterized in that, Step S4 is as follows: The multi-source remote sensing images of the landslides to be extracted are preprocessed to the same size and channels as the training samples; the images are cropped into 256×256 pixel patches according to the possible distribution of the landslides, and the three channels of green, blue and near-infrared are uniformly selected, and the pixel values ​​are normalized. Input the preprocessed pixel patch into the trained FMEformer landslide extraction model obtained in step S3; obtain the landslide extraction result for the patch.

Images