Landslide signal stable migration method crossing space-time device

Abstract

The invention provides a remote sensing image style migration method for landslide, and the method comprises the steps: firstly, obtaining a source style remote sensing image; then, obtaining an initial feature map based on the source style remote sensing image; then, capturing domain style spectrum features of the initial feature map in a frequency domain through a wavelet pyramid pooling module to obtain a first intermediate feature map; then, multi-directional context features of the source style remote sensing image are aggregated through a staggered and crossed fan-shaped boundary enhancement module, and a second intermediate feature map is obtained; thirdly, performing spatial feature refinement on the second intermediate feature map through depth separable convolution to obtain a third intermediate feature map; and finally, inputting the third intermediate feature map into a decoder to obtain a target style remote sensing image. According to the landslide remote sensing image style migration method provided by the invention, the quality of the generated image is remarkably improved, and the generated image can be synchronized with the target style while the key landslide features of the original image are accurately reserved.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and pattern recognition, and in particular to a method for the stable migration of landslide signals across time and space. Background Technology

[0002] Landslides are a common geological hazard in mountainous areas. They are extremely destructive and pose a significant threat, seriously endangering people's lives and property and hindering regional economic development. Therefore, accurate identification and monitoring of landslide areas are crucial, providing key support for post-disaster assessment, emergency response, and disaster prevention and mitigation.

[0003] Remote sensing technology boasts advantages such as wide coverage, high efficiency, and minimal terrain limitations, enabling large-scale landslide monitoring in mountainous areas with complex terrain. High-resolution remote sensing imagery provides detailed and accurate landslide feature characteristics, laying the data foundation for precise landslide identification. Therefore, landslide identification and monitoring are currently primarily accomplished through the analysis and processing of remote sensing image data. With the development of machine learning, deep learning methods have been introduced into the field of remote sensing image analysis and applied to landslide identification. Typical networks include U-Net, SegNet, Mask R-CNN, and improved Convolutional Neural Network (CNN) structures. These deep learning methods can automatically extract landslide features and significantly improve the accuracy of landslide identification.

[0004] However, while existing landslide detection methods based on remote sensing imagery can achieve high accuracy on single regions or homogeneous datasets, their generalization ability often declines significantly when applied across regions. This is because geographical features such as geology, topography, vegetation cover, and climate conditions vary considerably across regions, as do the feature spaces, statistical distributions, and annotation systems of different datasets. This cross-regional performance degradation has become a key bottleneck restricting the widespread application of remote sensing landslide detection models. Although style transfer technology has made some progress in natural image processing, its application in landslide extraction is still in the exploratory stage, with a lack of research on style transfer targeting the unique geographical and environmental characteristics of landslides. Existing style transfer techniques still have shortcomings in feature representation and region focusing, which may lead to the loss of key landslide features during the style transfer process, severely affecting the accuracy of landslide identification. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a style transfer method for remote sensing image data related to landslides. The method is based on a generator from a pre-trained generative adversarial network. The generator includes an encoder and a decoder. The encoder includes a wavelet pyramid pooling module and an interleaved fan-shaped boundary enhancement module. The method comprises:

[0006] Acquire source-style remote sensing imagery;

[0007] The source style remote sensing image is input into the generator of a pre-trained generative adversarial network to obtain the target style remote sensing image.

[0008] The source-style remote sensing image is input into the generator of a pre-trained generative adversarial network to obtain the target-style remote sensing image, specifically including:

[0009] The source style remote sensing image is used to extract features through a convolutional layer to obtain an initial feature map;

[0010] The initial feature map is extracted in the frequency domain by using a wavelet pyramid pooling module, and the frequency domain feature extraction result is reconstructed in the spatial domain to obtain the first intermediate feature map.

[0011] The second intermediate feature map is obtained by aggregating the multi-directional contextual features of the first intermediate feature map through the interlaced fan-shaped boundary enhancement module.

[0012] The second intermediate feature map is spatially refined by depthwise separable convolution to obtain the third intermediate feature map;

[0013] The third intermediate feature map is input into the decoder, and the decoder generates a target-style remote sensing image based on the third intermediate feature map.

[0014] In some embodiments, the step of extracting features from the initial feature map in the frequency domain using a wavelet pyramid pooling module and reconstructing the frequency domain feature extraction result in the spatial domain to obtain a first intermediate feature map specifically includes:

[0015] The initial feature map is decomposed into multiple frequency sub-bands using wavelet transform; adaptive average pooling is performed on different frequency sub-bands using pooling kernels of different sizes, and the pooling results are input into a convolutional layer. The output of the convolutional layer is added element-wise to the original sub-band features of the frequency sub-band to obtain the feature extraction result of the frequency sub-band; the feature extraction results of the multiple frequency sub-bands are reconstructed using inverse wavelet transform to obtain the first intermediate feature map.

[0016] In some embodiments, the plurality of frequency sub-bands includes a low-frequency sub-band and a high-frequency sub-band in multiple directions; after decomposing the initial feature map into a plurality of frequency sub-bands, the method further includes: decomposing the low-frequency sub-band into a plurality of deep frequency components; extracting features from the deep frequency components through a convolutional layer, and reconstructing the feature extraction results of the deep frequency components together with the feature extraction results of the plurality of frequency sub-bands through inverse wavelet transform to obtain a first intermediate feature map.

[0017] In some embodiments, the interlaced cross-fan boundary enhancement module aggregates multi-directional contextual features of the source style remote sensing image to obtain a second intermediate feature map. Specifically, the interlaced cross-fan boundary enhancement module uses the Sobel operator to extract boundary features of the source style remote sensing image, and fuses the boundary features with the first intermediate feature map through convolution to obtain a boundary enhancement feature map; the boundary enhancement feature map is input into interlaced convolution, cross convolution, and fan-shaped convolution paths respectively to obtain interlaced feature representation, cross feature representation, and fan-shaped feature representation; and the second intermediate feature map is obtained based on the interlaced feature representation, cross feature representation, fan-shaped feature representation, and the first intermediate feature map.

[0018] In some more specific embodiments, the boundary enhancement feature map is input into staggered convolution, cross convolution, and fan-shaped convolution paths respectively to obtain staggered feature representation, cross feature representation, and fan-shaped feature representation, specifically including:

[0019] The boundary enhancement feature map is input into the staggered convolution path, which asymmetrically fills the boundary enhancement feature map in different directions. Different convolution kernels are used to convolve the filled feature map, and the convolution results are fused to obtain the staggered feature representation.

[0020] The boundary enhancement feature map is input into the cross convolution path. The cross convolution path uses convolution kernels in the horizontal and vertical directions to extract features from the boundary enhancement feature map, and then fuses the feature extraction results of the two to obtain the cross feature representation.

[0021] The boundary enhancement feature map is input into a fan-shaped convolution path. The fan-shaped convolution path uses multiple fan-shaped convolution kernels with different directional offsets to extract features from the boundary enhancement feature map and fuses the extraction results to obtain a fan-shaped feature representation.

[0022] In some more specific embodiments, a second intermediate feature map is obtained based on the interlaced feature representation, the cross feature representation, the sector feature representation, and the first intermediate feature map, specifically including:

[0023] The interleaved feature representation, cross feature representation, sector feature representation, and the first intermediate feature map are fused to obtain a fused feature map. The fused feature map is input into a convolutional layer to obtain a residual feature map. Global average pooling is performed on the residual feature map and channel attention weights are calculated. The residual feature map is multiplied element-wise by the attention weights and added to the fused feature map to obtain a second intermediate feature map.

[0024] In some embodiments, the step of refining the spatial features of the second intermediate feature map through depthwise separable convolution to obtain a third intermediate feature map specifically includes: inputting the second intermediate features into a depthwise separable convolution module, performing depthwise separable convolution on the second intermediate feature map using multiple convolution kernels of different sizes, and fusing the convolution results to obtain a third intermediate feature map.

[0025] In some embodiments, the generator is trained based on a target-style remote sensing image and includes target-style background feature data; the decoder includes a content masking module and an attention masking module; the third intermediate feature map is input into the decoder to obtain the target-style remote sensing image, specifically including:

[0026] The decoder divides the third intermediate feature map into two parts through a Split operation, and inputs them into the content mask module and the attention mask module respectively to obtain content features; it then fuses the target style background features with the source style remote sensing image to obtain background image features; finally, it fuses the background image features with the content features to obtain the target style remote sensing image.

[0027] In some embodiments, the training method for the generative adversarial network includes:

[0028] Obtain a multi-source remote sensing image dataset, which includes at least two different styles of remote sensing images, referred to as first-style remote sensing images and second-style remote sensing images, respectively.

[0029] The generative adversarial network includes a first generator, a second generator, and a discriminator;

[0030] The first style remote sensing image is input into the first generator to obtain the first generated image; the first generated image and the second style remote sensing image are used to train the discriminator.

[0031] The first style remote sensing image is input into the first generator to obtain the first generated image; the first generated image is judged by the discriminator, and the model parameters of the first generator are iterated according to the judgment result;

[0032] The first generated image is input into the second generator to obtain the second generated image; the model parameters of the first generator and the second generator are iterated according to the difference between the second generated image and the first style remote sensing image.

[0033] Repeat the above steps, alternatingly training the discriminator, the first generator, and the second generator of the generative adversarial network until the preset conditions are met, and obtain the trained generative adversarial network.

[0034] In some more specific embodiments, the discriminator employs a PatchGAN discriminator network structure.

[0035] In some embodiments, the style transfer method is used for landslide extraction.

[0036] The landslide remote sensing image style transfer method provided by this invention enhances the multi-scale feature representation and regional focusing of landslides through wavelet pyramid pooling and staggered fan-shaped boundary enhancement, significantly improving the quality of the generated images. This allows the generated images to accurately preserve key landslide features of the original images while achieving synchronization with the target style. The style transfer method provided by this invention can unify the style of remote sensing images from different regions, or transfer remote sensing images from which landslides are difficult to extract to a target style with higher landslide image quality, thereby improving the accuracy and generalization of landslide identification. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A schematic diagram illustrating the training process of the LMGAN generative adversarial network provided by this invention;

[0039] Figure 2 A schematic diagram of the discriminator structure of the LMGAN generative adversarial network provided by the present invention;

[0040] Figure 3 A schematic diagram of the generator structure of the LMGAN generative adversarial network provided by the present invention;

[0041] Figure 4 This is a schematic diagram of the wavelet pyramid pooling module structure;

[0042] Figure 5 A schematic diagram of the intersecting sector-shaped boundary reinforcement module structure;

[0043] Figure 6 A schematic diagram of the decoder structure of the LMGAN generative adversarial network provided by this invention;

[0044] Figure 7 This is a visualization of the style transfer results obtained using different style transfer models (CycleGAN, AttentionGAN, StegoGAN, and LMGAN provided by this invention) in an embodiment of the present invention, with Lushan landslide as the target style and Jiuzhaigou (JZG), Taiwan (TW), Luding (LD), Wenchuan (WC), and Zhouqu (ZQ) as the source styles. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described below with reference to the accompanying drawings.

[0046] The applicant found that existing style transfer techniques fail to fully utilize the multi-scale, complex, and local saliency of landslide features when performing style transfer on landslides, resulting in significant deficiencies in the feature representation and region focusing of existing models. Based on this, this invention designs a stable landslide signal transfer method across time and space by introducing frequency domain features, boundary enhancement, and a joint mechanism of content masking and attention masking. The aim is to overcome the limitations of existing transfer methods and achieve high accuracy and robustness in landslide detection under different regions and multi-source image conditions.

[0047] The concept of this invention has been introduced above. The following, in conjunction with specific embodiments, details the style transfer method for remote sensing image data provided by this invention. The style transfer method provided by this invention includes a model training stage and a style transfer stage.

[0048] S1, Model Training Phase

[0049] S11. Obtain a multi-source remote sensing image dataset, the dataset including at least two different styles of remote sensing images, referred to as first-style remote sensing images and second-style remote sensing images, respectively.

[0050] The multi-source remote sensing image dataset includes various landslide remote sensing images from different regions and sensors, each with its own style. In this invention, the style of remote sensing image data refers to the comprehensive visual characteristics of an image determined by the sensor, imaging conditions, acquisition time, and image processing methods. It is reflected in features such as illumination, color, texture, resolution, and spectral density. Different sensors and imaging conditions in different regions lead to variations in image style. These differences in style result in inconsistencies in illumination, color, texture, resolution, and spectral density, affecting the model's ability to grasp these features during landslide identification.

[0051] After acquiring a multi-source remote sensing image dataset, the remote sensing image data can be preprocessed to obtain remote sensing image data samples of uniform specifications for model training. The preprocessing includes cropping, rotating, flipping, and enhancing the remote sensing images.

[0052] S12. Train the LMGAN generative adversarial network based on the multi-source remote sensing image dataset.

[0053] This invention proposes an LMGAN network framework for style transfer of landslide remote sensing images. This framework is based on a Generative Adversarial Network (GAN) and includes a generator G and a discriminator D. The generator G converts the source-style remote sensing image to the target style while preserving the landslide features in the source-style image. The discriminator D distinguishes between the generated image from the real target-style remote sensing image and provides correction feedback. During training, adversarial training and cycle consistency training are performed using LG and LC loss functions. The generator G and discriminator D iteratively optimize each other to generate style-consistent images while maintaining necessary spatial and spectral features.

[0054] The training process of the LMGAN generative adversarial network is as follows: Figure 1 As shown, two sets of generators (G) are used during training. XY With G YX They have the same network structure, differing only in their training process and style transfer direction, with the first generator G... XY Used to generate a second-style image based on a first-style remote sensing image X; Second generator G YX Used to generate a first-style image based on a second-style remote sensing image Y. During training, two sets of generators are used simultaneously to cross-validate each other, ensuring that the generated images retain key landslide features from the source-style remote sensing imagery. The specific training process includes the following steps:

[0055] S121. Input the real first-style remote sensing image X into the first generator G. XY The first generated image is obtained. Using the first generated image The discriminator D compares the real second-style remote sensing image Y with the real second-style remote sensing image Y. Y Train the discriminator D Y Capable of distinguishing the first generated image The ability to work with second-style remote sensing imagery Y.

[0056] The discriminator of the LMGAN generative adversarial network uses, for example... Figure 2 The diagram shows the PatchGAN structure. PatchGAN is a discriminator architecture in generative adversarial networks, widely used in image generation tasks, especially image transformation tasks. Unlike traditional global discriminators, the PatchGAN discriminator focuses on the local texture and details of an image. It performs local evaluation of the image, judging the realism of each image region (called a "patch") separately, rather than making a uniform realism judgment on the entire image, thus enabling it to focus on local details in the image.

[0057] During training, the discriminator segments the input 256×256×3 image into multiple N×N local regions using a multi-layer convolutional network and evaluates the realism of each region individually. The PatchGAN discriminator network progressively reduces the image size through convolutional layers, normalized BatchNorm layers, and the LeakyReLU activation function, ultimately outputting a 30×30×1 prediction map that includes the discrimination result for each region, where each region is predicted independently. In style transfer tasks, the PatchGAN discriminator network effectively captures local texture features and maintains regional consistency. Its local evaluation method not only reduces computational complexity but also improves the realism of the generated image and the consistency of local style.

[0058] S122, Input the first-style remote sensing image X into the first generator G XY The first generated image is obtained. Using the discriminator D Y For the first generated image The model parameters of the first generator are iterated using the LG loss function based on the discrimination result.

[0059] S123, the first generated image Input the second generator G YX The second generated image is obtained. According to the second generated image The difference between the first style remote sensing image X and the first generator G is determined by iterating the first generator G using the loss function LC. XY Second generator G YX Model parameters.

[0060] The second generator G YX Used to generate a generated image with a first style based on a second-style remote sensing image, if the second generator G YX Able to generate based on the first image Reconstructing the first-style remote sensing image X can then corroborate the first generated image. The data includes the target landslide features in the first-style remote sensing image X.

[0061] S124. Repeat steps S121-S123, alternately training the discriminator D of the generative adversarial network. Y Generator G XY and G YX The generator and discriminator are iteratively optimized until they reach a balance, resulting in a well-trained generative adversarial network (LMGAN).

[0062] In order to pass the discriminator's judgment, the generator continuously iterates its own parameters according to the loss function, so that the style of the generated image is close to the real target style remote sensing image. Through continuous alternating training iterations, the generator G adjusts its own generation parameters according to the feedback of the discriminator, so that the background of the generated image is close to the background features of the target style image.

[0063] It should be understood that the aforementioned "first style" and "second style" can be interchanged, such as... Figure 1 As shown at the bottom, during training, the "second-style remote sensing image Y" and "first-style remote sensing image X" in the above training process can be replaced with each other, and the corresponding generator can be replaced. The model can be trained and optimized from two directions to improve training efficiency.

[0064] The training process of the LMGAN generative adversarial network has been described above. The following describes the landslide remote sensing image style transfer method provided by this invention. This method is based on the generator of a trained LMGAN network and includes the following steps:

[0065] S2, Style Transfer Stage

[0066] S21. Acquire source style remote sensing imagery;

[0067] The source style remote sensing image is the original remote sensing image that needs to be style transferred. The source style remote sensing image contains landslide features or signals that need to be transferred. The purpose of the style transfer method provided by this invention is to transfer landslide features from the source style image to the target style, so that the target style remote sensing image can be used for further identification and extraction of landslides.

[0068] S22. Input the source style remote sensing image into the generator G of the pre-trained LMGAN generative adversarial network to obtain the target style remote sensing image.

[0069] The generator of the LMGAN network is a spatial-frequency hybrid encoder-decoder architecture specifically designed to enhance robust landslide feature extraction under complex terrain conditions and cross-domain migrations. Its structural diagram is shown below. Figure 3 As shown, it includes an encoder and a decoder.

[0070] The LMGAN network's encoder includes a Wavelet Pyramid Pooling (WPPM) module and an Interleaved Cross Fan (ICF) boundary enhancement module. The Wavelet Pyramid Pooling module aims to capture domain-specific spectral variations, such as differences in texture intensity, hue, and illumination across multiple resolutions and directions. WPPM enables fine-tuning of spectral features like hue, texture, and contrast in the frequency domain, ensuring that style adaptation does not compromise the underlying structural content. The ICF boundary enhancement module is introduced to improve the model's boundary recognition accuracy and morphological awareness. This module is specifically designed to learn multi-directional contours and shape descriptors to address the challenges of irregular landslide geometries and improve the recognition performance of blurred or low-contrast boundaries.

[0071] The following is combined Figure 3 This section details the generation process of target-style remote sensing images. First, it introduces the feature extraction process of the LMGAN network encoder.

[0072] S221, Initial Feature Extraction

[0073] like Figure 3 As shown, after the input image (i.e., the source style remote sensing image) enters the generator, it first extracts preliminary features through a 7×7 convolutional layer, and then normalizes and activates the features through batch normalization and activation functions (RELU) to obtain the initial feature map.

[0074] S222, Frequency Domain Pooling

[0075] This step is performed based on the Wavelet Pyramid Pooling (WPPM) module in the generator, which operates entirely in the frequency domain and aims to leverage the hierarchical characteristics of wavelet subbands for multi-scale spectral learning to enhance domain-relevant style features while preserving the morphology of the image content.

[0076] The structure of the WPPM module is as follows: Figure 4 As shown, the input feature map (i.e., the initial feature map obtained in the previous step) is first decomposed into four frequency subbands LL, LH, HL and HH by a two-dimensional Haar wavelet transform (WT). These subbands represent the low-frequency fundamental frequency component (LL) and the directional high-frequency components (horizontal LH, vertical HL and diagonal HH) of the feature map, respectively. This decomposition is performed using a fixed Haar filter bank, as shown in Equations (1) and (2).

[0077]

[0078] [X LL ,X LH ,X HL ,X HH ] = Conv([fLL ,f LH ,f HL ,f HH ],X) (2)

[0079] Among them, f LL ,f LH ,f HL ,f HH These represent the Haar filters in four directions; X represents the input feature map for this step, and Conv represents the convolution operation. LL ,X LH ,X HL ,X HH These represent the frequency sub-bands of the feature map X in four directions, i.e., the decomposition results.

[0080] Next, adaptive average pooling is performed on different frequency subbands using pooling kernels of different sizes. The pooling results are then input into a 5×5 convolutional layer. The output of the convolutional layer is added element-wise to the original subband features of the frequency subband to obtain the feature extraction result of the frequency subband.

[0081] In this step, each subband obtained from the decomposition is independently passed through a convolutional layer, preceded by an adaptive average pooling operation with pooling kernel sizes of 1, 3, 5, and 7. These multi-scale pooling layers enable the network to capture domain-specific style features at different resolutions and frequency directions, such as texture intensity and illumination, thereby enhancing the flexibility of cross-domain spectral distribution alignment. Element-wise addition of the pooled and convolutional outputs back to their original subband features enhances spectral contrast without distorting the underlying content structure.

[0082] Furthermore, the low-frequency subband LL undergoes a second Haar wavelet decomposition to obtain four deeper frequency components (LLL, LLH, LHL, LLH), which are then refined using 3×3 convolution to yield feature extraction results for these deeper frequency components. This deep frequency enhancement provides finer spectral granularity, improving the network's ability to handle subtle style variations while maintaining structural coherence.

[0083] Finally, the feature extraction results of all subbands (including frequency subbands in four directions and deep frequency components) are reconstructed by inverse wavelet transform (IWT) implemented by transposed convolution according to formula (3) to obtain the output feature map of the wavelet transform module, namely the first intermediate feature map.

[0084] X = Conv-transposed([f LL ,f LH ,f HL ,f HH ],[X LL ,XLH ,X HL ,X HH (3)

[0085] like Figure 3 As shown, in the LMGAN network, the initial feature map can be input into a 3×3 convolutional layer to perform conventional fine feature extraction in the spatial domain. The features extracted by the convolutional layer in the spatial domain are then fused with the feature map extracted by the WPPM module in the frequency domain to obtain a fused first intermediate feature map, which can replace the first intermediate feature map as the input feature map for subsequent steps.

[0086] In the LMGAN network, multiple frequency domain feature extractions can be achieved by setting up multiple WPPM modules, gradually capturing frequency domain information at multiple scales, and further optimizing the features through the residual module, fusing features from different levels to generate a high-level feature representation.

[0087] The above describes the feature extraction process of the WPPM module in the LMGAN network. The following describes the ICF boundary enhancement module.

[0088] S223, Boundary Enhancement

[0089] This step is performed based on the Interlaced-Cross-Fan (ICF) boundary enhancement module, designed to enhance landslide boundary features. Maintaining fine-scale landslide boundary structure is particularly challenging during domain transfer, as many generative models tend to blur or distort sharp edges during style transfer. To address this issue, this invention proposes the ICF boundary enhancement module, a boundary-aware module designed to emphasize morphological consistency while aggregating multi-directional contextual features. The structure of the ICF boundary enhancement module is as follows: Figure 5 As shown, this module combines edge prior information with three custom convolution types: Interleaved Conv, Cross Conv, and Fan Conv, each capturing unique geometric patterns crucial for enhancing landslide boundaries. The following section combines... Figure 5 This section provides a detailed introduction to the boundary enhancement process.

[0090] First, the Sobel operator is used to extract the boundary features of the source style remote sensing image, and then the features are fused with the first intermediate feature map obtained in the previous step through a 1×1 convolution according to formula (4) to obtain the boundary enhancement feature map X. e This operation acts as a soft guidance mechanism, ensuring that subsequent convolutions remain sensitive to boundary structures.

[0091] X e =X+Conv 1×1 (Sobel(X0)) (4)

[0092] Where X0 is the source style remote sensing image, and X is the input feature map for this step, i.e., the first intermediate feature map. e Enhanced feature map of the fused boundary.

[0093] Next, the boundary enhancement feature map X is processed through multiple convolutional paths. e Feature extraction is performed using multiple convolutional paths, including staggered convolution, cross convolution, and fan-shaped convolution. Each path aims to capture different feature relationships. The architecture of the three convolutional paths is as follows: Figure 5 The lower half shows the description of each convolution path.

[0094] Interleaved Conv paths aim to capture directional features from the horizontal and vertical axes using asymmetric padding convolutions. Interleaved convolution uses four independent 1×3 and 3×1 convolutional kernels and performs directional padding to simulate directional filtering. As shown in equations (5)-(9), the interleaved convolution path first performs asymmetric padding on the input boundary enhancement feature map in different directions, and then uses convolutional kernels W in different directions. i Convolution is performed on the padded feature map, and finally the convolution results X in different directions are combined. i1 -X i4 The connection yields an interleaved feature representation X. io This design allows the network to explicitly model anisotropic boundary structures, such as cliff-like or horizontally layered landslides.

[0095] X i1 =SiLU(BN(X) P(1,0,0,3) *W1 1×3 (5)

[0096] X i2 =SiLU(BN(X) P(0,3,0,1) *W2 3×1 (6)

[0097] X i3 =SiLU(BN(X) P(0,1,3,0) *W3 1×3 (7)

[0098] X i4 =SiLU(BN(X) P(3,0,1,0) *W4 3×1 (8)

[0099] X io =Concat(X) i1 ,X i2 ,X i3 ,X i4 (9)

[0100] Where P(a,b,c,d) represents the padding applied to the left (a), right (b), top (c), and bottom (d) of feature map X, and W i Here, represents the directional convolution kernel, BN represents normalization, SiLU represents the activation function, and Concat represents the connection.

[0101] Cross-convolution paths learn orthogonal dependencies by applying convolutional kernels along the horizontal and vertical axes. Unlike staggered convolutions that process different directions separately, cross-convolution fuses vertical boundary responses into a unified representation. Formally, this operation is particularly effective for capturing common intersections, edge crossings, and T-shaped patterns (e.g., ridge junctions and erosion boundaries) in landslide topographic structures. As shown in Equation (10), the cross-convolution path uses convolutional kernels in the horizontal and vertical directions respectively to enhance the boundary feature map X. e Feature extraction is performed, and the feature extraction results from both are fused to obtain the cross-feature representation X. cross .

[0102] X cross =Concat(X) e *W hor 3×1 ,X e *W ver 1×3 (10)

[0103] Among them, W hor 3×1 The convolution kernel is 3×1 horizontally, W ver 1×3 It is a 1×3 vertical convolution kernel.

[0104] The Fan Conv path extends the concept of directional filtering by aggregating features from multi-directional fan-shaped regions. Each convolution uses a fan-shaped kernel with directional offset to simulate propagation along diagonals and curve edges. The output of the Fan Conv path is computed as shown in Equation (11), where W i P represents a fan-shaped convolution kernel running from the top left to the bottom right. i This indicates the corresponding fill. This design simulates the diffusion of features along radial patterns, enabling the network to follow the curved boundaries commonly found in natural landslide profiles.

[0105]

[0106] The feature extraction process for the three convolutional paths has been described above. By inputting the boundary enhancement feature map into the three convolutional paths, we obtain interleaved feature representations, cross feature representations, and sector feature representations. The second intermediate feature map is then obtained based on these interleaved feature representations, cross feature representations, sector feature representations, and the input first intermediate feature map.

[0107] like Figure 5 As shown, after obtaining the feature representation, the interleaved feature representation X is... io Cross-feature representation X cross and sector feature representation X fan The first intermediate feature map X of the input ICF boundary enhancement module is connected according to formula (12) to obtain the fused feature map X. fuse Then, the fused representation X fuse Input a 3×3 convolutional layer to aggregate the spatial context and obtain the residual feature map X. fused Next, global average pooling is performed on the residual feature map, and channel attention weights are calculated through a 1×1 convolutional layer and the Sigmoid activation function (σ), as shown in formula (13). Finally, the residual feature map is multiplied element-wise with the attention weights, and then multiplied with the original fused feature X. fuse The summation yields a second intermediate feature map. This structure ensures that only the most salient boundary enhancement features are forward-propagated, while residual connections help preserve the global content structure. This combined effect enables precise, morphology-based style synchronization, especially at the edges of landslides.

[0108] X fuse =Concat(X) io ,X cross ,X fan ,X) (12)

[0109] X attn =σ(Conv 1×1 (AvgPool(X fused (13)

[0110] In addition, to enhance the boundary features, the boundary features extracted by the Sobel operator can be further extracted using a 1×1 convolutional layer, and the extraction result can be connected with the second intermediate feature map mentioned above to obtain an enhanced second intermediate feature map, which can replace the second intermediate feature map as the input feature map for subsequent steps.

[0111] like Figure 3As shown, in the LMGAN network, features can be extracted from the first intermediate feature map through multiple 3×3 convolutional layers and transposed convolutional layers. The feature extraction results are then fused with the second intermediate feature map output by the ICF boundary enhancement module to obtain a fused second intermediate feature map. This feature map aggregates multi-directional contextual features on the basis of conventional feature extraction results, which significantly improves the landslide boundary generation effect.

[0112] The above describes the feature extraction process of the ICF boundary enhancement module in the LMGAN network. The following describes the scale-depth separable convolution module of the LMGAN network generator.

[0113] S224, Depthwise Separable Convolution

[0114] like Figure 3 As shown, the features enhanced by the ICF boundary enhancement module are adaptively refined using a multi-scale depthwise separable convolution module. This module contains depthwise separable convolutions with kernel sizes of 3×3, 5×5, and 7×7. The feature extraction results from these three depthwise separable convolutions are fused to obtain the third intermediate feature map output by this module. This depthwise separable convolution design achieves lightweight, scale-aware spatial feature refinement, which helps identify landslides of different sizes and morphologies without introducing significant computational overhead.

[0115] The above describes the feature extraction process of the encoder in the LMGAN network generator. The following describes the image reconstruction process of the LMGAN network decoder.

[0116] S225, Decoding Stage

[0117] This step is implemented based on the decoder of the LMGAN network, which includes a content masking module and an attention masking module. The content masking module focuses on preserving semantically consistent and structurally accurate slide morphology, while the attention masking module adaptively adjusts the importance of pixel-level features to match the style of the target domain. This method of explicitly separating content processing and style processing helps to achieve domain-invariant morphological encoding, which is crucial for maintaining the structural integrity of slide features in different heterogeneous domains.

[0118] The structural diagram of the decoder is as follows: Figure 6As shown, the input feature map (i.e., the third intermediate feature map output by the encoder) is first divided into two parts by a Split operation. One part is input to the content masking module to extract content features, resulting in feature maps CM1, CM2, and CM3. The other part is input to the attention masking module to extract background features, resulting in feature maps FA1, FA2, and FA3, and the source style background feature map BA. Next, the content features CM1, CM2, and CM3 are fused with the features FA1, FA2, and FA3 to obtain the content-enhanced image. The background features of the target style are then fused with the remote sensing image of the source style to obtain the background image features, thereby improving the distinction between the background and foreground. Finally, the content-enhanced image and the background image features are fused to generate the output image, i.e., the target style remote sensing image. The output image retains the basic morphological features of the target area (i.e., the landslide) while maintaining style consistency with the target image and reducing background interference.

[0119] The background features of the target style can be obtained by training the generator using remote sensing images of the target style. As mentioned earlier, the decoder can extract the content features and background features of the input image respectively. As long as real remote sensing images of the target style are used as training samples, the generator can extract the background features of the target style from the training samples and store them in the generator as the basis for image generation.

[0120] The above describes the steps of the remote sensing image style transfer method provided by this invention.

[0121] To demonstrate the effectiveness of the remote sensing image style transfer method provided by this invention in the field of landslide recognition, style transfer and landslide recognition tests were conducted on a real landslide remote sensing image dataset using the LMGAN generative adversarial network provided by this invention. The specific test process and results are described below.

[0122] The remote sensing imagery used in the test consisted of GF1 and GF2 satellite imagery and Google imagery, including landslide samples from Lushan, Taiwan, Jiuzhaigou, Wenchuan, Luding, and Zhouqu. Specific information about the remote sensing imagery data is shown in Table 1. In the test, Lushan was selected as the target style region, while Taiwan, Jiuzhaigou, Wenchuan, and Luding were selected as source style regions for landslide style migration.

[0123] Table 1. Landslide Remote Sensing Image Dataset Information

[0124]

[0125] Before model training, the remote sensing image data needs to be preprocessed. Due to limitations in computer computing power, each image is randomly cropped into 256x256 pixel patches based on the actual location of the landslide. The image channels selected are green, blue, and near-infrared. Three data augmentation methods were used during preprocessing: random 90° and 180° rotations with a 1 / 3 probability, random vertical flips with a 1 / 5 probability, and horizontal flips with a 1 / 4 probability.

[0126] During training, the input data is first encoded into 256-channel features, and then decoded step by step to a target style map of the same size as the input image by concatenating it with the features generated during the encoding process. In testing, the AdamW learning strategy was used to train the LMGAN network model with an initial learning rate of 0.0002. A linear decay strategy was used to update the learning rate with a decay weight of 0.5, and the minimum learning rate was set to 0. Training was conducted for 200 epochs, with the learning rate remaining constant for the first 100 epochs and then decaying linearly for the last 100 epochs.

[0127] After training, the style transfer performance of the trained LMGAN network model was tested using samples from each region in the test set, yielding style transfer results for each region with Lushan as the target style. Simultaneously, other existing image generation models (such as CycleGAN, AttentionGAN, and StegoGAN) were used to perform the same style transfer task under the same conditions as a control. Figure 7 To visualize the style transfer results obtained using different style transfer models with Lushan landslide as the target style and Jiuzhaigou (JZG), Taiwan (TW), Luding (LD), Wenchuan (WC), and Zhouqu (ZQ) as source styles, it can be seen that in most areas, the LMGAN network model provided by this invention can achieve accurate style synchronization while preserving the key features of the landslide. To quantify the style transfer effect, the FID (Fréchet Inception Distance) evaluation index was used to evaluate the style transfer results of the LMGAN network and other models, and the evaluation results are shown in Table 2. The FID evaluation index measures the quality and diversity of generated images by comparing the feature distribution differences between generated images and real images, and is widely used to evaluate the performance of generation models (especially GANs). FID assumes that the feature distribution is a multivariate Gaussian distribution, and calculates the Fréchet distance between the mean and covariance matrices of the generated image and the real image. The lower the FID value, the closer the distribution of the generated image is to the distribution of the real image, and the better the generation effect. The FID value is calculated as shown in formula (14).

[0128] FID = ||μ r -μ g || 2 +Tr(Σr +Σ g -2(Σ r Σ g ) 1 / 2 (14)

[0129] Where, μ r μ is the mean vector of the features of the real image. g To generate the mean vector of image features, Σ r Let Σ be the covariance matrix of the features of the real image. g To generate the covariance matrix of image features, Tr represents the trace of the matrix.

[0130] Table 2. FID values ​​of landslide style migration results

[0131]

[0132]

[0133] As can be seen, the FID values ​​of the LMGAN network model provided by this invention are lower than those of other existing models in each region, indicating that the style transfer method provided by this invention can better preserve the key landslide features in real images.

[0134] In addition, the landslide extraction model SwinUNet was used to extract landslides from images without style transfer and from the transfer results of different models. The precision, recall, F1 score, and IoU (Intersection over Union) of the landslide extraction were evaluated, and the test results are shown in Tables 3-6. The above indicators are calculated based on the number of true positives (TP), false positives (FP), false negatives (FN), and true negatives (TN) in the landslide extraction results, as shown in formulas (15)-(18).

[0135] Precision = TP / (TP + FP) (15)

[0136] Recall = TP / (TP + FN) (16)

[0137] IoU=TP / (TP+FP+FN) (17)

[0138] F1-score=(2×Precision×Recall) / (Precision+Recall) (18)

[0139] Table 3. Statistical table of landslide extraction accuracy before and after landslide style migration (Taiwan-Lushan)

[0140] method IoU F1 precision recall Not migrated 6.01％ 11.35％ 40.77％ 6.59％ CycleGAN 7.52％ 13.99％ 14.31％ 13.68％ AttentionGAN 18.73％ 31.55％ 23.18％ 49.37％ StegoGAN 13.68％ 24.06％ 16.35％ 45.54％ LMGAN 15.07％ 26.19％ 25.35％ 27.09％

[0141] As shown in Table 3, in the Taiwan-Lushan landslide extraction test results, the F1 score of the landslide extraction results without style transfer was only 11.35%. After using CycleGAN and StegoGAN style transfer, the F1 scores of the landslide extraction results increased to 13.99% and 24.06%, respectively, but the overall level was still low. AttentionGAN achieved an F1 score of 31.55%, showing outstanding recall (49.37%), but due to insufficient precision, it resulted in a large number of false positives. LMGAN, on the other hand, had the most stable overall performance, with an IoU of 15.07% and an F1 score of 26.19%, achieving a good balance between precision and recall, thus ensuring the reliability of the results.

[0142] Table 4. Statistical table of landslide extraction accuracy before and after landslide style migration (Jiuzhaigou-Lushan)

[0143] method IoU F1 precision recall Not migrated 4.71％ 9.00％ 12.52％ 7.02％ CycleGAN 11.58％ 20.75％ 34.76％ 14.79％ AttentionGAN 8.36％ 15.43％ 33.22％ 10.05％ StegoGAN 4.50％ 8.62％ 9.96％ 7.60％ LMGAN 9.62％ 17.55％ 20.12％ 15.56％

[0144] As shown in Table 4, in the Jiuzhaigou-Lushan landslide extraction test, the differences in data style were more significant, with the F1 score for landslide extraction results from non-migrated images being only 9.00%. CycleGAN achieved an F1 score of 20.75% in this scenario, outperforming other comparative methods, but exhibiting significant fluctuations. AttentionGAN and StegoGAN both achieved performance below 16%. In contrast, LMGAN maintained high stability while achieving an F1 score of 17.55% and an IoU of 9.62%, demonstrating strong adaptability even under complex cross-domain conditions.

[0145] Table 5. Statistical table of landslide extraction accuracy before and after landslide style migration (Luding-Lushan)

[0146] method IoU F1 precision recall Not migrated 1.21％ 2.40％ 1.76％ 3.79％ CycleGAN 3.76％ 7.25％ 4.04％ 35.26％ AttentionGAN 3.60％ 6.96％ 3.80％ 40.93％ StegoGAN 3.29％ 6.38％ 3.47％ 39.80％ LMGAN 4.26％ 8.17％ 4.68％ 31.81％

[0147] As shown in Table 5, in the Luding-Lushan landslide extraction test, the performance of all style transfer models declined due to complex terrain and significant sample differences. The F1 score for the untransferred image was only 2.40%, while CycleGAN and AttentionGAN achieved 7.25% and 6.96%, respectively, and StegoGAN achieved 6.38%, indicating that the overall accuracy remained low. LMGAN achieved an F1 score of 8.17% and an IoU of 4.26% in this scenario. Although the values ​​are still not high, it outperformed other methods under extremely unfavorable conditions, demonstrating its robustness.

[0148] Table 6. Statistical table of landslide extraction accuracy before and after landslide style migration (Wenchuan-Lushan)

[0149] method IoU F1 precision recall Not migrated 1.33％ 2.62％ 2.37％ 2.94％ CycleGAN 7.23％ 13.51％ 7.63％ 58.80％ AttentionGAN 7.39％ 13.77％ 7.93％ 52.46％ StegoGAN 7.58％ 14.09％ 8.03％ 57.56％ LMGAN 8.32％ 15.36％ 13.97％ 17.05％

[0150] As shown in Table 6, in the Wenchuan-Lushan landslide extraction test, the complex geological and environmental conditions led to the generally poor performance of traditional methods. The F1 score of the untransferred raw data was 2.62%, while CycleGAN, AttentionGAN, and StegoGAN were around 13–14%, but their precision was low and their false positive rate was high. LMGAN achieved an F1 score of 15.36% and an IoU of 8.32%, and significantly improved precision to 13.97%, achieving a better balance between precision and recall, indicating that it has a stronger ability to identify landslide features in complex mountainous scenarios.

[0151] Table 7. Statistical table of landslide extraction accuracy before and after landslide style migration (Wenchuan-Lushan)

[0152]

[0153]

[0154] As shown in Table 7, in the Zhouqu-Lushan landslide extraction test, LMGAN performed exceptionally well in the landslide extraction task, significantly outperforming other models. Its IoU was 8.32%, F1 score was 15.36%, precision was 13.97%, and recall was 17.05%. Compared to other transfer learning methods, LMGAN's IoU was 0.74% higher than StegoGAN, its F1 score was 1.27% higher, its precision was 6.04% higher than AttentionGAN, and its recall was 0.25% higher than CycleGAN. This indicates that our proposed LMGAN method is more effective in transferring landslide extraction data.

[0155] As can be seen, in the cross-regional landslide extraction experiment, the LMGAN network proposed in this invention outperformed or nearly matched the best comparative methods in all five test regions, demonstrating a strong advantage in landslide extraction. Compared with other typical generative adversarial network methods, this method not only achieved higher scores in IoU and F1, but also achieved a better balance between precision and recall, demonstrating strong cross-domain generalization ability and robustness. Compared with traditional GAN ​​methods, the style transfer method proposed in this invention effectively improves the accuracy and robustness of cross-domain landslide detection by leveraging frequency domain feature modeling and attention mechanisms, and demonstrates strong generalization ability under multi-source data conditions.

[0156] In summary, compared with existing technologies, the landslide remote sensing image style transfer method provided by this invention enhances the multi-scale feature representation and regional focusing of landslides through wavelet pyramid pooling and staggered fan-shaped boundary enhancement, significantly improving the quality of the generated images. This allows the generated images to accurately retain key landslide features of the original images while achieving synchronization with the target style. The style transfer method provided by this invention can unify the style of remote sensing images from different regions, or transfer remote sensing images from which landslides are difficult to extract to a target style with higher landslide image quality, thereby improving the accuracy and generalization of landslide identification.

[0157] In the description of the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0158] In the description of the embodiments of this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, and A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more.

[0159] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0160] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for style transfer of remote sensing images, characterized in that, The method is executed based on a generator of a pre-trained generative adversarial network, the generator including an encoder and a decoder, and the method includes: Acquire source-style remote sensing imagery; The source style remote sensing image is used to extract features through a convolutional layer to obtain an initial feature map; The domain-style spectral features of the initial feature map are captured in the frequency domain and reconstructed in the spatial domain to obtain the first intermediate feature map; By aggregating the multi-directional contextual features of the source style remote sensing image, a second intermediate feature map is obtained; The second intermediate feature map is spatially refined by depthwise separable convolution to obtain the third intermediate feature map; The third intermediate feature map is input into the decoder to obtain the target style remote sensing image.

2. The method according to claim 1, characterized in that, The process of capturing the neighborhood style spectral features of the initial feature map in the frequency domain and reconstructing them in the spatial domain to obtain the first intermediate feature map, executed through a wavelet pyramid pooling module, specifically includes: The initial feature map is decomposed into multiple frequency sub-bands using wavelet transform; adaptive average pooling is performed on different frequency sub-bands using pooling kernels of different sizes, and the pooling results are input into a convolutional layer. The output of the convolutional layer is added element-wise to the original sub-band features of the frequency sub-band to obtain the feature extraction result of the frequency sub-band; the feature extraction results of the multiple frequency sub-bands are reconstructed using inverse wavelet transform to obtain the first intermediate feature map.

3. The method according to claim 2, characterized in that, The multiple frequency sub-bands include low-frequency sub-bands and high-frequency sub-bands in multiple directions; After decomposing the initial feature map into multiple frequency sub-bands, the method further includes: decomposing the low-frequency sub-bands into multiple deep frequency components; The deep frequency components are feature-extracted by a convolutional layer, and the feature extraction results of the deep frequency components are reconstructed together with the feature extraction results of the multiple frequency sub-bands by inverse wavelet transform to obtain a first intermediate feature map.

4. The method according to claim 1, characterized in that, The multi-directional contextual features of the source-style remote sensing image are aggregated to obtain a second intermediate feature map. This is then processed by an interlaced fan-shaped boundary enhancement module, specifically including: the interlaced fan-shaped boundary enhancement module using the Sobel operator to extract the boundary features of the source-style remote sensing image, and fusing the boundary features with the first intermediate feature map to obtain a boundary enhancement feature map; inputting the boundary enhancement feature map into interlaced convolution, cross convolution, and fan-shaped convolution paths respectively to obtain interlaced feature representation, cross feature representation, and fan-shaped feature representation; and obtaining the second intermediate feature map based on the interlaced feature representation, cross feature representation, fan-shaped feature representation, and the first intermediate feature map.

5. The method according to claim 4, characterized in that, The boundary enhancement feature map is input into staggered convolution, cross convolution, and fan-shaped convolution paths respectively to obtain staggered feature representation, cross feature representation, and fan-shaped feature representation, specifically including: The boundary enhancement feature map is input into the staggered convolution path, which asymmetrically fills the boundary enhancement feature map in different directions. Different convolution kernels are used to convolve the filled feature map, and the convolution results are fused to obtain the staggered feature representation. The boundary enhancement feature map is input into the cross convolution path. The cross convolution path uses convolution kernels in the horizontal and vertical directions to extract features from the boundary enhancement feature map, and then fuses the feature extraction results of the two to obtain the cross feature representation. The boundary enhancement feature map is input into a fan-shaped convolution path. The fan-shaped convolution path uses multiple fan-shaped convolution kernels with different directional offsets to extract features from the boundary enhancement feature map and fuses the extraction results to obtain a fan-shaped feature representation.

6. The method according to claim 4, characterized in that, The second intermediate feature map is obtained based on the interleaved feature representation, the cross feature representation, the sector feature representation, and the first intermediate feature map, specifically including: The interleaved feature representation, cross feature representation, sector feature representation, and the first intermediate feature map are fused to obtain a fused feature map. The fused feature map is input into a convolutional layer to obtain a residual feature map. Global average pooling is performed on the residual feature map and channel attention weights are calculated. The residual feature map is multiplied element-wise by the attention weights and added to the fused feature map to obtain a second intermediate feature map.

7. The method according to claim 1, characterized in that, The generator is trained based on target style remote sensing images and includes target style background feature data; the decoder includes a content masking module and an attention masking module. The third intermediate feature map is input into the decoder to obtain the target style remote sensing image, specifically including: The decoder divides the third intermediate feature map into two parts through a Split operation, and inputs them into the content mask module and the attention mask module respectively to obtain content features and source style background features; The background features of the target style are fused with the remote sensing image of the source style to obtain the background image features; the background image features are then fused with the content features to obtain the remote sensing image of the target style.

8. The method according to claim 1, characterized in that, The training method for the generative adversarial network includes: Obtain a multi-source remote sensing image dataset, which includes at least two different styles of remote sensing images, referred to as first-style remote sensing images and second-style remote sensing images, respectively. The generative adversarial network includes a first generator, a second generator, and a discriminator; The first style remote sensing image is input into the first generator to obtain the first generated image; the first generated image and the second style remote sensing image are used to train the discriminator. The first style remote sensing image is input into the first generator to obtain the first generated image; the first generated image is judged by the discriminator, and the model parameters of the first generator are iterated according to the judgment result; The first generated image is input into the second generator to obtain the second generated image; the model parameters of the first generator and the second generator are iterated according to the difference between the second generated image and the first style remote sensing image. Repeat the above steps, alternatingly training the discriminator, the first generator, and the second generator of the generative adversarial network until the preset conditions are met, and obtain the trained generative adversarial network.

9. The method according to claim 8, characterized in that, The discriminator adopts the PatchGAN discriminator network structure.

10. The method according to any one of claims 1-9, characterized in that, The method is used for landslide extraction.

Images