A multimodal remote sensing intelligent identification method for hidden geological hazards

Abstract

The present invention discloses a multimodal remote sensing intelligent identification method for hidden geological hazards, comprising: normalizing multimodal remote sensing data; constructing a pixel-level cross-modal visual converter; constructing a pixel-level cross-modal attention module, including a pixel self-attention submodule, a pixel mutual-attention submodule, and a layer-adaptive hybrid module; introducing a layer-adaptive noise mechanism to adaptively adjust the weight ratio of intra-modal self-attention and inter-modal mutual attention at different levels; constructing a relationship discriminator to modulate the attention calculation process by evaluating the inter-modal differences of spatially corresponding points; and training a model using a multi-task loss function, including a semantic segmentation loss and a boundary-aware loss, to jointly optimize and generate hidden geological hazard segmentation results. This method fully utilizes the complementary information of RGB optical images, InSAR deformation rate images, and DEM elevation data, and achieves accurate identification of hidden geological hazards through an efficient pixel-level cross-modal fusion mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of geological disaster monitoring, and in particular to a multi-modal remote sensing intelligent identification method for concealed geological disasters. Background Art

[0002] Hidden geological hazards, such as slow-moving landslides and ground subsidence, are often difficult to detect due to their slow onset and subtle manifestations. They can develop over time and suddenly transform into catastrophic events, posing a serious threat to people's lives and property. Traditional geological hazard investigation methods rely primarily on on-site surveys and expert judgment, but suffer from limited coverage, low efficiency, and low accuracy.

[0003] With the development of remote sensing technology, multimodal remote sensing data has been widely used in the field of geological disaster monitoring. Synthetic aperture radar interferometry (InSAR) can provide millimeter-level surface deformation information; optical remote sensing imagery can reflect surface cover characteristics; and digital elevation models (DEMs) can describe topographical features. This complementary information is of great value in identifying hidden geological hazards. However, existing multimodal data fusion methods have the following main problems:

[0004] 1. Existing methods use simple splicing or serial processing to perform information fusion, which makes it difficult to fully explore the complementarity between different modal data, resulting in limited fusion effects;

[0005] 2. The cross-modal fusion method based on cross-attention has high computational complexity when processing high-resolution remote sensing images, making it difficult to apply to large-scale geological disaster monitoring;

[0006] 3. The fusion method based on feature replacement may cause key information to be lost during the replacement process, affecting the final recognition accuracy;

[0007] 4. There is a lack of special design targeting the characteristics of hidden geological hazards, and it is impossible to effectively capture the complex relationship between slow deformation and surface characteristics.

[0008] Therefore, there is an urgent need to develop an efficient and accurate multimodal fusion method to fully utilize the complementary information of different remote sensing data to realize the identification of hidden geological hazards. Summary of the Invention

[0009] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, one object of the present invention is to propose a multimodal remote sensing intelligent identification method for hidden geological hazards. This method fully utilizes the complementary information of RGB optical images, InSAR deformation rate images, and DEM elevation data to achieve accurate identification of hidden geological hazards through an efficient pixel-level cross-modal fusion mechanism.

[0010] In order to solve the above problems, the present invention provides a multimodal remote sensing intelligent identification method for hidden geological hazards, comprising the following steps:

[0011] Normalize multimodal remote sensing data including optical RGB images, InSAR deformation maps and DEM terrain data, and map different modal data into a unified feature space;

[0012] Construct a pixel-level cross-modal visual converter, including a multimodal encoder and decoder. The encoder uses a multi-stage structure to extract multi-scale features, integrating pixel-level cross-modal fusion modules in each stage. The decoder integrates multimodal encoded features based on a multi-layer perceptron to generate segmentation predictions.

[0013] Construct a pixel-level cross-modal attention module, including a pixel self-attention submodule, a pixel mutual attention submodule, and a layer adaptation hybrid module, to achieve an efficient cross-modal attention mechanism with linear complexity through pixel alignment;

[0014] Construct a layer-adaptive noise mechanism to adaptively adjust the weight ratio of intra-modal self-attention and inter-modal mutual attention at different levels, and balance the degree of modal fusion at different depth layers;

[0015] Construct a relation discriminator to modulate the attention calculation process by evaluating the differences between modalities of spatially corresponding points, thereby enhancing the ability to extract complementary information;

[0016] The training model uses a multi-task loss function including semantic segmentation loss and boundary perception loss to jointly optimize and generate hidden geological hazard segmentation results.

[0017] Preferably, the step of normalizing the multimodal remote sensing data includes:

[0018] For RGB optical images, standard normalization is applied: the pixel values of the RGB image are subtracted from the mean and then divided by the standard deviation;

[0019] For the InSAR deformation rate map, robust normalization is applied: InSAR data are mapped to the range of [-1, 1], and 95% quantile clipping is applied to avoid the influence of outliers;

[0020] For DEM elevation data, logarithmic transformation is applied and then normalized: first calculate the logarithmic transformation log(1+DEM), and then apply standard normalization.

[0021] Preferably, the encoder of the pixel-level cross-modal visual converter includes four stages, each of which includes:

[0022] Overlapping image partitioning module, which divides the input image into serialized blocks through convolution operations;

[0023] Multiple pixel-level cross-modal Transformer blocks, each containing a pixel-level cross-modal attention module and a feed-forward neural network;

[0024] The downsampling module halves the spatial resolution and increases the channel dimension through convolution with a stride of 2;

[0025] The configuration of the four stages includes: the first stage has a block size of 7×7, a step size of 4, and 64 output channels; the second stage has a block size of 3×3, a step size of 2, and 128 output channels; the third stage has a block size of 3×3, a step size of 2, and 320 output channels; the fourth stage has a block size of 3×3, a step size of 2, and 512 output channels.

[0026] Preferably, the calculation formula of the pixel self-attention submodule is:

[0027]

[0028] in, are query, key, and value matrices, is the learnable projection weight, d is the dimension of the attention head, and T is the transposed sign; the pixel mutual attention submodule only establishes connections between pixels that are aligned in spatial positions, and its calculation formula is:

[0029]

[0030] Among them, Q i，n , K j，n and V j，n denote the query of modality i at position n and the key and value of modality j, respectively. i，j is the relation discriminator.

[0031] Preferably, the layer adaptive noise mechanism includes:

[0032] Create unique noise embeddings for each modality and each network layer: in, is the noise embedding vector, EmBed is the embedding function, which generates a specific noise vector according to the modality index i and layer index l;

[0033] The noise embedding is added to the key matrix in the self-attention process: in, is the bond matrix derived from the modality index i and the layer index l, is the key matrix updated after the noise embedding is added;

[0034] Noise injection guides self-attention computation:

[0035] Preferably, the structure of the relationship discriminator is designed as follows:

[0036] Ri,j (X i ,X j )=σ(MLP([X i ;X j ]));

[0037] Among them, [X i ;X j ] represents the feature connection operation, MLP is a multi-layer perceptron, and σ is a Softmax function; the relationship score is used to modulate the key value calculation in the mutual attention: Where ⊙ represents element-wise multiplication, Kj is the key value obtained by modal index j, To calculate the updated key value.

[0038] Preferably, the multi-task loss function includes:

[0039] Focal loss: L focal =-α t (1-p t ) γ log(p t ), where p t is the probability of predicting category t, α t is the category balance parameter, γ is the focus parameter;

[0040] Boundary-aware loss: Among them, CE is the cross entropy loss, w n is the boundary weight: w n =exp(-d n / θ), d n is the distance from the pixel to the nearest segmentation boundary, θ is the parameter that controls the weight decay speed, and y n represents the true value of pixel n, represents the predicted value of pixel n;

[0041] Combined loss: L = λ1L focal +λ2L boundary , where λ1 and λ2 are loss weights, set to ,λ1=0.7,λ2=0.3.

[0042] Preferably, the step of training the model includes:

[0043] Initialize the encoder using a pre-trained visual transformer to speed up training and improve stability;

[0044] The cosine annealing learning rate scheduling is used, and the initial learning rate is set to 2×10 -5 , the minimum learning rate is 1×10 -7 ;

[0045] The batch size is set to 32 and the total number of training epochs is 150.

[0046] Preferably, the method further comprises the steps of reasoning and practical application:

[0047] Acquire multimodal remote sensing data of the target area, including RGB images, InSAR deformation maps, and DEM data;

[0048] Pre-process the data, including registration, cropping and normalization;

[0049] Input the processed multimodal data into the trained model for inference;

[0050] Process the segmentation results output by the model to generate a map of hidden geological hazard risk areas;

[0051] Combined with on-site investigation, verify and refine the geological hazard risk assessment results.

[0052] The advantages of the present invention compared with the prior art are:

[0053] 1. This paper proposes a novel pixel-level cross-modal attention mechanism. Compared with the global cross-attention method, the computational complexity is reduced from quadratic to linear, significantly reducing the computational resource requirements while maintaining the fusion effect.

[0054] 2. This paper designs a layer-adaptive noise mechanism to automatically adjust the ratio of intra-modality self-attention and inter-modality mutual attention in different network depth layers, optimize feature representation, and improve fusion efficiency;

[0055] 3. The present invention designs a special relationship discriminator to enhance the acquisition of complementary information and avoid information redundancy and homogenization;

[0056] 4. Based on the complementarity of multimodal data, the present invention significantly improves the accuracy of identifying hidden geological hazards, especially the deformation monitoring capability in vegetation-covered areas;

[0057] 5. The method of the present invention has strong generalization ability and can be applied to multi-source remote sensing data in different terrain conditions and different seasons, meeting the needs of actual monitoring applications.

[0058] The present invention can achieve accurate identification of hidden geological disasters by efficiently fusing multimodal remote sensing data. It can be widely used in the fields of early warning of geological disasters, risk assessment, disaster prevention and mitigation, and has important theoretical significance and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 Schematic diagram of the overall framework of the method of the present invention (complete processing flow from multimodal data input to hidden geological hazard identification);

[0061] Figure 2 This is the network architecture diagram of the pixel-level cross-modal visual converter (structural design of the model encoder and decoder);

[0062] Figure 3 Schematic diagram of the structure of the pixel-level cross-modal attention module (the connection relationship between the pixel self-attention submodule, the pixel mutual attention submodule and the layer adaptation hybrid module). DETAILED DESCRIPTION

[0063] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0064] In the description of this application, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; they can refer to internal communication between two components or the interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0065] The present invention will be described in further detail below with reference to the accompanying drawings.

[0066] Reference Figure 1 , Figure 1 Schematic diagram of the overall framework of an embodiment of the present invention.

[0067] like Figure 1As shown in the figure, the intelligent identification method of hidden geological hazards based on pixel-level cross-modal visual converter proposed in the present invention includes five main steps: multimodal remote sensing data preprocessing, construction of pixel-level cross-modal visual converter, construction of pixel-level cross-modal attention module, construction of layer adaptive noise mechanism, construction of relationship discriminator and model training and application.

[0068] The pixel-level cross-modal visual converter (PixCrossTrans) of the present invention adopts an encoder-decoder architecture. Figure 2 As shown in the figure, the encoder has a four-stage hierarchical structure for extracting multi-scale features; the decoder adopts a multi-layer perceptron-based design to fuse the multi-scale features of the encoder to generate the final hidden geological hazard segmentation prediction.

[0069] In terms of multimodal input, the present invention mainly processes three types of remote sensing data:

[0070] 1. RGB optical imagery: provides surface cover types and visible spectrum characteristics;

[0071] 2. InSAR deformation rate map: provides centimeter to millimeter level surface displacement information;

[0072] 3. DEM elevation data: provides terrain features such as terrain undulation and slope.

[0073] These three modal data types each have their own unique characteristics and complement each other: RGB imagery can intuitively display surface cover but is susceptible to weather and lighting effects; InSAR deformation data accurately reflects subtle surface deformations but is susceptible to temporal decoherence; and DEM data reflects topographic features but is less timely. The PixCrossTrans model in this paper fully exploits the complementary nature of these three modal data types, extracting comprehensive features through an efficient pixel-level cross-modal fusion mechanism.

[0074] In this embodiment, the method for intelligently identifying hidden geological hazards using a pixel-level cross-modal visual converter includes the following steps:

[0075] Step S1: Multimodal remote sensing data preprocessing.

[0076] Before model processing, different modal data need to be normalized to facilitate network learning. For each modality, a modality-specific normalization strategy is used:

[0077] For RGB optical images, apply standard normalization:

[0078]

[0079] Among them, μ RGB and σ RGB are the mean and standard deviation of the image, respectively.

[0080] For the InSAR deformation rate map, due to the presence of outliers, robust normalization is used:

[0081]

[0082] Among them, min(X InSAR ) and max(X InSAR ) Use robust statistical methods such as 95% quantile to avoid the influence of outliers.

[0083] For DEM elevation data, taking into account the distribution characteristics of elevation data, logarithmic transformation is used and then normalized:

[0084]

[0085] Among them, μ log( DEM ) and σ log(DEM) are the mean and standard deviation of the DEM data after logarithmic transformation.

[0086] The above normalization process ensures that different modal data are mapped to the appropriate numerical range, which is beneficial to network training and feature fusion between modalities.

[0087] Step S2: Build a pixel-level cross-modal visual converter.

[0088] Encoder structure design:

[0089] PixCrossTrans's encoder adopts a hierarchical architecture consisting of four stages, each of which contains overlapping image blocks, multiple pixel-level cross-modal Transformer blocks, and a downsampling module.

[0090] The overlapping image block module divides the input image into sequential blocks

[0091]

[0092] in, Represents the feature representation of the output of the i-th modality in the l-th layer of the Transformer encoder, Proj l is the convolution projection operation, and M is the number of modes. For the first stage, is the i-th modal input data after the normalization process in step S1. The normalization process is to normalize the original modal data according to the normalization formula in step S1 so that the numerical ranges of different modal data are consistent.

[0093] The pixel-level cross-modal Transformer block processes the input features at each stage:

[0094]

[0095] in, represents the output of the k-th Transformer block of the i-th modality in the l-th layer, PCM is the pixel-level cross-modal attention module, LN is the layer normalization, and FFN is the feedforward neural network.

[0096] The feature map resolution generated at each stage is 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input, respectively. It should be understood that the above configuration can be adjusted according to the specific application scenario.

[0097] Step S3: Pixel Cross-Modal Attention Module (PCM).

[0098] The pixel-level cross-modal attention module overcomes the quadratic complexity problem of traditional cross-attention and provides an efficient cross-modal fusion mechanism with linear complexity.

[0099] Pixel self-attention submodule construction:

[0100] For each modality, self-attention is calculated to capture contextual information:

[0101]

[0102] in, are query, key, and value matrices, is the learnable projection weight and d is the dimension of the attention head.

[0103] Pixel Cross-Attention (PCA) submodule construction:

[0104] Unlike traditional global cross attention, pixel mutual attention only establishes connections between pixels that are spatially aligned:

[0105]

[0106] Among them, Q i，n , K j，n and V j,n denote the query of modality i at position n and the key and value of modality j, respectively. i，j is the relation discriminator.

[0107] Layer-Adaptive Mixing Module (LAM) construction

[0108] Combine the outputs of self-attention and mutual attention, and introduce a layer-adaptive noise mechanism:

[0109]

[0110] Among them, N i l is the adaptive noise of the lth layer, α l i,j is a learnable mixing weight that controls the contribution of mutual attention between different modalities.

[0111] Through the above design, the computational complexity of the PCM module is reduced from O(N) to O(N) of the traditional cross attention. 2 ) is reduced to O(N), where N is the number of input tokens. Taking typical remote sensing image processing (512×512 pixels) as an example, traditional cross-attention requires approximately 17G FLOPs, while the PCM module of the present invention only requires 0.14G FLOPs, improving computational efficiency by approximately 99%.

[0112] Step S4: Layer-adaptive noise mechanism construction

[0113] In order to solve the problem that different network depth layers have different requirements for modal fusion, this paper constructs a layer-adaptive noise mechanism (LAN)

[0114] Create unique noise embeddings for each modality and each network layer:

[0115]

[0116] Among them, EmBed is the embedding function, which generates a specific noise vector according to the modality index i and layer index l.

[0117] The noise embedding is added to the key matrix in the self-attention process:

[0118]

[0119] Noise injection guides self-attention computation:

[0120]

[0121] Layer-adaptive noise modulates the soft attention distribution: at shallow layers, noise guides the model to focus more on intra-modal information; at deeper layers, noise encourages the model to focus more on cross-modal information. This adaptive mechanism automatically balances intra-modal feature extraction and inter-modal information integration at different depths.

[0122] Step S5: Build a relation discriminator

[0123] In order to enhance the extraction of complementary information between modalities, the present invention designs a relation discriminator (RD) to evaluate the differences between modalities at corresponding spatial positions.

[0124] Relationship discriminator structure design:

[0125] R i,j (X i ,X j )=σ(MLP([X i ;X j );

[0126] Among them, [X i ;X j ] represents the feature connection operation, MLP is the multi-layer perceptron, and σ is the Softmax function.

[0127] The relationship score is used to modulate the key-value calculation in the mutual attention:

[0128]

[0129] Where ⊙ represents element-wise multiplication

[0130] Through the relationship discriminator, the model can dynamically adjust attention allocation based on the degree of complementarity between different modal data at specific locations, avoiding redundant information exchange and enhancing meaningful cross-modal interactions. For example, in areas with significant terrain changes but consistent optical image coverage, the relationship discriminator guides the model to pay more attention to InSAR deformation data; while in areas with less deformation but distinct optical features, it utilizes more RGB image information.

[0131] Step S6: Model training and application

[0132] Loss function design:

[0133] In order to improve the segmentation accuracy of hidden geological hazards, especially the accuracy of the boundary area, the present invention adopts a combined loss function:

[0134] Focal Loss:

[0135] L focal =-α t (1-p t ) γ log(p t ));

[0136] Among them, p t is the probability of predicting category t, α t is the category balance parameter and γ is the focus parameter, which is used to solve the foreground-background category imbalance problem.

[0137] Boundary-Aware Loss:

[0138]

[0139] Among them, CE is the cross entropy loss, w n is the boundary weight:

[0140]

[0141] d n is the distance from the pixel to the nearest segmentation boundary, and θ is a parameter that controls the speed of weight decay.

[0142] Combined loss:

[0143] L=λ1L focal +λ2L boundary ;

[0144] Where λ1 and λ2 are loss weights. In this example, λ1 = 0.7 and λ2 = 0.3. The model training adopts the following strategy to optimize performance:

[0145] Initializing the encoder with a pre-trained visual transformer (such as the Transformer pre-trained on ImageNet) speeds up training and improves stability.

[0146] Cosine annealing learning rate scheduling is used, with the initial learning rate set to 2×10^-5 and the minimum learning rate to 1×10^-7.

[0147] The AdamW optimizer with weight decay set to 0.05 and betas parameter in [0.9, 0.999] was used.

[0148] A learning rate warmup of 1×10^-6 is used in the first 3000 iterations.

[0149] The batch size is set to 32 and the total number of training epochs is 150.

[0150] After the model training is completed, it can be applied to actual hidden geological disaster monitoring:

[0151] 1. Obtain multimodal remote sensing data of the target area, including RGB images, InSAR deformation maps, and DEM data.

[0152] 2. Pre-process the data, including registration, cropping and normalization.

[0153] 3. Input the processed multimodal data into the trained PixCrossTrans model for inference.

[0154] 4. Process the segmentation results output by the model to generate a map of hidden geological hazard risk areas.

[0155] 5. Combined with on-site investigation, verify and refine the geological hazard risk assessment results.

[0156] Variants and extensions of the invention

[0157] The PixCrossTrans method of the present invention can have many variations and extensions:

[0158] S01: The number of modalities can be flexibly adjusted. In addition to RGB, InSAR, and DEM, other remote sensing data sources such as thermal infrared and SAR intensity maps can also be integrated.

[0159] S02: The encoder can use different backbone networks, such as Swin Transformer or other visual converters, and choose the most suitable architecture according to the specific application scenario.

[0160] S03: The design of the layer-adaptive noise mechanism can be further optimized, such as using a dynamic parameter generation network to adaptively adjust the noise parameters according to the characteristics of the input data.

[0161] In summary, the intelligent identification method for hidden geological hazards based on pixel-level cross-modal visual converter proposed in this paper realizes the efficient fusion of multimodal remote sensing data through an efficient pixel-level attention mechanism, while maintaining linear computational complexity, improving the accuracy and robustness of hidden geological hazard identification, and providing strong support for early warning and risk management of geological hazards.

[0162] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A multimodal remote sensing intelligent identification method for hidden geological hazards, characterized by: The following steps are involved: Normalize multimodal remote sensing data including optical RGB images, InSAR deformation maps and DEM terrain data, and map different modal data into a unified feature space; Construct a pixel-level cross-modal visual converter, including a multimodal encoder and decoder; the encoder uses a multi-stage structure to extract multi-scale features, and integrates a pixel-level cross-modal fusion module in each stage; The decoder integrates multimodal encoding features based on a multi-layer perceptron to generate segmentation predictions; Construct a pixel-level cross-modal attention module, including a pixel self-attention submodule, a pixel mutual attention submodule, and a layer adaptation hybrid module, to achieve an efficient cross-modal attention mechanism with linear complexity through pixel alignment; Construct a layer-adaptive noise mechanism to adaptively adjust the weight ratio of intra-modal self-attention and inter-modal mutual attention at different levels, and balance the degree of modal fusion at different depth layers; Construct a relation discriminator to modulate the attention calculation process by evaluating the differences between modalities of spatially corresponding points, thereby enhancing the ability to extract complementary information; The training model uses a multi-task loss function including semantic segmentation loss and boundary perception loss to jointly optimize and generate hidden geological hazard segmentation results; The calculation formula of the pixel self-attention submodule is: in, are query, key, and value matrices, is the learnable projection weight, d is the dimension of the attention head, and T is the transposed sign; the pixel mutual attention submodule only establishes connections between pixels that are aligned in spatial positions, and its calculation formula is: Among them, Q i，n , K j，n and V j，n denote the query of modality i at position n and the key and value of modality j, respectively. i，j is the relation discriminator; The structure of the relationship discriminator is designed as follows: R i,j (X i ,X j )=σ(MLP([X i ;X j ])); Among them, [X i ;X j ] represents the feature connection operation, MLP is a multi-layer perceptron, and σ is a Softmax function; the relationship score is used to modulate the key value calculation in the mutual attention: Where ⊙ represents element-wise multiplication, Kj is the key value obtained by modal index j, To calculate the updated key value.

2. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1 is characterized by: The step of normalizing the multimodal remote sensing data includes: For RGB optical images, standard normalization is applied: the pixel values of the RGB image are subtracted from the mean and then divided by the standard deviation; For the InSAR deformation rate map, robust normalization is applied: InSAR data are mapped to the range of [-1, 1], and 95% quantile clipping is applied to avoid the influence of outliers; For DEM elevation data, logarithmic transformation is applied and then normalized: first calculate the logarithmic transformation log(1+DEM), and then apply standard normalization.

3. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1 is characterized by: The encoder of the pixel-level cross-modal visual converter consists of four stages, each of which contains: Overlapping image partitioning module, which divides the input image into serialized blocks through convolution operations; Multiple pixel-level cross-modal Transformer blocks, each containing a pixel-level cross-modal attention module and a feed-forward neural network; The downsampling module halves the spatial resolution and increases the channel dimension through convolution with a stride of 2; The configuration of the four stages includes: the first stage has a block size of 7×7, a step size of 4, and 64 output channels; the second stage has a block size of 3×3, a step size of 2, and 128 output channels; the third stage has a block size of 3×3, a step size of 2, and 320 output channels; the fourth stage has a block size of 3×3, a step size of 2, and 512 output channels.

4. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1 is characterized by: The layer adaptive noise mechanism includes: Create unique noise embeddings for each modality and each network layer: in, is the noise embedding vector, EmBed is the embedding function, which generates a specific noise vector according to the modality index i and layer index l; The noise embedding is added to the key matrix in the self-attention process: in, is the bond matrix derived from the modality index i and the layer index l, is the key matrix updated after the noise embedding is added; Noise injection guides self-attention computation:

5. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1 is characterized by: The multi-task loss function includes: Focal loss: L focal =-α t (1-p t ) γ log(p t ), where p t is the probability of predicting category t, α t is the class balance parameter, γ is the focus parameter; Boundary-aware loss: Among them, CE is the cross entropy loss, w n is the boundary weight: w n =exp(-d n / θ), d n is the distance from the pixel to the nearest segmentation boundary, θ is the parameter that controls the weight decay speed, and y n represents the true value of pixel n, represents the predicted value of pixel n; Combined loss: L = λ1L focal +λ2L boundary , where λ1 and λ2 are loss weights, set to ,λ1=0.7,λ2=0.

3.

6. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1, characterized in that: The steps of training the model include: Initialize the encoder using a pre-trained visual transformer to speed up training and improve stability; The cosine annealing learning rate scheduling is used, and the initial learning rate is set to 2×10 -5 , the minimum learning rate is 1×10 -7 ; The batch size is set to 32 and the total number of training epochs is 150.

7. The multimodal remote sensing intelligent identification method for hidden geological hazards according to claim 1, characterized in that: The method also includes the steps of reasoning and practical application: Acquire multimodal remote sensing data of the target area, including RGB images, InSAR deformation maps, and DEM data; Perform pre-processing on the data, including registration, cropping and normalization; Input the processed multimodal data into the trained model for inference; Process the segmentation results output by the model to generate a map of hidden geological hazard risk areas; Combined with on-site investigation, verify and refine the geological hazard risk assessment results.

Images