Geological disaster identification method and system based on multi-modal semi-supervised learning

Abstract

This invention discloses a method and system for geological hazard identification based on multimodal semi-supervised learning. The method includes the following steps: acquiring labeled and unlabeled orthophotos and elevation models of geological hazard areas; constructing teacher and student networks containing dual-branch encoders, extracting features from the orthophotos and elevation models respectively, and linearly fusing them, generating ensemble predictions using learnable dynamic weights; in semi-supervised training, calculating the uncertainty of the student network's prediction results in real time, and triggering an exponential moving average update of the teacher network using the student network parameters only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold; optimizing the student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss; inputting the test data into the trained student network, and outputting fine-grained geological hazard segmentation results. This invention can achieve high-precision automatic identification of geological hazards under small sample conditions.

Description

Technical Field

[0001] This invention belongs to the field of geological hazard identification and machine learning technology, and specifically relates to a geological hazard identification method and system based on multimodal semi-supervised learning. Background Technology

[0002] Benggang is a typical and severe soil erosion landform in the red soil region of southern my country, characterized by steep slopes and intense gully erosion, seriously threatening regional ecological security and the lives and property of residents. With the popularization of UAV mapping technology, the automated identification and monitoring of enggang using high-resolution digital orthophoto maps (DOM) and digital elevation models (DEM) has become a research hotspot.

[0003] Traditional methods for identifying landslides often rely on manual field surveys or visual interpretation of single images, resulting in high costs, long lead times, and limited coverage. In recent years, the introduction of deep learning technology has significantly improved recognition efficiency. For example, Chinese patent application CN120747802A discloses an intelligent landslide recognition method based on a coordinate attention mechanism from UAV images. This method enhances the model's encoding ability for horizontal and vertical features by embedding a coordinate attention mechanism into the U-Net model, thereby improving the recognition of long-distance gullies and steep edges in landslide areas. However, this method primarily focuses on improving the network structure to enhance feature extraction capabilities, and its training process typically relies on a large amount of finely labeled sample data. In practical engineering, landslide terrain is complex and severely affected by vegetation occlusion and shadows, making the acquisition of large-scale, high-precision pixel-level annotations extremely costly and time-consuming.

[0004] To improve recognition accuracy using multi-source data, Chinese patent application CN114972991A discloses an automatic identification method and system for landslides. This method employs the RefineNet framework and fuses DOM and DSM data through a Channel Exchange network, making the information from the two modalities complementary to enhance the model's robustness to terrain changes. Although this method addresses the problem of insufficient information from a single data source, it essentially still falls under the category of fully supervised learning and does not solve the problem of high dependence on labeled data.

[0005] To reduce labeling costs, semi-supervised learning is increasingly being applied to geological hazard identification, with the Mean Teacher framework being one of the mainstream frameworks. This framework utilizes unlabeled samples by imposing consistency constraints on unlabeled data. However, existing semi-supervised methods still face the following challenges in landslide identification scenarios: Traditional Mean Teacher models use a fixed Exponential Moving Average (EMA) strategy to update the teacher network. In the early stages of training or when model performance fluctuates, incorrect predictions from the student network are directly passed to the teacher network, leading to a decline in the quality of pseudo-labels generated by the teacher, thus creating a vicious cycle and resulting in severe confirmation bias. DOM provides texture information, while DEM provides geometric information, and their statistical distributions differ significantly. In the absence of strong supervisory signals, simple splicing or channel swapping is difficult to form stable complementary representations in a semi-supervised environment, easily leading to blurred boundaries or false detections. In long-term semi-supervised training, with the introduction of unlabeled data, the model is prone to overfitting the noise in the pseudo-labels, causing the prediction distribution to deviate from the initial correct semantic direction, i.e., model drift.

[0006] Therefore, designing a collapse identification method that can work with very few labeled samples, effectively suppress false label noise, achieve robust fusion of multimodal data, and prevent model drift is a technical challenge that urgently needs to be solved. Summary of the Invention

[0007] This invention provides a geological hazard identification method and system based on multimodal semi-supervised learning. By constructing a dual-branch teacher-student network architecture that includes linear feature fusion, adopting a dynamic teacher update strategy based on student evolution state evaluation, and combining a multidimensional loss function with confidence weighting and historical distribution constraints for collaborative optimization, this invention aims to solve the problems in existing semi-supervised geological hazard identification methods, such as serious confirmation bias caused by the accumulation of false label noise, unstable model training, insufficient feature expression of a single data modality in complex surface environments, and difficulty in efficient complementary fusion of heterogeneous data.

[0008] To address the aforementioned technical problems, this invention proposes a geological hazard identification method based on multimodal semi-supervised learning, comprising the following steps: Acquire labeled and unlabeled orthophotos and elevation models of geological disaster areas; A teacher and student network containing a dual-branch encoder is constructed to extract features from orthophotos and elevation models respectively and perform linear fusion. Learnable dynamic weights are used to generate ensemble predictions. In semi-supervised training, the uncertainty of the student network's prediction results is calculated in real time. Only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold is an exponential moving average update of the teacher network using the student network parameters triggered; otherwise, the teacher network parameters are kept unchanged or retraining is triggered. Optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss; The data to be tested is input into the trained student network, and the fine segmentation results of geological disasters are output.

[0009] Preferably, the method for constructing the teacher-student network is as follows: The SegFormer architecture is used as the backbone. The encoder calculates multi-head self-attention for orthophoto features and elevation model features respectively, and fuses the features of each branch through linear mapping. The method of generating ensemble predictions using learnable dynamic weights specifically involves: During the decoding stage, the prediction results of the orthophoto branch and the elevation model branch are output separately. Learnable dynamic weights are introduced to perform weighted integration of the prediction results of the orthophoto branch and the elevation model branch.

[0010] Preferably, each level of the dual-branch encoder includes an overlapping block embedding module; The overlapping block embedding module uses a convolution operation with a stride smaller than the kernel size to map the input feature map, in order to maintain the local neighborhood continuity of geological disaster landforms on orthophoto texture and elevation model terrain.

[0011] Preferably, the specific method for updating the exponential moving average is as follows: Calculate the mean of the binary classification information entropy of the current training batch samples as the current uncertainty value, and maintain a historical uncertainty mean; Determine whether the difference between the current uncertainty value and the historical average uncertainty value is greater than a preset evolution threshold; If the value is greater than the value, the student network is considered to have undergone effective evolution, triggering an exponential moving average update of the teacher network. If the value is not greater than the specified value, it is determined that no effective evolution has occurred, and the teacher network parameters remain unchanged.

[0012] Preferably, the method further includes the following steps: A historical buffer is set up to store the current uncertainty value. When the historical buffer is full and no effective evolutionary decision is triggered within the entire window period, the model is determined to be trapped in a local suboptimal solution. Trigger a selective retraining mechanism to reinitialize the student network or retrain it from its historical best state.

[0013] Preferably, the method for calculating the student historical consistency loss is as follows: After completing the initial supervised training phase, the student network parameters at this point are frozen as the initial anchor network. During semi-supervised training, the predicted distribution of the current student network is compared with the predicted distribution of the initial anchor network. The KL divergence between the two is calculated as a historical consistency loss term to constrain the predicted distribution of the current student network from deviating excessively from the distribution structure formed in the initial supervision stage.

[0014] Preferably, the confidence-weighted consistency loss is calculated as follows: Obtain the predicted probability distribution of the teacher network for unlabeled data, and take the maximum class probability as the confidence index; Set a confidence threshold; if the confidence level of a sample is lower than the confidence threshold, the loss weight of the corresponding sample will be reset to 0. If the sample confidence level is higher than or equal to the confidence threshold, a nonlinear transformation of the confidence level is performed using a power exponent to obtain weighting coefficients, which are then combined with cross-entropy to calculate the loss value. Where the power exponent... Nonlinear enhancement is applied to high-confidence samples.

[0015] Preferably, in the semi-supervised training, an asymmetric data augmentation operation is performed before the data is input into the teacher network and the student network: Weak augmentation is performed on the data input to the teacher network, and strong augmentation is performed on the data input to the student network. The enhanced features include geometric transformation and photometric / elevation perturbation. During the geometric transformation, a set of shared geometric transformation parameters are generated and synchronously applied to the orthophoto, elevation model, and labels. When performing photometric / elevation perturbations, a photometric transformation is applied to the orthophoto, and elevation noise or downsampling perturbation is applied to the elevation model data.

[0016] Preferably, the method for adjusting the strength of the enhanced strength is as follows: Define the enhancement intensity coefficient as it changes with the current training steps, using a linear or exponential growth function to gradually increase the enhancement intensity from 0 to a preset maximum value; In the early stages of training, low-intensity augmentations are maintained to avoid strong image perturbations that could disrupt the student network's initial learning of semantic features.

[0017] A second aspect of the present invention also proposes a geological hazard identification system based on multimodal semi-supervised learning, the system being used to implement the geological hazard identification method as described in the first aspect of the present invention, comprising: The data acquisition module is used to acquire labeled and unlabeled digital orthophotos and digital elevation models of geological disaster areas; The network construction module is used to build teacher and student networks containing dual-branch encoders, extract features from digital orthophotos and digital elevation models respectively and perform linear fusion, and generate ensemble predictions using learnable dynamic weights. The dynamic update module is used to calculate the uncertainty of the student network's prediction results in real time during semi-supervised training. It triggers an exponential moving average update of the teacher network using the student network parameters only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold; otherwise, it keeps the teacher network parameters unchanged or triggers retraining. The parameter optimization module is used to optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss. The recognition output module is used to input the test data into the trained student network and output the fine segmentation results of geological disasters.

[0018] Compared with the prior art, the present invention has the following technical effects: 1. The geological hazard identification method proposed in this invention constructs a SegFormer-based dual-branch encoder and adopts a hierarchical linear feature fusion and dynamic integration prediction strategy at the decoding end. It can adaptively balance the texture information of digital orthophotos and the geometric information of digital elevation models, achieving deep interaction of heterogeneous features while maintaining pixel-level spatial alignment.

[0019] 2. The geological hazard identification method proposed in this invention constructs a composite objective function comprising supervised loss, confidence-weighted consistency loss, and student-historical consistency loss. The confidence-weighted consistency loss effectively filters out interference from low-quality pseudo-labels through a nonlinear weighting strategy; the student-historical consistency loss, by introducing distribution constraints into the initial anchor network, effectively prevents semantic drift or catastrophic forgetting in long-term unsupervised training. The synergistic effect of these multiple strategies significantly improves the model's generalization performance and robustness under small sample conditions.

[0020] 3. The geological hazard identification method proposed in this invention introduces a dynamic update mechanism based on the assessment of prediction uncertainty (information entropy). The teacher network's EMA update is triggered only when the student network exhibits effective evolution, coupled with a selective retraining mechanism for long-term stagnation. This effectively cuts off the propagation path of early errors or fluctuating noise from the student network to the teacher network, suppresses the accumulation of confirmation bias common in semi-supervised learning, ensures the quality of pseudo-labels generated by the teacher, and thus guarantees stable convergence of the model even with very little labeled data.

[0021] 4. The geological hazard identification method proposed in this invention combines asymmetric data augmentation with a semi-supervised algorithm, enabling full utilization of massive amounts of unlabeled remote sensing data to mine geomorphic features. This reduces the manual annotation costs in geological hazard monitoring operations while ensuring identification accuracy. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the geological hazard identification method described in this invention; Figure 2 This is a schematic diagram of the structure of the teacher-student network described in an embodiment of the present invention; Figure 3 This is a schematic diagram of the mIoU change curve during the training process described in the embodiment of the present invention; Figure 4 This is a visualization result shown in the embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with specific embodiments of the present application and with reference to the accompanying drawings.

[0024] Example 1 This embodiment describes a geological hazard identification method based on multimodal semi-supervised learning. Figure 1 As shown, it includes the following steps one through five: Step 1: Obtain labeled and unlabeled orthophotos and elevation models of the geological disaster area.

[0025] In this embodiment, taking Benggang as an example, the data acquisition and preprocessing includes data collection and raw processing, data annotation, image cropping and dataset construction, and semi-supervised dataset partitioning sub-processes.

[0026] Data Acquisition and Raw Processing: This study selected a region in southern China as the research area, characterized by typical gully erosion landforms. Low-altitude photogrammetry was conducted using an unmanned aerial vehicle (UAV) equipped with a high-resolution digital camera. Aerial triangulation, dense matching, and orthorectification were performed on the acquired aerial images using photogrammetry software to generate a high-resolution digital orthophoto image with a spatial resolution of 0.5 meters and a digital elevation model (DEM) of the same resolution. Strict spatial registration between the DOM and DEM was ensured, meaning their pixel coordinate systems were consistent, enabling pixel-level overlay and correspondence.

[0027] Data annotation: Experts in the field of geological hazards visually interpreted the DOM imagery of the study area using GIS software. Combining the topographic features of the DEM (such as slope steepness and gully texture), the boundaries of the erosion areas were manually delineated, generating corresponding binary mask labels. A pixel value of 1 represents the erosion area, and a pixel value of 0 represents the non-erosion background area.

[0028] Image cropping and dataset construction: Because the original remote sensing image is too large to be directly input into a deep neural network, a sliding window cropping strategy is adopted. In this embodiment, the cropping window size is set to 1200×1200 pixels. To avoid the collapsed hillside target being truncated at the cropping edge, this embodiment uses an overlapping cropping method. For example, corresponding to the cropping window size set in this embodiment, the cropping step size is set to 600-1000 pixels to retain a certain overlap rate. For the cropped image, image patches with too much pure background or invalid areas are removed, and finally, valid samples containing the collapsed hillside target are selected. In this embodiment, 366 samples are obtained as an example.

[0029] Semi-supervised dataset partitioning: To construct a training environment suitable for semi-supervised learning, the above dataset is divided into training and test sets according to a preset ratio (e.g., 7:3). Furthermore, a labeled / unlabeled partitioning operation is performed on the training set to simulate a scenario where labels are scarce in real-world engineering projects. Labeled Data: A small percentage of samples (e.g., 5%, 10%, or 30%) are randomly selected from the training set, and their corresponding DOM, DEM, and expert-annotated mask labels are retained. This data is used to provide definite supervision signals.

[0030] Unlabeled Data: This refers to the remaining large proportion of samples in the training set (e.g., 95%, 90%, or 70%) as unlabeled data. For this portion, only the DOM and DEM images are retained, and their corresponding mask labels are hidden or removed. During training, the model can only access its image features and cannot directly obtain the ground truth.

[0031] This step completes the construction of a system containing a small number of labeled sample pairs. and a large number of unlabeled samples The multimodal dataset of geological disasters provides a data foundation for the subsequent training of the dual-branch teacher-student network.

[0032] Step 2: Construct a teacher and student network containing a dual-branch encoder, extract orthophoto and elevation model features respectively and perform linear fusion, and generate ensemble predictions using learnable dynamic weights.

[0033] The method for constructing the teacher-student network is as follows: The SegFormer architecture is used as the backbone. The encoder calculates multi-head self-attention for orthophoto features and elevation model features respectively, and fuses the features of each branch through linear mapping.

[0034] The method of generating ensemble predictions using learnable dynamic weights specifically involves: During the decoding stage, the prediction results of the orthophoto branch and the elevation model branch are output separately. Learnable dynamic weights are introduced to perform weighted integration of the prediction results of the orthophoto branch and the elevation model branch.

[0035] Specifically, each level of the dual-branch encoder includes an overlapping block embedding module; The overlapping block embedding module uses a convolution operation with a stride smaller than the kernel size to map the input feature map, in order to maintain the local neighborhood continuity of geological disaster landforms on orthophoto texture and elevation model terrain.

[0036] In this embodiment, combined with Figure 2 Describe the specific implementation details of network construction and forward inference.

[0037] Basic Network Architecture Setup: In this embodiment, the Teacher Network and Student Network have identical network structures, both with two identical branches, but their parameters are stored independently. In branch one, DOM data sequentially passes through a Multi-Head Self-Attention (MHA) layer and a Feed-Forward Network (FFN), and then undergoes linear fusion with the DEM data from branch two, which also sequentially passes through MHA and FFN. The linear fusion unit receives features from the DOM branch. and the characteristics of DEM branches The two were interactively fused to output two features. and .in, It is an image feature that incorporates terrain information and is sent to the DOM branch FFN above; The terrain features, incorporating texture information, are fed into the DEM branch FFN below. The two FFNs after the linear fusion unit receive features that have undergone cross-modal enhancement but still retain their respective modal attributes. Prediction is the prediction result, a pixel-level classification probability map with the same size as the original image, output by the decoder. The Prediction output by the teacher network is used to generate pseudo-labels, while the Prediction output by the student network is used to calculate the loss with the real or pseudo-labels. The Pseudo-label filtering mechanism filters out low-confidence predicted pixels to prevent incorrect pseudo-labels from misleading the student network, as the predictions generated by the teacher network are not always completely accurate. This implementation uses thresholds and weighting functions to filter out low-quality pseudo-labels and weights high-confidence samples. ConsistencyCost represents the consistency loss, a loss function used in semi-supervised learning to measure the difference between the predictions of the student network and the teacher network, aiming to make the student network's output as close as possible to the teacher network's output. Finally, student network parameters are updated via backpropagation, while teacher network parameters are updated via the exponential moving average (EMA) of student network parameters.

[0038] The backbone network uses SegFormer, specifically the MiT-B2 version, which employs a hierarchical Transformer structure. The overall architecture is designed as a dual-branch parallel encoder, with the first branch taking the digital orthophoto image (DOM) as input. The second branch inputs a digital elevation model (DEM). In this embodiment, to efficiently extract multi-scale semantic information, the encoder uses SegFormer's Mix Vision Transformer (MiT) as its backbone and employs a four-stage pyramid structure to generate multi-scale features layer by layer. Let... For the first Layer output features, The encoder is then recursively represented as:

[0039] When i=0, the encoder receives the original DOM and DEM data as input. Each layer consists of an Overlap PatchEmbedding module, multiple Transformer Blocks, and layer normalization and feature rearrangement operations.

[0040] Implementation of the Overlap Patch Embedding module: At the beginning of each stage of the encoder, the feature map is downsampled and serialized using the Overlap Patch Embedding module. Unlike the non-overlapping slicing method of the traditional Vision Transformer, this module uses convolutional layers for embedding. The convolutional kernel size is set to k×k, the stride is s, and the padding is p. To ensure overlap, the stride s is set to be smaller than the convolutional kernel size k (e.g., k=7, s=4 or k=3, s=2).

[0041] The calculation process is as follows:

[0042] in, This represents the overlapping block embedding features, derived from the output of the previous layer. Obtained through Overlap PatchEmbedding and LN; These are the output features of the previous layer. Presentation layer normalization operation, This represents a convolution operation with a kernel size of k×k and a stride of s. This overlapping design preserves the continuity of local neighborhoods between pixels, making it particularly suitable for landforms like eroded hills with continuous erosion edges, thus avoiding the jagged edge effect caused by non-overlapping slices.

[0043] Subsequently Context information is extracted through several Transformer Blocks; each Block can be formalized as a residual structure:

[0044]

[0045] in, For attention aggregation features, The final output feature of the current Transformer Block is represented by ATTE, which represents multi-head sub-attention and subsequent linear mapping operation, and MLP represents multilayer perceptron processing.

[0046] In the encoder Transformer Block of this embodiment, two branches separately compute multi-head self-attention. To achieve intermodal information interaction, while maintaining consistency in channel dimensions, a linear feature fusion module is introduced to explicitly interact with the attention responses of different modalities, thereby achieving cross-modal integration of mid-level semantic information. Specifically, the ATTE function, representing the multi-head sub-attention and subsequent linear mapping operation, computes the multi-head attention output for each modality m branch in the self-attention computation. :

[0047]

[0048] in, The result after fusion This indicates two different data processing branches: DOM and DEM. MHA represents the multi-head attention mechanism. This represents the linear fusion calculation of DOM and DEM features. The attention output of the layer DOM branch is The attention output of the DEM branch is The fused result is obtained by explicitly interacting and fusing the two through a linear fusion layer. This operation allows the network to fuse the spectral texture information of the DOM and the topographic geometry information of the DEM during the mid-layer feature extraction stage.

[0049] Decoding and Dynamic Ensemble Prediction: The decoder receives fused features from the encoder's four-level feature pyramid, restores the resolution through an MLP (Multi-Layer Perceptron) layer, and finally outputs the prediction result. To address the issue of varying modal contributions in different scenarios (such as DOM failure in vegetated areas or indistinct DEM features in flat areas), this embodiment designs a learnable dynamic ensemble strategy.

[0050] The decoder first generates prediction graphs for each DOM branch. Predicted plot of DEM branches Then define a learnable scalar weight. Its value range is limited to [0,1] by the Sigmoid function. The weighted fusion is then used to obtain the integrated prediction result using the following formula. :

[0051] During training, the network will automatically adjust. Value. For example, in a well-textured, exposed, collapsed area, It may tend towards 1; in areas with dense vegetation cover and hidden landslides, It may tend to 0, thereby achieving adaptive perception of complex surface environments.

[0052] After obtaining the predictions for each modality and their ensemble results, all predictions from both the training and evaluation phases are uniformly upsampled to the original input resolution using bilinear interpolation. Finally, during the training phase, the model calculates the supervised loss based on these predictions, while during the inference phase, it simultaneously outputs the predictions for each modality and the ensemble prediction for generating the hillslide segmentation results. The ensemble prediction strategy enhances the algorithm's ability to identify hillslides in areas with unclear boundaries (such as vegetation restoration areas), further improving the model's generalization performance under single-modality loss conditions. Subsequent experiments demonstrate that the ensemble prediction significantly improves segmentation accuracy in areas with blurred boundaries, while maintaining better robustness under single-modality loss.

[0053] Step 3: In semi-supervised training, the uncertainty of the student network prediction results is calculated in real time. Only when the decrease in uncertainty compared with the historical mean exceeds the preset evolution threshold is the exponential moving average update of the teacher network using the student network parameters triggered; otherwise, the teacher network parameters are kept unchanged or retraining is triggered.

[0054] In this embodiment, the semi-supervised training phase is an iterative optimization process aimed at improving the model's generalization ability using unlabeled data. The specific implementation process includes three sub-processes: asymmetric data augmentation and dynamic intensity adjustment, uncertainty calculation and dynamic update determination, and an escape mechanism for local optima.

[0055] In the asymmetric data augmentation and dynamic intensity adjustment sub-process of this embodiment, before inputting data into the teacher network and student network, an asymmetric data augmentation operation is also performed: Weak augmentation is performed on the data input to the teacher network, and strong augmentation is performed on the data input to the student network. The enhanced features include geometric transformation and photometric / elevation perturbation. During the geometric transformation, a set of shared geometric transformation parameters are generated and synchronously applied to the orthophoto, elevation model, and labels. When performing photometric / elevation perturbations, photometric transformations, such as color jitter, Gaussian blur, or grayscale, are applied to orthophotos to simulate different lighting and weather conditions; elevation noise or downsampling perturbations are applied to elevation model data to simulate scenarios of measurement errors or low-precision terrain data.

[0056] Weak augmentation involves only random horizontal flips or very small translations; its purpose is to ensure the teacher network receives clean, stable images, thereby generating high-quality pseudo-labels. Strong augmentation aims to teach the student network feature invariance under various perturbations; in practice, a set of geometric transformation parameters (such as rotation angle and scaling ratio) are randomly sampled and applied simultaneously to the DOM image and DEM data to ensure strict spatial alignment of multimodal data and prevent geometric misalignment from causing training divergence.

[0057] Specifically, the method for adjusting the intensity of the enhanced strength is as follows: Define the enhancement intensity coefficient as it changes with the current training steps, using a linear or exponential growth function to gradually increase the enhancement intensity from 0 to a preset maximum value; In the early stages of training, low-intensity augmentations are maintained to avoid strong image perturbations that could disrupt the student network's initial learning of semantic features.

[0058] Set the total number of training steps to The number of preheating steps to enhance strength is If the current step number is Then the strength coefficient The calculation formula can be selected as follows: Linear growth:

[0059] Exponential growth:

[0060] in, This is an adjustment factor, for example, 5.0. Used to control the trigger probability and perturbation amplitude of strong enhancement operations.

[0061] After data augmentation, the data is input into the network for forward propagation. The process then proceeds to the uncertainty calculation and dynamic update determination sub-process. In this sub-process, the specific method for updating the exponential moving average is as follows: Calculate the mean binary classification information entropy of the current training batch samples as the current uncertainty value, and maintain a historical uncertainty mean.

[0062] For any input sample The prediction results of the student network are normalized using Softmax to obtain the positive class (collapse) probability. The prediction uncertainty of this sample is defined as the binary classification information entropy. :

[0063] For all samples in the current training batch, calculate the average of their uncertainties, and denot it as the current uncertainty value. At the same time, maintain a historical mean of uncertainty. This mean is the average value over the past several iterations. The moving average.

[0064] Determine whether the difference between the current uncertainty value and the historical average uncertainty value is greater than a preset evolution threshold; if it is, determine that the student network has undergone effective evolution and trigger the exponential moving average update of the teacher network.

[0065] Set evolution threshold (For example Calculate the difference. . This indicates that the predictive certainty of the student network has significantly improved, meaning the model has learned effective features. At this point, perform the EMA update operation:

[0066] in, For teacher network parameters, For student network parameters, The momentum coefficient (e.g.) ).

[0067] If the current uncertainty value is not greater than the difference between the historical uncertainty average and the current uncertainty value, then it is determined that no effective evolution has occurred, and the teacher network parameters remain unchanged. This is to prevent incorrect parameters from being passed to the teacher network when the student network performance fluctuates or deteriorates, thereby suppressing the accumulation of confirmation bias.

[0068] As a preferred embodiment of the present invention, an escape mechanism for local optima is also included; specifically, the exponential moving average update method further includes the following steps: A historical buffer is set up to store the current uncertainty value. When the historical buffer is full and no effective evolutionary decision is triggered within the entire window period, the model is determined to be trapped in a local suboptimal solution. A selective retraining mechanism is triggered to reinitialize the student network or start retraining from the historical best state.

[0069] A history buffer size B is set. During training, if the decision result is no effective evolution for B consecutive iterations, the model is considered to have stagnated (a local suboptimal solution). At this point, a selective retraining mechanism is triggered: the student network parameters are... Reset to the state when effective evolution was last triggered, or re-initialize randomly; selectively reduce the learning rate or adjust the enhancement intensity to help the model escape the current local extrema and find a new optimization path.

[0070] Step 4: Optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss.

[0071] In this embodiment, the parameter update of the student network is achieved by minimizing a global loss function consisting of three parts. This is achieved by using the backpropagation algorithm to calculate the gradient and an optimizer (such as AdamW) to update the parameters.

[0072] global loss function Defined as:

[0073] in, To monitor losses, The consistency loss is calculated using confidence-weighted methods. For the loss of historical consistency among students, and To balance the weighting coefficients of each loss term.

[0074] The method for calculating the supervision loss is as follows: For labeled data, the difference between the student network's predictions and the ground truth is calculated. To address the difficulty in identifying the boundary of the collapse, this embodiment employs Online Hard Example Mining (OHEM) with cross-entropy loss. The student network outputs three prediction results: DOM branch prediction. DEM branch prediction and integrated prediction Calculate the OHEM loss for each of the three prediction results, and take the average as the final supervision loss:

[0075] in, Gradient backpropagation is performed only on a predetermined proportion of pixels with the largest loss values ​​to force the model to focus on edge pixels that are difficult to classify.

[0076] The confidence-weighted consistency loss is calculated using unlabeled data, specifically as follows: Obtain the prediction probability distribution of the teacher network on unlabeled data, and take the maximum class probability as the confidence index; assuming the teacher network at pixel location... The predicted probability distribution at is It contains The probability value of each category. Then the confidence level of that pixel. Defined as .

[0077] A confidence threshold is set. If the sample confidence level is lower than the threshold, the loss weight of the corresponding sample is reset to 0. If the sample confidence level is higher than or equal to the threshold, a power-law transformation is applied to the confidence level to obtain weighting coefficients, which are then combined with cross-entropy to calculate the loss value. Nonlinear enhancement is applied to high-confidence samples.

[0078] when When the weighting coefficient is calculated, the formula is:

[0079] in, As a power exponent, this embodiment sets At this point, high-confidence samples will receive greater weights than linear weighting, allowing the model to focus more on learning regions where the teacher network is very certain, further suppressing noise.

[0080] final, Calculated as weighted cross-entropy:

[0081] Where N represents the total number of pixels involved in the loss calculation, or the total dimension of the unlabeled data; Let CE represent the probability distribution of the student network's predictions for the current unlabeled input image, and let CE denote the cross-entropy function. The predicted pseudo-label categories for teachers.

[0082] The student history consistency loss is used to prevent catastrophic forgetting or model drift during semi-supervised training. The specific calculation method is as follows: After completing the initial supervised training phase, the student network parameters are frozen as the initial anchor network. In the training process, several rounds of purely supervised training are first performed using the limited amount of labeled data available. At the end of this phase, a copy of the student network is made and its parameters are frozen, denoted as . This network is no longer updated during subsequent training; it is only used to provide a reference distribution for forward inference.

[0083] During semi-supervised training, the predicted distribution of the current student network is compared with the predicted distribution of the initial anchor network; the KL (Kullback-Leibler) divergence between the two is calculated as the historical consistency loss term to constrain the predicted distribution of the current student network from deviating excessively from the distribution structure formed in the initial supervision stage.

[0084] For unlabeled samples, calculate the current student network prediction distribution. With anchor network prediction distribution KL divergence between:

[0085] To prevent limiting the model's learning ability in the early stages of training, this embodiment also includes a Warmup mechanism: Warmup only occurs when the global iteration count reaches a certain threshold. Only then will Enable this loss term by setting it to a non-zero value (such as 0.1 or 1.0). This ensures that the student network is subjected to the regularization constraint of the historical distribution after fully exploring the features of the unlabeled data, thus achieving a balance between learning new knowledge and retaining old knowledge.

[0086] Step 5: Input the data to be tested into the trained student network and output the fine segmentation results of geological hazards.

[0087] Obtain the digital orthophoto and digital elevation model of the region to be detected. Perform the same preprocessing operations as in step one, including normalization (based on the mean and standard deviation of the training set statistics) and sliding window cropping.

[0088] During the inference phase, only the test data needs to be input into the student network. The teacher network is only used for auxiliary supervision during the training phase and does not participate in the calculation during the inference phase, thus ensuring the inference speed.

[0089] The preprocessed DOM image blocks and DEM data blocks are input into the trained student network, respectively. The student network's dual-branch encoder extracts texture and terrain features, respectively, and performs cross-modal feature interaction through a linear fusion module. The decoder outputs the DOM branch prediction map. and DEM branch prediction map Meanwhile, the network internally adjusts the learned dynamic weights... Automatic calculation of ensemble prediction results:

[0090] The integrated prediction results It combines the advantages of both spectral and topographical approaches, maintaining robustness in both vegetation-covered areas that primarily rely on topography and flat terrain areas that primarily rely on texture, thus serving as the final basis for inference.

[0091] Since the output of network structures such as SegFormer is typically 1 / 4 of the input resolution, this embodiment uses bilinear interpolation to convert the prediction results... The image is uniformly upsampled to the original input resolution. Then, an Argmax operation or a probability threshold is set on the upsampled probability map to classify each pixel as either a collapsed area (pixel value 1) or the background (pixel value 0), generating a binarized mask.

[0092] The binarized masks of each clipping window are restored according to their coordinate positions at the time of clipping and stitched together to form a complete map of the regional landslide distribution. For overlapping areas, a voting method or an averaging method can be used to eliminate stitching gaps. The final output is a detailed segmentation result of geological hazards in the region, which can be used for quantifying the area of ​​landslide erosion, soil and water conservation monitoring, and disaster risk assessment.

[0093] To verify the effectiveness of the method described in this embodiment, a real-world hill collapse dataset was used for testing. The data was collected from a typical hill collapse area in southern China, and DOM and DEM data with a spatial resolution of 0.5m were generated using a drone. A total of 366 samples (size 1200×1200) were obtained through overlapping cropping, and divided into training and test sets in a 7:3 ratio.

[0094] The experimental hardware environment consisted of two NVIDIA GeForce RTX 4090 GPUs, an Ubuntu operating system, and a development framework based on PyTorch 1.12.1 and CUDA 11.8. The training parameters are shown in Table 1 below. Table 1 Training Parameter Settings

[0095] To demonstrate the superiority of the method in this embodiment, it was compared with several mainstream semi-supervised semantic segmentation models, including UPCol, MMS, UAMT, and CCT. Experiments were conducted with three different proportions of labeled data: 5%, 10%, and 30%. The evaluation metrics were the mean intersection-union ratio (mIoU / Mean-IoU) and the pixel accuracy (Mean-Acc).

[0096] The experimental results are shown in Table 2 below: Table 2 Evaluation metrics results for different algorithms

[0097] As shown in Table 2, all semi-supervised learning schemes exhibit significant performance improvements (average accuracy and average intersection-over-union ratio) with the increase of labeled datasets. Under different proportions of labeled data, the method in this embodiment achieves the best performance compared to other mainstream semi-supervised models.

[0098] With an extremely low labeling ratio (5%), existing technologies such as MMS and CCT exhibit mIoU of only 0.2563 and 0.3660, respectively, demonstrating a significant performance degradation. In contrast, the scheme in this embodiment achieves an mIoU of 0.6061 and a Mean-Acc of 0.7288, representing a significant performance leap compared to the suboptimal model. This demonstrates that, under conditions of extreme label scarcity, the dynamic update strategy of this embodiment can effectively suppress the negative impact of noisy pseudo-labels.

[0099] With 10% of the labeled data, the performance of all models was significantly limited by the amount of labeled data. Only UPCoL and the scheme in this embodiment achieved an mIoU of over 60%, with the mIoU and Mean-ACC of this embodiment being 0.71802 and 0.83379, respectively. This result indicates that the scheme in this embodiment has stronger robustness under conditions with very few labels and can effectively alleviate the performance degradation caused by unreliable early pseudo-labels. As the dataset increases, the performance of all models improves significantly. UAMT, UPCoL, and the scheme in this embodiment outperform CCT and MMS, indicating that introducing an uncertainty filtering strategy into the mean model can effectively suppress the interference of unreliable parts in the model prediction.

[0100] With a 30% annotation ratio, the mIoU of the scheme in this embodiment is improved to 0.8481 and the Mean-Acc reaches 0.9158, which is still significantly better than UAMT (mIoU 0.5942) and UPCoL (mIoU 0.7462).

[0101] The above data shows that the solution in this embodiment is not only highly robust under small sample conditions, but also has a higher performance ceiling as the amount of data increases, and can better utilize unlabeled data to improve recognition accuracy.

[0102] To verify the effectiveness of each module in this embodiment, the basic Mean Teacher (MT) model was used as a baseline, and dynamic update, weighted loss, and asymmetric data augmentation were gradually added for testing. The comparison results are shown in Table 3. Table 3 Ablation Test Results

[0103] The results indicate that basic consistency regularization can utilize unlabeled data to some extent, but it is still limited by the quality of pseudo-labels and training stability in low-label scenarios.

[0104] The experiment used the original MT model as the performance baseline for subsequent improvements, achieving Mean-IoU of 0.5770, 0.6251, and 0.7317 under three labeling ratios. These results indicate that basic consistency regularization can utilize unlabeled data to some extent, but it remains limited by pseudo-label quality and training stability in low-label scenarios.

[0105] First, by introducing Dynamic Update, this embodiment utilizes a dynamic update strategy based on uncertainty assessment to replace the exponential moving average for maintaining the teacher model's updates. This improves the Mean-IoU to 0.6116, 0.6362, and 0.7627 for 5%, 10%, and 30% of the data, respectively. This result demonstrates that the dynamic update mechanism can effectively improve the stability of the teacher model, alleviate the false label bias problem caused by fixed or lagging update strategies, and thus provide more reliable supervision signals for the student model.

[0106] Secondly, this embodiment employs a multi-loss weighted optimization strategy consisting of supervised learning loss, confidence-weighted cross-entropy consistency loss, and student history consistency loss. Under 5%, 10%, and 30% labeling conditions, the Mean-IoU reaches 0.6236, 0.7006, and 0.8171, respectively, verifying the effectiveness of the synergistic effect of each loss term. Specifically, the confidence-weighted consistency loss significantly suppresses the interference of noisy pseudo-labels on model training by assigning higher supervision weights to high-confidence predictions. The student history consistency loss alleviates the problem of accumulated error from pseudo-labels in the middle and later stages of training by constraining the current student model prediction to maintain consistency with its historical prediction distribution. This ensures that the algorithm exhibits stronger robustness and generalization ability in scenarios with limited labeled data.

[0107] Finally, ramp-up data augmentation further improved the model's generalization ability, achieving best performance under the current settings across all labeled ratios, with Mean-IoU increasing to 0.6355, 0.7188, and 0.8481, respectively. The data augmentation strategy and consistency constraints worked synergistically to help the model learn more robust feature representations.

[0108] The ablation experiments fully validated the effectiveness of each optimization strategy in this embodiment. The dynamic update strategy based on uncertainty assessment provided stable teacher guidance, the optimized weighted loss effectively mitigated pseudo-label noise, data augmentation improved the model's generalization ability, and multimodal fusion fully utilized the complementary information of different modalities. Experiments show that the synergistic effect of multiple optimization strategies enabled the model to achieve continuous, stable, and significant performance improvements under different labeling ratios, verifying the rationality and effectiveness of the proposed overall method design.

[0109] To verify the necessity of the dual-branch linear fusion architecture, the recognition performance was compared using only DOM, only DEM, and DOM+DEM fusion. The performance comparison results are shown in Table 4 below: Table 4 Performance Comparison of Different Data Combinations

[0110] The visualization results also show that the fusion model can effectively combine the texture details of the DOM and the terrain structure of the DEM, and can accurately identify the boundaries of ridge collapse in both vegetation-covered areas and flat terrain areas, proving the effectiveness of multimodal complementary fusion.

[0111] Comparing the Mean-ACC and Mean-IoU results under different data annotations and three different data inputs, it is clear that the DOM performance improves more significantly with increasing data volume. Even with 30% DOM (Only) input, the algorithm achieves a Mean-IoU of 74%. Therefore, compared to DEM, the modal information contained in DOM has a stronger discriminative ability for semantic category differentiation in the current task, providing a more stable supervision signal under limited annotation conditions. However, DEM (Only) also achieves a score of 0.5965, indicating that although the single-modal performance of DEM is relatively low, it still provides important complementary information to DOM in terms of spatial structure, terrain undulation, or height changes, laying the foundation for subsequent multimodal fusion.

[0112] Secondly, under all three annotation ratio settings, the DOM+DEM fused input achieved the best performance, and significantly improved the Mean-ACC and Mean-IoU metrics compared to the two single-modal inputs, with improvements of approximately +17.91%, +9.65%, and +9.95% respectively compared to the DOM single-modal input. This result fully demonstrates that multimodal fusion can effectively integrate the appearance semantic information of the DOM with the structural geometric information of the DEM, forming a more discriminative and robust joint representation in pixel-level prediction.

[0113] Finally, the performance gains from multimodal fusion are particularly significant with extremely low annotation ratios (5%). In this scenario, the Mean-IoU of the fusion model is improved by approximately +17.91% and +19.75% compared to the DOM and DEM single-modal models, respectively, indicating that when supervision information is extremely limited, the complementarity between different modalities can effectively compensate for the performance bottleneck caused by insufficient information in a single modality.

[0114] The mIoU change curve during the training process is as follows: Figure 3 As shown, compared to the comparison model, the scheme in this embodiment exhibits a faster convergence speed in the early stages of training and maintains a steady performance improvement in the later stages of training, without significant oscillations or overfitting. This further verifies the role of history consistency loss in stabilizing the training process.

[0115] To visually compare the model's recognition performance under different landslide distribution states, this embodiment randomly selected several sets of recognition results for visualization. For example... Figure 4 As shown, each row represents a collapse sample, from top to bottom: Collapse 1, Collapse 2, and Collapse 3.Figure 4 In the image, the first column displays the DOM image of the collapsed area, the second column displays the binary mask labels of the collapsed area (white represents the collapsed area, black represents the non-collapsed area), and the third column presents the segmentation results predicted by the model. By comparing the predicted results with the ground truth labels, it can be seen that the proposed solution can accurately extract the boundaries of collapsed areas in images with different vegetation cover levels. In these three images, collapsed areas 1 and 2 show clearly exposed collapsed areas, so the algorithm's recognition is relatively complete. In contrast, collapsed area 3 has denser vegetation cover. Especially in the DOM image in the third row, although only part of the edge area is captured, the algorithm effectively extracts the approximate location of the collapsed area by combining the unique texture features of the collapsed area with the DEM topographic features and capturing multi-scale features from the tilt angle. However, the semi-supervised model still has room for improvement in recognizing the edges of collapsed areas. Due to the complexity of vegetation cover and the limitations of semi-supervised labels, the algorithm is subject to some significant interference, failing to learn complete features, resulting in edge information not being effectively transmitted to the decoder, thus causing some errors in local edge recognition. In summary, the experimental results demonstrate that the proposed solution has unique advantages and outstanding capabilities in processing geographic information and accurately identifying landslide areas in complex environments.

[0116] Example 2 This embodiment is a geological hazard identification system based on multimodal semi-supervised learning. The system is used to implement the geological hazard identification method as described in Embodiment 1, including: The data acquisition module is used to acquire labeled and unlabeled digital orthophotos and digital elevation models of geological disaster areas; The network construction module is used to build teacher and student networks containing dual-branch encoders, extract features from digital orthophotos and digital elevation models respectively and perform linear fusion, and generate ensemble predictions using learnable dynamic weights. The dynamic update module is used to calculate the uncertainty of the student network's prediction results in real time during semi-supervised training. It triggers an exponential moving average update of the teacher network using the student network parameters only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold; otherwise, it keeps the teacher network parameters unchanged or triggers retraining. The parameter optimization module is used to optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss. The recognition output module is used to input the test data into the trained student network and output the fine segmentation results of geological disasters.

[0117] It should be understood that although the specific embodiments of this specification describe the identification of landslide erosion landforms as an example in detail, the multimodal semi-supervised learning method based on dynamic updating and mean teacher network proposed in this invention is not limited to this. Based on the cross-modal feature fusion and robust semi-supervised training concept of this application, those skilled in the art can extend the method described in this embodiment to the automatic identification of other geological disasters such as landslides, debris flows, and gully erosion, as well as various remote sensing image segmentation scenarios that require comprehensive judgment by combining surface texture and topographic geometric information, such as soil erosion monitoring and ecological restoration assessment.

[0118] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. A geological hazard identification method based on multimodal semi-supervised learning, characterized in that, Includes the following steps: Acquire labeled and unlabeled orthophotos and elevation models of geological disaster areas; A teacher and student network containing a dual-branch encoder is constructed to extract features from orthophotos and elevation models respectively and perform linear fusion. Learnable dynamic weights are used to generate ensemble predictions. In semi-supervised training, the uncertainty of the student network's prediction results is calculated in real time. Only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold is an exponential moving average update of the teacher network using the student network parameters triggered; otherwise, the teacher network parameters are kept unchanged or retraining is triggered. Optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss; The data to be tested is input into the trained student network, and the fine segmentation results of geological disasters are output.

2. The method according to claim 1, characterized in that, The method for constructing the teacher-student network is as follows: The SegFormer architecture is used as the backbone. The encoder calculates multi-head self-attention for orthophoto features and elevation model features respectively, and fuses the features of each branch through linear mapping. The method of generating ensemble predictions using learnable dynamic weights specifically involves: During the decoding stage, the orthophoto branch prediction results and the elevation model branch prediction results are output separately. Learnable dynamic weights are introduced to perform weighted integration of the prediction results of the orthophoto branch and the elevation model branch.

3. The method according to claim 2, characterized in that, Each level of the dual-branch encoder includes an overlapping block embedding module; The overlapping block embedding module uses a convolution operation with a stride smaller than the kernel size to map the input feature map, in order to maintain the local neighborhood continuity of geological disaster landforms on orthophoto texture and elevation model terrain.

4. The method according to claim 1, characterized in that, The specific method for updating the exponential moving average is as follows: Calculate the mean of the binary classification information entropy of the current training batch samples as the current uncertainty value, and maintain a historical uncertainty mean; Determine whether the difference between the current uncertainty value and the historical average uncertainty value is greater than a preset evolution threshold; If the value is greater than the value, the student network is considered to have undergone effective evolution, triggering an exponential moving average update of the teacher network. If the value is not greater than the specified value, it is determined that no effective evolution has occurred, and the teacher network parameters remain unchanged.

5. The method according to claim 4, characterized in that, The method further includes the following steps: A historical buffer is set up to store the current uncertainty value. When the historical buffer is full and no effective evolutionary decision is triggered within the entire window period, the model is determined to be trapped in a local suboptimal solution. Trigger a selective retraining mechanism to reinitialize the student network or retrain it from its historical best state.

6. The method according to claim 1, characterized in that, The method for calculating the student historical consistency loss is as follows: After completing the initial supervised training phase, the student network parameters at this point are frozen as the initial anchor network. During semi-supervised training, the predicted distribution of the current student network is compared with the predicted distribution of the initial anchor network. The KL divergence between the two is calculated as a historical consistency loss term to constrain the predicted distribution of the current student network from deviating excessively from the distribution structure formed in the initial supervision stage.

7. The method according to claim 1, characterized in that, The confidence-weighted consistency loss is calculated as follows: Obtain the predicted probability distribution of the teacher network for unlabeled data, and take the maximum class probability as the confidence index; Set a confidence threshold; if the confidence level of a sample is lower than the confidence threshold, the loss weight of the corresponding sample will be reset to 0. If the sample confidence level is higher than or equal to the confidence threshold, a nonlinear transformation of the confidence level is performed using a power exponent to obtain weighting coefficients, which are then combined with cross-entropy to calculate the loss value. Where the power exponent... Nonlinear enhancement is applied to high-confidence samples.

8. The method according to claim 1, characterized in that, In the semi-supervised training, asymmetric data augmentation is performed before the data is input into the teacher network and the student network: Weak augmentation is performed on the data input to the teacher network, and strong augmentation is performed on the data input to the student network; The enhanced features include geometric transformation and photometric / elevation perturbation. During the geometric transformation, a set of shared geometric transformation parameters are generated and synchronously applied to the orthophoto, elevation model, and labels. When performing photometric / elevation perturbations, a photometric transformation is applied to the orthophoto, and elevation noise or downsampling perturbation is applied to the elevation model data.

9. The method according to claim 1, characterized in that, The method for adjusting the strength of the enhanced strength is as follows: Define the enhancement intensity coefficient as it changes with the current training steps, using a linear or exponential growth function to gradually increase the enhancement intensity from 0 to a preset maximum value; In the early stages of training, low-intensity augmentations are maintained to avoid strong image perturbations that could disrupt the student network's initial learning of semantic features.

10. A geological hazard identification system based on multimodal semi-supervised learning, characterized in that, The system is used to implement the geological hazard identification method as described in any one of claims 1-9, including: The data acquisition module is used to acquire labeled and unlabeled digital orthophotos and digital elevation models of geological disaster areas; The network construction module is used to build teacher and student networks containing dual-branch encoders, extract features from digital orthophotos and digital elevation models respectively and perform linear fusion, and generate ensemble predictions using learnable dynamic weights. The dynamic update module is used to calculate the uncertainty of the student network's prediction results in real time during semi-supervised training. It triggers an exponential moving average update of the teacher network using the student network parameters only when the decrease in uncertainty compared to the historical mean exceeds a preset evolution threshold; otherwise, it keeps the teacher network parameters unchanged or triggers retraining. The parameter optimization module is used to optimize student network parameters by combining supervised loss, confidence-weighted consistency loss, and student historical consistency loss. The recognition output module is used to input the test data into the trained student network and output the fine segmentation results of geological disasters.

Images