Method for detecting landslides on both sides of reservoir based on global-local feature perception collaboration

Abstract

The invention discloses a method for detecting landslides on both sides of a reservoir based on global-local feature perception collaboration. The method comprises the following steps: acquiring landslide images by adopting air-water collaboration and constructing a landslide detection data set on both sides of the reservoir; extracting and fusing the overall landslide terrain trend, crack details and multi-scale edge textures step by step through a four-level layered structure of the constructed landslide feature extraction network, and outputting high-robustness landslide features; and finally, the detection head outputs a bounding box and a confidence coefficient by using the landslide features, and performs landslide detection and deformation judgment in combination with a multi-period data intersection-union ratio. For landslide scenes on both sides of a reservoir, the overall landslide terrain trend and crack details are cooperatively perceived, and global-local feature complementation is realized. The detection head locates a landslide area through a classification and regression network, quantifies the existence probability, completes landslide detection and deformation determination of two banks of the reservoir, achieves accurate monitoring of an existing landslide body in a complex reservoir area environment, improves the reliability and timeliness of deformation identification, and provides a guarantee for safe operation of the reservoir.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geological disaster monitoring, in particular to a reservoir bank landslide detection method based on global-local feature perception cooperation. BACKGROUND

[0002] With the large-scale promotion of water conservancy construction and the increasing demand for water resource regulation, reservoirs, as key water conservancy infrastructure, play an irreplaceable role in flood control and disaster reduction, water supply guarantee, and hydropower generation. However, the bank slopes of reservoirs are affected by multiple factors such as hydrodynamic action, geological structure activity, and meteorological disasters, and landslides occur frequently. Such disasters not only cause bank collapse and sediment accumulation, threatening the safety of dam structures and the normal operation of reservoirs, but also may trigger secondary floods, blockages of waterways, and other chain risks, posing a serious threat to the safety of life and property of surrounding residents and regional social and economic development. Therefore, accurate and efficient detection of landslides on both banks of reservoirs and timely understanding of landslide dynamics are of great practical significance and strategic value for disaster warning, emergency disposal, and safe operation of reservoirs.

[0003] Traditional methods for detecting landslides on both banks of reservoirs mainly rely on manual inspection, geological drilling, and visual interpretation of remote sensing images. These methods not only consume a lot of manpower and resources, but also have problems such as long detection period, limited coverage, and poor timeliness, making it difficult to meet the needs of large-scale reservoirs for full coverage and high-frequency monitoring. With the rapid development of remote sensing technology, data acquisition methods such as high-resolution optical images and laser radar are becoming more mature, providing rich data sources for landslide detection. At the same time, artificial intelligence technologies represented by deep learning have made breakthroughs in image recognition, providing technical support for automatic landslide detection.

[0004] In recent years, deep learning technology has been increasingly applied in the field of landslide detection, which can automatically extract landslide features from complex images, significantly improving detection efficiency and accuracy. In view of the particularity of the reservoir scene, related researches have enhanced the adaptability of the model to complex environments by optimizing the network structure, introducing attention mechanisms, and improving feature fusion methods, such as improving the resistance to interference factors such as water surface reflection and vegetation obstruction, and the ability to identify landslides of different scales. However, the environment on both banks of reservoirs is complex, with varying terrain and many ground object interferences, and the above methods have not solved the problem of global-local feature imbalance in high-humidity, high-reflectivity, and broken terrain scenes in the reservoir area. How to more effectively extract the global distribution pattern and local detail features of landslides and realize their cooperative perception to further improve the detection performance is still a key direction that needs to be broken through. SUMMARY

[0005] To solve the above technical problems, the present application proposes the following technical solutions: In a first aspect, the embodiments of the present application provide a reservoir landslide detection method based on global-local feature perception collaboration, comprising: Air-water collaborative collection of landslide images and construction of a reservoir landslide detection dataset; Through the four-level hierarchical structure of the constructed landslide feature extraction network, the overall terrain trend, crack details and multi-scale edge textures of the landslide are extracted and fused step by step, and the landslide features with high robustness are output; Finally, the detection head uses the landslide feature to output the bounding box and confidence, and combines the intersection over union of multiple periods to detect the landslide and determine the deformation.

[0006] In a possible implementation, the air-water collaborative collection of landslide images and the construction of the reservoir landslide detection dataset comprise: Deploying a UAV above the reservoir area to take multi-angle aerial photographs of typical landslide areas to obtain RGB high-resolution remote sensing images for capturing the overall shape of large-scale landslides; On the water surface, an unmanned ship is used to carry an optical camera, a GPS and an IMU, and to continuously take images along the near-water area on both banks to obtain images for supplementing small-scale crack details near the shore; All devices are synchronized through timestamps to ensure the spatio-temporal consistency of image, position and attitude data, and finally form a multi-source heterogeneous dataset containing visual, geographic and physical parameters; The collected raw data is cleaned and standardized to eliminate noise interference and unify the format, thereby forming the reservoir landslide detection dataset.

[0007] In a possible implementation, the cleaning and standardization of the collected raw data to eliminate noise interference and unify the format to form the reservoir landslide detection dataset comprise: First, geometric correction is performed based on GPS coordinates and attitude sensor data, and a perspective transformation algorithm is used to correct the spatial deviation of the image to eliminate image stretching or deviation caused by ship tilting and rocking, so that each pixel point in the image can accurately correspond to the actual geographic coordinates; Then, a region recognition-differentiation processing two-step strategy is used to suppress water surface interference noise and retain key detection features, including: The optical property difference between the water surface and the bank slope is used to realize region segmentation, and for each pixel point (x, y) in the image, the minimum value of the RGB three channels in a preset size neighborhood centered on it is taken to obtain the dark channel feature of the pixel : ; Wherein, is the color channel, is the neighborhood range, is the channel pixel value. The Otsu algorithm is used again to binarize the dark channel image, and the optimal threshold T is automatically determined to maximize the variance between the water surface and the bank slope; The adaptive Gaussian filter is used to smooth the reflection and wave shadow for water surface area denoising, ensuring that the large-area reflection is effectively suppressed in strong light environment, and avoiding excessive filtering to cause the water surface and bank slope boundary to be blurred in weak light environment; The Sobel edge detection is used for the bank slope area to retain the crack and edge key features; Finally, data standardization processing is performed to eliminate the interference of data format and scale difference on model training, and input consistency is realized.

[0008] In a possible implementation, the landslide feature extraction network includes a multi-level backbone network, wherein a first-level backbone network receives an input reservoir landslide detection data set on both banks, and outputs to a second-level backbone network; the output of each subsequent N-1 level backbone network is used as the input of an N-level backbone network; and the output of the last-level backbone network is given to a detection head; Each level of the backbone network includes a block embedding and a bank mixed block. Each level first performs a block embedding on the input to obtain high-dimensional features of non-overlapping image blocks, and then sends the high-dimensional features to the bank mixed block. Each bank mixed block internally integrates a landslide terrain-crack collaborative perception LTFS module and a detail perception feature fusion DAFF module in sequence, and a normalization layer LayerNorm is arranged before the LTFS module and the DAFF module. The LTFS module collaboratively perceives the overall terrain trend and crack details of the landslide, realizes global-local feature complementation, the DAFF module fuses double-branch features, adjusts through dynamic convolution and feature weighting strategy, strengthens the perception of the landslide boundary contour and detail texture, integrates the advantages of global-local features, and outputs high-discrimination fusion features to improve the landslide feature recognition.

[0009] In a possible implementation, the four-level hierarchical structure of the constructed landslide feature extraction network is used to extract and fuse the overall terrain trend, crack details and multi-scale edge texture of the landslide step by step, and output high-robustness landslide features, including: The input of each level of the backbone network is first subjected to block embedding processing, the image is divided into a fixed-size image block feature sequence that does not overlap each other, and the pixels in the block are mapped into a high-dimensional feature vector; Subsequently, the image block feature sequences of different levels of the backbone network are sent to the bank mixed block for deeper feature processing; The LTFS module and the DAFF module integrated in the bank mixed block in sequence, the LTFS module is composed of a local attention branch and a frequency self-attention branch, which refines the global branch, and realizes collaborative perception of the overall terrain trend and local crack details of the landslide.

[0010] In a possible implementation, the image block feature sequence of the subsequent different levels of backbone network is sent to the bank mixed block for deeper feature processing, including: The first level of backbone network receives an input with a resolution of , embedded by 16x16 pixel block division, and extracts large-scale bank terrain features by the bank mixed block; The second level of backbone network is based on resolution, embedded by 8x8 pixel block division, and extracts medium-scale landslide structure features by the bank mixed block; The third level is for resolution, embedded by 4x4 pixel block division, and captures small-scale crack texture features by the bank mixed block; The fourth level performs 2x2 pixel block division on resolution, and integrates the multi-scale features of the previous three levels by the bank mixed block to output a complete feature expression covering large-scale terrain, medium-scale structure, and small-scale texture, capturing large-scale terrain and small-scale crack details.

[0011] In a possible implementation, in the local attention branch, a lightweight attention convolution operator is introduced to reduce computational complexity and improve operation efficiency; The input feature is first generated into query vector , key vector and value vector by a fully connected layer; Depthwise separable convolution is applied to query vector and key vector respectively; Then, layer normalization is applied to the features after convolution to generate enhanced query vector and key vector ; Next, the Hadamard product of the enhanced query vector and key vector is calculated to capture the local interaction between the query and the key and enhance the context awareness of the network, and the result of the Hadamard product is passed through a fully connected layer, combined with Gaussian error linear unit (GELU) and hyperbolic tangent (Tanh) activation function, to improve the model's ability to represent complex remote sensing features; For value vector , adaptive average pooling is first used to aggregate global information, and then depthwise separable convolution is used to further capture local spatial features to generate enhanced value vector ;, dynamically adjust the importance of features, so that the network can effectively focus on the key information in the remote sensing image; then these attention weights and a set of learnable parameters Element-wise multiplication, dynamically adjust the kernel weight, so that the network can adaptively optimize different remote sensing input features; learnable parameters Initialize using the Xavier strategy to ensure the stability of the gradient flow at the beginning of training; update through backpropagation and optimize using Adam to enable the model to adaptively refine the convolutional weights for different remote sensing image features; Finally, perform Hadamard product operation between the enhanced value vector And attention weight Attn, to deeply integrate information and produce the final output, the whole process is represented as follows: ; ; ; ; ; ; ; ; Where: is the local branch input, FC represents the fully connected layer, is the number of feature channels, is the Hadamard product.

[0012] In one possible implementation, the FSAR module is introduced in the global branch, which introduces frequency domain feature processing in the self-attention SA mechanism. The self-attention mechanism only adjusts the weighting parameters through the interaction of internal information; Input features First, process through the fully connected layer to generate the query vector , key vector and value vector ; The FSAR module extracts low-frequency features from the feature matrix through channel-wise adaptive pooling, and extracts high-frequency features from the feature matrix using depth separable convolution; Subsequently, perform matrix operation on the query vector and low-frequency features LK, divide by the square root of the feature map dimension for scaling, and normalize using the softmax function; Finally, the normalized results are matrix multiplied with the high-frequency feature HV to obtain the output feature of the FSAR module, and the entire process is represented by the formula: ; ; ; ; ; wherein: represents the low-frequency feature, represents the high-frequency feature, represents the matrix of each channel, represents the number of channels, represents the range of the pooling step.

[0013] In one possible implementation, the DAFF module enhances the directional edge response through the edge-aware convolution BorderConv, reduces the computational overhead through the depthwise separable convolution DWConv module, and combines the GELU-Tanh nonlinear activation function and residual cross-branch fusion to improve the recognition degree and consistency of the crack endpoint and bank collapse boundary fine-grained features and suppress background noise; Input feature First, the feature is mapped through the fully connected layer FC layer, and then the GELU activation function is used for nonlinear transformation to enhance the detail perception; Subsequently, the and modules are further refined to capture the boundary details and spatial structure of the image, and finally the fully connected layer is output as the main branch feature ; The DAFF module introduces a parallel auxiliary branch, in which the input feature is first extracted through DWConv, then normalized through LayerNorm, and further refined through the fully connected layer to form the auxiliary feature ; The final main branch feature and the auxiliary feature are fused through the residual connection to generate the final output feature , and the process is represented by the formula as follows: ; ; .

[0014] In a possible implementation, the last detection head outputs a bounding box and a confidence level with the landslide feature, and combines a multi-period data intersection ratio to perform landslide detection and deformation determination, including: The dual-branch detection head adopts a parallel structure of a classification branch and a regression branch, analyzes the features output by the landslide feature extraction network, cooperatively optimizes through a multi-dimensional loss function, and realizes accurate discrimination of the landslide category and fine positioning of the boundary; The classification branch distinguishes the specific type of the landslide and suppresses misjudgment caused by interference factors. After the input features are reduced in dimension through a 1x1 convolution layer, the classification branch outputs a probability distribution of each category through a Softmax activation function. The loss function is defined as: ; wherein N is the number of samples, C is the number of landslide categories, is the true label of the i-th sample belonging to the j-th category, is the predicted probability of the corresponding category; the loss function is used to distinguish different types of landslides; The regression branch is used to predict the boundary box parameters of the landslide area. After the features are mapped through a convolution layer, the regression branch outputs normalized boundary box parameters through a Sigmoid activation function, and uses a smooth L1 loss function to optimize the positioning accuracy: ; wherein: is the i-th boundary box parameter predicted by the model, is the corresponding true boundary box parameter, and the loss function is used to predict the boundary box error; The positioning loss function is introduced to associate the classification confidence and the boundary box positioning accuracy, and the total loss formula is: ; ; wherein, is the intersection over union of the predicted box and the true box, reflecting the positioning accuracy, is the confidence of the true box, 、 、 is a hyperparameter for balancing each loss term.

[0015] In the embodiments of the present application, by constructing a data set based on air-water cooperative collection of landslide images, the overall terrain trend and crack details of the landslide are cooperatively perceived for the landslide scene on both sides of the reservoir, and global-local feature complementation is realized. The detection head locates the landslide area through the classification and regression network, quantifies the existence probability, completes the landslide detection and deformation determination on both sides of the reservoir, adapts to complex environments, and ensures the monitoring accuracy. The existing landslide body is accurately monitored in the complex reservoir environment, the reliability and timeliness of deformation recognition are improved, and the safety of the reservoir operation is ensured. Attached Figure Description

[0016] Figure 1 A flowchart illustrating a method for detecting landslides on both banks of a reservoir based on global-local feature perception collaboration, provided in an embodiment of this application; Figure 2 This is a schematic diagram of the landslide feature extraction network provided in an embodiment of this application; Figure 3 This is a schematic diagram of the LTFS module provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of the DAFF module provided in an embodiment of this application. Detailed Implementation

[0017] The present solution will now be described in conjunction with the accompanying drawings and specific embodiments.

[0018] See Figure 1 The landslide detection method for both banks of a reservoir based on global-local feature perception collaboration provided in this embodiment includes: S101, adopts air-water collaborative acquisition of landslide images and constructs a landslide detection dataset on both sides of the reservoir.

[0019] In this embodiment, an air-water collaborative approach is used to acquire landslide images. UAVs acquire large-scale, high-resolution remote sensing images from the air, while unmanned surface vessels collect detailed images near the shore. Both types of platforms are equipped with optical cameras, GPS, and attitude sensors. Spatial deviations are eliminated through spatiotemporal synchronization and geometric correction, and reflective noise is removed through a water surface interference suppression algorithm. Finally, a standardized multi-source sample dataset is constructed to provide high-quality input for subsequent detection.

[0020] Specifically, to improve the coverage and resolution of landslide samples on both banks of the reservoir, this embodiment adopts an air-water collaborative acquisition strategy. UAVs are deployed over the reservoir area to conduct multi-angle aerial photography of typical landslide areas, acquiring 512×512–2048×2048 pixel RGB high-resolution remote sensing images to capture the overall morphology of large-scale landslides. On the water surface, unmanned surface vessels equipped with optical cameras, GPS, and IMUs continuously photograph along the near-water areas on both banks, acquiring 640×640 pixel images to supplement details of small-scale cracks near the shore.

[0021] All devices are synchronized via timestamps to ensure the spatiotemporal consistency of image, location, and attitude data, ultimately forming a multi-source heterogeneous dataset containing visual, geographic, and physical parameters. All images and their metadata (GPS coordinates, IMU attitude, timestamps) are cached in real time on the airborne edge computing node to avoid data loss caused by unstable networks in mountainous areas.

[0022] The collected raw data is cleaned and standardized. Addressing the characteristics of reservoir bank slope images—"strong water surface noise interference, easy loss of bank slope details, and heterogeneous data formats"—noise interference is eliminated and the format is standardized to provide high-quality data for model input. The specific process is as follows: First, geometric correction is performed. Based on GPS coordinates and attitude sensor data, a perspective transformation algorithm is used to correct the spatial deviation of the image, eliminating image stretching or offset caused by ship tilting and turbulence, so that each pixel in the image can accurately correspond to the actual geographic coordinates.

[0023] In reservoir bank slope images, noise such as reflections and wave shadows in the water surface area differs significantly from landslide features (such as cracks and edges) in the bank slope area. A two-step strategy of "region identification - differential processing" is required to accurately suppress noise while retaining key detection features. The specific process is as follows: Based on the "dark channel prior" theory, region segmentation is achieved by utilizing the difference in optical properties between the water surface and the bank slope. Specifically, for each pixel (x, y) in the image, a region segmentation is performed centered on that pixel. Within the neighborhood, the minimum value of the RGB three channels is taken to obtain the dark channel feature of the pixel. : ; in, For color channels, For the neighborhood range, This represents the channel pixel value.

[0024] The Otsu algorithm is then used to binarize the dark channel image, automatically determining the optimal threshold T to maximize the variance between the water surface and bank slope classes. This method is highly adaptable to different lighting conditions, providing a precise basis for region segmentation in subsequent differential processing.

[0025] To address the different characteristics of the water surface and the bank slope, targeted algorithms are employed to achieve a balance between noise suppression and feature enhancement. For noise reduction in the water surface area, adaptive Gaussian filtering is used to smooth reflections and wave shadows. This design ensures effective suppression of large-area reflections in strong light conditions and avoids over-filtering that could blur the water surface and bank slope boundaries in low light conditions. For the bank slope area, Sobel edge detection is used to preserve key features such as cracks and edges.

[0026] To eliminate the interference of data format and scale differences on model training and achieve input consistency, bilinear interpolation is first used to uniformly scale all images to 640×640 pixels. Then, Min-Max normalization is used to map pixel values ​​to the [0,1] interval to eliminate the influence of illumination changes. The specific calculation formula is as follows: , wherein, , respectively are the global minimum and maximum pixel values of the image, and the normalized pixel mean is stabilized at 0.5±0.1, facilitating fast convergence of the model.

[0027] In S102, the four-level hierarchical structure of the landslide feature extraction network is used to extract and fuse the overall terrain trend, crack details and multi-scale edge texture of the landslide, and output high-robustness landslide features.

[0028] In this embodiment, the landslide feature extraction network includes a multi-level backbone network, wherein the first-level backbone network receives the input reservoir landslide detection data set and outputs to the second-level backbone network; the output of the subsequent N-1 level backbone network is used as the input of the N level backbone network; and the output of the last level backbone network is given to the detection head.

[0029] Each level of the backbone network includes a block embedding and a bank mixed block. After a block embedding is performed on the input to obtain high-dimensional features of non-overlapping image blocks, the high-dimensional features are sent to the bank mixed block. The bank mixed block is internally integrated with an LTFS (Landslide Terrain-Fissure Synergy) module and a DAFF (Detail-Aware Feature Fusion) module in sequence. A normalization layer LayerNorm is arranged before the LTFS module and the DAFF module. The LTFS module cooperatively perceives the overall terrain trend and crack details of the landslide, realizes global-local feature complementation, the DAFF module fuses double-branch features, adjusts through dynamic convolution and weighting strategy, strengthens landslide boundary contour and detail texture perception, integrates global-local feature advantages, outputs high-discrimination fusion features, and improves landslide feature recognition. Figure 2 In this embodiment, a four-level backbone network is constructed to extract multi-scale basic features layer by layer.

[0030] For the reservoir landslide detection scene, the air-water cooperative collection image is often affected by factors such as water surface reflection and platform shaking, resulting in detail loss, large scale difference, and difficulty in adapting to complex scenes for traditional feature extraction. Therefore, in this embodiment, a four-level hierarchical backbone network is used to perform multi-level feature extraction on the unified coordinate system image after geometric correction and water surface interference suppression.

[0031] Each level first performs block embedding processing, divides the image into fixed-size blocks that do not overlap each other, and maps the pixels in the blocks to high-dimensional feature vectors for more detailed feature extraction.

[0032] ​Subsequently, the input of each level of the backbone network first performs block embedding processing, divides the image into a sequence of fixed-size image blocks that do not overlap each other, and maps the pixels within the block to a high-dimensional feature vector.

[0033] The image block feature sequences of different levels of the backbone network are sent to the bankside hybrid block for deeper feature processing. Specifically, the first level receives an input with a resolution of , performs block embedding by 16x16 pixel blocks, and extracts large-scale bankside terrain features such as overall slope direction and macroscopic terrain trend by the bankside hybrid block; the second level performs block embedding by 8x8 pixel blocks based on a resolution of , and extracts medium-scale landslide structure features such as landslide body contour and patchy bank collapse area by the bankside hybrid block; the third level performs block embedding by 4x4 pixel blocks for a resolution of , and captures small-scale crack texture features by the bankside hybrid block; and the fourth level performs block embedding by 2x2 pixel blocks for a resolution of , and fuses the multi-scale features of the previous three levels by the bankside hybrid block to output a complete feature expression covering "large-scale terrain - medium-scale structure - small-scale texture", realizes progressive extraction of multi-scale features from coarse to fine for the bankside image, fully captures large-scale terrain and small-scale crack details, and provides high-robustness basic feature support for subsequent landslide detection.

[0034] The LTFS module and the DAFF module are integrated in the bankside hybrid block in sequence, while maintaining lightweight, adaptively compensating for sub-pixel displacement caused by ship body sway, significantly improving feature robustness in complex reservoir environments, and providing high-quality multi-scale basic features for subsequent landslide detection and dynamic early warning.

[0035] To solve the problem of insufficient coordination between global spatial distribution and local detail features in landslide detection on both banks of the reservoir, the LTFS module is introduced. The module mainly consists of a local attention branch and a frequency self-attention refined global branch, realizes the coordinated perception of landslide overall terrain trend and local crack details, effectively balances multi-scale features, and improves detection accuracy in complex environments. The LTFS module is shown in FIG. 3.

[0036] In the local branch, a lightweight attention convolution operator is introduced to reduce computational complexity and improve operation efficiency. Specifically, the input feature is first passed through a fully connected layer to generate a query vector , a key vector , and a value vector , which retains the structure of the classic attention mechanism and maintains overall simplicity.

[0037] In order to effectively capture local information in remote sensing images while reducing computational overhead, the query vector is first passed through a 1x1 convolution layer to reduce the dimensionality of the feature, and then the dimensionality of the feature is restored by a 1x1 convolution layer. and key vectors Depthwise Convolution (DWConv) is applied. Subsequently, LayerNorm is applied to the convolved features to eliminate the scale differences between channels and ensure stable and fast convergence during training. This produces enhanced query vectors and key vectors .

[0038] Next, the Hadamard product of the enhanced query vectors and key vectors is calculated, effectively capturing local interactions between queries and keys, highlighting important features, and enhancing the network's contextual awareness. The result of the Hadamard product is passed through a fully connected layer and combined with Gaussian Error Linear Unit (GELU) and hyperbolic tangent (Tanh) activation functions to further improve the model's ability to represent complex remote sensing features.

[0039] For value vectors , adaptive average pooling is first used to aggregate global information, and then depthwise separable convolution is used to further capture local spatial features, generating enhanced value vectors . Subsequently, the softmax function is used to calculate attention weights , dynamically adjusting the importance of features, allowing the network to effectively focus on key information in remote sensing images. These attention weights are then element-wise multiplied by a set of learnable parameters , dynamically adjusting the convolution kernel weights, allowing the network to adaptively optimize for different remote sensing input features. The learnable parameters are initialized using the Xavier strategy to ensure the stability of gradient flow at the beginning of training. They are updated through backpropagation and optimized using Adam, allowing the model to adaptively refine convolution weights for different remote sensing image features.

[0040] Finally, the Hadamard product operation is performed between the enhanced value vectors and attention weights Attn to deeply integrate information and produce the final output. This mechanism effectively enhances the model's local perception while maintaining sensitivity to global contextual features, making it well suited for lightweight slope image detection tasks. The entire process can be represented by the following formula: ; ; ; ; ; ; ; ; wherein: is the local branch input, FC denotes a fully connected layer, is the number of feature channels, is a Hadamard product. Unlike traditional self-attention that only relies on , the embodiment integrates and GELU to enhance nonlinearity and better capture complex spatial dependencies. Stable attention weights, while GELU improves feature extraction in remote sensing images.

[0041] In the global branch, the embodiment introduces a frequency self-attention refinement module, which introduces frequency domain feature processing in the self-attention (SA) mechanism. The self-attention mechanism adjusts the weighting parameters only through the interaction of internal information. Given that target detection in slope bank images is disturbed by complex backgrounds, the existing SA mechanism only performs similarity attention feature map calculation through the self-similarity of the matrix without processing global information to achieve more effective global feature extraction.

[0042] Input features are first processed by a fully connected layer to generate query vectors , key vectors and value vectors . The FSAR module extracts low-frequency features from the feature matrix through channel-wise adaptive pooling, and extracts high-frequency features from the feature matrix using a depth separable convolution. This method not only specifically separates features of different frequencies, but also lays the foundation for subsequent feature fusion and processing.

[0043] Subsequently, the FSAR module extracts low-frequency features from the feature matrix through channel-wise adaptive pooling, and extracts high-frequency features from the feature matrix using a depth separable convolution. To ensure that the distribution of the operation result is more reasonable, avoid entering the saturation zone of the softmax function, and allow the model to better focus on important features.

[0044] Finally, the normalized results are multiplied by the high-frequency feature HV to obtain the output feature of the FSAR module. In this way, the high and low frequency features can be fused, the extraction effect of global information in slope image target detection can be improved, and the interference of complex background can be reduced.

[0045] The whole process can be represented by the following formula: ; ; ; ; ; Among them: represents the low-frequency feature, represents the high-frequency feature, represents the matrix of each channel, represents the number of channels, represents the range of pooling step. In the experiment, since the number of feature channels is 6, the number of channels is divided into two groups, which are set to 1, 2 and 3 respectively.

[0046] By adopting this double-branch structure design, the model can effectively process multi-scale information without stacking deep complex modules, thereby significantly reducing the overall complexity. In order to effectively integrate the output of the local and global branches, the output feature of the local branch is spliced with the output feature of the global branch along the channel dimension, and a residual connection mechanism is introduced. This method not only improves the processing efficiency of the model, but also enhances the performance of slope image detection without significantly increasing the number of model parameters.

[0047] Since the landslide boundary often appears fuzzy and intermittent in the environment of high reflectivity and vegetation shelter in the reservoir area, it is difficult for existing networks to maintain the integrity and consistency of the crack-landslide edge. Therefore, the DAFF module is introduced in this embodiment. This module enhances the directional edge response through Border-aware convolution (BorderConv), reduces the computational overhead through Depthwise convolution (DWConv), and combines GELU-Tanh nonlinear activation function and residual cross-branch fusion, which can significantly improve the recognition and consistency of fine-grained features such as crack endpoints and landslide boundaries, while suppressing background noise. The detail-aware feature fusion module is shown in Figure 4.

[0048] Specifically, the input feature First, the feature is mapped through a fully connected layer, and then a GELU activation function is used for nonlinear transformation to enhance the perception of details. Subsequently, the feature is further refined through a fully connected layer and output as the main branch feature and The module is further refined to capture the boundary details and spatial structure of the image, and finally output through a fully connected layer as the main branch feature .

[0049] The main purpose of the operation is to specifically enhance the edge perception ability of the image. In remote sensing images, many key land feature details are located in the edge area, and boundary details are particularly important for target detection and classification. The core purpose is to reduce the computational complexity of convolution operations while retaining efficient feature extraction capabilities, which is particularly effective for resource-constrained devices.

[0050] In addition, in order to further improve the efficiency of the network in extracting detailed features, The module introduces a parallel auxiliary branch. In this branch, the input feature is first extracted through , then normalized through LayerNorm, and further refined through a fully connected layer to form the auxiliary feature . Finally, the main branch feature and the auxiliary feature are fused through residual connection to generate the final output feature Z output. The specific formula is as follows: , , .

[0051] S103, finally the detection head outputs the bounding box and confidence with the landslide feature, and combines the intersection over union of multiple periods to detect and deform the landslide.

[0052] In view of the scene characteristics of "terrain fragmentation, large scale difference, and strong water surface reflection interference" in reservoir landslide detection, the dual-branch detection head adopts a classification and regression parallel structure, analyzes the "global-local collaborative features" output by the LTFS-DAFF module, and optimizes through a multi-dimensional loss function to realize accurate discrimination of landslide categories and fine positioning of boundaries.

[0053] The classification branch focuses on distinguishing specific types of landslides (such as crack type, collapse type, and composite type) and suppressing misjudgments caused by water surface reflection, vegetation shadows, and other interference factors. After dimensionality reduction through a 1x1 convolution layer (the number of channels matches the number of pre-set landslide categories), the input features are output through a Softmax activation function to obtain the probability distribution of each category. The loss function is defined as: , where N is the number of samples, C is the number of landslide categories, is the true label of the i-th sample belonging to the j-th category (0 or 1), is the predicted probability of the corresponding category. This loss function can effectively enhance the model's ability to distinguish between different types of landslides, such as "small-scale cracks" and "large-area collapses." Especially in high-reflectivity areas with similar spectral characteristics to landslides, it reduces the misjudgment of non-landslide areas as landslides.

[0054] The regression branch focuses on accurately predicting the bounding box parameters of the landslide area (center coordinates (x, y) and width and height (w, h), which adapt to the characteristics of irregular boundaries and large terrain undulations on reservoir slopes. After feature mapping through a convolution layer, the normalized bounding box parameters are output through a Sigmoid activation function, and a smooth L1 loss function is used to optimize the positioning accuracy: ; where is the i-th bounding box parameter predicted by the model, is the corresponding true bounding box parameter (based on the landslide range labeled by field survey). This loss function is more robust to bounding box shifts and can effectively handle prediction errors caused by boat movement and broken terrain, especially suitable for positioning the boundaries of small landslides on near-shore shoals and large landslides on steep slopes.

[0055] To further improve the reliability of the model in complex reservoir environments, a positioning loss function is introduced to associate the classification confidence and bounding box positioning accuracy, reducing the "high confidence but large positioning deviation" situation. The total loss formula is: ; ; where is the intersection over union (IOU) of the predicted and true bounding boxes (reflecting the positioning accuracy), is the true bounding box confidence (fixed at 1), , , are hyperparameters for balancing each loss term, = 1.0, = 2.5, = 2.0. Through joint optimization, the model can maintain the accuracy and robustness of landslide detection in dynamic scenarios such as water level fluctuation leading to shoreline changes and seasonal vegetation coverage.

[0056] To achieve monitoring of landslide deformation trends, the detection head integrates a multi-period feature comparison module, which combines historical data to build a closed-loop early warning mechanism.

[0057] The module input includes the current period landslide detection results (including bounding boxes, confidence, and category) and historical period landslide detection results (stored in a cloud database, associated with GPS coordinates and timestamps). During processing, the same slope monitoring area in the current period and the historical period is matched through GPS coordinates and timestamps to ensure spatial and temporal consistency of the comparison.

[0058] The deformation determination and early warning link is achieved by calculating the intersection over union (IoU) of the current period and historical period landslide areas. Let be the current period landslide area, be the historical period landslide area, be the set intersection operator, be the set union operator, then the IoU calculation formula is: , where represents the number of pixels in the overlapping part of the two areas, represents the total number of pixels after merging the two areas. When the IoU change rate exceeds the preset threshold, the system determines that the landslide has deformed and outputs a "yes / no" deformation warning.

[0059] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association between the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B together, and the existence of B alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, and c can represent: a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0060] The above is only a specific implementation of the present application, and any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. The protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A reservoir landslide detection method based on global-local feature perception collaboration, characterized in that, The method comprises the following steps: The water-air cooperative collection of landslide images is adopted to construct a landslide detection data set of the reservoir bank; The four-level hierarchical structure of the landslide feature extraction network is used to extract and fuse the overall terrain trend, crack details and multi-scale edge texture of the landslide, and output high-robustness landslide features; Finally, the detection head uses the landslide feature output boundary box and confidence, and combines the intersection ratio of multiple data to detect the landslide and determine the deformation.

2. The reservoir landslide detection method based on global-local feature perception synergy according to claim 1, characterized in that, The water-air cooperative collection of landslide images and the construction of the landslide detection data set of the reservoir bank comprise the following steps: Deploying a UAV above the reservoir area to take multi-angle photos of the typical landslide area to obtain high-resolution RGB remote sensing images for capturing the overall shape of the large-scale landslide; On the water surface, an unmanned ship is used to carry an optical camera, GPS and IMU to continuously take images along the near-water area of the bank to obtain images for supplementing the small-scale crack details of the near-shore area; All devices are synchronized through timestamps to ensure the spatio-temporal consistency of the image, position and attitude data, and finally form a multi-source heterogeneous data set containing visual, geographical and physical parameters; The collected raw data is cleaned and standardized to eliminate noise interference and unify the format, and the landslide detection data set of the reservoir bank is formed.

3. The reservoir landslide detection method based on global-local feature perception synergy according to claim 2, characterized in that, The cleaning and standardization of the collected raw data to eliminate noise interference and unify the format to form the landslide detection data set of the reservoir bank comprise the following steps: First, geometric correction is performed based on the GPS coordinates and attitude sensor data, and a perspective transformation algorithm is used to correct the spatial deviation of the image to eliminate the image stretching or deviation caused by the inclination and rocking of the ship body, so that each pixel point in the image can accurately correspond to the actual geographical coordinates; Then, a region recognition-differentiation processing two-step strategy is adopted to suppress water surface interference noise and retain key detection features, including: The water surface and the bank slope are segmented by using the optical characteristic difference. For each pixel (x, y) in the image, the minimum value of the RGB three channels in a preset size neighborhood centered on the pixel is taken to obtain the dark channel feature of the pixel : ； wherein, is a color channel, is a neighborhood range, is a channel pixel value; The Otsu algorithm is then used to binarize the dark channel image to automatically determine the optimal threshold T to maximize the variance between the water surface and the bank slope; Adaptive Gaussian filtering is used to smooth the reflection and wave shadow of the water surface area to ensure that the large-area reflection is effectively suppressed in strong light environment and the water surface and bank slope boundary is not blurred due to excessive filtering in weak light environment; The Sobel edge detection is used for the bank slope area to retain crack and edge key features; Finally, data standardization processing is performed to eliminate the interference of data format and scale difference on model training to realize input consistency.

4. The reservoir landslide detection method based on global-local feature perception synergy according to claim 1, characterized in that, The landslide feature extraction network comprises a multi-level backbone network, wherein the first-level backbone network receives the input landslide detection data set of the reservoir bank and outputs to the second-level backbone network; the output of the N-1 level backbone network is input to the N level backbone network; and the output of the last level backbone network is input to the detection head. Each stage of the backbone network comprises a patch embedding and a bank hybrid block, the input is first subjected to a patch embedding to obtain high-dimensional features of non-overlapping image blocks, and then sent to the bank hybrid block; each bank hybrid block is internally integrated with a landslide terrain-fracture collaborative perception LTFS module and a detail perception feature fusion DAFF module in sequence, and a LayerNorm normalization layer is arranged before the LTFS module and the DAFF module; the LTFS module collaboratively perceives the overall terrain trend and fracture details of the landslide, realizes global-local feature complementation, the DAFF module fuses double-branch features, adjusts the features through a dynamic convolution and a feature weighting strategy, strengthens the perception of the landslide boundary contour and detail texture, integrates the advantages of global-local features, and outputs high-discrimination fusion features to improve the landslide feature recognition.

5. The reservoir landslide detection method based on global-local feature perception synergy according to claim 4, characterized in that, The four-stage hierarchical structure of the landslide feature extraction network is constructed to extract and fuse the overall terrain trend, fracture details and multi-scale edge texture of the landslide step by step, and output high-robustness landslide features, including: The input of each stage of the backbone network is first subjected to patch embedding processing to divide the image into a fixed-size image block feature sequence that does not overlap, and map the pixels in the block to a high-dimensional feature vector; Subsequently, the image block feature sequence of the backbone network at different stages is sent to the bank hybrid block for deeper feature processing; The LTFS module and the DAFF module are integrated in the bank hybrid block in sequence, the LTFS module is composed of a local attention branch and a frequency self-attention branch, and the collaborative perception of the overall terrain trend and the local fracture details of the landslide is realized.

6. The reservoir landslide detection method based on global-local feature perception synergy according to claim 5, characterized in that, The image block feature sequence of the backbone network at different stages is subsequently sent to the bank hybrid block for deeper feature processing, including: The first level backbone network receives input with resolution of 16x16 pixel patches, and large-scale bank topography features are extracted by bank hybrid patches. The second level backbone network is based on The resolution is refined to 8×8 pixel blocks, and the mesoscale landslide structure features are extracted by using the slope mixed block. Third level targeting Resolution, 4x4 pixel block partitioning is used for embedding, and small-scale crack texture features are captured by slope mixed blocks. The fourth level is for The resolution is 2×2 pixel block embedding, and the first three levels of multi-scale features are fused with slope and bank hybrid blocks to output a complete feature expression covering large-scale terrain, medium-scale structure and small-scale texture, capturing large-scale terrain and small-scale crack details.

7. The global-local feature perception based collaborative reservoir landslide detection method according to claim 5, characterized in that, In the local attention branch, a lightweight attention convolution operator is introduced to reduce the computational complexity and improve the operation efficiency; input features First, a query vector is generated by a fully connected layer , a key vector and a value vector ; a depthwise separable convolution is applied to the query vector and the key vector respectively; Subsequently applying layer normalization to the convolved features produces an enhanced query vector and key vectors ; Next, the enhanced query vector is calculated and the Hadamard product of the key vector captures the local interaction between the query and the key and enhances the context awareness ability of the network, and the result of the Hadamard product is passed through a fully connected layer, combined with Gaussian error linear unit (GELU) and hyperbolic tangent (Tanh) activation functions, to improve the ability of the model to represent complex remote sensing features. For value vector First, the global information is aggregated using adaptive average pooling, and then the local spatial features are further captured using depthwise separable convolution to generate an enhanced value vector ; Then, the attention weight is calculated using the softmax function , dynamically adjusting the importance of the features, so that the network can effectively focus on the key information in the remote sensing image; These attention weights are then multiplied with a set of learnable parameters Element-wise multiplication, dynamically adjusting the kernel weights, enabling the network to adaptively optimize for different remote sensing input features; learnable parameters Initialized using Xavier strategy to ensure stability of gradient flow at the beginning of training; The model can adaptively refine the convolution weights for different remote sensing image features through back propagation and optimization using Adam; Finally, the enhanced value vector and attention weight Attn are Hadamard product to deeply fuse information, and the final output is generated, and the whole process is represented as follows: ； ； ； ； ； ； ； ； wherein: is a local branch input, FC denotes a fully connected layer, is the number of feature channels, is a Hadamard product.

8. The reservoir landslide detection method based on global-local feature perception synergy according to claim 5, characterized in that, In the global branch, the FSAR module is introduced, which introduces frequency domain feature processing in the self-attention SA mechanism, and the self-attention mechanism only adjusts the weighting parameters through internal information interaction; input features are first processed by a fully connected layer to generate a query vector , a key vector , and a value vector ; The FSAR module extracts low-frequency features from the feature matrix through channel-wise adaptive pooling , the feature matrix extracts high-frequency features using depthwise separable convolution ; Subsequently, the query vector is used to query the feature map The matrix operation is performed with the low-frequency feature LK, and then divided by the square root of the feature map dimension for scaling and normalized by the softmax function. Finally, the normalized result is subjected to matrix multiplication operation with the high-frequency feature HV to obtain the output feature of the FSAR module, and the whole process is represented by the formula: ； ； ； ； ； wherein: represents a low frequency feature, represents a high frequency feature, represents a matrix for each channel, represents a number of channels, represents a range of pooling steps.

9. The global-local feature perception based collaborative reservoir landslide detection method according to claim 4 or 5, characterized in that, The DAFF module strengthens the directional edge response through the edge perception convolution BorderConv, reduces the computational overhead through the depth separable convolution DWConv module, and combines the GELU-Tanh nonlinear activation function and residual cross-branch fusion to improve the recognition degree and consistency of the fracture endpoint and bank collapse boundary fine-grained features and suppress background noise; Input features The feature mapping is first performed by a full connection layer FC layer, and then a non-linear transformation is performed by using a GELU activation function to enhance the detail perception; Subsequently through And The module is further refined to capture the boundary details and spatial structure of the image, and finally outputs as the main branch feature through the fully connected layer ; The DAFF module introduces a parallel auxiliary branch in which input features are processed First, preliminary feature extraction is performed by DWConv, then normalization is performed by LayerNorm, and further refinement is performed by a fully connected layer to form auxiliary features ; final main branch feature and auxiliary feature fusion by residual connection, generate final output feature , the process formula is as follows: ； ； 。 10. The reservoir landslide detection method based on global-local feature perception synergy according to claim 1, characterized in that, The final detection head utilizes the landslide feature output boundary box and confidence to combine the multiple data intersection-over-union ratio to perform landslide detection and deformation determination, including: The double-branch detection head adopts a parallel structure of classification branch and regression branch, analyzes the features output by the landslide feature extraction network, and realizes accurate discrimination of landslide categories and fine positioning of boundaries through multi-dimensional loss function collaborative optimization. The classification branch distinguishes the specific types of landslides and suppresses misjudgments caused by interference factors. After the input features are reduced in dimension by a 1×1 convolution layer, the Softmax activation function outputs the probability distribution of each category. The loss function is defined as: ； Wherein: N is the number of samples, C is the number of landslide categories, is the true label of the i-th sample belonging to the j-th category, is the predicted probability of the corresponding category; the loss function is used to distinguish different types of landslides; The regression branch is used to predict the boundary box parameters of the landslide area. After the features are mapped by a convolution layer, the Sigmoid activation function outputs the normalized boundary box parameters. The smooth L1 loss function is used to optimize the positioning accuracy: ； wherein: is the i-th bounding box parameter predicted by the model, is the corresponding real bounding box parameter, the loss function predicts the bounding box error; The positioning loss function is introduced to associate the classification confidence and boundary box positioning accuracy. The total loss formula is: ； ； wherein, For the Intersection over Union of the predicted box and the ground truth box, reflecting the positioning accuracy, For the ground truth confidence, , , is a hyperparameter for balancing the loss terms.

Images