Pixel-level feature fusion and recognition method of optical image and radar image in coal mine power supply line geological disaster monitoring

Abstract

The present application relates to the technical field of remote sensing image processing, geological disaster identification and deep learning, and discloses a pixel-level feature fusion identification method for optical images and radar images in coal mine power supply line geological disaster monitoring, comprising: S1: acquiring spatially matched optical remote sensing images and multi-temporal SAR remote sensing images to form optical-SAR image pairs; S2: inputting the optical-SAR image pairs into optical texture feature extraction branches and radar deformation feature extraction branches respectively and generating optical global tokens and SAR global tokens; S3: guiding the features extracted by the other branch using the optical global tokens and the SAR global tokens respectively, and adaptively fusing the guided results to generate a fusion global token; S4: reversely injecting the fusion global token into a decoder to output a geological disaster hidden danger identification result; and S5: model training and optimization. The present application improves the identification accuracy and boundary clarity of geological disaster hidden dangers such as goaf collapse and landslides.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of remote sensing image processing, geological disaster identification, and deep learning, and particularly to a pixel-level feature fusion and identification method for monitoring geological disasters in coal mine power supply lines. Background Technology

[0002] Coal mine power supply lines often traverse complex terrains such as goaf areas and mountainous regions. Single optical images are easily affected by lighting and fog, while single SAR images lack detailed surface textures, leading to high false alarm rates and blurred boundaries in hazard identification. Existing technologies mainly suffer from the following drawbacks:

[0003] Insufficient fusion depth: Most methods only perform shallow feature stitching or simple weighting of optical and SAR data, without considering the differences in spatial distribution of the two modal features, resulting in the fused features not being fully complementary.

[0004] Lack of heterogeneous feature collaboration mechanism: Optical features are represented by dense pixel-level data, while SAR deformation features are represented by sparse superpixel-level data. Existing technologies have not solved the problem of dimensional alignment and fusion of the two heterogeneous features.

[0005] Poor pixel-level spatial correspondence: Failure to strictly maintain pixel-level spatial alignment throughout the feature extraction and fusion process leads to blurry outlines and loss of details in the fusion results.

[0006] SAR deformation physical resolution is often confused with mathematical grids: Existing technologies often confuse "mathematical grid alignment" with "physical detection resolution", mistaking the resampled interpolation results as the true deformation refinement information, which violates the underlying principles of InSAR. Summary of the Invention

[0007] To address the problems of insufficient fusion depth of optical and SAR images, lack of heterogeneous feature collaboration mechanisms, and mixed use of physical resolution and mathematical grids in existing technologies, this invention provides a pixel-level recognition method based on dual-branch heterogeneous feature adaptive token fusion. By constructing optical texture feature extraction branches and radar deformation feature extraction branches, it clearly distinguishes between mathematical grid alignment and physical detection resolution. A cross-modal adaptive token fusion module is designed to deeply fuse dense pixel-level features and sparse superpixel-level graph structure features, achieving collaborative optimization of the two heterogeneous features at the pixel level. Finally, it outputs high-precision and high-definition geological hazard identification results, thereby improving the identification accuracy and boundary clarity of geological hazard risks such as mining subsidence and landslides.

[0008] To achieve the above objectives, the following technical solution is adopted:

[0009] This invention provides a pixel-level feature fusion and recognition method for optical and radar images in geological disaster monitoring of coal mine power supply lines, comprising: S1: acquiring spatially matched optical remote sensing images and multi-temporal SAR remote sensing images, and performing geometric registration and spatial resolution unification on the optical and SAR remote sensing images to form spatially one-to-one optical-SAR image pairs; S2: inputting the optical-SAR image pairs into an optical texture feature extraction branch and a radar deformation feature extraction branch respectively; the optical texture feature extraction branch is used to extract pixel-level texture features of the optical remote sensing images and generate an optical global token; the radar deformation feature extraction branch is used to invert temporal deformation information based on the multi-temporal SAR remote sensing images, and extract superpixel-level spatial structure features through superpixel segmentation and graph neural networks, and generate SAR... S3: Using a cross-attention mechanism, the optical global token guides the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch, and the SAR global token guides the pixel-level texture features extracted by the optical texture feature extraction branch. The guidance results are then adaptively fused to generate a fused global token. S4: The fused global token is injected back into the decoder to guide pixel-level feature reconstruction and output the geological hazard identification result. S5: Model training and optimization, including: constructing a sample library, wherein the samples in the sample library include optical-SAR image pairs composed of optical remote sensing images and SAR remote sensing images, and corresponding pixel-level geological hazard labels; and using a weighted combination of Dice loss and cross-entropy loss as the total loss function to train the model end-to-end.

[0010] Furthermore, in step S1, forming a spatially one-to-one optical-SAR image pair specifically includes: using the optical remote sensing image as a reference, employing a multimodal registration method based on mutual information constraints, and combining scale-invariant feature transformation features for auxiliary matching, to perform geometric fine correction on the SAR remote sensing image; unifying the registered optical remote sensing image and SAR remote sensing image to the same spatial grid coordinate system, and after multi-view processing and geocoding, resampling the SAR remote sensing image to the spatial grid coordinate system using the nearest neighbor interpolation method to achieve mathematical grid alignment; and cropping the optical remote sensing image and SAR remote sensing image after unifying spatial resolution into image blocks of fixed size to form a spatially one-to-one optical-SAR image pair.

[0011] Furthermore, in step S2, the optical texture feature extraction branch adopts a hybrid architecture combining convolutional neural networks and Transformers as the backbone network, and a learnable global token aggregation module is set at the end of the encoder. The learnable global token aggregation module is used to average pool all tokens output by the last layer of the backbone network, then map them to a preset dimension through a linear projection layer, and perform weighted fusion with the token vector initialized by learnable parameters to generate the optical global token.

[0012] Furthermore, the feature extraction process of the radar deformation feature extraction branch includes: performing temporal processing on the multi-temporal SAR remote sensing images using a temporal radar interferometry method to output a surface deformation rate map and a deformation time-series curve; performing superpixel segmentation based on the surface deformation rate map, treating each superpixel as a graph node, constructing node feature vectors based on the deformation information, backscattering coefficient, and deformation time-series curve of pixels within the superpixel, and constructing edges according to the spatial adjacency relationship between superpixels to form a graph structure; performing convolution operations on the graph structure using a graph convolutional network to extract superpixel-level spatial structural features; inputting the node feature sequence after graph convolution into a Transformer encoder to establish long-range dependencies between superpixels and generate the SAR global token.

[0013] Furthermore, the method for constructing the node feature vector includes: calculating the mean deformation rate and the mean backscattering coefficient of all pixels within the superpixel, and performing median aggregation on the deformation time-series curves of all pixels within the superpixel, and compressing them through a one-dimensional convolutional neural network to obtain time-series aggregated features; and concatenating the mean deformation rate, the mean backscattering coefficient, and the time-series aggregated features to form the node feature vector.

[0014] Further, step S3 specifically includes: performing cross-attention calculation on the optical global token and the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch to obtain the optically guided SAR global features; performing cross-attention calculation on the SAR global token and the pixel-level texture features extracted by the optical texture feature extraction branch to obtain the SAR-guided optical global features; and performing weighted fusion of the optically guided SAR global features and the SAR-guided optical global features using learnable weight parameters to generate the fused global token.

[0015] Furthermore, in step S4, the fused global token is injected backward into the decoder to guide pixel-level feature reconstruction. Specifically, this includes: transforming the fused global token through a multilayer perceptron, expanding it into a bias tensor with the same spatial size as the current level feature map of the decoder through a pixel-by-pixel copying operation, and superimposing the bias tensor with the decoder feature map pixel by pixel; in each level of the decoder, the feature weights at different spatial locations are dynamically adjusted through a feature recalibration module to strengthen regions semantically consistent with the fused global token and suppress irrelevant background.

[0016] Furthermore, in step S4, the decoder adopts a dual-stream decoder architecture, including: an optical decoder, used to progressively upsample the pixel-level texture features output by the optical texture feature extraction branch and output an optical decoded feature map; and a SAR decoder, used to restore the superpixel-level spatial structure features output by the radar deformation feature extraction branch to a raster feature map.

[0017] Furthermore, the SAR decoder restores the superpixel-level spatial structure features output by the radar deformation feature extraction branch into a raster feature map, specifically including: for each raster pixel, finding the nearest preset number of superpixel nodes, and performing a weighted summation of the features of the superpixel nodes with the inverse of the distance as the weight to generate an initial dense feature map; and smoothing the initial dense feature map through a convolutional layer to output a raster feature map.

[0018] Furthermore, at the end of the decoder, the optical decoding feature map output by the optical decoder and the raster feature map output by the SAR decoder are weighted and summed using learnable pixel-level weights to obtain the final fused feature map, and the hazard identification result is output based on the final fused feature map.

[0019] Compared with the prior art, the present invention achieves the following beneficial effects:

[0020] (1) Dual-branch heterogeneous feature collaboration: construct optical texture branch (pixel-level dense features) and SAR deformation branch (superpixel-level sparse graph structure), realize dense mapping through inverse distance weighted interpolation, and realize cross-modal collaboration through adaptive token fusion.

[0021] (2) Effective fusion of temporal information: The SBAS-InSAR temporal deformation curve is embedded into the superpixel node features by “taking the median point by point + 1D-CNN compression”, which solves the engineering problem of fusion of temporal information and single-temporal phase diagram structure.

[0022] (3) Global guidance for local reconstruction: The fused global token is injected into the decoder through spatial broadcasting and feature recalibration to achieve global correction of pixel-level features and improve the clarity of potential hazard boundaries.

[0023] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of the invention. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0025] Figure 1 This is a flowchart illustrating the pixel-level feature fusion and recognition method of optical images and radar images in geological disaster monitoring of coal mine power supply lines according to an embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of the network structure for dual-branch heterogeneous feature extraction and cross-modal fusion in an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of how the SAR decoder restores superpixel features to a raster feature map in an embodiment of the present invention;

[0028] Figure 4 This is a schematic diagram of landslide data according to an embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0031] This invention constructs a technical system encompassing multi-source data acquisition and spatial alignment, dual-branch heterogeneous feature extraction, cross-modal adaptive token fusion, and pixel-level feature reconstruction and recognition. The core process is as follows: 1. Data Preparation: Acquire spatiotemporally matched optical and SAR remote sensing images, completing geometric registration, radiometric correction, and spatial resolution unification. Clear distinction: Optical images maintain their original high resolution, while SAR deformation data maintains its original physical resolution (approximately 10m × 10m). Spatial alignment is achieved through multi-scale grid matching, without physical information interpolation. 2. Dual-branch Heterogeneous Feature Extraction: Optical texture feature extraction branch: Employing a CNN+Transformer hybrid structure to extract pixel-level texture, spectral, and edge details from optical images. Radar deformation feature extraction branch: Using the SBAS-InSAR method to invert multi-temporal SAR images, obtaining temporal deformation information, and extracting superpixel-level spatial structural features through superpixel map structure construction + GCN + Transformer. 3. Cross-modal adaptive token fusion: The features output from the two branches are mapped to modality-specific global tokens with uniform dimensions. A cross-attention mechanism is used to enable interaction between the tokens and the feature sequences, generating a fused global token. 4. Pixel-level feature reconstruction and recognition: The fused global token is used to guide pixel-level feature reconstruction through spatial broadcasting and feature recalibration, outputting a hazard identification result with the same size as the input.

[0032] Figure 1 This is a flowchart illustrating the pixel-level feature fusion and recognition method of optical and radar images in geological disaster monitoring of coal mine power supply lines according to an embodiment of the present invention. Specifically, see... Figure 1 The method includes:

[0033] S1: Acquire spatially matched optical remote sensing images and multi-temporal SAR remote sensing images, and perform geometric registration and spatial resolution unification on the optical remote sensing images and SAR remote sensing images to form spatially one-to-one optical-SAR image pairs.

[0034] Step S1 is used to achieve multi-source remote sensing data acquisition and spatial alignment. Specifically, it includes the following steps:

[0035] S11, Data Acquisition:

[0036] Collect high-resolution optical remote sensing images (such as Gaofen-2, with a spatial resolution better than 1m) covering the power supply corridor of coal mines.

[0037] Simultaneous acquisition of multi-temporal SAR remote sensing images (such as Sentinel-1 and Gaofen-3) is used to extract surface deformation information. The native resolution of SAR images is approximately 10m × 10m, and the physical detection limit is determined by the satellite imaging system and cannot be improved through post-processing interpolation.

[0038] S12. Spatial Alignment and Mesh Unification:

[0039] Step S12 is used to form spatially one-to-one optical-SAR image pairs, specifically including: using the optical remote sensing image as a reference, employing a multimodal registration method based on mutual information constraints, and combining scale-invariant feature transformation features for auxiliary matching, to perform geometric fine correction on the SAR remote sensing image; unifying the registered optical remote sensing image and SAR remote sensing image to the same spatial grid coordinate system; after multi-view processing and geocoding, the SAR remote sensing image is resampled to the spatial grid coordinate system using the nearest neighbor interpolation method to achieve mathematical grid alignment; cropping the optical remote sensing image and SAR remote sensing image after unifying spatial resolution into image blocks of fixed size to form spatially one-to-one optical-SAR image pairs. The specific implementation process is as follows:

[0040] Using optical images as a reference, a multimodal registration method based on mutual information constraints is adopted, and SIFT features are combined for auxiliary matching to perform geometric fine correction on SAR images;

[0041] The registered optical and SAR images are unified to the same spatial grid coordinate system (grid resolution set to 1m×1m). After multi-look processing and geocoding, the SAR data is resampled to this grid using nearest neighbor interpolation. It should be noted that this resampling operation is only for mathematical grid alignment and does not change the physical detection resolution of the SAR deformation data (which remains approximately 10m), nor does it create actual 1m deformation information.

[0042] Image blocks are cropped to a fixed size (e.g., 512×512 pixels) to form spatially corresponding "optical-SAR" image pairs. The registration error is controlled within one SAR pixel (approximately 10m) to ensure that the spatial correspondence between the optical and SAR images meets the requirements for subsequent fusion.

[0043] S2: Input the optical-SAR image pair into the optical texture feature extraction branch and the radar deformation feature extraction branch respectively; the optical texture feature extraction branch is used to extract the pixel-level texture features of the optical remote sensing image and generate an optical global token; the radar deformation feature extraction branch is used to invert the temporal deformation information based on the multi-temporal SAR remote sensing image, and extract the superpixel-level spatial structure features through superpixel segmentation and graph neural network, and generate a SAR global token;

[0044] Step S2 is used to extract heterogeneous features from the two branches. For example... Figure 2 The diagram shown is a schematic of the network structure for dual-branch heterogeneous feature extraction and cross-modal fusion in an embodiment of the present invention. Step S2 specifically includes the following steps:

[0045] S21, Optical Texture Feature Extraction Branch (OTB) – Pixel-level Dense Feature Extraction:

[0046] The optical texture feature extraction branch adopts a hybrid architecture combining convolutional neural networks and Transformers as the backbone network, and a learnable global token aggregation module is set at the end of the encoder. The learnable global token aggregation module is used to average pool all tokens output from the last layer of the backbone network, then map them to a preset dimension through a linear projection layer, and perform weighted fusion with the token vector initialized with learnable parameters to generate the optical global token. The details are as follows:

[0047] Using the Swing Transformer-Base as the backbone network, a learnable global token aggregation module (256 dimensions) is added to the original network to aggregate global texture and structural information of optical images at the encoder end. Specifically, this module performs average pooling on all tokens output from the last layer of the Swing Transformer, then maps them to 256 dimensions via a linear projection layer, and finally weights and fuses them with the token vector initialized with learnable parameters to generate the final optical global token. ;

[0048] Input an optical image patch, extract multi-scale, globally perceived pixel-level features, and output an optical pixel-level feature map. Where H and W are the feature map space dimensions (related to the input image block size, and generally 1 / 4 or 1 / 8 of the original size after downsampling).

[0049] S22, Radar Deformation Feature Extraction Branch (STB) – Superpixel-level sparse map structure feature extraction:

[0050] The feature extraction process of the radar deformation feature extraction branch is as follows: Temporal processing of the multi-temporal SAR remote sensing images is performed using a temporal radar interferometry method to output a surface deformation rate map and deformation time-series curves; superpixel segmentation is performed based on the surface deformation rate map, with each superpixel treated as a graph node. Node feature vectors are constructed based on the deformation information, backscattering coefficient, and deformation time-series curves of pixels within each superpixel, and edges are constructed according to the spatial adjacency relationships between superpixels to form a graph structure; a graph convolutional network is used to perform convolution operations on the graph structure to extract superpixel-level spatial structural features; the node feature sequence after graph convolution is input into a Transformer encoder to establish long-range dependencies between superpixels and generate the SAR global token. Specifically, the steps include:

[0051] S221. Temporal Deformation Feature Inversion: The SBAS-InSAR method is used to perform temporal processing on multi-temporal SAR images (≥15 scenes). The key engineering parameters are set as follows:

[0052] Time baseline threshold: ≤60 days;

[0053] Spatial baseline threshold: ≤150 meters;

[0054] Number of interference pairs: ≥30 pairs;

[0055] Phase unwrapping: using the minimum cost flow algorithm;

[0056] Timing filtering: Adaptive Gaussian filtering is used, with a filtering window of 5×5;

[0057] Coherence screening: coherence coefficient threshold ≥ 0.3;

[0058] Output: Map of surface deformation rate in the study area (unit: mm / year, native physical resolution approximately 10m×10m) and deformation time series curves (one curve per pixel).

[0059] S222, Superpixel image structure construction:

[0060] The deformation rate map is normalized to the gray range of 0~255 and used as a single-channel image input to the SLIC algorithm. The deformation rate is used as the pixel intensity feature for superpixel segmentation.

[0061] The deformation rate map is subjected to median filtering (3×3 window) to suppress noise, and the filtered image is used as the input for SLIC segmentation.

[0062] The SLIC algorithm is used to divide the filtered deformation rate map into superpixel regions, with the number of superpixels controlled between 200 and 500, and each superpixel serving as a graph node;

[0063] Node feature construction: For each superpixel node, aggregate the information of all pixels within it to form a multimodal feature vector:

[0064] (1) Mean deformation rate (1D): the arithmetic mean of the deformation rates of all pixels within a superpixel;

[0065] (2) Mean backscattering coefficient (1D): the arithmetic mean of the backscattering coefficients of all pixels in the superpixel, with Sentinel-1 VV polarization as the polarization mode;

[0066] (3) Temporal deformation curve aggregation feature (32-dimensional): Take the median of the temporal deformation curve of all pixels in the superpixel point by point to obtain the aggregated temporal curve of the superpixel (the length is the same as the number of time series of the image), and then compress it into a 32-dimensional feature vector by 1D-CNN+global average pooling.

[0067] The node feature vector is obtained by concatenating the three features: the mean deformation rate, the mean backscattering coefficient, and the temporal aggregation feature. (1+1+32=34 dimensions);

[0068] A graph structure is constructed by using superpixel nodes as vertices and establishing edges between adjacent superpixels. Here, G is the graph structure, used to represent the topological relationships between superpixel nodes, and is the basic data structure for subsequent Graph Convolutional Network (GCN) processing; V is the vertex set, where each vertex corresponds to a superpixel node, and the node features are a concatenated vector (34-dimensional) of the mean deformation rate, mean backscattering, and time-series deformation curve compression features of the pixels within the superpixel. E is the edge set, representing the adjacency relationships between superpixel nodes. If two superpixels are spatially adjacent (share a boundary), an edge is established between them.

[0069] S223, GCN Feature Extraction: A 3-layer Graph Convolutional Network (GCN) is used to perform convolution operations on the graph structure to extract superpixel-level spatial structural features. The configuration of each layer is as follows:

[0070] First layer: Input dimension 34, output dimension 64 (mapped via a linear projection layer), activation function GELU;

[0071] Second layer: Input dimension 64, output dimension 128, activation function GELU;

[0072] The third layer has an input dimension of 128, an output dimension of 256, and uses the GELU activation function.

[0073] S224, Transformer Global Encoding: Converts the node feature sequence after graph convolution. (N is the number of superpixel nodes, 200-500) Input a 4-layer Transformer encoder (embedding dimension 256, attention head 8, feedforward network dimension 1024) to establish long-range dependencies between superpixels and generate a learnable SAR global token at the end of the encoder. It is used to aggregate global deformation and structural information of SAR images.

[0074] Output SAR superpixel feature sequence With SAR global token .

[0075] S3: Through the cross-attention mechanism, the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch are guided by the optical global token, and the pixel-level texture features extracted by the optical texture feature extraction branch are guided by the SAR global token. The guidance results are adaptively fused to generate a fused global token.

[0076] Step S3: Implement cross-modal adaptive token fusion through the cross-modal adaptive token fusion module. Step S3 specifically includes: performing cross-attention calculation on the optical global token and the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch to obtain the optically guided SAR global features; performing cross-attention calculation on the SAR global token and the pixel-level texture features extracted by the optical texture feature extraction branch to obtain the SAR-guided optical global features; and weighting and fusing the optically guided SAR global features with learnable weight parameters to generate a fused global token. Specifically, this includes the following steps:

[0077] S31, Cross-attention between global tokens and feature sequences:

[0078] Optical global token With SAR characteristic sequences Cross-attention calculation is performed to guide the local features of SAR with global optical information; the SAR global token is then used. Flattening sequence of optical feature maps Cross-attention calculation is performed to correct local optical features using global SAR information. All tokens and feature dimensions are unified to 256 to ensure the validity of matrix operations.

[0079] S311, optically guided SAR characteristics:

[0080]

[0081] : Optically guided SAR global features, representing the aggregation result of optical global information on SAR superpixel features, with dimensions of Softmax: A normalized exponential function used to map attention weights to... The interval contains elements whose sum is 1. Optical global token, dimension It aggregates global texture and structural information from optical images; SAR superpixel feature sequence, dimension ,in The number of superpixel nodes (200-500); : Scaling factor, value 16, used to alleviate the problem of excessively large dot product values; 256: Feature embedding dimension, the unified dimension of all tokens and feature vectors.

[0082] S312, SAR guiding optical features:

[0083]

[0084] SAR-guided optical global features represent the aggregation result of SAR global information on optical pixel features, with dimensions... ; SAR global token, dimension It aggregates global deformation and structural information from SAR images; The sequence after flattening the optical feature map, dimension ,in The spatial height and width of the optical feature map; : Height of the optical feature map (number of pixels); Width (in pixels) of the optical feature map

[0085] S313 Adaptive Fusion: Through learnable weight parameters and Adaptive weighted fusion of the bidirectional guidance results:

[0086]

[0087] in Adaptive fusion weights, satisfying It is obtained through Softmax normalization, that is These are learnable parameters, all initialized to 0 (corresponding to...) ), together with the model parameters, are optimized through gradient descent. The merged global token, dimension ; Global characteristics of SAR after optical guidance; : Global optical characteristics after SAR guidance; Learnable parameters, all initialized to 0, are mapped to Softmax. ; Exponential functions are used to convert learnable parameters into non-negative weights.

[0088] S4: Inject the fused global token into the decoder in reverse to guide pixel-level feature reconstruction and output the geological hazard identification result.

[0089] Step S4 is used to achieve pixel-level feature reconstruction and hazard identification.

[0090] S41, Global token reverse guidance for pixel-level feature reconstruction:

[0091] Step S41 is used to inversely inject the fused global token into the decoder to guide pixel-level feature reconstruction. Specifically, it includes: at each upsampling level of the decoder, after feature transformation of the fused global token through a multilayer perceptron, expanding it into a bias tensor with the same spatial size as the feature map of that level through a pixel-by-pixel copying operation, and then superimposing the bias tensor onto the feature map of that level of the decoder pixel-by-pixel. In each level of the decoder, the feature weights at different spatial locations are dynamically adjusted through a feature recalibration module to strengthen regions semantically consistent with the fused global token and suppress irrelevant background. Details are as follows:

[0092] The merged global token As "prior knowledge," it is injected into the decoder in reverse:

[0093] (1) Space broadcasting: via MLP ( Perform feature transformation, and then repeat the copying operation to... Expanded to a bias tensor with the same spatial size as the feature map of the current level of the decoder. The broadcast rule is to copy pixel by pixel, copying the 1×256 token vector. The input dimension is 256, and the output dimension is 256, which is used for feature transformation. The bias tensor after spatial broadcasting, dimension ; : The height of the feature map at the current level of the decoder; : Width of the feature map of the current layer in the decoder;

[0094] (2) Pixel-by-pixel overlay: Pixel-by-pixel addition is performed with the decoder feature map to correct local features using global information;

[0095] (3) Feature recalibration: In each layer of the decoder, the feature weights at different spatial locations are dynamically adjusted through a lightweight feature recalibration module (1×1 convolution → Sigmoid). The input is the feature map of the current layer, and the output is a spatial weight map (with the same size as the input). This strengthens regions that are consistent with the semantics of the global token and suppresses irrelevant background. This module is connected after the output of each decoding layer and before upsampling.

[0096] S42, Dual-stream decoder architecture:

[0097] Preferably, the decoder in this embodiment of the invention adopts a dual-stream decoder architecture, including: an optical decoder, used to progressively upsample the pixel-level texture features output by the optical texture feature extraction branch and output an optical decoded feature map; and a SAR decoder, used to restore the superpixel-level spatial structure features output by the radar deformation feature extraction branch to a raster feature map. Specifically:

[0098] S421, Optical Decoder: It adopts a progressive upsampling UNet structure, and the upsampling method is bilinear interpolation. After upsampling at each layer, it is spliced ​​with the features of the corresponding layer of the encoder through skip connections, and then fused by 3×3 convolution.

[0099] S422, SAR Decoder: Restores the SAR superpixel feature sequence to a raster feature map. The SAR decoder restores the superpixel-level spatial structure features output by the radar deformation feature extraction branch to a raster feature map, specifically including: for each raster pixel, finding the nearest preset number of superpixel nodes, and performing a weighted summation of the features of the superpixel nodes using the inverse of the distance as the weight to generate an initial dense feature map; smoothing the initial dense feature map through a convolutional layer to output a raster feature map. For example... Figure 3 The diagram shown illustrates how the SAR decoder recovers superpixel features into a raster feature map in an embodiment of the present invention. The specific method is as follows:

[0100] S4221. Inverse distance weighted interpolation to generate the initial dense map: For each raster pixel, find the k nearest superpixel nodes (preferably, k=3, which can be adjusted between 2 and 5 depending on the actual application scenario), and perform weighted summation of the node features with the inverse distance as the weight (weight = 1 / d, where d is the Euclidean distance) to generate the initial dense feature map. This method can effectively eliminate superpixel boundary discontinuities.

[0101] S4222, Feature Smoothing: Through 3 layers of convolution ( The initial dense feature map is smoothed with a step size of 1 and padding of 1. After each layer, batch normalization and ReLU activation are performed to output the final SAR dense feature map (i.e., the raster feature map). .

[0102] S423, Pixel-wise weighted fusion: At the end of the decoder, through learnable weights... (After Softmax normalization) The final fused feature map is obtained by pixel-level weighted summation of the optical decoder feature map (optical features) output by the optical decoder and the raster feature map (SAR dense feature map) output by the SAR decoder:

[0103]

[0104] The final fused feature map, dimension This is the result of pixel-level weighted fusion of optical and SAR features; Learnable pixel-level weight tensors, dimensions After Softmax normalization, along the channel dimension (2 channels), it satisfies Softmax: Normalized exponential function, applied along the channel dimension. Normalize the data so that the sum of the weights at each pixel position is 1. Optical feature fusion weights, dimensions , representing the contribution of optical features at each pixel location to the fusion; SAR feature fusion weights, dimensions , representing the contribution of SAR features at each pixel location to the fusion; Element-wise multiplication (Hadamard product) multiplies the weight tensor with the corresponding position in the feature map; The optical decoded feature map output by the optical decoder, in terms of dimensions. It is obtained after optical branch processing in the dual-stream decoder; : The raster feature map output by the SAR decoder, dimension It is obtained after processing by the SAR branch in the dual-stream decoder (which includes inverse range weighted interpolation and convolutional smoothing).

[0105] S43, Hazard Output:

[0106] The decoder outputs a hazard probability map (binary classification: hazard / background) with the same size as the input through a 1×1 convolutional layer, achieving pixel-level hazard identification.

[0107] The embodiments of the present invention also include step S5: model training and optimization, including: constructing a sample library, wherein the samples in the sample library include optical-SAR image pairs composed of optical remote sensing images and SAR remote sensing images, and corresponding pixel-level geological disaster hazard labels; and using a weighted combination of Dice loss and cross-entropy loss as the total loss function to train the model end-to-end.

[0108] Step S5: Model Training and Optimization

[0109] S51, Hardware Configuration and Training Parameters:

[0110] Hardware: GPU (NVIDIA RTX 4090, 24GB VRAM) ×1, CPU (Intel i9-13900K), 64GB RAM;

[0111] Batch Size: 4 (for 24GB video memory);

[0112] Training rounds: 100 rounds;

[0113] Optimizer: AdamW, initial learning rate 1e-4, cosine annealing decay;

[0114] Early stopping mechanism: If the validation set loss does not decrease for 10 consecutive rounds, the process will stop.

[0115] S52. Sample library construction:

[0116] Sample collection area: a 110kV power supply line corridor (25km long) in a coal mining area, collecting Gaofen-2 optical images (0.8m) and Sentinel-1 SAR time-series images (2019-2023, 45 scenes in total);

[0117] Geological hazard accuracy data sourced from historical geological hazard records, on-site inspection records, and detailed UAV surveys. Pixel-level polygon annotations were performed by remote sensing geologists using ArcGIS / QGIS software. The annotation protocol conforms to the industry standard "Technical Specification for Comprehensive Remote Sensing Monitoring of Geological Hazards." Hazard types include mining subsidence areas (12 locations) and landslides (8 locations). A schematic diagram of the landslide data is shown below. Figure 4 As shown;

[0118] Total sample size: The original image blocks (512×512 pixels) consisted of 412 positive samples and 1240 negative samples. After geometric enhancement (random rotation and flipping) and radiometric enhancement (brightness and contrast fine-tuning), the number of positive samples increased to 2472 and the number of negative samples to 7440. These were then divided into training, validation, and test sets in a ratio of 7:2:1.

[0119] S53. Loss Function and Evaluation Metrics:

[0120] Loss function: a weighted combination of Dice loss and cross-entropy loss (both with a weight of 0.5).

[0121] Evaluation metrics: mean inter-integration ratio (mIoU), marginal F1 score, false alarm rate.

[0122] Experimental results:

[0123] Comparative Experiment: Compared with existing technologies, which are dual-stream CNN + feature concatenation models (each stream uses ResNet18 as the encoder, and the feature maps are concatenated and then processed by two convolutional layers for classification), the training, validation, and test sets were used exactly the same as those used in this invention. The experimental results are shown in Table 1.

[0124] Table 1

[0125]

[0126] The aforementioned mIoU, edge F1 score, and false alarm rate metrics are calculated based on an independent test set (30% of the samples are randomly selected from 7440 negative samples and merged with all positive sample test sets). The test samples do not overlap with the training samples, ensuring the authenticity and reliability of the evaluation results. While maintaining the physical resolution of SAR deformation, this invention extracts spatial structure features through superpixel map structure and achieves cross-modal collaboration through adaptive token fusion. The mIoU is improved by approximately 5.3 percentage points, the edge F1 score is improved by approximately 0.11, and the false alarm rate is reduced by approximately 7.6 percentage points compared to existing technologies. The above improvements are based on real physical resolution SAR deformation data and do not use interpolation to fabricate information. The performance gains have a physical and geological basis.

[0127] It should also be noted that, in the embodiments of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0128] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in the embodiments of this application may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown in this application, but is to be accorded the widest scope consistent with the principles and novel features disclosed in the embodiments of this application.

Claims

1. A pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines, characterized in that... include: S1: Acquire spatially matched optical remote sensing images and multi-temporal SAR remote sensing images, and perform geometric registration and spatial resolution unification on the optical remote sensing images and SAR remote sensing images to form spatially one-to-one optical-SAR image pairs. S2: Input the optical-SAR image pairs into the optical texture feature extraction branch and the radar deformation feature extraction branch respectively; The optical texture feature extraction branch is used to extract pixel-level texture features of the optical remote sensing image and generate an optical global token; the radar deformation feature extraction branch is used to invert temporal deformation information based on the multi-temporal SAR remote sensing image, and extract superpixel-level spatial structure features through superpixel segmentation and graph neural network, and generate a SAR global token. S3: Through the cross-attention mechanism, the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch are guided by the optical global token, and the pixel-level texture features extracted by the optical texture feature extraction branch are guided by the SAR global token. The guidance results are adaptively fused to generate a fused global token. S4: Inject the fused global token into the decoder in reverse to guide pixel-level feature reconstruction and output the geological hazard identification result; S5: Model training and optimization, including: constructing a sample library, wherein the samples in the sample library include optical-SAR image pairs consisting of optical remote sensing images and SAR remote sensing images, and corresponding pixel-level geological hazard labels; and using a weighted combination of Dice loss and cross-entropy loss as the total loss function to train the model end-to-end.

2. The pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines according to claim 1, characterized in that, In step S1, forming spatially corresponding optical-SAR image pairs specifically includes: Using the optical remote sensing image as a reference, a multimodal registration method based on mutual information constraints is adopted, and scale-invariant feature transformation features are combined for auxiliary matching to perform geometric fine correction on the SAR remote sensing image. The registered optical remote sensing images and SAR remote sensing images are unified into the same spatial grid coordinate system. After multi-view processing and geocoding, the SAR remote sensing images are resampled to the spatial grid coordinate system using the nearest neighbor interpolation method to achieve mathematical grid alignment. The optical remote sensing images and SAR remote sensing images with unified spatial resolution are cropped into image blocks of fixed size to form spatially corresponding optical-SAR image pairs.

3. The pixel-level feature fusion and recognition method for optical images and radar images in geological disaster monitoring of coal mine power supply lines according to claim 2, characterized in that, In step S2, the optical texture feature extraction branch adopts a hybrid architecture combining convolutional neural networks and Transformers as the backbone network, and a learnable global token aggregation module is set at the end of the encoder. The learnable global token aggregation module is used to average pool all tokens output by the last layer of the backbone network, then map them to a preset dimension through a linear projection layer, and perform weighted fusion with the token vector initialized by learnable parameters to generate the optical global token.

4. The pixel-level feature fusion and recognition method for optical images and radar images in geological disaster monitoring of coal mine power supply lines according to claim 1 or 3, characterized in that, The feature extraction process of the radar deformation feature extraction branch includes: The multi-temporal SAR remote sensing images are processed using a time-series radar interferometry method to output a surface deformation rate map and a deformation time-series curve. Superpixel segmentation is performed based on the surface deformation rate map. Each superpixel is treated as a graph node. Node feature vectors are constructed based on the deformation information, backscattering coefficient, and deformation time-series curve of pixels within the superpixel. Edges are constructed according to the spatial adjacency relationship between superpixels to form a graph structure. A graph convolutional network is used to perform convolution operations on the graph structure to extract superpixel-level spatial structural features. The node feature sequence after graph convolution is input into the Transformer encoder to establish long-range dependencies between superpixels and generate the SAR global token.

5. The pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines according to claim 4, characterized in that, The methods for constructing the node feature vectors include: The mean deformation rate and mean backscattering coefficient of all pixels in a superpixel are calculated, and the median aggregation of the deformation time-series curves of all pixels in a superpixel is performed. The time-series aggregated features are then compressed by a one-dimensional convolutional neural network. The mean deformation rate, the mean backscattering coefficient, and the temporal aggregation feature are concatenated to form the node feature vector.

6. The pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines according to claim 1, characterized in that, Step S3 specifically includes: The optical global token and the superpixel-level spatial structure features extracted by the radar deformation feature extraction branch are cross-attention calculated to obtain the optically guided SAR global features. The SAR global token and the pixel-level texture features extracted by the optical texture feature extraction branch are cross-attention calculated to obtain the SAR-guided optical global features; The fused global token is generated by weighting and fusing the optically guided SAR global features with the optically guided SAR global features using learnable weight parameters.

7. The pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines according to claim 1, characterized in that, In step S4, the fused global token is injected backward into the decoder to guide pixel-level feature reconstruction, specifically including: After the fused global token is transformed by a multilayer perceptron, it is expanded into a bias tensor with the same spatial size as the current layer feature map of the decoder through a pixel-by-pixel copying operation, and the bias tensor is superimposed on the decoder feature map pixel by pixel. In each layer of the decoder, the feature weights at different spatial locations are dynamically adjusted through the feature recalibration module to strengthen regions that are consistent with the semantics of the fused global token and suppress irrelevant background.

8. The pixel-level feature fusion and recognition method for optical images and radar images in geological disaster monitoring of coal mine power supply lines according to claim 1 or 7, characterized in that, In step S4, the decoder adopts a dual-stream decoder architecture, including: An optical decoder is used to progressively upsample the pixel-level texture features output from the optical texture feature extraction branch and output an optical decoded feature map. The SAR decoder is used to restore the superpixel-level spatial structure features output by the radar deformation feature extraction branch into a raster feature map.

9. The pixel-level feature fusion and recognition method for monitoring geological disasters in coal mine power supply lines according to claim 8, characterized in that, The SAR decoder restores the superpixel-level spatial structure features output by the radar deformation feature extraction branch into a raster feature map, specifically including: For each grid pixel, find the nearest preset number of superpixel nodes, and sum the features of the superpixel nodes with the reciprocal of the distance as the weight to generate an initial dense feature map; The initial dense feature map is smoothed by a convolutional layer to output a raster feature map.

10. The pixel-level feature fusion and recognition method for optical images and radar images in geological disaster monitoring of coal mine power supply lines according to claim 8, characterized in that, At the end of the decoder, the optical decoding feature map output by the optical decoder and the raster feature map output by the SAR decoder are weighted and summed using learnable pixel-level weights to obtain the final fused feature map, and the hazard identification result is output based on the final fused feature map.

Images