A mine remote sensing scene classification method and system and a storage medium

Abstract

The application provides a mine remote sensing scene classification method and system and a storage medium, relates to the technical field of remote sensing image processing, and comprises the following steps: acquiring a remote sensing image of a mine target area; inputting the remote sensing image into a double-flow feature representation network; enhancing global features and local features in the remote sensing image respectively according to the double-flow feature representation network, and obtaining a global feature map and a local feature map of the mine target area respectively; obtaining a fusion feature map according to the global feature map and the local feature map; and obtaining a classification label of the mine target area according to the fusion feature map. The double-flow feature representation network integrates global and local information of the image, enhances the distinguishing ability of the features, and significantly improves the accuracy and efficiency of mine remote sensing scene classification.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and more specifically, to a method, system, and storage medium for classifying remote sensing scenes in mines. Background Technology

[0002] Mining activities are often accompanied by environmental changes, ecosystem damage, and safety hazards. Therefore, monitoring land cover types in mining areas is crucial for ecological and environmental development. Mines and surrounding forests, farmland, and other land cover areas constitute a complex geological environment, typically requiring remote sensing technology to obtain large-scale surface information.

[0003] In related technologies, the surface features of mines are usually irregular in shape and vary greatly in scale, resulting in complex and variable surface features in mining areas. In addition, remote sensing data itself has a lot of noise and complexity, which leads to errors in the identification of land cover types in remote sensing scenes of mines, thus affecting the classification accuracy of remote sensing scenes of mines. Summary of the Invention

[0004] The problem addressed by this invention is how to improve the classification effect of remote sensing scenes in mines.

[0005] To address the aforementioned problems, this invention provides a method, system, and storage medium for classifying remote sensing scenes in mines.

[0006] In a first aspect, the present invention provides a method for classifying remote sensing scenes in mines, including:

[0007] Acquire remote sensing images of the target area of ​​the mine;

[0008] The remote sensing image is input into a dual-stream feature representation network. Based on the dual-stream feature representation network, the global and local features in the remote sensing image are enhanced respectively to obtain the global feature map and local feature map of the target area of ​​the mine.

[0009] A fused feature map is obtained based on the global feature map and the local feature map;

[0010] Based on the fused feature map, the classification label of the target area of ​​the mine is obtained.

[0011] Optionally, the dual-stream feature representation network includes two branches, namely a SwinTransformer branch and a ConvNeXt branch; the step of enhancing the global and local features in the remote sensing image according to the dual-stream feature representation network to obtain the global feature map and local feature map of the target area of ​​the mine includes:

[0012] The global feature map is obtained by aggregating global information from the remote sensing image based on the Swin Transformer branch.

[0013] The local feature map is obtained by performing local information enhancement on the remote sensing image based on the ConvNeXt branch.

[0014] Optionally, the step of aggregating global information from the remote sensing image based on the Swing Transformer branch to obtain the global feature map includes:

[0015] The remote sensing image is segmented into multiple image patches based on the embedding layer in the Swing Transformer branch;

[0016] The image blocks are transformed to obtain the embedded representation of each image block;

[0017] The embedded representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-range dependencies of the remote sensing image are captured through the window mechanism to obtain the output feature map;

[0018] The output feature maps are merged to obtain the global feature map.

[0019] Optionally, the Swin Transformer branch has multiple sequentially connected SwinTBlock layers;

[0020] The process involves sequentially inputting the embedded representation into the SwinTBlock layer of the Swin Transformer branch, capturing the long-range dependencies of the remote sensing image through a windowing mechanism, and obtaining an output feature map, including:

[0021] The embedding representation is input into the first SwinTBlock layer, and features are extracted by moving a window of a preset size within the embedding representation of the image patch to obtain the feature map output by the first SwinTBlock layer. The feature map output by each SwinTBlock layer is used as the input to the next SwinTBlock layer.

[0022] After the last SwinTBlock layer outputs the feature map, the feature map output by the last SwinTBlock layer is used as the output feature map.

[0023] Optionally, the Swin Transformer branch may further include multiple global channel attention modules;

[0024] The step of merging the output feature maps to obtain the global feature map includes:

[0025] The embedded representation is input into the first global channel attention module. The output features of the global channel attention module are merged with the feature map output by the first SwinTBlock layer. The merged feature map is then input into the second global channel attention module. Starting from the second global channel attention module, the features output by each global channel attention module are sequentially merged with the feature map output by a SwinTBlock layer and then input into the next global channel attention module.

[0026] The feature map output by the last SwinTBlock layer is merged with the feature map output by the last global channel attention module to obtain the global feature map.

[0027] Optionally, it also includes:

[0028] After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector.

[0029] The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

[0030] Optionally, the ConvNeXt branch includes multiple sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module;

[0031] The step of performing local information enhancement on the remote sensing image based on the ConvNeXt branch to obtain the local feature map includes:

[0032] The remote sensing image is input into the first feature extraction block of the ConvNeXt branch for local feature extraction, and the feature map output by the feature extraction block is used as the input for the next feature extraction block until the last feature extraction block.

[0033] Simultaneously, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block for local feature enhancement, thereby obtaining the enhanced local feature map output by each local feature enhancement module;

[0034] The enhanced local feature maps are concatenated by stitching together the output order of each enhanced local feature map to obtain the local feature map.

[0035] Optionally, obtaining the fused feature map based on the global feature map and the local feature map includes:

[0036] The global feature map and the local feature map are input into the dual-branch deep feature fusion module for global pooling, to obtain pooled feature maps corresponding to the global feature map and the local feature map, respectively.

[0037] The pooled feature maps corresponding to the global feature map and the local feature map are weighted and fused to obtain a weighted feature map.

[0038] The pooled feature maps corresponding to the global feature map and the local feature map, as well as the weighted feature map, are respectively input into the corresponding depthwise separable convolutional network, and then merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

[0039] Secondly, the present invention provides a mine remote sensing scene classification system, comprising:

[0040] The remote sensing image acquisition unit is used to acquire remote sensing images of the target area of ​​the mine;

[0041] The feature processing unit is used to input the remote sensing image into a dual-stream feature representation network, and enhance the global and local features in the remote sensing image according to the dual-stream feature representation network to obtain the global feature map and local feature map of the target area of ​​the mine respectively.

[0042] A fusion unit is used to obtain a fused feature map based on the global feature map and the local feature map;

[0043] A classification unit is used to obtain a classification label for the target area of ​​the mine based on the fused feature map.

[0044] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the above-described mine remote sensing scene classification method.

[0045] The present invention provides a method, system, and storage medium for classifying remote sensing scenes in mines. By acquiring remote sensing images of the target area of ​​a mine, it provides raw data for the classification task. Next, the remote sensing images are input into a dual-stream feature representation network (DMN). The DMN effectively captures long-range dependencies in the images, thus obtaining a global feature map. Simultaneously, it focuses on extracting local details of the images, obtaining local feature maps. By utilizing the DMN, which considers both the global structure and local details of the image, it provides richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global and local feature maps, a fused feature map is obtained, further integrating the global and local information of the image. Fusing local and global features enhances the discriminative power of the features. In mine remote sensing scenes, different land cover types may overlap in local features, but global features can provide additional discriminative information, helping the model better distinguish similar categories. Finally, the fused feature map is mapped to specific classification labels, achieving accurate classification of the target area of ​​the mine and significantly improving the accuracy and efficiency of mine remote sensing scene classification. Attached Figure Description

[0046] Figure 1 This is a flowchart of a mine remote sensing scene classification method according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the branch structure of a two-stream feature representation network in one embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of a two-stream feature representation network in one embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of the Swin Transformer branch structure in one embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of the collaborative dictionary learning module in one embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the structure of a local feature enhancement module in one embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of the structure of a dual-branch deep feature fusion module in one embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram of the structure of a mine remote sensing scene classification system in one embodiment of the present invention. Detailed Implementation

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0055] Combination Figure 1 As shown, the present invention provides a method for classifying remote sensing scenes in mines, including:

[0056] Acquire remote sensing images of the target area of ​​the mine.

[0057] Specifically, remote sensing images of the target mining area are acquired, providing raw data for the classification task. Remote sensing images can capture surface features of the mining area, such as vegetation cover, topographic relief, and ore distribution, which are crucial for subsequent classification. Through remote sensing technology, detailed visual information of the mining area can be obtained, providing a rich data foundation for the classification algorithm. In a preferred embodiment of the invention, multi-source data is fused based on different types of band information, including optical, synthetic aperture radar, digital elevation models, and topographic images, to obtain remote sensing images of the target mining area.

[0058] The remote sensing image is input into a dual-stream feature representation network. Based on the dual-stream feature representation network, the global and local features in the remote sensing image are enhanced respectively, resulting in the global feature map and local feature map of the target area of ​​the mine.

[0059] Specifically, remote sensing images are input into a two-stream feature representation network (DVR). The DVR effectively captures long-range dependencies in the image, thus obtaining a global feature map. Simultaneously, it focuses on extracting local details of the image, obtaining local feature maps. In a preferred embodiment of the invention, the DVR has two branches: a Swing Transformer branch focusing on global features and a ConvNeXt branch focusing on local features. These two branches enhance the global and local features of the image, respectively, enabling the network to simultaneously capture both the macroscopic structure and microscopic details of the image. In the DVR, the global feature map is obtained through the Swing Transformer branch, which utilizes a windowing mechanism and hierarchical processing to capture long-range dependencies in the image, thereby understanding the global structure of the image. The local feature map is obtained through the ConvNeXt branch, which extracts local details of the image through multi-layer convolutional operations, providing more comprehensive feature information for classification tasks.

[0060] A fused feature map is obtained based on the global feature map and the local feature map.

[0061] Specifically, based on the global and local feature maps, the network obtains a fused feature map through specific fusion strategies, such as weighted fusion or feature map concatenation. This fused feature map integrates global and local features, enhancing the discriminative power of the features and enabling the model to more accurately identify and distinguish different land cover types, ensuring that the model fully utilizes all relevant information from the image during the decision-making process.

[0062] Based on the fused feature map, the classification label of the target area of ​​the mine is obtained.

[0063] Specifically, classification labels for the target areas of the mine are obtained based on the fused feature maps. The most suitable classification label is then assigned to each area using the fused feature maps. This step is the final goal of the classification task, transforming the features extracted and fused in the previous steps into specific classification results, thus achieving accurate classification of the remote sensing scene of the mine.

[0064] The mine remote sensing scene classification method of this invention obtains remote sensing images of the target area of ​​the mine, providing raw data for the classification task. Next, the remote sensing images are input into a dual-stream feature representation network (DVR). The DVR effectively captures long-range dependencies in the image, thus obtaining a global feature map. Simultaneously, it focuses on extracting local details of the image, obtaining a local feature map. The DVR considers both the global structure and local details of the image, providing richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global and local feature maps, a fused feature map is obtained, further integrating the global and local information of the image. Fusing local and global features enhances the discriminative power of the features. In mine remote sensing scenes, different land cover types may overlap in local features, but global features can provide additional discriminative information, helping the model better distinguish similar categories. Finally, the fused feature map is mapped to specific classification labels, achieving accurate classification of the target area of ​​the mine and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0065] Optionally, the dual-stream feature representation network includes two branches, namely a SwinTransformer branch and a ConvNeXt branch; the step of enhancing the global and local features in the remote sensing image according to the dual-stream feature representation network to obtain the global feature map and local feature map of the target area of ​​the mine includes:

[0066] The global feature map is obtained by aggregating global information from the remote sensing image based on the Swin Transformer branch.

[0067] The local feature map is obtained by performing local information enhancement on the remote sensing image based on the ConvNeXt branch.

[0068] Specifically, in combination Figure 2 As shown, in the mine remote sensing scene classification method of this invention, the dual-stream feature representation network includes two branches: the Swing Transformer branch and the ConvNeXt branch. The Swing Transformer branch is specifically used for global information aggregation of remote sensing images. Through windowing mechanisms and hierarchical processing, it effectively captures long-range dependencies in the image, thereby obtaining a global feature map. This global feature map contains the overall layout and structural information of the mine target area, such as the distribution of mines and the undulation of terrain. On the other hand, the ConvNeXt branch focuses on local information enhancement of remote sensing images. Through multi-layer convolutional operations, it extracts local details of the image, such as the texture of vegetation and the surface features of ore, to obtain a local feature map. This design enables the network to consider both the global structure and local details of the image simultaneously, providing more comprehensive feature information for the classification task.

[0069] In this embodiment of the invention, through the collaborative work of a dual-stream feature representation network, the model can more accurately identify and distinguish different land cover types in mine remote sensing images. The global feature map provides a macroscopic view of the image, helping to understand the overall structure of the scene, while the local feature map provides a microscopic view, helping to capture detailed features. This combination of global and local features enables the model to make more accurate classification decisions when faced with complex and ever-changing mine remote sensing scenes.

[0070] Optionally, the step of performing global information aggregation on the remote sensing image based on the Swing Transformer branch to obtain the global feature map includes:

[0071] The remote sensing image is segmented into multiple image patches based on the embedding layer in the Swing Transformer branch;

[0072] The image blocks are transformed to obtain the embedded representation of each image block;

[0073] The embedded representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-range dependencies of the remote sensing image are captured through the window mechanism to obtain the output feature map;

[0074] The output feature maps are merged to obtain the global feature map.

[0075] Specifically, the remote sensing image is segmented into multiple image patches through the embedding layer in the Swing Transformer branch, where each patch is a local region of the image. Next, each image patch is transformed to obtain its embedding representation, which is a high-level feature of the image patch and captures key information within it. These embedding representations are then sequentially input into the SwingTBlock layer in the Swing Transformer branch. Here, a window mechanism is used to process the image patches, and a self-attention mechanism is employed to capture dependencies within the image patches. Simultaneously, by moving the window and hierarchical processing, the receptive field is gradually expanded, thereby capturing long-range dependencies in the image. Finally, by merging the output features, a global feature map is obtained, which contains global information about the entire remote sensing image. Figure 3 As shown, in a preferred embodiment of the present invention, the remote sensing image is segmented into fixed-size image patches by the Patch Embedding layer, and these image patches are converted into embedding vectors to prepare for subsequent SwinTBlock layer processing.

[0076] In this embodiment of the invention, through global information aggregation via the Swing Transformer branch, the model can obtain a comprehensive and rich global feature map, containing the global structure and long-range dependencies of the image. This is because features such as terrain, landforms, and vegetation distribution in mining areas often exhibit clear patterns and regularities at the global level. Obtaining the global feature map allows the model to better understand these patterns and regularities, thereby making more accurate judgments during classification.

[0077] Optionally, the Swin Transformer branch has multiple sequentially connected SwinTBlock layers;

[0078] The process involves sequentially inputting the embedded representation into the SwinTBlock layer of the Swin Transformer branch, capturing the long-range dependencies of the remote sensing image through a windowing mechanism, and obtaining an output feature map, including:

[0079] The embedding representation is input into the first SwinTBlock layer, and features are extracted by moving a window of a preset size within the embedding representation of the image patch to obtain the feature map output by the first SwinTBlock layer. The feature map output by each SwinTBlock layer is used as the input to the next SwinTBlock layer.

[0080] After the last SwinTBlock layer outputs the feature map, the feature map output by the last SwinTBlock layer is used as the output feature map.

[0081] Specifically, the Swin Transformer branch design employs multiple sequentially connected SwinTBlock layers, forming a deep feature extraction pipeline. Image patches with embedded representations are input to the first SwinTBlock layer, which moves through the embedded representations of the image patches via a pre-sized window, performing a self-attention mechanism to extract local features and capture dependencies within local regions. The output feature map of each SwinTBlock layer then serves as the input to the next SwinTBlock layer, and so on, with each layer further extracting and processing features based on the previous one. This stacked structure allows the model to progressively integrate and process information, enabling each subsequent layer to understand image content at a higher level. After the last SwinTBlock layer outputs its feature map, this map is used as the output feature map, containing global feature information from low to high levels. Figure 3 As shown, in a preferred embodiment of the invention, the SwinTransformer branch includes three SwinTBlock layers. The embedding vector is input to the first SwinTBlock layer. This layer processes the embedding representation of image patches through a self-attention mechanism and a feedforward neural network, capturing local features and performing feature extraction. The output feature map of this layer is passed to the next SwinTBlock layer. The second SwinTBlock layer continues to process the feature map processed by the previous SwinTBlock layer, further extracting and processing features. This layer also includes a self-attention mechanism and a feedforward neural network to enhance the expressive power of the features. The third SwinTBlock layer performs the final feature extraction on the feature map processed by the second SwinTBlock layer. The output feature map of this layer contains global feature information from low to high levels.

[0082] In this embodiment of the invention, through the sequential connection and processing of multiple SwinTBlock layers, the model can achieve layer-by-layer abstraction and deepening of remote sensing image features. The layer-by-layer feature extraction method enables the model to more effectively capture long-distance dependencies from local to global perspectives, thereby containing rich hierarchical information in the global feature map.

[0083] Optionally, the Swin Transformer branch may further include multiple global channel attention modules;

[0084] The step of merging the output feature maps to obtain the global feature map includes:

[0085] The embedded representation is input into the first global channel attention module, the output feature of the global channel attention module is merged with the feature map output by the first SwinTBlock layer, and the merged feature map is input into the second global channel attention module. Starting from the second global channel attention module, the output feature of each global channel attention module is merged with the feature map output by a SwinTBlock layer and then input into the next global channel attention module.

[0086] The feature map output by the last SwinTBlock layer is merged with the feature map output by the last global channel attention module to obtain the global feature map.

[0087] Specifically, in combination Figure 3 As shown, the output of each SwinTBlock layer in the Swin Transformer branch is connected to a Global Channel Attention Module (GCAM). This design aims to further refine and enhance the global information of the feature maps. After extracting local features, the feature map output by each SwinTBlock layer is input into the corresponding GCAM. The GCAM aggregates the global information of the feature maps through global average pooling and global max pooling, and then learns channel attention through a multilayer perceptron (MLP) to obtain weighted channels. These weighted channels are used to guide the network's learning process and enhance the key information in the feature maps. Following the output order of the feature maps, the feature maps output by each SwinTBlock layer are processed by GCAM and then merged. Finally, the feature map output by the last SwinTBlock layer is merged with the merged feature map output by the GCAM corresponding to the previous SwinTBlock layer to obtain the final global feature map.

[0088] In a preferred embodiment of the present invention, combined with Figure 4As shown, firstly, the input feature map undergoes Global Max Pooling (GMP) and Global Average Pooling (GAP) to aggregate the global statistical properties of each channel. Next, these pooled features undergo a non-linear transformation using a Multilayer Perceptron (MLP) to learn the complex relationships between channels. Then, the MLP output is passed through a Sigmoid activation function to generate attention weights for each channel, reflecting the importance of each channel. By multiplying these weights with the original feature map, the feature map is weighted, highlighting key features and suppressing unimportant information. Finally, the feature maps processed by GCAM are merged to form an output feature map containing rich global information, providing a more accurate and robust feature representation for subsequent classification tasks. For a feature map F of size H×W×C, the global channel attention module first performs global average pooling and global max pooling in parallel to aggregate the global information of the feature map, resulting in two 1×1×C channel maps. Then, a Multilayer Perceptron (MLP) with hidden layers is added separately to obtain the weighted channels of the features, used to learn the final channel attention feature map. To reduce parameter overhead, the size of the hidden layer is set to C / r, where r is a reduction factor. To effectively combine these two distinct features, their respective results are numerically summed. Finally, the effective features are improved by multiplying the input feature map by the channel weights. The weight coefficients are obtained using the Sigmoid function, a commonly used non-linear activation function in neural networks. The mathematical expression for the non-linear activation function is:

[0089] ;

[0090] Among them, is x The input to the neuron. The output of the Sigmoid function ranges between (0, 1), which means it can map any real value to a probability value between 0 and 1.

[0091] In this embodiment of the invention, the global representation of features is optimized through a global channel attention module, enabling the network to capture and utilize global information more effectively. Specifically, through the layer-by-layer application of the global channel attention module and the merging of feature maps, the model can gradually construct a comprehensive and rich global feature map. This feature map not only contains the local details of the image, but more importantly, it also contains the global structure and long-range dependencies of the image.

[0092] Optionally, it also includes:

[0093] After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector.

[0094] The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

[0095] Specifically, the feature map output from the last SwinTBlock layer of the Swin Transformer branch, after global pooling, is fed into the Collaborative Dictionary Learning Module (CDLM) to reconstruct the feature vector. The global pooling steps, including Global Average Pooling (GAP) and Global Max Pooling (GMP), aim to reduce the spatial dimensionality of the feature map while preserving the most important global information. These pooled feature maps are then used to reconstruct the feature vector, which is achieved through CDLM, combined with... Figure 5 As shown, CDLM aims to learn a dictionary D that can collaboratively represent each sample in the sample set. Figure 3 As shown, firstly, given a dictionary D and an eigenvector x, a set of coefficient vectors is obtained through collaborative representation, defined as follows:

[0096] ;

[0097] Where W is a learnable linear transformation matrix, For custom parameters, I is the identity matrix vector. Then, the coefficient vector is used. We use a dictionary to represent x. This is done through y = WD. To reconstruct the feature vectors, the distance loss L between the reconstructed feature vectors y and x is minimized. c To optimize dictionary D. To stabilize the learning of dictionary D, the loss function L is used. c Backpropagation on x is not involved, and the dictionary D is randomly initialized.

[0098] In this embodiment of the invention, through the reconstruction process of the collaborative dictionary learning module, the model can learn an optimized feature dictionary that more accurately represents the key features of the mine remote sensing image. Global pooling helps extract the global statistical properties of the image, while CDLM further ensures that these properties can be effectively used for feature reconstruction. By minimizing the distance loss between the reconstructed feature vector and the original feature vector, the model can improve the accuracy and robustness of feature representation.

[0099] Optionally, the ConvNeXt branch includes multiple sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module;

[0100] The step of performing local information enhancement on the remote sensing image based on the ConvNeXt branch to obtain the local feature map includes:

[0101] The remote sensing image is input into the first feature extraction block of the ConvNeXt branch for local feature extraction, and the feature map output by the feature extraction block is used as the input for the next feature extraction block until the last feature extraction block.

[0102] Simultaneously, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block for local feature enhancement, thereby obtaining the enhanced local feature map output by each local feature enhancement module;

[0103] The enhanced local feature maps are concatenated by stitching together the output order of each enhanced local feature map to obtain the local feature map.

[0104] Specifically, the ConvNeXt branch enhances the local information of remote sensing images through multiple sequentially connected feature extraction blocks. These feature extraction blocks are specifically designed to capture local features in the image, such as texture, edges, and small geological structures. The output of each feature extraction block is passed to the next, forming a feature extraction pipeline until the last feature extraction block. Simultaneously, the output of each feature extraction block is also fed into the corresponding Local Feature Enhancement Module (LFEM). The LFEM further enhances local features through Collaborative Dictionary Learning (CDLM). Through the multi-layer feature extraction and local feature enhancement of the ConvNeXt branch, the model can deeply mine local details in remote sensing images. This enhancement of local features is crucial for identifying small targets and complex backgrounds in mine remote sensing images, as these details often contain key information distinguishing different land cover types. By concatenating each enhanced local feature map in the output order, the model can construct a local feature map that integrates all local features. Figure 3 As shown, in a preferred embodiment of the invention, the ConvNeXt branch includes three sequentially connected feature extraction blocks (ConvBlocks), each ConvBlock corresponding to an LFEM. After the last ConvBlock, the feature map passes through a global pooling layer, which helps reduce the spatial dimensionality of the feature map while preserving the most important local information.

[0105] After receiving the feature map output by ConvBlock, LFEM performs feature transformation on the dictionary D through a fully connected layer (FC), and then performs a dot product with the feature map F to adjust and align the feature representation. Next, the dot-producted feature map is passed through an average pooling (AVG) layer to reduce dimensionality and extract statistical information from local features. Subsequently, these features are transformed into a probability distribution through a softmax layer, which helps to identify the confidence levels of different classes in the feature map. The output of the softmax layer is multiplied with the original feature map F, where the feature map F is weighted according to class confidence, thus highlighting key local features. Finally, the weighted feature map is added to the original feature map F to form an enhanced local feature map, which helps to fuse and strengthen local information. In this way, LFEM not only improves the model's ability to recognize local details, but also provides more accurate and robust feature representations for subsequent classification tasks through feature enhancement, thereby achieving higher accuracy in mine remote sensing scene classification. Figure 6 As shown, the feature map F is first reshaped from H×W×C to N×C. Then, a fully connected (FC) layer is applied to F and Z for feature mapping. Next, the contribution of each local feature vector is estimated based on the correlation between the local feature vectors and the key semantic set Z, and the column mean is calculated. Then, normalization is performed using the Softmax function to obtain the attention coefficient vector. Finally, the attention coefficient vector is multiplied by the original feature map, and residual connections are applied to obtain the feature map as the output of LFEM.

[0106] In this embodiment of the invention, the multi-layer feature extraction and local feature enhancement of the ConvNeXt branch improve the model's ability to recognize local details. Moreover, the splicing of features helps to maintain the spatial consistency of features, enabling the model to make more accurate use of local information during classification.

[0107] Optionally, obtaining the fused feature map based on the global feature map and the local feature map includes:

[0108] The global feature map and the local feature map are input into the dual-branch deep feature fusion module for global pooling, to obtain pooled feature maps corresponding to the global feature map and the local feature map, respectively.

[0109] The pooled feature maps corresponding to the global feature map and the local feature map are weighted and fused to obtain a weighted feature map.

[0110] The pooled feature maps corresponding to the global feature map and the local feature map, as well as the weighted feature map, are respectively input into the corresponding depthwise separable convolutional network, and then merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

[0111] Specifically, the process of fusing global and local feature maps is implemented through a dual-branch deep feature fusion module. First, the global and local feature maps are input into this module for global pooling. This step is achieved through Global Average Pooling (GAP) and Global Max Pooling (GMP), resulting in pooled feature maps corresponding to the two feature maps. These pooled feature maps capture the global statistical characteristics of the image, providing a foundation for subsequent fusion. Next, these pooled feature maps are weighted and fused. This process may involve using a sigmoid activation function to obtain weights. Then, the weighted feature map is multiplied by the original feature map to obtain a weighted feature map. Finally, the pooled features corresponding to the global and local feature maps, along with the weighted feature map, are input into a depthwise separable convolutional network. This network structure further extracts and fuses features through a combination of depthwise convolution and pointwise convolution. Finally, the outputs of the depthwise separable convolutional network are merged to obtain a fused feature map. Figure 7 As shown, the global and local feature maps are processed using Global Max Pooling (GMP) and Global Average Pooling (GAP), respectively, to aggregate global information from the feature maps. Next, these pooled feature maps are processed by Depthwise Separable Convolution (DSConv), an operation that improves computational efficiency by decomposing standard convolution into depthwise and pointwise convolutions, while extracting deeper features. Then, the feature maps processed by DSConv are further processed by standard convolutional layers (Conv) to enhance their expressive power. The output feature maps of these convolutional layers are fused using an addition operation to form a weighted feature map. Finally, the fused feature map is classified using a fully connected layer (FC) to output the final classification result. This dual-branch deep feature fusion module effectively combines global and local information through this structure, improving the discriminative power of the features.

[0112] In this embodiment of the invention, through the processing of the dual-branch deep feature fusion module, the model can deeply fuse the effective information from the global feature map and the local feature map, thereby obtaining a fused feature map that contains both global structural information and local detailed information. This fused feature map can provide a more comprehensive feature representation for classification tasks, enabling the model to more accurately identify and distinguish different land cover types when facing complex mine remote sensing scenes. Global pooling helps capture the global statistical characteristics of the image, while depthwise separable convolution can further extract and fuse features, enhancing the discriminative power of the features.

[0113] Combination Figure 8 As shown, the present invention also provides a mine remote sensing scene classification system, comprising:

[0114] The remote sensing image acquisition unit is used to acquire remote sensing images of the target area of ​​the mine;

[0115] The feature processing unit is used to input the remote sensing image into a dual-stream feature representation network, and enhance the global and local features in the remote sensing image according to the dual-stream feature representation network to obtain the global feature map and local feature map of the target area of ​​the mine respectively.

[0116] A fusion unit is used to obtain a fused feature map based on the global feature map and the local feature map;

[0117] A classification unit is used to obtain a classification label for the target area of ​​the mine based on the fused feature map.

[0118] The mine remote sensing scene classification system of this invention acquires remote sensing images of the target area of ​​the mine, providing raw data for the classification task. Next, the remote sensing images are input into a dual-stream feature representation network (DMN). The DMN effectively captures long-range dependencies in the images, thus obtaining a global feature map. Simultaneously, it focuses on extracting local details of the image, obtaining a local feature map. By utilizing the DMN, which considers both the global structure and local details of the image, it provides richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global and local feature maps, a fused feature map is obtained, further integrating the global and local information of the image. The fusion of local and global features enhances the discriminative power of the features. In mine remote sensing scenes, different land cover types may overlap in local features, but global features can provide additional discriminative information, helping the model better distinguish similar categories. Finally, the fused feature map is mapped to specific classification labels, achieving accurate classification of the target area of ​​the mine and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0119] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the above-described mine remote sensing scene classification method.

[0120] The computer-readable storage medium of this invention provides raw data for classification tasks by acquiring remote sensing images of target areas in mines. Next, the remote sensing images are input into a two-stream feature representation network (DMR), which effectively captures long-range dependencies in the images to obtain a global feature map. Simultaneously, it focuses on extracting local details of the images to obtain local feature maps. By utilizing the DMR to consider both the global structure and local details of the image, it provides richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global and local feature maps, a fused feature map is obtained, further integrating the global and local information of the image. Fusing local and global features enhances the discriminative power of the features. In mine remote sensing scenarios, different land cover types may overlap in local features, but global features can provide additional discriminative information, helping the model better distinguish similar categories. Finally, the fused feature map is mapped to specific classification labels, achieving accurate classification of the target areas in mines and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0122] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for classifying remote sensing scenes in a mine, characterized in that, include: Acquire remote sensing images of the target area of ​​the mine; The remote sensing image is input into a dual-stream feature representation network. Based on the dual-stream feature representation network, the global and local features in the remote sensing image are enhanced respectively to obtain the global feature map and local feature map of the target area of ​​the mine. The dual-stream feature representation network includes two branches: a Swing Transformer branch and a ConvNeXt branch. The step of enhancing global and local features in the remote sensing image using the dual-stream feature representation network to obtain a global feature map and a local feature map of the target mining area includes: aggregating global information in the remote sensing image using the Swing Transformer branch to obtain the global feature map; and enhancing local information in the remote sensing image using the ConvNeXt branch to obtain the local feature map. The step of aggregating global information from the remote sensing image based on the Swin Transformer branch to obtain the global feature map specifically includes: segmenting the remote sensing image into multiple image patches according to the embedding layer in the Swin Transformer branch; transforming each image patch to obtain an embedding representation of the image patch; sequentially inputting the embedding representation into the SwinTBlock layer in the Swin Transformer branch, capturing the long-distance dependencies of the remote sensing image through a window mechanism to obtain an output feature map; and merging the output feature maps to obtain the global feature map. The Swin Transformer branch contains multiple sequentially connected SwinTBlock layers. The step of sequentially inputting the embedding representation into the SwinTBlock layers of the Swin Transformer branch, and capturing the long-range dependencies of the remote sensing image through a window mechanism to obtain an output feature map, includes: inputting the embedding representation into the first SwinTBlock layer; performing feature extraction by moving a window of a preset size within the embedding representation of the image patch to obtain the feature map output by the first SwinTBlock layer; using the feature map output by each SwinTBlock layer as the input to the next SwinTBlock layer; and using the feature map output by the last SwinTBlock layer as the output feature map after the last SwinTBlock layer outputs its feature map. The Swin Transformer branch further includes multiple global channel attention modules; the step of merging the output feature maps to obtain the global feature map includes: inputting the embedding representation into the first global channel attention module; merging the output features of the global channel attention module with the feature map output by the first SwinTBlock layer; inputting the merged feature map into the second global channel attention module; starting from the second global channel attention module, sequentially merging the features output by each global channel attention module with the feature map output by one SwinTBlock layer and inputting the merged feature map into the next global channel attention module; merging the feature map output by the last SwinTBlock layer with the features output by the last global channel attention module to obtain the global feature map. A fused feature map is obtained based on the global feature map and the local feature map; Based on the fused feature map, the classification label of the target area of ​​the mine is obtained.

2. The mine remote sensing scene classification method according to claim 1, characterized in that, Also includes: After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector. The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

3. The mine remote sensing scene classification method according to claim 1, characterized in that, The ConvNeXt branch includes multiple sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module. The step of performing local information enhancement on the remote sensing image based on the ConvNeXt branch to obtain the local feature map includes: The remote sensing image is input into the first feature extraction block of the ConvNeXt branch for local feature extraction, and the feature map output by the feature extraction block is used as the input for the next feature extraction block until the last feature extraction block. Simultaneously, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block for local feature enhancement, thereby obtaining the enhanced local feature map output by each local feature enhancement module; The enhanced local feature maps are concatenated by stitching together the output order of each enhanced local feature map to obtain the local feature map.

4. The mine remote sensing scene classification method according to claim 1, characterized in that, The step of obtaining the fused feature map based on the global feature map and the local feature map includes: The global feature map and the local feature map are input into the dual-branch deep feature fusion module for global pooling, to obtain pooled feature maps corresponding to the global feature map and the local feature map, respectively. The pooled feature maps corresponding to the global feature map and the local feature map are weighted and fused to obtain a weighted feature map. The pooled feature maps corresponding to the global feature map and the local feature map, as well as the weighted feature map, are respectively input into the corresponding depthwise separable convolutional network, and then merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

5. A mine remote sensing scene classification system applied to the mine remote sensing scene classification method of claim 1, characterized in that, include: The remote sensing image acquisition unit is used to acquire remote sensing images of the target area of ​​the mine; The feature processing unit is used to input the remote sensing image into a dual-stream feature representation network, and enhance the global and local features in the remote sensing image according to the dual-stream feature representation network to obtain the global feature map and local feature map of the target area of ​​the mine respectively. A fusion unit is used to obtain a fused feature map based on the global feature map and the local feature map; A classification unit is used to obtain a classification label for the target area of ​​the mine based on the fused feature map.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the mine remote sensing scene classification method according to any one of claims 1 to 4.

Images