Mine remote sensing scene classification method and system and storage medium

Abstract

The invention provides a mine remote sensing scene classification method and system and a storage medium, and relates to the technical field of remote sensing image processing, and the method comprises the steps: obtaining a remote sensing image of a mine target region; inputting the remote sensing image into a double-flow feature representation network, and according to the double-flow feature representation network, respectively enhancing global features and local features in the remote sensing image to respectively obtain a global feature map and a local feature map of the mine target area; obtaining a fusion feature map according to the global feature map and the local feature map; and obtaining a classification label of the mine target area according to the fusion feature map. According to the method, the global and local information of the image is integrated through the double-flow feature representation network, the feature distinguishing capability is enhanced, and the mine remote sensing scene classification precision and efficiency are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a mine remote sensing scene classification method, system and storage medium. Background Art

[0002] Mining activities are often associated with environmental changes, ecosystem damage, and safety hazards. Therefore, monitoring land cover types in mining areas is crucial for ecological and environmental development. Mines and surrounding land cover areas, such as forests and cultivated land, create a complex geological environment, often requiring remote sensing technology to capture large-scale surface information.

[0003] In related technologies, since the surface elements of mines are usually irregular in shape and have large scale differences, the surface characteristics of the mining area are complex and changeable. At the same time, the remote sensing data itself has a lot of noise and complexity, which leads to errors in the identification of land cover types in mine remote sensing scenes, thereby affecting the classification accuracy of mine remote sensing scenes. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the classification effect of mine remote sensing scenes.

[0005] To solve the above problems, the present invention provides a mine remote sensing scene classification method, system and storage medium.

[0006] In a first aspect, the present invention provides a mine remote sensing scene classification method, comprising:

[0007] Acquire remote sensing images of target areas of mines;

[0008] Inputting the remote sensing image into a dual-stream feature representation network, and enhancing the global features and local features in the remote sensing image according to the dual-stream feature representation network to obtain a global feature map and a local feature map of the target mine area;

[0009] Obtaining a fused feature map according to the global feature map and the local feature map;

[0010] According to the fused feature map, a classification label of the mine target area is obtained.

[0011] Optionally, the dual-stream feature representation network includes two branches, the two branches including a SwinTransformer branch and a ConvNeXt branch; and enhancing the global features and local features in the remote sensing image according to the dual-stream feature representation network to obtain a global feature map and a local feature map of the mine target area, respectively, includes:

[0012] Performing global information aggregation on the remote sensing image according to the Swin Transformer branch to obtain the global feature map;

[0013] Local information enhancement is performed on the remote sensing image according to the ConvNeXt branch to obtain the local feature map.

[0014] Optionally, performing global information aggregation on the remote sensing image according to the Swin Transformer branch to obtain the global feature map includes:

[0015] Segmenting the remote sensing image into a plurality of image blocks according to the embedding layer in the Swin Transformer branch;

[0016] Performing a transformation on each of the image blocks to obtain an embedded representation of the image block;

[0017] The embedded representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-range dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map;

[0018] The output feature maps are merged to obtain the global feature map.

[0019] Optionally, the Swin Transformer branch has a plurality of SwinTBlock layers connected in sequence;

[0020] The embedding representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-distance dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map, including:

[0021] Input the embedded representation into the first SwinTBlock layer, and extract features by moving a window of a preset size in the embedded representation of the image block to obtain a feature map output by the first SwinTBlock layer. The feature map output by each SwinTBlock layer serves as the input of the next SwinTBlock layer;

[0022] After the last SwinTBlock layer outputs the feature map, the feature map output by the last SwinTBlock layer is used as the output feature map.

[0023] Optionally, the Swin Transformer branch further includes a plurality of the global channel attention modules;

[0024] The merging according to the output feature maps to obtain the global feature map includes:

[0025] Input the embedded representation into the first global channel attention module, merge the output features of the global channel attention module with the feature map output by the first SwinTBlock layer, and input the merged feature map into the second global channel attention module. Starting from the second global channel attention module, merge the features output by each global channel attention module with the feature map output by a SwinTBlock layer in turn, and then input them into the next global channel attention module;

[0026] The feature map output by the last SwinTBlock layer is merged with the feature output by the last global channel attention module to obtain the global feature map.

[0027] Optionally, it also includes:

[0028] After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector;

[0029] The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

[0030] Optionally, the ConvNeXt branch includes a plurality of sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module;

[0031] The performing local information enhancement on the remote sensing image according to the ConvNeXt branch to obtain the local feature map includes:

[0032] Input the remote sensing image into the first feature extraction block of the ConvNeXt branch to perform local feature extraction, and use the feature map output by the feature extraction block as the input of the next feature extraction block until the last feature extraction block;

[0033] At the same time, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block to perform local feature enhancement, thereby obtaining an enhanced local feature map output by each local feature enhancement module;

[0034] The enhanced local feature maps are spliced in the output order of each of the enhanced local feature maps to obtain the local feature map.

[0035] Optionally, obtaining a fused feature map according to the global feature map and the local feature map includes:

[0036] Inputting the global feature map and the local feature map into a dual-branch deep feature fusion module for global pooling, thereby obtaining pooled feature maps corresponding to the global feature map and the local feature map respectively;

[0037] Performing weighted fusion on the global feature map and the pooled feature map corresponding to the local feature map to obtain a weighted feature map;

[0038] The pooled feature maps corresponding to the global feature map and the local feature map and the weighted feature map are respectively input into the corresponding depthwise separable convolutional network, and are merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

[0039] In a second aspect, the present invention provides a mine remote sensing scene classification system, comprising:

[0040] A remote sensing image acquisition unit, used to acquire remote sensing images of a target area of a mine;

[0041] a feature processing unit, configured to input the remote sensing image into a dual-stream feature representation network, and enhance the global features and local features in the remote sensing image according to the dual-stream feature representation network, thereby obtaining a global feature map and a local feature map of the target mine area;

[0042] A fusion unit, configured to obtain a fused feature map based on the global feature map and the local feature map;

[0043] A classification unit is used to obtain a classification label of the mine target area according to the fused feature map.

[0044] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned mine remote sensing scene classification method is implemented.

[0045] The mine remote sensing scene classification method, system, and storage medium of the present invention provide raw data for the classification task by acquiring remote sensing images of the mine target area. Next, the remote sensing image is input into a dual-stream feature representation network, which effectively captures long-range dependencies in the image, thereby obtaining a global feature map. At the same time, it focuses on extracting local details of the image to obtain a local feature map. The dual-stream feature representation network simultaneously considers the global structure and local details of the image, providing richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global feature map and the local feature map, a fused feature map is obtained, further integrating the global and local information of the image. The fusion of local and global features can enhance the feature distinguishing ability. In the mine remote sensing scene, different land cover types may overlap in local features, but global features can provide additional distinguishing information, helping the model to better distinguish similar categories. Finally, the fused feature map is mapped to a specific classification label, achieving accurate classification of the mine target area and significantly improving the accuracy and efficiency of mine remote sensing scene classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a mine remote sensing scene classification method according to an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of a branch structure of a dual-stream feature representation network in one embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the structure of a dual-stream feature representation network in one embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of the structure of a Swin Transformer branch in one embodiment of the present invention;

[0050] Figure 5 A schematic diagram of the structure of a collaborative dictionary learning module in one embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the structure of a local feature enhancement module in one embodiment of the present invention;

[0052] Figure 7 2 is a schematic diagram of the structure of a dual-branch deep feature fusion module in one embodiment of the present invention;

[0053] Figure 8 Schematic diagram of the structure of a mine remote sensing scene classification system in one embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0055] Combine Figure 1 As shown, the present invention provides a mine remote sensing scene classification method, comprising:

[0056] Acquire remote sensing images of target areas of mines.

[0057] Specifically, remote sensing images of the target mining area are obtained, and these images provide the raw data for the classification task. Remote sensing images can capture the surface features of the mining area, such as vegetation cover, terrain undulations, ore distribution, etc. These features are crucial for subsequent classification. Through remote sensing technology, detailed visual information of the mining area can be obtained, providing a rich data foundation for the classification algorithm. In a preferred embodiment of the present invention, multi-source data is fused based on different types of band information such as optical, synthetic aperture radar, digital elevation model, and terrain imagery to obtain a remote sensing image of the target mining area.

[0058] The remote sensing image is input into a dual-stream feature representation network, and global features and local features in the remote sensing image are enhanced according to the dual-stream feature representation network to obtain a global feature map and a local feature map of the mine target area.

[0059] Specifically, the remote sensing image is input into the dual-stream feature representation network, which effectively captures the long-distance dependencies in the image, thereby obtaining a global feature map. At the same time, it focuses on extracting local details of the image to obtain a local feature map. In a preferred embodiment of the present invention, the dual-stream feature representation network has two branches, including a Swin Transformer branch that focuses on global features and a ConvNeXt branch that focuses on local features. These two branches enhance the global features and local features in the image respectively, so that the network can capture the macroscopic structure and microscopic details of the image at the same time. In the dual-stream feature representation network, the global feature map is obtained through the Swin Transformer branch. This branch uses a window mechanism and hierarchical processing to capture the long-distance dependencies in the image, thereby understanding the global structure of the image. The local feature map is obtained through the ConvNeXt branch. This branch extracts local details of the image through multi-layer convolution operations, providing more comprehensive feature information for the classification task.

[0060] A fused feature map is obtained according to the global feature map and the local feature map.

[0061] Specifically, the network uses a specific fusion strategy, such as weighted fusion or feature map concatenation, to generate a fused feature map based on the global and local feature maps. This fused feature map combines global and local features, enhancing the ability to distinguish features. This allows the model to more accurately identify and distinguish different land cover types, ensuring that the model fully utilizes all relevant information from the image during its decision-making process.

[0062] According to the fused feature map, a classification label of the mine target area is obtained.

[0063] Specifically, based on the fused feature map, classification labels are obtained for the target mining areas. The fused feature map is used to assign the most appropriate classification label to each area. This step, the ultimate goal of the classification task, transforms the features extracted and fused in the previous steps into specific classification results, achieving accurate classification of the mining remote sensing scene.

[0064] The mine remote sensing scene classification method of the present invention provides raw data for the classification task by acquiring remote sensing images of the mine target area. Then, the remote sensing image is input into the dual-stream feature representation network, which effectively captures the long-distance dependencies in the image, thereby obtaining a global feature map. At the same time, it focuses on extracting local details of the image to obtain a local feature map. The dual-stream feature representation network simultaneously considers the global structure and local details of the image, providing richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global feature map and the local feature map, a fused feature map is obtained, which further integrates the global and local information of the image. The fusion of local and global features can enhance the feature differentiation ability. In the mine remote sensing scene, different land cover types may overlap in local features, but global features can provide additional differentiation information to help the model better distinguish similar categories. Finally, the fused feature map is mapped to a specific classification label, achieving accurate classification of the mine target area and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0065] Optionally, the dual-stream feature representation network includes two branches, the two branches including a SwinTransformer branch and a ConvNeXt branch; and enhancing the global features and local features in the remote sensing image according to the dual-stream feature representation network to obtain a global feature map and a local feature map of the mine target area, respectively, includes:

[0066] Performing global information aggregation on the remote sensing image according to the Swin Transformer branch to obtain the global feature map;

[0067] Local information enhancement is performed on the remote sensing image according to the ConvNeXt branch to obtain the local feature map.

[0068] Specifically, combined Figure 2 As shown, in the mine remote sensing scene classification method of the present invention, the dual-stream feature representation network includes two branches: the Swin Transformer branch and the ConvNeXt branch. The Swin Transformer branch is specifically used to perform global information aggregation on remote sensing images. Through the window mechanism and hierarchical processing, it effectively captures the long-distance dependencies in the image, thereby obtaining a global feature map. This global feature map contains the overall layout and structural information of the mine target area, such as the distribution of mines, the undulations of the terrain, etc. On the other hand, the ConvNeXt branch focuses on local information enhancement of remote sensing images, and extracts local details of the image through multi-layer convolution operations, such as the texture of vegetation, the surface features of ore, etc., to obtain a local feature map. This design enables the network to consider both the global structure and local details of the image, providing more comprehensive feature information for the classification task.

[0069] In this embodiment of the present invention, the collaborative work of a two-stream feature representation network enables the model to more accurately identify and distinguish different land cover types in mine remote sensing imagery. The global feature map provides a macroscopic view of the image, helping to understand the overall structure of the scene, while the local feature map provides a microscopic view, helping to capture detailed features. This combination of global and local features enables the model to make more accurate classification decisions when faced with complex and changing mine remote sensing scenes.

[0070] Optionally, performing global information aggregation on the remote sensing image according to the Swin Transformer branch to obtain the global feature map includes:

[0071] Segmenting the remote sensing image into a plurality of image blocks according to the embedding layer in the Swin Transformer branch;

[0072] Performing a transformation on each of the image blocks to obtain an embedded representation of the image block;

[0073] The embedded representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-range dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map;

[0074] The output feature maps are merged to obtain the global feature map.

[0075] Specifically, the remote sensing image is divided into multiple image blocks through the embedding layer in the Swin Transformer branch, where each block is a local area of the image. Next, each image block is transformed to obtain its embedded representation. These embedded representations are high-level features of the image block, which can capture the key information within the image block. These embedded representations are then input into the SwinTBlock layer in the Swin Transformer branch in sequence. Here, the image blocks are processed using a window mechanism, and the dependencies within the image blocks are captured through a self-attention mechanism. At the same time, the receptive field is gradually expanded through window movement and hierarchical processing, thereby capturing long-distance dependencies in the image. Finally, by merging the output features, a global feature map is obtained. This feature map contains the global information of the entire remote sensing image. Combined with Figure 3 As shown, in a preferred embodiment of the present invention, the remote sensing image is divided into fixed-size image patches (patches) through the Patch Embedding layer, and these image patches are converted into embedding vectors to prepare for subsequent SwinTBlock layer processing.

[0076] In this embodiment of the present invention, by aggregating global information within the Swin Transformer branch, the model is able to obtain a comprehensive and rich global feature map that encompasses the global structure and long-range dependencies of the image. This is because features such as the topography, landforms, and vegetation distribution of mining areas often exhibit distinct patterns and regularities at the global level. Acquiring a global feature map enables the model to better understand these patterns and regularities, leading to more accurate classification decisions.

[0077] Optionally, the Swin Transformer branch has a plurality of SwinTBlock layers connected in sequence;

[0078] The embedding representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-distance dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map, including:

[0079] Input the embedded representation into the first SwinTBlock layer, and extract features by moving a window of a preset size in the embedded representation of the image block to obtain a feature map output by the first SwinTBlock layer. The feature map output by each SwinTBlock layer serves as the input of the next SwinTBlock layer;

[0080] After the last SwinTBlock layer outputs the feature map, the feature map output by the last SwinTBlock layer is used as the output feature map.

[0081] Specifically, the design of the Swin Transformer branch adopts multiple SwinTBlock layers connected in sequence to form a deep feature extraction pipeline. The embedded image block is input to the first SwinTBlock layer, which moves in the embedded representation of the image block through a window of preset size, performs a self-attention mechanism, and thus extracts local features and captures dependencies within local areas. The output feature map of each SwinTBlock layer is then used as the input of the next SwinTBlock layer, and so on. Each layer further extracts and processes features based on the previous layer. This stacked structure allows the model to gradually integrate and process information, so that each subsequent layer can understand the image content at a higher level. After the last SwinTBlock layer outputs a feature map, this feature map is used as the output feature map, which contains global feature information from low to high levels. Combined with Figure 3 As shown, in a preferred embodiment of the present invention, the SwinTransformer branch includes three SwinTBlock layers, and the embedding vector is input to the first SwinTBlock layer. This layer processes the embedded representation of the image block through a self-attention mechanism and a feedforward neural network, captures local features and performs feature extraction. The output feature map of this layer is passed to the next SwinTBlock layer. The second SwinTBlock layer continues to process the feature map processed by the previous SwinTBlock layer, further extracting and processing features. This layer also contains a self-attention mechanism and a feedforward neural network to enhance the expressiveness of features. The third SwinTBlock layer performs the final feature extraction on the feature map processed by the second SwinTBlock layer. The output feature map of this layer contains global feature information from low-level to high-level.

[0082] In this embodiment of the present invention, by sequentially connecting and processing multiple SwinTBlock layers, the model can achieve layer-by-layer abstraction and deepening of remote sensing image features. This layer-by-layer feature extraction method enables the model to more effectively capture long-range dependencies from local to global, thereby incorporating rich hierarchical information into the global feature map.

[0083] Optionally, the Swin Transformer branch further includes a plurality of the global channel attention modules;

[0084] The merging according to the output feature maps to obtain the global feature map includes:

[0085] Input the embedded representation into the first global channel attention module, merge the output features of the global channel attention module with the feature map output by the first SwinTBlock layer, and input the merged feature map into the second global channel attention module. Starting from the second global channel attention module, merge the features output by each global channel attention module with the feature map output by a SwinTBlock layer in turn, and then input them into the next global channel attention module;

[0086] The feature map output by the last SwinTBlock layer is merged with the feature output by the last global channel attention module to obtain the global feature map.

[0087] Specifically, combined Figure 3 As shown in the figure, the output of each SwinTBlock layer in the Swin Transformer branch is connected to a global channel attention module (GCAM). This design aims to further refine and enhance the global information of the feature map. After extracting local features, each SwinTBlock layer outputs a feature map that is input into the corresponding GCAM. The GCAM aggregates the global information of the feature map through global average pooling and global maximum pooling. It then learns channel attention through a multi-layer perceptron (MLP) to obtain weight channels. These weight channels are used to guide the network's learning process and enhance the key information in the feature map. The feature maps output by each SwinTBlock layer are processed by the GCAM and merged according to the output order of the feature maps. Finally, the feature map output by the last SwinTBlock layer is merged with the merged feature map output by the GCAM corresponding to the previous SwinTBlock layer to obtain the final global feature map.

[0088] In a preferred embodiment of the present invention, Figure 4As shown in the figure, the input feature map first undergoes global max pooling (GMP) and global average pooling (GAP) to aggregate the global statistical characteristics of each channel. These pooled features are then transformed nonlinearly through a multilayer perceptron (MLP) to learn the complex relationships between channels. The output of the MLP then passes through a sigmoid activation function to generate per-channel attention weights, which reflect the importance of each channel. By multiplying these weights with the original feature map, the feature map is weighted, highlighting key features and suppressing unimportant information. Finally, the GCAM-processed feature maps are merged to form an output feature map that contains rich global information, providing a more accurate and robust feature representation for subsequent classification tasks. The global channel attention module first performs global average pooling and global max pooling on the feature map F of size H × W × C in parallel to aggregate the global information of the feature map, resulting in two channel maps of size 1 × 1 × C. A multilayer perceptron (MLP) with a hidden layer is then added separately to obtain the weighted channels of the features, which are used to learn the final channel attention feature map. To reduce parameter overhead, the size of the hidden layer is set to C / r, where r is the reduction factor. To effectively combine the two different features, the respective results are numerically summed. Finally, the effective features are improved by multiplying the input feature map by the channel weight. The weight coefficient is obtained by the Sigmoid function, which is a nonlinear activation function commonly used in neural networks. The mathematical expression of the nonlinear activation function is: ; Among them, x The input of the neuron. The output range of the Sigmoid function is between (0, 1), which means that it can map any real value to a probability value between 0 and 1.

[0089] In this embodiment of the present invention, the global channel attention module optimizes the global representation of features, enabling the network to more effectively capture and utilize global information. Specifically, through the layer-by-layer application of the global channel attention module and the merging of feature maps, the model is able to gradually build a comprehensive and rich global feature map. This feature map not only captures the local details of the image, but more importantly, also captures the global structure and long-range dependencies of the image.

[0090] Optionally, it also includes:

[0091] After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector;

[0092] The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

[0093] Specifically, the feature maps output by the last SwinTBlock layer of the Swin Transformer branch are globally pooled and then input into the Collaborative Dictionary Learning Module (CDLM) to reconstruct the feature vector. The global pooling step, including global average pooling (GAP) and global maximum pooling (GMP), aims to reduce the spatial dimension of the feature map while retaining the most important global information. These pooled feature maps are then used to reconstruct the feature vector, which is achieved by CDLM, combined with Figure 5 As shown in , CDLM aims to learn a dictionary D that can collaboratively represent each sample in the sample set. Figure 3 As shown, first, given the dictionary D and the feature vector x, a set of coefficient vectors is obtained through collaborative representation, which is defined as follows: ; Among them, W is a learnable linear transformation matrix, is a custom parameter, and I is the unit matrix vector. Then, use the coefficient vector Combine dictionaries to represent x. By y = WD To reconstruct the feature vector, and minimize the distance loss L between the reconstructed feature vector y and x c To optimize the dictionary D. In order to stabilize the learning of the dictionary D, the loss function L c No backpropagation on x is involved and the dictionary D is initialized randomly.

[0094] In this embodiment of the present invention, through the reconstruction process of the collaborative dictionary learning module, the model is able to learn an optimized feature dictionary that more accurately represents the key features of mine remote sensing images. Global pooling helps extract the global statistical characteristics of the image, while CDLM further ensures that these characteristics are effectively used for feature reconstruction. By minimizing the distance loss between the reconstructed feature vector and the original feature vector, the model can improve the accuracy and robustness of feature representation.

[0095] Optionally, the ConvNeXt branch includes a plurality of sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module;

[0096] The performing local information enhancement on the remote sensing image according to the ConvNeXt branch to obtain the local feature map includes:

[0097] Input the remote sensing image into the first feature extraction block of the ConvNeXt branch to perform local feature extraction, and use the feature map output by the feature extraction block as the input of the next feature extraction block until the last feature extraction block;

[0098] At the same time, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block to perform local feature enhancement, thereby obtaining an enhanced local feature map output by each local feature enhancement module;

[0099] The enhanced local feature maps are spliced in the output order of each of the enhanced local feature maps to obtain the local feature map.

[0100] Specifically, the ConvNeXt branch enhances the local information of remote sensing images through multiple sequentially connected feature extraction blocks. These feature extraction blocks are specially designed to capture local features in the image, such as textures, edges, and small geological structures. The output of each feature extraction block is passed to the next feature extraction block to form a feature extraction pipeline until the last feature extraction block. At the same time, the output of each feature extraction block is also input into the corresponding local feature enhancement module (LFEM). LFEM further enhances local features through collaborative dictionary learning (CDLM). Through the multi-layer feature extraction and local feature enhancement of the ConvNeXt branch, the model can deeply explore the local details in the remote sensing image. This enhancement of local features is crucial for identifying small targets and complex backgrounds in mine remote sensing images, because these details often contain key information for distinguishing different land cover types. By splicing each enhanced local feature map in the output order, the model can construct a local feature map that integrates all local features. Combined with Figure 3 As shown in FIG, in a preferred embodiment of the present invention, the ConvNeXt branch includes three sequentially connected feature extraction blocks (ConvBlocks), each of which is connected to a corresponding LFEM. After the last ConvBlock, the feature map passes through a global pooling layer, which helps to reduce the spatial dimension of the feature map while retaining the most important local information.

[0101] When LFEM receives the feature map output by ConvBlock, the dictionary D is transformed through the fully connected layer (FC) and then multiplied with the feature map F to adjust and align the feature representation. Next, the feature map after the dot multiplication is reduced in dimension and the statistical information of the local features is extracted through the average pooling (AVG) layer. Subsequently, these features are converted into probability distributions through the Softmax layer, which helps to identify the confidence of different categories in the feature map. The output of the Softmax layer is multiplied with the original feature map F, where the feature map F is weighted according to the category confidence to highlight the key local features. Finally, the weighted feature map is added to the original feature map F to form an enhanced local feature map, which helps to fuse and strengthen local information. In this way, LFEM not only improves the model's ability to recognize local details, but also provides more accurate and robust feature representations for subsequent classification tasks through feature enhancement, thereby achieving higher accuracy in mine remote sensing scene classification. Combined with Figure 6 As shown in Figure 1, the feature map F is first reshaped from H×W×C to N×C. A fully connected (FC) layer is then applied to F and Z for feature mapping. Next, the contribution of each local feature vector is estimated based on the correlation between the local feature vector and the key semantic set Z, and the column mean is calculated. This is then normalized using the Softmax function to obtain the attention coefficient vector. Finally, the attention coefficient vector is multiplied with the original feature map, and a residual connection is applied to obtain the feature map as the output of the LFEM.

[0102] In an embodiment of the present invention, the multi-layer feature extraction and local feature enhancement of the ConvNeXt branch improve the model's ability to recognize local details, and the splicing of features also helps to maintain the spatial consistency of features, so that the model can more accurately utilize local information during classification.

[0103] Optionally, obtaining a fused feature map according to the global feature map and the local feature map includes:

[0104] Inputting the global feature map and the local feature map into a dual-branch deep feature fusion module for global pooling, thereby obtaining pooled feature maps corresponding to the global feature map and the local feature map respectively;

[0105] Performing weighted fusion on the global feature map and the pooled feature map corresponding to the local feature map to obtain a weighted feature map;

[0106] The pooled feature maps corresponding to the global feature map and the local feature map and the weighted feature map are respectively input into the corresponding depthwise separable convolutional network, and are merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

[0107] Specifically, the process of fusing global feature maps and local feature maps is implemented through a dual-branch deep feature fusion module. First, the global feature map and the local feature map are input into the module for global pooling. This step is achieved through global average pooling (GAP) and global maximum pooling (GMP), and the pooled feature maps corresponding to the two feature maps are obtained respectively. These pooled feature maps capture the global statistical characteristics of the image and provide a basis for subsequent fusion. Next, these pooled feature maps are weighted fused. This process may involve using a sigmoid activation function to obtain weights, and then multiplying the weighted feature map with the original feature map to obtain a weighted feature map. Finally, the global feature map, the pooled features corresponding to the local feature map, and the weighted feature map are input into a deep separable convolutional network. This network structure further extracts and fuses features through a combination of deep convolution and point-by-point convolution, and finally merges them according to the output of the deep separable convolutional network to obtain a fused feature map. Combined Figure 7 As shown in the figure, the global feature map and the local feature map are processed by global maximum pooling (GMP) and global average pooling (GAP), respectively, to aggregate the global information in the feature map. Next, these pooled feature maps pass through depthwise separable convolution (DSConv). This convolution operation improves computational efficiency by decomposing standard convolution into depthwise convolution and pointwise convolution, while extracting deeper features. The feature maps processed by DSConv are then further processed by standard convolution layers (Conv) to enhance the expressive power of the features. The output feature maps of these convolution layers are fused through addition operations to form weighted feature maps. Finally, the fused feature maps are classified through a fully connected layer (FC), and the final classification result is output. Through this structure, the entire dual-branch deep feature fusion module effectively combines global and local information and improves the distinguishing power of features.

[0108] In an embodiment of the present invention, through the processing of a dual-branch deep feature fusion module, the model is able to deeply fuse the effective information of the global feature map and the local feature map, thereby obtaining a fused feature map that contains both global structural information and local detail information. This fused feature map can provide a more comprehensive feature representation for classification tasks, enabling the model to more accurately identify and distinguish different land cover types when faced with complex mine remote sensing scenes. Global pooling helps capture the global statistical characteristics of the image, while deep separable convolution can further extract and fuse features, enhancing the ability to distinguish features.

[0109] Combine Figure 8 As shown, the present invention also provides a mine remote sensing scene classification system, comprising:

[0110] A remote sensing image acquisition unit, used to acquire remote sensing images of a target area of a mine;

[0111] a feature processing unit, configured to input the remote sensing image into a dual-stream feature representation network, and enhance the global features and local features in the remote sensing image according to the dual-stream feature representation network, thereby obtaining a global feature map and a local feature map of the target mine area;

[0112] A fusion unit, configured to obtain a fused feature map based on the global feature map and the local feature map;

[0113] A classification unit is used to obtain a classification label of the mine target area according to the fused feature map.

[0114] The mine remote sensing scene classification system of the present invention provides raw data for the classification task by acquiring remote sensing images of the mine target area. Then, the remote sensing image is input into the dual-stream feature representation network, which effectively captures the long-range dependencies in the image, thereby obtaining a global feature map. At the same time, it focuses on extracting local details of the image to obtain a local feature map. The dual-stream feature representation network simultaneously considers the global structure and local details of the image, providing richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global feature map and the local feature map, a fused feature map is obtained, which further integrates the global and local information of the image. The fusion of local and global features can enhance the feature differentiation ability. In the mine remote sensing scene, different land cover types may overlap in local features, but global features can provide additional distinguishing information to help the model better distinguish similar categories. Finally, the fused feature map is mapped to a specific classification label, achieving accurate classification of the mine target area and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0115] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned mine remote sensing scene classification method is implemented.

[0116] The computer-readable storage medium of the present invention provides raw data for the classification task by acquiring remote sensing images of the target area of the mine. Then, the remote sensing image is input into the dual-stream feature representation network, which effectively captures the long-distance dependencies in the image, thereby obtaining a global feature map. At the same time, it focuses on extracting local details of the image to obtain a local feature map. The dual-stream feature representation network simultaneously considers the global structure and local details of the image, providing richer and more comprehensive feature information for the classification task. Subsequently, by fusing the global feature map and the local feature map, a fused feature map is obtained, which further integrates the global and local information of the image. The fusion of local and global features can enhance the feature differentiation ability. In the mine remote sensing scene, different land cover types may overlap in local features, but global features can provide additional differentiation information to help the model better distinguish similar categories. Finally, the fused feature map is mapped to a specific classification label, achieving accurate classification of the mine target area and significantly improving the accuracy and efficiency of mine remote sensing scene classification.

[0117] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-described methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0118] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0119] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A mine remote sensing scene classification method, characterized in that: include: Acquire remote sensing images of target areas of mines; Inputting the remote sensing image into a dual-stream feature representation network, and enhancing the global features and local features in the remote sensing image according to the dual-stream feature representation network to obtain a global feature map and a local feature map of the target mine area; Obtaining a fused feature map according to the global feature map and the local feature map; According to the fused feature map, a classification label of the mine target area is obtained.

2. The mine remote sensing scene classification method according to claim 1, characterized in that: The dual-stream feature representation network includes two branches, the two branches including a Swin Transformer branch and a ConvNeXt branch; according to the dual-stream feature representation network, the global features and local features in the remote sensing image are enhanced respectively to obtain a global feature map and a local feature map of the mine target area respectively, including: Performing global information aggregation on the remote sensing image according to the Swin Transformer branch to obtain the global feature map; Local information enhancement is performed on the remote sensing image according to the ConvNeXt branch to obtain the local feature map.

3. The mine remote sensing scene classification method according to claim 2, characterized in that: The performing global information aggregation on the remote sensing image according to the SwinTransformer branch to obtain the global feature map includes: Segmenting the remote sensing image into a plurality of image blocks according to the embedding layer in the Swin Transformer branch; Performing a transformation on each of the image blocks to obtain an embedded representation of the image block; The embedded representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-range dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map; The output feature maps are merged to obtain the global feature map.

4. The mine remote sensing scene classification method according to claim 3, characterized in that: The Swin Transformer branch has multiple SwinTBlock layers connected in sequence; The embedding representation is sequentially input into the SwinTBlock layer in the Swin Transformer branch, and the long-distance dependency of the remote sensing image is captured through a window mechanism to obtain an output feature map, including: Input the embedded representation into the first SwinTBlock layer, and extract features by moving a window of a preset size in the embedded representation of the image block to obtain a feature map output by the first SwinTBlock layer. The feature map output by each SwinTBlock layer serves as the input of the next SwinTBlock layer; After the last SwinTBlock layer outputs the feature map, the feature map output by the last SwinTBlock layer is used as the output feature map.

5. The mine remote sensing scene classification method according to claim 4, characterized in that: The Swin Transformer branch also includes a plurality of global channel attention modules; The merging according to the output feature maps to obtain the global feature map includes: Input the embedded representation into the first global channel attention module, merge the output features of the global channel attention module with the feature map output by the first SwinTBlock layer, and input the merged feature map into the second global channel attention module. Starting from the second global channel attention module, merge the features output by each global channel attention module with the feature map output by a SwinTBlock layer in turn, and then input them into the next global channel attention module; The feature map output by the last SwinTBlock layer is merged with the feature output by the last global channel attention module to obtain the global feature map.

6. The mine remote sensing scene classification method according to claim 5, characterized in that: Also includes: After the last SwinTBlock layer outputs the feature map, the feature map is globally pooled and then input into the collaborative dictionary learning module to reconstruct the feature vector; The local feature enhancement module of the ConvNeXt branch is optimized by minimizing the distance loss between the reconstructed feature vector and the original feature vector.

7. The mine remote sensing scene classification method according to claim 4, characterized in that: The ConvNeXt branch includes a plurality of sequentially connected feature extraction blocks, and the output of each feature extraction block corresponds to a local feature enhancement module; The performing local information enhancement on the remote sensing image according to the ConvNeXt branch to obtain the local feature map includes: Input the remote sensing image into the first feature extraction block of the ConvNeXt branch to perform local feature extraction, and use the feature map output by the feature extraction block as the input of the next feature extraction block until the last feature extraction block; At the same time, the feature map output by each feature extraction block is input into the local feature enhancement module corresponding to each feature extraction block to perform local feature enhancement, thereby obtaining an enhanced local feature map output by each local feature enhancement module; The enhanced local feature maps are spliced in the output order of each of the enhanced local feature maps to obtain the local feature map.

8. The mine remote sensing scene classification method according to claim 1, characterized in that: The step of obtaining a fused feature map according to the global feature map and the local feature map includes: Inputting the global feature map and the local feature map into a dual-branch deep feature fusion module for global pooling, thereby obtaining pooled feature maps corresponding to the global feature map and the local feature map respectively; Performing weighted fusion on the global feature map and the pooled feature map corresponding to the local feature map to obtain a weighted feature map; The pooled feature maps corresponding to the global feature map and the local feature map and the weighted feature map are respectively input into the corresponding depthwise separable convolutional network, and are merged according to the output of the depthwise separable convolutional network to obtain the fused feature map.

9. A mine remote sensing scene classification system, characterized in that: include: A remote sensing image acquisition unit, used to acquire remote sensing images of a target area of a mine; a feature processing unit, configured to input the remote sensing image into a dual-stream feature representation network, and enhance the global features and local features in the remote sensing image according to the dual-stream feature representation network, thereby obtaining a global feature map and a local feature map of the target mine area; A fusion unit, configured to obtain a fused feature map based on the global feature map and the local feature map; A classification unit is used to obtain a classification label of the mine target area according to the fused feature map.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the mine remote sensing scene classification method according to any one of claims 1 to 8 is implemented.

Images