Strip mine area automatic extraction method based on convolutional neural network and attention mechanism

Abstract

The invention provides a strip mine area automatic extraction method based on a convolutional neural network and an attention mechanism, and the method comprises the steps: carrying out the data preprocessing, and constructing multispectral data containing a vegetation index and a plurality of wavebands; a strip mining area extraction model is set, a double-branch encoder structure is constructed in the strip mining area extraction model, a main encoder in the double-branch encoder structure inputs the visible light wave band of the remote sensing image to extract geometric and texture features, and an auxiliary encoder fuses the near-infrared wave band and the vegetation index to extract vegetation features; high-level features output by the double-branch encoder are input into a cavity space convolution pooling pyramid module, and multi-scale context information is integrated; constructing a decoder containing a depth supervision mechanism, and setting an attention segmentation head at the tail end of the decoder; and training the strip mining area extraction model by adopting the dynamic weighted mixed loss function, and outputting an automatic extraction result of the strip mining area based on the trained model. According to the invention, rapid and accurate extraction of the strip mine area can be realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of remote sensing image information extraction, and specifically provides a kind of open pit mining area automatic extraction technical scheme based on convolutional neural network and attention mechanism. BACKGROUND

[0002] The exploitation of mineral resources is a double-edged sword, which not only promotes the rapid development of economy, but also has a series of negative impacts on regional ecology, especially in open pit mining areas. With the development of mineral resources, the earth has been left with huge "scars". A study reported in Nature in 2024 shows that at least half of the world's mines are still unrecorded. The lack of spatio-temporal distribution information of open pit mines has seriously hampered the process of global mine ecological restoration and sustainable management of resources. Therefore, it is urgent to build intelligent means to accurately identify large-scale open pit mines, provide data support for mine ecological quality monitoring and ecological restoration, and help the sustainable development of global open pit mines.

[0003] Currently, the extraction methods for open pit mining areas mainly include three categories: 1) traditional extraction methods, which manually interpret satellite images and identify them by combining spectral features and geometric shapes of the mining area. This method has the advantage of high extraction accuracy, but the disadvantages are that it relies on expert experience, has low processing efficiency, and is difficult to deal with misjudgment problems in complex topography. 2) Machine learning extraction method, with the rise of machine learning algorithms, open pit extraction has entered the semi-automated stage. The focus of the research has shifted to multi-feature fusion and classifier optimization: by constructing a multi-dimensional feature set of spectral, texture and terrain features, and combining support vector machine (SVM), random forest (RF) and other algorithms to realize classification, this method can reduce the workload of visual interpretation, but the feature engineering relies on manual design, and the model generalization ability is limited, making it difficult to adapt to different geological environments. 3) Deep learning extraction method, deep learning has promoted open pit extraction into the intelligent era. Models represented by convolutional neural networks can learn multi-level features directly from images, thus achieving accurate extraction of open pit mines. This method can effectively reduce the workload of manual work, saving manpower and material resources, and is the mainstream technology for extracting mines and tailings.

[0004] Deep learning networks represented by convolutional neural networks have strong non-linear representation ability and high image recognition ability, and have significant advantages in remote sensing image information extraction. However, the types of ground objects in open pit mining areas are complex, and the terrain is complex and variable, which brings certain challenges to the extraction of mining areas. The existing deep learning models have the following problems in the process of extracting open pit mines: 1) the differences in vegetation features between open pit mines and unexploited areas are ignored; 2) the feature expression ability of the decoder is limited, and the detail recovery effect is poor; 3) the foreground target and background object categories are unbalanced. The above problems have brought challenges to the automatic extraction of open pit mining areas. SUMMARY

[0005] In view of the deficiencies of the existing deep learning model in the extraction process of open-pit mining area, the present application aims to provide a new open-pit mining area extraction model, and realize an automatic extraction method of open-pit mining area based on convolutional neural network and attention mechanism.

[0006] The technical scheme of the present application provides an automatic extraction method of open-pit mining area based on convolutional neural network and attention mechanism, which includes the following processes, performing data preprocessing to construct multispectral data containing vegetation index and multiple bands; setting an open-pit mining area extraction model, constructing a double-branch encoder structure in the open-pit mining area extraction model, inputting the visible light band of the remote sensing image into the main encoder in the double-branch encoder structure to extract geometric and texture features, and inputting the near-infrared band and vegetation index into the auxiliary encoder to extract vegetation features; inputting the high-level features output by the double-branch encoder into a hollow spatial convolution pyramid module to integrate multi-scale context information; constructing a decoder containing a deep supervision mechanism, fusing the multi-level features of the encoder through a skip connection, and gradually restoring the spatial resolution by adopting cascaded upsampling, and setting an attention segmentation head at the end of the decoder; training the open-pit mining area extraction model using a dynamic weighted hybrid loss function, and outputting the automatic extraction result of the open-pit mining area based on the trained open-pit mining area extraction model.

[0007] Moreover, a feature cross fusion module is embedded in the middle stage of the double-branch encoder structure to enhance the double-branch feature interaction capability.

[0008] Moreover, the feature cross fusion module uses adaptive average pooling to respectively decompose the input feature tensor into perception feature maps along the width and height directions, so as to realize the acquisition of global statistics in the width and height directions.

[0009] Moreover, the hollow spatial convolution pyramid module contains multiple branches arranged in parallel, including an original resolution convolution branch, a hollow convolution branch with different expansion rates, and a global pooling branch; the outputs of each branch are spliced along the channel dimension, and feature fusion is performed through a convolution layer.

[0010] Moreover, the operation of the deep supervision mechanism includes generating intermediate prediction results at multiple levels of the decoder, and fusing the multi-level outputs through adaptive weights.

[0011] Moreover, the attention segmentation head contains two layers of channel attention modules connected in series.

[0012] Moreover, the dynamic weighted hybrid loss function combines cross-entropy loss and regional overlap metric loss, and dynamically adjusts the weights according to the real-time category distribution to balance the contributions of the foreground and the background, wherein the foreground is an open-pit mine.

[0013] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the open-pit mine automatic extraction method based on convolutional neural networks and attention mechanisms as described above.

[0014] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the open-pit mine automatic extraction method based on convolutional neural networks and attention mechanisms as described above.

[0015] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the open-pit mine automatic extraction method based on convolutional neural networks and attention mechanisms as described above.

[0016] The aforementioned solution addresses the problems of existing deep learning models in extracting open-pit mine areas, such as ignoring the differences in vegetation features between open-pit and unmined areas, limited decoder feature representation capabilities leading to poor detail recovery, and imbalance between foreground and background feature categories. The automatic open-pit mine extraction method disclosed in this invention, based on convolutional neural networks and attention mechanisms, overcomes these problems by constructing an innovative model structure, a cross-feature fusion module, an attention segmentation head, and a mixed loss function. This achieves rapid and accurate extraction of open-pit mine areas, providing a new technical solution for mine extraction research.

[0017] Compared to existing methods, the advantages and positive effects of this invention are as follows: 1) Accurate vegetation feature differentiation capability: By using a feature cross-fusion module to perform spatial-channel joint modeling of RGB data and near-infrared + vegetation index, the model's ability to capture the differences in vegetation features between open-pit mines and unmined areas is significantly enhanced. 2) Optimized multi-scale detail recovery performance: The constructed attention segmentation head dynamically calibrates channel weights through a two-level SEAttention module and combines it with a cascaded upsampling structure, solving the problem of insufficient feature representation capability of traditional decoders. 3) Enhanced robustness to class imbalance problems: A hybrid loss function (Dice Loss + Cross-Entropy Loss) and a deep supervision mechanism are adopted to balance the contributions of the foreground (mine area) and background (vegetation, roads) through learnable weight parameters. 4) Generalization performance in complex scenes: Through multi-level feature fusion design and attention-guided feature interaction, the model can adapt to complex terrain, lighting changes, and multi-scale target challenges in mining areas. Attached Figure Description

[0018] Figure 1 This is a flowchart of the automatic extraction process in an open-pit mine according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the network structure according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the dual-branch encoder structure according to an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the feature cross-fusion module structure constructed according to an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of the ASPP feature fusion module structure according to an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the upsampling stage structure in an embodiment of the present invention.

[0024] Figure 7 This is a structural diagram of the attention segmentation head according to an embodiment of the present invention.

[0025] Figure 8 A partial sample image of the dataset constructed for an embodiment of the present invention.

[0026] Figure 9 This is a visual comparison diagram of the extraction results from an open-pit mine, as shown in this embodiment of the invention. Detailed Implementation

[0027] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0028] To address the problems existing in the extraction of open-pit mine areas using current deep learning models, this invention provides an automatic extraction method for open-pit mine areas based on convolutional neural networks and attention mechanisms. By constructing an innovative model structure, a cross-feature fusion module, an attention segmentation head, and a mixed loss function, this method overcomes three problems in the extraction of open-pit mine areas: ignoring the differences in vegetation features between open-pit and unmined areas; limited feature representation capabilities of the decoder, resulting in poor detail recovery; and an imbalance between foreground targets and background features. This enables rapid and accurate extraction of open-pit mine areas, providing a new technical solution for mine extraction research.

[0029] Specifically, this invention constructs a dual-branch encoder and a feature cross-fusion module to enhance the model's learning of vegetation features in open-pit mining areas. It further mines multi-scale information extracted by the two encoders using the Spatial Convolutional Pooling Pyramid (ASPP) model. A decoder with deconvolution, a deep supervision module, and an attention segmentation head is built to improve the accuracy of mining area extraction. Finally, a hybrid loss function is constructed to overcome the feature learning bias caused by the imbalance between foreground and background categories. By constructing the model structure, proposing innovative modules, and building a loss function adapted to mining area extraction, this invention effectively solves the problems existing in current mining area extraction research and significantly improves the accuracy of mining area extraction.

[0030] See Figure 1 This invention provides an automatic extraction method for open-pit mining areas based on convolutional neural networks and attention mechanisms, specifically including the following process: Data preprocessing involves radiometric calibration, atmospheric correction, and orthorectification of the GF-2 multispectral data, as well as radiometric calibration and orthorectification of the panchromatic data. Finally, the multispectral orthorectification results and the panchromatic orthorectification results are fused to obtain multispectral data with a spatial resolution of 1 meter.

[0031] The dataset construction begins with calculating the vegetation index NDVI. The NDVI is then overlaid with preprocessed GF-2 multispectral data to obtain multispectral data with five bands: R, G, B, NIR, and NDVI. Mining area labels are created through visual interpretation. The multispectral images and corresponding labels are then segmented into 512×512 pixel samples. Finally, the constructed samples are divided into training, testing, and validation data in an 8:1:1 ratio. The training and testing data are used to train the model, while the validation data is used to verify the model's extraction results. In practice, data processing and sample construction can be performed in advance.

[0032] The core of this invention is the modeling and training process, which involves constructing an open-pit mine extraction model. The model includes a dual-branch network, a cross-feature fusion module, an attention segmentation head, and a deep supervision module, and incorporates a hybrid loss function. The dataset is input into the model for training, and the model parameters are optimized based on the training results to obtain the optimal trained model. Using this trained model, the data to be analyzed can be input to automatically extract the open-pit mine extraction results.

[0033] In practice, performance evaluation can also be conducted, and various accuracy evaluation indicators can be used to evaluate the accuracy of the model prediction results.

[0034] The present invention further proposes: 1) Design the network structure for the open-pit mining area extraction model. The network structure is designed as a mainstream "encoder-decoder" structure. The encoder is used to downsample the input image and mine the deep features contained in the image. The decoder is used to upsample the deep features to obtain the target extraction result.

[0035] 2) Design of a dual-branch encoder feature cross-fusion module. To fully fuse the deep features from the downsampling process of the dual-branch encoder, a feature cross-fusion module is designed. This module consists of both positional attention and spatial attention. The feature cross-fusion module is embedded in the feature fusion layer to enhance the feature interaction capability between the RGB band and NIR+NDVI. The high-level features output from the dual-branch encoder are input into a dilated spatial convolutional pooling pyramid module to integrate multi-scale contextual information.

[0036] 3) Construct a decoder with a deep supervision mechanism, fuse multi-level features of the encoder through skip connections, and gradually restore spatial resolution by using cascaded upsampling; set an attention segmentation head at the end of the decoder, and dynamically calibrate feature weights through a series of channel attention modules; train the open-pit mining area extraction model using a dynamic weighted hybrid loss function, and output the automatic extraction results of the open-pit mining area based on the trained open-pit mining area extraction model.

[0037] This invention incorporates a multi-level supervision mechanism into the decoder, fusing deep semantic features with shallow detail features through learnable weight parameters to suppress noise interference. An attention segmentation head is designed, utilizing an attention mechanism module to dynamically calibrate channel feature weights, improving detail recovery capabilities. Furthermore, a loss function is designed and used in the model training process to overcome feature learning bias caused by foreground / background class imbalance. Ultimately, this achieves automatic and accurate extraction of open-pit mining area data.

[0038] To enable the constructed model to better learn vegetation features, a dual-encoder structure was designed for the network, collaboratively mining geometric, textural, and vegetation features through a primary and secondary dual-path approach. The primary encoder takes RGB (visible light band) images as input to extract geometric and textural features; the secondary encoder fuses NIR (near-infrared) and NDVI bands to focus on vegetation features. In the third and fourth stages, the two branches achieve multi-scale interaction (including 1x1 convolutional dimensionality reduction and spatial-positional attention enhancement) through a feature cross-fusion module. In the fifth stage, after feature concatenation, the ASPP module integrates multi-scale context using multi-diffraction dilated convolutions and global pooling. The decoder employs a four-level upsampling structure, dynamically weighting multi-level prediction results with a deep supervision mechanism, and uses an SE attention segmentation head for dual-channel feature recalibration. To address the class imbalance problem, a dynamic weighted hybrid loss function (cross-entropy + Dice) is proposed to balance global optimization and regional overlap measurement. This model achieves multi-dimensional fusion and accurate extraction of mining area features.

[0039] Appendix Figure 2 This is a schematic diagram of the network structure in an embodiment of the present invention. The network is a dual-branch semantic segmentation network with an encoder-decoder structure. RGB bands and NIR+NDVI are used as inputs to two encoders (the main encoder and the auxiliary encoder) in the feature encoder, respectively. These are upsampled by the same decoder to obtain the prediction result for the open-pit mine. In the workflow, the feature encoder first acquires key features of the open-pit mine, such as geometry, texture, spectrum, and vegetation. A feature cross-fusion module is embedded within the feature encoder to enhance feature fusion between the two encoders. The encoder downsampling result is input to the ASPP module to further mine the multi-scale contextual information extracted by the two encoders. Finally, the mined deep features are input to the feature decoder for upsampling, and the final prediction result for the mine is output after passing through an attention segmentation head.

[0040] The specific implementation of the network in this embodiment is described below: 1) The main encoder takes RGB bands from the remote sensing image as input to mine geometric and textural features. The auxiliary encoder takes near-infrared (NIR) bands from the remote sensing image and the calculated Normalized Difference Vegetation Index (NDVI) to learn the vegetation features of the mining area. Both the main and auxiliary encoders contain multiple encoding stages, with each encoder preferably having five encoding stages. The outputs of the third and fourth encoding stages are processed by a feature cross-fusion module before being input to the next stage. The downsampling results are then concatenated and input into Atrous Spatial Pyramid Pooling (ASPP) to further mine the multi-scale information extracted by the two encoders. Finally, the results are input into the decoder, which contains multiple decoding stages to gradually recover the model's extraction results of the mining area.

[0041] The encoder structure of the embodiment is shown in the attached figure. Figure 3 As shown, the main and auxiliary encoders are implemented as follows: The first stage of the main encoder (Encoder layer 1) consists of two convolutional layers (conv) and a max-pooling layer (MaxPool2d) set sequentially. The kernel size of the convolutional layers is 3×3, the window size of the max-pooling layer is 2, and the stride size is also 2. The input data size is 512×512×3 (3-channel RGB image), the number of output channels is 64, and the output size is 256×256×64.

[0042] The second encoding stage of the main encoder consists of two convolutional layers and one max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 256×256×64, the number of output channels is 128, and the output size is 128×128×128.

[0043] The main encoder's encoding stage 3 consists of three convolutional layers and one max pooling layer set sequentially. The convolutional kernel size is 3×3, the max pooling layer has a window size of 2 and a stride size of 2. The input data size is 128×128×128, the number of output channels is 256, and the output size is 64×64×256.

[0044] The main encoder's encoding stage 4 consists of three convolutional layers and one max pooling layer set sequentially. The convolutional kernel size is 3×3, the max pooling layer has a window size of 2 and a stride size of 2. The input data size is 64×64×256, the number of output channels is 512, and the output size is 32×32×512.

[0045] The fifth stage of the main encoder consists of three convolutional layers and one max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 32×32×512, the number of output channels remains unchanged at 512, and the output size is 16×16×512.

[0046] The first stage of the auxiliary encoder (Encoder layer 1) consists of two convolutional layers and a max pooling layer set sequentially. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 512×512×2 (2-channel NIR+NDVI image), the number of output channels is 64, and the output size is 256×256×64.

[0047] The second stage of the auxiliary encoder (Encoder layer 2) consists of two convolutional layers and a max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 256×256×64, the number of output channels is 128, and the output size is 128×128×128.

[0048] The third stage of the auxiliary encoder consists of three convolutional layers and one max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 128×128×128, the number of output channels is 256, and the output size is 64×64×256.

[0049] The fourth stage of the auxiliary encoder consists of three convolutional layers and one max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 64×64×256, the number of output channels is 512, and the output size is 32×32×512.

[0050] The fifth stage of the auxiliary encoder consists of three convolutional layers and one max pooling layer set in sequence. The convolutional kernel size is 3×3, the window size of the max pooling layer is 2, the stride size is also 2, the input data size is 32×32×512, the number of output channels remains unchanged at 512, and the output size is 16×16×512.

[0051] 2) Based on existing feature fusion technology, this invention improves upon existing technology by incorporating a feature cross-fusion module used in dual-branch encoders. Figure 2 The winning designation is F), and the structure of the feature cross-fusion module is as follows: Figure 4 As shown. The feature cross-fusion module is implemented based on an attention mechanism. First, unlike existing technologies that use global pooling, this invention utilizes two adaptive average pooling methods to pool the input feature tensor along the width and height directions respectively. Decomposed into two directional sensing feature maps and This allows us to obtain global statistics in the width and height directions, respectively.

[0052] Subsequent processing is similar to existing technologies. For ease of implementation and reference, the implementation method is described below: Direction-aware feature map and obtained by splicing Then, spatial attention weights are generated through nonlinear transformation. ,inr This represents the compression ratio, which balances model complexity and feature representation capability by controlling the degree of compression of intermediate feature channels. The mathematical expression is:

[0053] f The intermediate feature that contains both horizontal and vertical spatial information is called... ; Represents the ReLU activation function, which enhances the nonlinear expressive power; This represents the weight matrix of the fully connected layer; This indicates the splicing of spatial dimensions.

[0054] Then Decomposed into height weights along the spatial dimension and width weight Spatial attention maps are generated using independent convolutional layers and the sigmoid function, respectively. and width weight The mathematical expression is:

[0055] in This represents the Sigmoid function, which maps weights to the interval [0,1].

[0056] Finally, attention weights are applied to the original input features through element-wise multiplication. By decoupling spatial orientation dependencies, this module enables fine-grained position awareness, accurately locates the geometric structure of irregular targets, and achieves complementary fusion of deep features from the dual-branch encoder.

[0057] 3) The features from the third output of the main encoder and the third output of the auxiliary encoder are fused using the feature cross-fusion module described in 2). The fusion result is then input into the fourth stage of both the main encoder and the auxiliary encoder for downsampling. Similarly, the features from the fourth output of the main encoder and the downsampled results from the fourth output of the auxiliary encoder are fused using the feature cross-fusion module and then input into the fifth stage of both the main encoder and the auxiliary encoder for further downsampling. The features from the fifth output of the main encoder and the fifth output of the auxiliary encoder, along with the features fused by the feature cross-fusion module, are concatenated and input into the Atrous Spatial Pyramid Pooling (ASPP) module to further capture the multi-scale contextual information extracted by the two encoders. The ASPP module structure set in the embodiment is as follows: Figure 5As shown, the dilated spatial convolutional pooling pyramid module includes multiple branches set in parallel, including a convolutional branch at the original resolution, dilated convolutional branches with different dilation rates, and a global pooling branch; the outputs of each branch are concatenated along the channel dimension, and feature fusion is performed through convolutional layers. Specifically, it includes the following parts: The 1x1 convolution branch, which includes sequentially set 1×1 convolutional layers (conv), batch normalization (BN), and activation function ReLU, is used to directly reduce or transform the input features while preserving the original resolution information. The multi-dilution rate dilated convolution branch contains three branches, which use 3×3 dilated convolutions with dilation rates of 6, 12, and 18, respectively. The purpose is to expand the receptive field by different dilation rates to capture multi-scale local contextual information (small dilation rate focuses on details, while large dilation rate captures long-range dependencies). Specifically, the three branches include 3×3 convolutional layers (conv) and BN and ReLU layers set sequentially, but with different dilation rates.

[0058] The global pooling branch includes a global average pooling layer, a 1×1 convolutional layer (conv), and an upsampling layer, which are set in sequence. Its function is to obtain image-level global context information through global average pooling and then upsample to restore spatial resolution. Feature concatenation and fusion involves concatenating the outputs of the five branches along the channel dimension, and then performing channel fusion through a 1×1 convolution (here, a 1×1 convolutional layer conv, batch normalization BN, activation function ReLU, and upsampling layer are set in sequence) to obtain the output layer, which integrates multi-scale features to generate a unified contextual representation.

[0059] 4) The upsampling stage of the open-pit mining area extraction model consists of three core parts: decoder, depth supervision, and segmentation head (the overall structure of the upsampling stage is as follows). Figure 6 (As shown), the specific content is as follows: The decoder consists of four decoding layers, from Decoder Layer 4 to Decoder Layer 1, progressively restoring the resolution of the feature maps and fusing them with the skip connections of the encoder. Each Decoder Layer includes upsampling, feature fusion, and convolution processing. Upsampling uses a transpose convolution (2×2) to double the feature map resolution. The upsampled features are then skip-connected with the features of the corresponding encoder layers. This includes adding the outputs of MainEncoder1 and Auxiliary Encoder1 to obtain output Fused1, which is then skip-connected to Decoder Layer 1; adding the outputs of Main Encoder2 and Auxiliary Encoder2 to obtain output Fused2, which is then skip-connected to Decoder Layer 2; fusing the outputs of Main Encoder3 and Auxiliary Encoder3 to obtain output Fused3, which is then skip-connected to Decoder Layer 3; and fusing the outputs of Main Encoder4 and Auxiliary Encoder4 to obtain output Fused4, which is then skip-connected to Decoder Layer 2. Layer 4; the features after skip connections are further refined by two convolutional layers Conv3×3 (including batch normalization BN and ReLU); finally, final upsampling is performed.

[0060] Deep Supervision Module: To alleviate the vanishing gradient problem and improve the model's ability to model multi-scale features, a multi-level deep supervision mechanism is introduced. Intermediate prediction results are generated at multiple levels of the decoder, and the multi-level outputs are fused using adaptive weights. In the example, intermediate prediction maps DS_1 to DS_4 are generated using 1×1 convolutional layers in Decoder layer 4 to Decoder layer 1, respectively. These are then upsampled to the same spatial resolution as the original labels using bilinear interpolation. The loss for each intermediate prediction result is calculated, and the final segmentation result is a weighted sum of the predictions from each layer. Softmax normalization is used to ensure the weight sum is 1. The final loss is a weighted sum of the main output loss Loss out and the intermediate supervision losses Loss 1 to Loss 4.

[0061] 5) In the final segmentation stage of the last upsampling layer, to further suppress noise interference and improve the segmentation accuracy of open-pit mine areas, this invention designs an attention segmentation head (MASH). The structure of the attention segmentation head in the embodiment is as follows: Figure 7 As shown: MASH connects two layers of Squeeze-and-Excitation attention modules (SE), which act on the input features and intermediate features respectively, forming a progressive attention optimization mechanism to improve the robustness of feature representation. An intermediate convolutional layer is set between the two SE modules.

[0062] Each SE attention module consists of a Squeeze layer, an Excitation layer, and a Scale layer, set sequentially. The Squeeze layer uses global average pooling to directly compress the original feature map X, which contains global information, into a 1×1×C feature vector Z, in order to fuse local information to obtain global information.

[0063] The excitation operation is used to capture channel dependencies and consists of two fully connected layers (FC), a ReLU activation function, and a cascaded sigmoid activation. The first FC compresses C channels into C / r channels to reduce computation; ReLU adds a non-linear transformation between the two fully connected layers, enabling the model to capture complex dependencies between channels. The second fully connected layer restores the number of channels to C, and then uses sigmoid activation to obtain the weights S, where the dimension of S is... .

[0064] The role of the scale layer is to apply the obtained attention weights to the features of each channel. Specifically, the original feature X is multiplied by the channel weight S channel by channel to enhance the response of important channels, resulting in the final output of the first SE module. .

[0065] Output of the first-level SE module As input to the intermediate convolutional layers of the attention segmentation head, 3×3 convolutional layers are used in the intermediate convolutional layers to preserve spatial details, while normalization and non-linear activation are used to improve the stability of the model. The output result... . Will The second-layer SE module further refines the channel weights to obtain the final attention-weighted feature map. Finally, a 1×1 convolutional layer maps the channel features to the number of target classes, generating a pixel-wise classification probability map. The mathematical expression for MASH is:

[0066] in, This represents the final pixel-by-pixel classification probability map. Represents a 1×1 convolution. and These represent the first and second layer SE attention modules, respectively. B represents the ReLU activation function, and B represents batch normalization. This represents a 3×3 convolution, where X represents the original input feature.

[0067] Based on the aforementioned network structure, this invention constructs a Dynamic Weighted Hybrid Loss (DWH) function to overcome the feature learning bias caused by the imbalance between foreground (open-pit mine) and background (other land features) in the data. The DWH Loss mainly consists of three parts: dynamic class weights, an improved Cross-Entropy Loss, and an improved Dice Loss (overlap metric loss). Specifically, it combines Cross-Entropy Loss and regional overlap metric loss, and dynamically adjusts the weights according to the real-time class distribution to balance the contributions of the foreground and background.

[0068] First, dynamic class weights are calculated based on the class distribution within each batch size. Specifically, the class weight is calculated using inverse frequency based on the number of pixels in each class. The calculation formula is as follows:

[0069] in Represents the number of pixels in a certain category. This indicates the strength of the weight adjustment; the default value is 0.5, representing a medium-strength adjustment. This represents the inverse frequency weight for a certain category. Since the number of pixels in each category is random in each batch size image, the calculated inverse frequency weights need to be normalized to avoid weight scaling out of control.

[0070] in, and These represent the inverse frequency weights of the background pixels and the open-pit mine pixels, respectively. These are the inverse frequency weights after normalization for class c. It should be noted that the dynamic adjustment of the weights means that they are updated in real time based on the class distribution for each batch size. Using inverse frequency to calculate the weights allows the model to focus more on land cover features with a lower proportion of classes in the data, overcoming the class imbalance problem.

[0071] Secondly, a dynamically weighted cross-entropy loss is constructed based on the cross-entropy loss. Cross-entropy loss is currently widely used in semantic segmentation tasks; combining it with the dynamic class weights constructed in this invention can, to some extent, compensate for its sensitivity to class imbalance. The mathematical expression for the dynamically weighted cross-entropy loss (DWCE) is:

[0072] Where c represents the category, c = 0 indicates the category is background, and c = 1 indicates the category is open-pit mine. Indicates the first i The normalized weights of class c in the sub-image, where N represents the number of images and CE represents the standard cross-entropy loss. and They represent the first i The true label and prediction results of the sub-image.

[0073] To further improve the model's performance in addressing feature learning bias caused by class imbalance during training, a Dynamic Weighted Dice Loss (DWD) is constructed based on Dice Loss. Its mathematical formula is as follows:

[0074] Here, Dice represents the standard Dice Loss.

[0075] Ultimately, the DWH Loss can be expressed as:

[0076] in To control and The weights are between 0 and 1, and their optimal weights are determined through multiple experiments based on the accuracy evaluation index.

[0077] To facilitate understanding of the technical effects of this invention, based on the automatic extraction method for open-pit mining areas based on convolutional neural networks and attention mechanisms provided in the above embodiments, specific experimental data is provided, including the following steps: S1. Construct a dataset of open-pit mining areas: An Open-pit Mining Areas Dataset (OPMSet) was built based on 115 scenes of Gaofen-2 multispectral data. The dataset covers open-pit mining areas in 6 regions across 5 continents, including coal mines and other metal mines.

[0078] The acquired Gaofen-2 image is an L1A grade product, requiring preprocessing including radiometric calibration, atmospheric correction, and orthorectification. The preprocessed image includes four bands: Red, Green, Blue, and NIR, with a spatial resolution of 4 meters. Since there is a clear vegetation boundary between the open-pit mine and the unexcavated surface, this invention calculates the Normalized Difference Vegetation Index (NDVI) for each image and overlays it with the preprocessed multispectral data to obtain multispectral data with five bands: Red, Green, Blue, NIR, and NDVI, thus providing more vegetation features. Because the open-pit mine covers a large area in the image, to ensure the accuracy of label creation, this invention uses visual interpretation on the ArcGIS platform to create sample labels, ultimately constructing 2026 TIFF format samples with a size of 512*512. Some samples from the OPMSet dataset are shown below. Figure 8 As shown.

[0079] PMSet has the following characteristics: 1) High spatial resolution: The data has a spatial resolution of 4 meters, which can better depict the details of the mining area. 2) Significant multispectral characteristics: It retains the spectral characteristics of remote sensing data and incorporates vegetation features in combination with the characteristics of the target land cover, making the dataset more feature-rich. 3) Scene diversity and target diversity: The dataset includes samples from different climatic environments, different geological conditions, and different mineral types, which helps to improve the model's generalization ability.

[0080] S2. Experimental parameter settings: All experiments in this invention were performed on a desktop computer with two Intel(R) Xeon(R) Gold 5118 CPUs, 64.0 GB of memory, and an NVIDIA GeForce RTX 4090 D graphics card, running Windows 10 Professional. All computations were performed using Python (v3.12.9) on the PyCharm 2025.1 platform. Both the model built for this experiment and the comparison model used the PyTorch (v.2.5.1) deep learning framework.

[0081] The initial learning rate was set to 0.001, and cosine annealing was used to dynamically adjust the learning rate. The warm-up step size was 20 epochs, the first epoch step size was 100 epochs, the doubling factor for each epoch was set to 1.5, the minimum learning rate was 1e-5, the maximum learning rate was 0.001, and the decay coefficient gamma was set to 0.98. The epoch and batch size were set to 600 and 10, respectively, and the Adam optimizer was used to optimize the network parameters. Furthermore, in the constructed loss function, experiments showed that setting alpha to 0.86 and power to 0.5 yielded the best results in overcoming feature learning bias caused by foreground-background imbalance in the data. The 2026 samples were divided into training, test, and validation sets in an 8:1:1 ratio, with 1620 samples in the training set, and 203 samples each in the test and validation sets. To improve the robustness of the training results, data augmentation operations are used during the data loading process. Specifically, a random flip operation is added, which flips the image and its corresponding label horizontally or vertically with a 50% probability; a random rotation operation is added, which rotates the image and its corresponding label randomly by 0°, 90°, 180° or 270°, with the rotation axis being the height and width of the image.

[0082] S3. Comparison Method: To verify the effectiveness of this invention, it was compared with 10 representative methods, including classic semantic segmentation networks and semantic segmentation networks for remote sensing images. Based on their technical characteristics, the compared methods can be divided into three categories, specifically including classic semantic segmentation models based on convolutional neural networks: the UNet semantic segmentation network with a typical encoder-decoder structure; the classic fully convolutional semantic segmentation network FCN; the Deeplabv3+ semantic segmentation network with dilated convolution and spatial pyramid pooling; SegNet, which uses depooling in the decoder to upsample feature maps and maintain the integrity of high-frequency details during segmentation; and the PSPNet pyramid scene parsing network, which integrates global contextual information based on pyramid pooling modules. Semantic segmentation models combining convolutional neural networks and Transformers include: UNetFormer, a remote sensing image semantic segmentation network combining UNet and Transformer; SegFormer, which combines Transformer with a multilayer perceptron decoder; and DCSwin, a remote sensing image semantic segmentation network that introduces the Swing Transformer as a backbone to extract contextual information and designs a densely connected feature aggregation module (DCFAM) decoder. Semantic segmentation models coupling convolutional neural networks and attention mechanisms: MANet, a remote sensing image semantic segmentation network based on multi-attention; and A2FPN, a remote sensing image semantic segmentation network based on feature pyramid network (FPN) and attention aggregation module. Among them, UNetFormer, DCSwin, MANet, and A2FPN are all networks proposed in recent years for remote sensing image semantic segmentation. Comparing with these models better highlights the performance of the proposed OMENet in extracting open-pit mine areas from high-resolution remote sensing images.

[0083] S4. Input the dataset into the "Automatic Extraction Method for Open-Pit Mining Areas Based on Convolutional Neural Network and Attention Mechanism" for training: the main encoder inputs a 3-channel RGB image, the auxiliary encoder inputs a 2-channel "NIR+NDVI" image, and the two branches interact with each other by utilizing the constructed feature cross-fusion module. Based on their respective weights, they share weights to fully mine the spectral feature information contained in the high-resolution remote sensing image.

[0084] The downsampling results of the dual-branch encoder are input into the Atrous Spatial Pyramid Pooling (ASPP). ASPP can fully integrate the high-level features of the dual-branch encoder. At the same time, this module has multi-scale perception capabilities and can mine hidden features from contextual information.

[0085] Finally, the data is input to the decoder. In the decoder, the upsampling layer consists of deconvolution and convolution, and a depth supervision mechanism and skip connections are added to gradually restore the image dimensions and resolution. In the final segmentation stage of the last upsampling layer, in order to further suppress noise interference and improve the segmentation accuracy of open-pit mine areas, this invention designs a mine area attention segmentation head, ultimately obtaining high-precision open-pit mine extraction results.

[0086] S5. Results of open-pit mine extraction: To evaluate the performance of this invention, a comparative experiment was conducted on the constructed 5-channel OPMSet dataset, comparing this invention with 10 classic semantic segmentation networks and semantic segmentation networks for remote sensing images. To ensure the accuracy of the experimental results, all networks were trained on the same server using the same experimental parameter settings. Table 1 shows the performance of this invention and the 10 semantic segmentation methods.

[0087] Table 1. Performance comparison of the present invention with 10 semantic segmentation methods

[0088] As shown in Table 1, this invention outperforms the other 10 comparative methods across all seven accuracy evaluation metrics. Specifically, it improves IOU by 0.72%-4.29%, FWIOU by 0.44%-2.6%, Precision by 0.21%-2.0%, Recall by 0.06%-2.68%, F1 quantile by 0.38%-2.34%, OA by 0.23%-1.38%, and Kappa by 0.55%-3.32%. This indicates that the invention exhibits excellent overall performance. Furthermore, among the 10 comparative methods, MANet generally outperforms the other nine, while DCSwin performs relatively poorly.

[0089] (2) Comparison of prediction results To visually demonstrate the performance differences between the models, this invention uses the trained model and seven models with good accuracy to predict the "validation" data in OPMSet. Some prediction results are shown below. Figure 9 As shown. To visually illustrate the differences in prediction results between different models, Figure 9 In the prediction results of each model, the green and red portions represent the pixels that were "missed" and "false positives" relative to the true labels, respectively. It can be concluded that the prediction results of this invention have relatively fewer pixels with "missed" and "false positives," resulting in better visual performance than the other comparative models. Figure 9 The prediction results in (a) show that the model can adapt to open-pit mining under conditions of "foreground-background" category imbalance; Figure 9 (b) andFigure 9 (d) It can be seen that the model can meet the extraction task of open-pit mine under complex surface cover conditions; Figure 9 (f) performed relatively poorly, mainly because the mining area in the image is mainly a spoil heap, and the vegetation in both the spoil heap and the background is growing well, which to some extent led to the model's misjudgment.

[0090] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0091] The following embodiments describe the electronic device provided by the present invention. The electronic device described below can be referred to in correspondence with the automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms described above.

[0092] The electronic device may include a processor, a communications interface, memory, and a communication bus, wherein the processor, communications interface, and memory communicate with each other via the communication bus. The processor can call logical instructions in the memory to execute an automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms, mainly including the software processing part mentioned above.

[0093] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] On the other hand, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the software processing portion of the open-pit mine automatic extraction method based on convolutional neural networks and attention mechanisms provided by the above methods.

[0095] In another aspect, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the software processing portion of the open-pit mine automatic extraction method based on convolutional neural networks and attention mechanisms provided by the above methods.

[0096] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0097] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms, characterized in that, Includes the following processes, Data preprocessing was performed to construct multispectral data containing vegetation indices and multiple bands; An open-pit mining area extraction model is set up. In the open-pit mining area extraction model, a dual-branch encoder structure is constructed. The main encoder in the dual-branch encoder structure inputs the visible light band of the remote sensing image to extract geometric and texture features, and the auxiliary encoder fuses the near-infrared band and vegetation index to extract vegetation features. The high-level features output by the dual-branch encoder are input into the dilated spatial convolutional pooling pyramid module to integrate multi-scale contextual information; a decoder with a deep supervision mechanism is constructed, which fuses the multi-level features of the encoder through skip connections and gradually restores the spatial resolution by cascaded upsampling, and an attention segmentation head is set at the end of the decoder; A dynamic weighted hybrid loss function is used to train an open-pit mining area extraction model, and the automatic extraction results of the open-pit mining area are output based on the trained open-pit mining area extraction model.

2. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 1, characterized in that: The intermediate stage of the dual-branch encoder structure embeds a feature cross-fusion module to enhance the interaction capability of dual-branch features.

3. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 2, characterized in that: The feature cross-fusion module uses adaptive average pooling to decompose the input feature tensor into perceptual feature maps along the width and height directions, respectively, thereby obtaining global statistics in the width and height directions.

4. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 1, characterized in that: The dilated spatial convolutional pooling pyramid module includes multiple branches set in parallel, including the original resolution convolution branch, dilated convolution branches with different dilation rates, and a global pooling branch; the outputs of each branch are concatenated along the channel dimension and feature fusion is performed through convolutional layers.

5. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 1, characterized in that: The operation of the deep supervision mechanism includes generating intermediate prediction results at multiple levels of the decoder and fusing multi-level outputs through adaptive weights.

6. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 1, characterized in that: The attention segmentation head comprises two layers of cascaded channel attention modules.

7. The automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms according to claim 1, characterized in that: The dynamic weighted hybrid loss function combines cross-entropy loss and regional overlap metric loss, and dynamically adjusts the weights according to the real-time category distribution to balance the contributions of the foreground and background, wherein the foreground is an open-pit mine.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the automatic extraction method for open-pit mines based on convolutional neural networks and attention mechanisms as described in any one of claims 1 to 7.

Images