Weakly supervised open pit identification method based on adaptive SAM and automatic knowledge learning

Abstract

This invention discloses a weakly supervised open-pit mining site identification method based on adaptive SAM and automatic knowledge learning. The method includes: S1, collecting images of the study area and extracting three-band images using an input spectral adapter; extracting depth features using the feature extractor of the MobileSAM model; constructing a multi-scale perception and geometric adaptation module and obtaining image features by applying N² dilated convolutional units with different dilation rates to the depth features; S2, a scene classifier using predefined prototype features to guide image features in scene classification; then, a cue generator using class activation mapping to obtain a heatmap for scene classification and selecting a set of high-confidence cue points; and a pseudo-label generator constructing triplet contrast constraints; S3, the mask decoder of the MobileSAM model combining cue information and image features to perform cue decoding and obtain the open-pit mining site identification result. This invention combines a set of high-confidence cue points with image features for cue decoding to obtain efficient and high-precision open-pit mining site identification results.

Description

Technical Field

[0001] This invention relates to the field of open-pit mine identification technology, and in particular to a weakly supervised open-pit mine identification method based on adaptive SAM and automatic knowledge learning. Background Technology

[0002] While coal mining drives economic development, it also brings serious ecological and environmental problems, often resulting in illegal activities such as mining beyond permitted boundaries, mining under the guise of exploration, and overlapping mining rights. Therefore, conducting dynamic monitoring of open-pit mining, especially timely determination of the location and boundaries of open-pit mines, is of great significance for comprehensively understanding mining activities, assessing their impact on the ecological environment, and identifying illegal mining activities.

[0003] In recent years, with the rapid development of deep learning and remote sensing technologies, their integrated application in the field of ground object recognition has become increasingly widespread, including research on automatic recognition in open-pit mines. Related research has, to some extent, alleviated the burden of manpower and resources and the problem of delayed updates caused by traditional manual field surveys and visual interpretation methods, and also compensated for the shortcomings of traditional machine learning methods in recognition accuracy. In recent years, fundamental large-scale models represented by BERT and ChatGPT have triggered research revolutions in various fields. In the field of computer vision, Meta's Segment Anything Model (SAM) (developed by Facebook's Meta AI Lab. It is a deep learning-based image segmentation model that can recognize and segment any object based on text instructions or image recognition, comparable to GPT in the field of image segmentation) achieves flexible and efficient object segmentation by introducing cue mechanisms such as point annotation and bounding boxes, providing a new solution for ground object recognition. Especially under weakly supervised learning strategies, SAM's powerful recognition capabilities demonstrate broad application prospects. However, current research on applying SAM to weakly supervised ground feature recognition still faces several challenges, specifically: 1) Structural separation problem: Some existing methods do not combine SAM with scene classification networks to collaboratively generate pseudo-labels, resulting in relatively simple feature extraction from remote sensing images and an inability to obtain rich feature information; 2) Lack of prompting mechanism problem: Some existing methods directly use SAM to replace traditional segmentation networks in ground feature recognition processing without involving or utilizing prompting mechanisms, making it difficult to leverage the original advantages of SAM; 3) Adaptability problem: In remote sensing scenarios, SAM faces the dual challenges of spectral adaptation and pattern adaptation. On the one hand, SAM is mainly trained on RGB natural images, while remote sensing images often contain richer spectral information (such as near-infrared bands), and directly discarding them will lead to the loss of key information; on the other hand, open-pit mines have complex multi-scale features and irregular shapes, and existing SAM models cannot effectively model such patterns. In summary, there is an urgent need to design a weakly supervised open-pit mine identification method that can fully utilize SAM capabilities, comprehensively mine semantic knowledge of complex open-pit mines, and has strong generalization ability, so as to improve identification accuracy and application efficiency. Summary of the Invention

[0004] The purpose of this invention is to solve the technical problems pointed out in the background art and provide a weakly supervised open-pit mine identification method based on adaptive SAM and automatic knowledge learning. This method effectively reduces the dependence on high-cost pixel-level labeled data. Through image-level semantic learning identification, it uses a set of high-confidence point prompts to guide the capture of the overall semantic structure of the open-pit mine, thereby extracting its semantic prototype or discriminative features. It adopts a pseudo-label generation branch and a ground feature identification branch. The former generates pseudo-labels with the help of scene classification network and class activation mapping (CAM) technology, while the latter uses these pseudo-labels to train a semantic segmentation network, realizing knowledge sharing and collaborative optimization, and finally achieving ground feature identification of the open-pit mine, thus improving the identification accuracy of the open-pit mine.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A weakly supervised open-pit mining site identification method based on adaptive SAM and automatic knowledge learning, the method comprising:

[0007] S1. Collect N1 images of the study area and input them into the spectral adapter to extract N1 three-band images. Use the feature extractor of the MobileSAM model to extract depth features. Construct a multi-scale perception and geometric adaptation module and extract and correlate multi-scale information and geometric perception information through N2 dilated convolutional units with different dilation rates to obtain image features. ;

[0008] S2. Construct an automatic knowledge learning model that includes a scene classifier, a cue generator, and a pseudo-label generator. The scene classifier utilizes predefined prototype features. Guided image features The system classifies scenarios based on the presence or absence of open-pit mines. Then, a prompt generator uses class activation mapping to obtain a heatmap for the scene classification and filters a set of high-confidence prompts, which includes a foreground category feature set. and background category feature set The pseudo-label generator constructs a set of foreground category features. Background category feature set and prototype features Triple comparison constraints;

[0009] The S3 and MobileSAM model's cue encoder uses a set of high-confidence cue points to encode cue information, while the MobileSAM model's mask decoder combines the cue information with image features. The open-pit mine identification result is obtained by performing prompt decoding.

[0010] To better implement the present invention, in method S1, each of the N1 images contains four band channels: red, green, blue, and near-infrared. The spectral adapter includes a variational autoencoder (VAE). The VAE extracts features and semantic information from the images and compresses them into three band channels: red, green, and blue to obtain three-band images.

[0011] Preferably, the variational autoencoder (VAE) includes a VAE encoder, a latent space sampler, and a VAE decoder. The VAE encoder compresses the images from four band channels into a shallow latent space. The probability distribution parameters are further compressed and mapped to the deep latent space, and the latent space sampling samples latent variables from a Gaussian distribution; the VAE decoder reconstructs the shallow latent space based on the latent variables. The final output is a three-band image with red, green and blue channels.

[0012] Preferably, in method S1, the feature extractor of the MobileSAM model uses the lightweight visual transformer TinyViT to extract depth features from N1 three-band images.

[0013] Preferably, in method S1, the multi-scale perception and geometric adaptation module processes the depth features through five dilated convolutional units with different dilation rates. r represents the dilation rate, and dilated convolutional unit. Five different dilation rates r are set to 1, 3, 6, 12, and 18. These five dilated convolutional units form five branches arranged vertically. Each branch extracts contextual information from the depth features. The contextual information extracted by the dilated convolutional unit in one branch is then linked to the unprocessed features in the next branch for concatenation and fusion. Each of the five branches has a deformable convolutional module (DC) connected to its back end. These DCs learn the spatial offset o and importance weight w. The DC from one branch is passed to the next branch for concatenation and fusion of the spatial offset o and importance weight w. The outputs of the five DCs are then concatenated and stitched together to obtain the image features. .

[0014] Preferably, in method S2, the predefined prototype features of the scene classifier At the outset, predefined settings are implemented. The scene classifier includes a multi-head self-attention mechanism (MHSA) and a bidirectional cross-attention mechanism. The bidirectional cross-attention mechanism includes a multi-head cross-attention mechanism (PtoF-MHCA) and a multi-head cross-attention mechanism (PtoP-MHCA). The MHSA mechanism is used to process prototype features. Enhanced and input multi-head cross-attention mechanism PtoF-MHCA, which utilizes image features Optimize and enhance prototype features The head-to-P-MHCA attention mechanism utilizes optimized prototype features. Optimize image features Global average pooling and fully connected operations are used to process and optimize image features. Classify the scenarios.

[0015] Preferably, in method S2, the prompt generator provides information on the N1 image features. Scene classification utilizes class activation mapping to obtain heatmaps containing N1 foreground categories and N1 background categories. Filtering values ​​are set, including foreground and background filter values. High-confidence foreground and background category regions are selected from the heatmaps of both categories. Random selections are then made from these high-confidence foreground and background category regions. Foreground category sample points and The background category sample points are used as high-confidence point cues and average pooling is used for sampling. At the same time, the largest inscribed rectangle within the high-confidence foreground category region is constructed as a high-confidence cue box. The high-confidence point cues and high-confidence cue boxes are combined to obtain the high-confidence point cue set.

[0016] Preferably, in method S2, the pseudo-label generator employs a triplet contrast constraint and makes the prototype features To foreground category features Distance less than background category features And obtain the constrained prototype features. Utilizing constrained prototype features With image features A feature similarity map is generated and a threshold is set to obtain refined pseudo-labels. A total of N1 refined pseudo-labels are obtained, and the mean is fused to obtain the final refined pseudo-labels.

[0017] Preferably, in method S1, N1 is five, and five images of the same location in the study area are collected and input into the spectral adapter as a group, or one image of the study area is collected and expanded twice with weak perturbation and twice with strong perturbation to obtain four expanded images, which together constitute five images and input into the spectral adapter.

[0018] Preferably, a sample dataset is created, which includes image samples and corresponding label data. The image samples are images that have undergone image preprocessing, including radiometric calibration, atmospheric correction, orthorectification, and image fusion. The image samples after image preprocessing are cropped image blocks, which include two types: image blocks containing open-pit mines and image blocks not containing open-pit mines. Each image sample consists of a group of five image blocks, which includes one original image block and four expanded image blocks. The four expanded image blocks are obtained by expanding the original image block twice with weak perturbation and twice with strong perturbation, respectively. The sample dataset is used for learning and training according to methods S1 to S3.

[0019] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0020] (1) This invention utilizes prototype features trained with multiple constraints through a trained scene classifier. Guided image features This invention achieves scene classification and uses class activation mapping to obtain heatmaps and selects a set of high-confidence point prompts. It then uses the MobileSAM model and multi-scale perception and geometric adaptation modules to extract and correlate multi-scale information, geometric perception information, and features to obtain image features. Combining the set of high-confidence point prompts with the image features, it performs prompt decoding to obtain high-precision open-pit mining identification results. This invention can capture the overall and comprehensive semantic structure of open-pit mining, and the set of high-confidence point prompts can guide the learning of more comprehensive semantic prototypes and discriminative features.

[0021] (2) This invention compresses remote sensing images into three-band images with three channels through a spectral adapter, and uses the MobileSAM model to extract depth features. The multi-scale perception and geometric adaptation module uses multiple dilated convolutional units with different dilation rates to fully extract multi-scale information and geometric perception information, thus fully extracting the comprehensive feature information of the image. The automatic knowledge learning model realizes the generation of high-confidence prompts and pseudo-labels, which can fully adapt to multi-scale features and complex geometric structures, and generate a set of high-confidence point prompts and high-confidence pseudo-labels, providing an efficient and accurate identification method and data support for open-pit mining monitoring. Attached Figure Description

[0022] Figure 1 This is a flowchart of the weakly supervised open-pit mining identification method of the present invention;

[0023] Figure 2 This is a schematic diagram illustrating the principle of the weakly supervised open-pit mine identification method in the embodiment;

[0024] Figure 3 This is a schematic diagram illustrating the processing principle of the variational autoencoder in the embodiment;

[0025] Figure 4 This is a schematic diagram illustrating the processing principle of the multi-scale perception and geometric adaptation module in the embodiment;

[0026] Figure 5 This is a schematic diagram illustrating the processing principle of the deformable convolution module DC in the embodiment.

[0027] Figure 6 This is a schematic diagram illustrating the processing principle of the scene classifier in the embodiment. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to embodiments:

[0029] Example

[0030] like Figure 1 , Figure 2 As shown, a weakly supervised open-pit mine identification method based on adaptive SAM and automatic knowledge learning is proposed, the method comprising:

[0031] S1. Collect N1 images of the study area and input them into the spectral adapter to extract N1 three-band images. In this embodiment, N1 is typically five. Five images from the same location within the study area are collected and input into the spectral adapter as a group, or one image from the study area is collected, augmented twice with weak perturbations, and then augmented twice with strong perturbations to obtain four augmented images, which are then combined to form a group of five images input into the spectral adapter. In some embodiments, such as... Figure 3 As shown, each of the N1 images contains four band channels: red, green, blue, and near-infrared. The spectral adapter includes a variational autoencoder (VAE). The VAE extracts feature and semantic information from the images and compresses it into three band channels (red, green, and blue) to obtain a three-band image. A further preferred technical solution is that the VAE includes a VAE encoder, a latent space sampling module, and a VAE decoder. The VAE encoder compresses the images from the four band channels into a shallow latent space. The probability distribution parameters are further compressed and mapped to the deep latent space, and the latent space sampling samples latent variables from a Gaussian distribution; the VAE decoder reconstructs the shallow latent space based on the latent variables. The final output is a three-band image with red, green, and blue channels. Preferably, during the pre-training phase, the variational autoencoder (VAE) uses the shallow latent space in the VAE encoder and VAE decoder (…). and The reconstruction process and the KL divergence loss between the probability distribution parameters of the deep potential space and the standard normal distribution are optimized and trained. Then, fine-tuning is performed in the weakly supervised open-pit mining identification framework to further improve the identification performance.

[0032] Depth features are extracted using the feature extractor of the MobileSAM model (the MobileSAM model of this invention includes a feature extractor, a cue encoder, and a mask decoder); in some embodiments, the feature extractor of the MobileSAM model uses the lightweight visual transformer TinyViT to extract depth features from N1 three-band images respectively (a total of N1 depth features corresponding to the three-band images are obtained).

[0033] A multi-scale perception and geometric adaptation module is constructed, and image features are obtained by extracting and correlating multi-scale information and geometric perception information from depth features through N2 dilated convolutional units with different dilation rates. Preferably, while processing with N2 dilated convolutional units of different dilation rates, a dynamic perception mechanism for ground feature boundaries and structural deformation can be introduced, which can effectively enhance the model's segmentation accuracy and generalization ability in complex mining scenarios. In some embodiments, the multi-scale perception and geometric adaptation module processes depth features through five dilated convolutional units of different dilation rates. r represents the dilation rate, and dilated convolutional unit. Five different dilation rates r are given, namely 1, 3, 6, 12, and 18. These five dilated convolutional units form five branches arranged vertically. Each branch extracts contextual information from the depth features. The contextual information extracted by the dilated convolutional unit in one branch is then connected to the unprocessed features in the next branch for concatenation and fusion. Each of the five branches has a deformable convolutional module DC connected to its back end. These DCs are used to learn the spatial offset o and importance weight w. The movement offset of each convolution kernel during the convolution operation was controlled. By controlling the sampling importance of each convolutional kernel, five deformable convolutional modules (DCs) dynamically adjust their receptive regions based on standard sampling positions, improving the selectivity of feature response intensity. The DCs from the previous branch are passed to the next branch and fused using spatial offsets (o) and importance weights (w). The outputs of the five branches are then correlated and stitched together to obtain image features. This invention employs a densely connected strategy to associate multi-scale information and geometrically sensed information between different branches. For example... Figure 5 As shown, a dilated convolutional unit with a dilation rate of r=1 and a deformable convolutional module DC constitute a branch, with a total of five branches, namely dilated convolutional units with a dilation rate of r=1. The first branch is formed by a dilated convolutional unit with a dilation rate r=3, which is combined with a deformable convolutional module DC. The second branch, consisting of a deformable convolutional module (DC), comprises dilated convolutional units with an dilation rate (r) of 6. The third branch, consisting of dilated convolutional units with an dilation rate r=12, is formed together with the deformable convolutional module DC. Together with the deformable convolutional module DC, it forms the fourth branch, a dilated convolutional unit with an dilation rate r=18. The fifth branch is formed with the deformable convolutional module DC. The first branch is a dilated convolutional unit. The extracted context information corresponds to a jump connection to the second branch of the undilated convolutional unit. The processed features are spliced ​​and fused, and then passed through two branch dilated convolutional units. Process and output; second branch dilated convolution unit The extracted context information corresponds to a jump connection to the third branch undilated convolutional unit. The processed features (features of the third branch are first compared with those of the second branch without dilated convolution units) After feature fusion, it is then combined with the second branch dilated convolution unit. The extracted context information corresponds to skip connections, which are then fused and passed through three branch dilated convolutional units. In the process of processing and output, the parts are spliced ​​and blended; according to Figure 4 The processing method follows the same logic, with the five branch dilated convolutional units corresponding to the splicing and fusion output. The first branch deformable convolutional module DC is passed to the second branch deformable convolutional module DC and spliced ​​and fused with spatial offset o and importance weight w, according to... Figure 4 The processing method follows the same principle: the five branch deformable convolutional modules (DCs) output their respective features, and the image features are obtained by feature association and stitching of the five branch deformable convolutional module DC outputs. .

[0034] S2. Construct an automatic knowledge learning model that includes a scene classifier, a cue generator, and a pseudo-label generator. The scene classifier utilizes predefined prototype features. Guided image features The system classifies scenarios based on the presence or absence of open-pit mines. Then, a prompt generator uses class activation mapping to obtain a heatmap for the scene classification and filters a set of high-confidence prompts, which includes a foreground category feature set. and background category feature set The pseudo-label generator constructs a set of foreground category features. Background category feature set and prototype features The triplet comparison constraint.

[0035] In some embodiments, such as Figure 6 As shown, the predefined prototype features of the scene classifier Predefined settings are made at the beginning. The scene classifier includes a multi-head self-attention mechanism (MHSA) and a bidirectional cross-attention mechanism. The bidirectional cross-attention mechanism (in this embodiment, the bidirectional cross-attention mechanism is used to implement image features) is used. With prototype features The deep fusion includes the multi-head cross-attention mechanism PtoF-MHCA and the multi-head cross-attention mechanism PtoP-MHCA, and utilizes the multi-head self-attention mechanism MHSA for prototype features. Enhanced and input multi-head cross-attention mechanism PtoF-MHCA, multi-head self-attention mechanism MHSA for prototype features Enhancement can be expressed as:

[0036] ;

[0037] ,in, (Right now ), (Right now )and (Right now ) represent the Query, Key, and Value matrices, respectively. For multilayer perceptrons, This is the SoftMax activation function. for The feature dimensions. The multi-head cross-attention mechanism PtoF-MHCA utilizes image features. Optimize and enhance prototype features The head-to-P-MHCA attention mechanism utilizes optimized prototype features. Optimize image features Global average pooling and fully connected operations are used to process and optimize image features. Classify scenarios, such as Figure 6 As shown, scene classification is finally achieved.

[0038] In some embodiments, the prompt generator provides five image features Scene classification utilizes class activation mapping to obtain heatmaps containing five foreground categories and five background categories (preferably, the Dempster-Shafer Evidence Fusion method is used to fuse the foreground and background heatmaps into unique foreground and background heatmaps, respectively). Filtering values ​​are set, including foreground and background filtering values. High-confidence foreground and background category regions are selected from the foreground and background heatmaps, respectively. Random selections are then made within these high-confidence foreground and background category regions. Foreground category sample points and A set of background category sample points is used as high-confidence point cues and average pooled for sampling (preferably, a local feature pooling sampling window is customized according to the purpose of this invention for local feature average pooling sampling). Simultaneously, the largest inscribed rectangle within the high-confidence foreground category region is constructed as a high-confidence cue box. The high-confidence point cues and the high-confidence cue boxes are combined to obtain a high-confidence point cue set, which includes the foreground category feature set. and background category feature set The number of features is indivual.

[0039] In some embodiments, the pseudo-label generator employs triple contrast constraints and makes the prototype features To foreground category features Distance less than background category features (make the prototype features) (more representative of the foreground category), and obtained constrained prototype features. Utilizing constrained prototype features With image features Generate a feature similarity map (which can be generated using cosine similarity calculation) and set a threshold to obtain refined pseudo-labels.

[0040] The S3 and MobileSAM model's cue encoder uses a set of high-confidence cue points to encode cue information (the cue information is denoted as...). The MobileSAM model's mask decoder combines cue information with image features. The prompt decoding yields the open-pit mine identification result, and the prompt information is then transmitted. With image features The mask decoder is input into the MobileSAM model. The mask decoder performs prompt decoding and generates open-pit mine identification results. The open-pit mine identification results are the results of open-pit mine identification in the study area (including the location and boundary range of the open-pit mine).

[0041] Methods S1 to S3 of this invention utilize a sample dataset for learning and training. First, a sample dataset is created, comprising image samples and corresponding label data. The image samples are images that have undergone preprocessing, including radiometric calibration, atmospheric correction, orthorectification, and image fusion. The preprocessed image samples are cropped image blocks, which fall into two categories: image blocks containing open-pit mines and image blocks not containing open-pit mines. Each image sample consists of a group of five image blocks, including one original image block and four expanded image blocks. The four expanded image blocks are obtained by expanding the original image block twice with weak perturbation and twice with strong perturbation, respectively. Learning and training are then performed using the sample dataset according to methods S1 to S3. The loss supervision employed during the learning and training process consists of four parts: cross-entropy loss for scene classification, triplet contrast loss for prototype feature learning, mean squared error loss of pseudo-labels obtained from transformed images, and cross-entropy loss for ground feature extraction. An Uncertainty Weighting strategy is used to optimize the weights of each loss value. Variational autoencoders (VAEs) use a shallow latent space during the pre-training phase. and The mean squared error loss and KL divergence loss are supervised. Hyperparameters such as the number of training iterations, learning rate, batch size, and optimizer are set, and multiple iterations of training are performed. In each iteration, the gradient descent algorithm is used to reduce the model loss value, while optimizing and updating the model parameters of this invention.

[0042] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A weakly supervised open-pit mining site identification method based on adaptive SAM and automatic knowledge learning, characterized in that: The methods include: S1. Collect N1 images of the study area and input them into the spectral adapter to extract N1 three-band images. Use the feature extractor of the MobileSAM model to extract depth features. Construct a multi-scale perception and geometric adaptation module and extract and correlate multi-scale information and geometric perception information through N2 dilated convolutional units with different dilation rates to obtain image features. The multi-scale perception and geometric adaptation module applies deep features through five dilated convolutional units with different dilation rates. r represents the dilation rate, and dilated convolutional unit. Five different dilation rates r are set to 1, 3, 6, 12, and 18. These five dilated convolutional units form five branches arranged vertically. Each branch extracts contextual information from the depth features. The contextual information extracted by the dilated convolutional unit in one branch is then linked to the unprocessed features in the next branch for concatenation and fusion. Each of the five branches has a deformable convolutional module (DC) connected to its back end. These DCs learn the spatial offset o and importance weight w. The DC from one branch is passed to the next branch for concatenation and fusion of the spatial offset o and importance weight w. The outputs of the five DCs are then concatenated and stitched together to obtain the image features. ; S2. Construct an automatic knowledge learning model that includes a scene classifier, a cue generator, and a pseudo-label generator. The scene classifier utilizes predefined prototype features. Guided image features The system classifies scenarios based on the presence or absence of open-pit mines. Then, a prompt generator uses class activation mapping to obtain a heatmap for the scene classification and filters a set of high-confidence prompts, which includes a foreground category feature set. and background category feature set The pseudo-label generator constructs a set of foreground category features. Background category feature set and prototype features Triple comparison constraints; The S3 and MobileSAM model's cue encoder uses a set of high-confidence cue points to encode cue information, while the MobileSAM model's mask decoder combines the cue information with image features. The open-pit mine identification result is obtained by performing prompt decoding.

2. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: In method S1, each of the N1 images contains four band channels: red, green, blue, and near-infrared. The spectral adapter includes a variational autoencoder (VAE). The VAE extracts features and semantic information from the images and compresses them into three band channels: red, green, and blue to obtain three-band images.

3. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 2, characterized in that: Variational autoencoders (VAEs) consist of a VAE encoder, a latent space sampler, and a VAE decoder. The VAE encoder compresses images from four band channels into a shallow latent space. The probability distribution parameters are further compressed and mapped to the deep latent space, and the latent space sampling samples latent variables from a Gaussian distribution; the VAE decoder reconstructs the shallow latent space based on the latent variables. The final output is a three-band image with red, green and blue channels.

4. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: In method S1, the feature extractor of the MobileSAM model uses the lightweight visual transformer TinyViT to extract depth features from N1 three-band images.

5. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: In method S2, the predefined prototype features of the scene classifier At the outset, predefined settings are implemented. The scene classifier includes a multi-head self-attention mechanism (MHSA) and a bidirectional cross-attention mechanism. The bidirectional cross-attention mechanism includes a multi-head cross-attention mechanism (PtoF-MHCA) and a multi-head cross-attention mechanism (PtoP-MHCA). The MHSA mechanism is used to process prototype features. Enhanced and input multi-head cross-attention mechanism PtoF-MHCA, which utilizes image features Optimize and enhance prototype features The multi-head cross-attention mechanism PtoP-MHCA utilizes optimized prototype features. Optimize image features Global average pooling and fully connected operations are used to process and optimize image features. Classify the scenarios.

6. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: In method S2, the prompt generator provides N1 image features. Scene classification utilizes class activation mapping to obtain heatmaps containing N1 foreground categories and N1 background categories. Filtering values ​​are set, including foreground and background filter values. High-confidence foreground and background category regions are selected from the heatmaps of both categories. Random selections are then made from these high-confidence foreground and background category regions. Foreground category sample points and The background category sample points are used as high-confidence point cues and average pooling is used for sampling. At the same time, the largest inscribed rectangle within the high-confidence foreground category region is constructed as a high-confidence cue box. The high-confidence point cues and high-confidence cue boxes are combined to obtain the high-confidence point cue set.

7. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1 or 6, characterized in that: In method S2, the pseudo-label generator employs triple contrast constraints and makes the prototype features... To foreground category features Distance less than background category features And obtain the constrained prototype features. Utilizing constrained prototype features With image features A feature similarity map is generated and a threshold is set to obtain refined pseudo-labels. A total of N1 refined pseudo-labels are obtained, and the mean is fused to obtain the final refined pseudo-labels.

8. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: In method S1, N1 is five. Five images from the same location in the study area are collected and input into the spectral adapter as a group, or one image from the study area is collected and expanded twice with weak perturbation and twice with strong perturbation to obtain four expanded images, which together form a group of five images that are input into the spectral adapter.

9. The weakly supervised open-pit mining identification method based on adaptive SAM and automatic knowledge learning according to claim 1, characterized in that: A sample dataset is created, consisting of image samples and corresponding label data. The image samples are images that have undergone image preprocessing, including radiometric calibration, atmospheric correction, orthorectification, and image fusion. The preprocessed image samples are cropped image patches, which are divided into two categories: image patches containing open-pit mines and image patches not containing open-pit mines. Each image sample consists of a group of five image patches, including one original image patch and four expanded image patches. The four expanded image patches are obtained by expanding the original image patch twice with weak perturbation and twice with strong perturbation, respectively. The sample dataset is used for learning and training according to methods S1 to S3.

Images