A Remote Sensing Target Recognition Method for Typical Tailings Ponds Based on Deep Learning and Random Forest

Abstract

This invention provides a remote sensing target identification method for typical tailings ponds based on deep learning and random forests. This method is implemented through a system of remote sensing target identification techniques for typical tailings ponds. This system mainly consists of: original image dataset creation; target detection based on Faster R-CNN; multi-dimensional feature extraction based on models such as U-Net; and feature selection and classification based on random forests. Specifically: the original image dataset creation is used to label the target in the identification area for training the target detection model; the Faster R-CNN model first performs coarse extraction of tailings ponds from large-scale remote sensing images; the U-Net model extracts multi-dimensional features from the identification results, such as the area of ​​the tailings pond, scene topology, and image aspect ratio, for classifying false alarm targets and real targets; and random forests are used for typical feature selection and real target classification, achieving high-precision identification of typical tailings ponds. This invention has advantages such as low cost, high accuracy, and wide applicability.

Description

Technical Field

[0001] This invention belongs to the technical field of remote sensing image processing and environmental protection measures, specifically a remote sensing target identification method for typical tailings ponds based on deep learning and random forest. In the safety risk management of tailings ponds, it enables the identification of tailings ponds of different scales, shapes, and distributions, providing technical support for the comprehensive management and remediation of tailings ponds. Background Technology

[0002] Tailings dams are sites used to store tailings and other industrial waste during the development and processing of mineral resources. They are generally used for the recovery of mineral components remaining in tailings and the recycling of water resources. During the tailings storage process, tailings dams can gradually become a dangerous source with high potential energy. The rapid flow of debris flow caused by the instability of the tailings dam structure can cause large-scale damage, and the pollution caused by tailings containing large amounts of heavy metals can also bring catastrophic consequences to the environment.

[0003] Traditional tailings dam management often relies on manual statistics and on-site inspections, which presents challenges in terms of high labor costs and long implementation cycles when collecting large-scale tailings dam information. With the rapid development of new-generation information technologies such as remote sensing and deep learning, tailings dam safety management is gradually moving towards informatization, digitalization, and intelligence. Achieving high-precision automatic identification of tailings dams using current mainstream technologies, particularly tailings dam identification based on remote sensing imagery and deep learning methods, is of significant practical importance for tailings dam safety management.

[0004] With the rapid development of remote sensing technology, remote sensing image data, due to its high spatial resolution, high spectral resolution, high temporal resolution, and wide range of data sources, is widely used for target object detection and scene feature extraction. Combining multi-source remote sensing imagery, many existing studies have investigated the characteristics of mineral resource development and production scenarios from different perspectives. For example, Baodong Ma et al. analyzed the spectral and texture features of low-quality iron tailings in tailings ponds using Landsat data, while Lina Hao et al. conducted spectral feature analysis on dried tailings, wet tailings, and water bodies in tailings ponds. These research results provide ideas and guidance for feature extraction in tailings pond identification. However, extracting tailings pond targets from large-scale remote sensing images requires combining more feature indicators. In practical research on tailings pond identification, Hao Lina et al. introduced structural and distribution features into the analysis based on spectral and texture features. Using WorldView-2, they completed the analysis of 572 tailings ponds in southeastern Hubei Province through human-computer interaction combined with visual interpretation. Gao Yongzhi et al. used remote sensing images provided by China's Yaogan-14, Yaogan-24, Yaogan-5, and Gaofen-2 satellites to interpret 585 tailings ponds in Shandong Province from the perspective of improving data resolution. To further reduce the manual cost of tailings pond identification, researchers subsequently began exploring automated tailings pond identification technologies. For example, Xiong Wencheng et al. studied and analyzed the relevant features of tailings ponds in SAR data and optical images, used eCognition software to segment SPOT5 multispectral images, and combined multi-feature analysis to conduct research on automated tailings pond identification technology. To achieve tailings dam target identification in large-scale imagery, Yan Kai et al. improved the SSD model in 2020, combining existing target detection methods, to achieve intelligent identification of tailings dams in North China, achieving a precision of 0.882 and a recall of 0.857. Compared with the original SSD model, the precision was improved by 10.0% and the recall by 14.4%. In a similar study, Qingting Li et al. used the SSD model to extract tailings dams in the Beijing-Tianjin-Hebei region, achieving a detection precision of 90.2% and a recall of 88.7%.

[0005] In summary, existing tailings dam identification research typically employs manual visual interpretation combined with high-resolution remote sensing image interpretation, which suffers from high workload and low automation. Deep learning-based tailings dam identification methods struggle to balance accuracy and completeness when dealing with limited samples. Tailings dams in China are widely and dispersed, and the features of reservoirs and tailings dams in natural landscapes are similar, while tailings dams in different mining areas exhibit significant differences in shape and characteristics. Manual visual interpretation requires strong professional knowledge, making the aforementioned methods time-consuming and labor-intensive for large-scale tailings dam identification within imagery. While existing target detection algorithms based on big data and deep learning can perform some tailings dam target detection based on remote sensing, research varies depending on factors such as dam classification, location, production stage, size, shape, and remote sensing image features. Achieving high accuracy and complete target identification for typical tailings dams across a large area requires further optimization and improvement of methods.

[0006] Since Hinton et al. used deep learning to conduct research on data dimensionality reduction and classification, deep learning-related target models and technologies have begun to be successfully applied in high-resolution remote sensing image target detection, high-resolution remote sensing image information mining, and hyperspectral image classification. The existing mainstream target detection methods are mainly divided into two types: one is anchor-based target detection, such as Two-Stage detection models represented by R-CNN (Region Convolutional Neural Network), SPP-Net (Spatial Pyramid Pooling - Net), and Faster R-CNN (Faster Regions with Convolutional Neural Network), and One-Stage detection models represented by YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector); the other is anchor-free detection, such as CornerNet model and FSAF (Feature Selective Anchor-Free Module) model. Based on the above models, it is possible to achieve complete detection of targets in tailings ponds. To distinguish between real and false alarm targets in the identification results, existing U-Net scene semantic segmentation and Res-Net classification methods provide technical support for extraction of classification features. Algorithms such as random forest can classify tailings pond identification results based on multi-dimensional features, extract real tailings ponds, and further improve the accuracy of tailings pond classification. Summary of the Invention

[0007] The technical problem to be solved by this invention is to provide a remote sensing target identification method for typical tailings ponds based on deep learning and random forests, so as to realize the identification of tailings ponds of different scales, shapes and distributions in a large target area in the safety risk management of tailings ponds, and provide technical support for the comprehensive management and remediation of tailings ponds.

[0008] The present invention solves its technical problem by adopting the following technical solution:

[0009] The present invention provides a remote sensing target identification method for typical tailings ponds based on deep learning and random forest. This method is implemented through a system of remote sensing target identification methods for typical tailings ponds. The system mainly consists of: original image dataset production, target detection based on Faster R-CNN, multi-dimensional feature extraction based on models such as U-Net, and feature selection and classification based on random forest. Specifically: the original image dataset is produced to mark the target in the original image of the identification area for training the target detection model; the Faster R-CNN model is used to first perform coarse extraction of tailings ponds in a large-scale remote sensing image; the U-Net model is used to extract multi-dimensional features such as the area of ​​the tailings pond, scene topology, and image aspect ratio from the identification results for classification of false alarm targets and real targets; and random forest is used for typical feature selection and real target classification to achieve high-precision identification of typical tailings ponds.

[0010] This method can use high-resolution remote sensing images to detect tailings pond targets over a large area. By extracting multidimensional features from the identification results and eliminating false alarm targets based on the random forest algorithm, it can extract typical tailings ponds in the region and provide certain technical support for the scientific management of tailings ponds.

[0011] The present invention provides a method for remote sensing target identification of typical tailings ponds based on deep learning and random forest. Specifically, the method involves: first, segmenting remote sensing images and labeling tailings ponds according to model requirements to create a target detection model and train the remote sensing dataset; then, training the target detection model based on samples, inputting data into the model to detect tailings pond targets, and obtaining tailings pond identification results containing both real and false alarm targets; extracting and quantifying features such as sample geometry, sample classification probability, scene element area, and scene element topological relationship features from the identification results; and using the random forest algorithm for combined feature selection and sample classification to obtain the optimal feature combination and the best classification result for removing false alarm targets in tailings ponds.

[0012] This method includes typical tailings dam target detection based on Faster R-CNN, feature extraction based on U-Net and Res-Net models, and tailings dam identification based on random forest, specifically:

[0013] (1) Object detection based on Faster R-CNN:

[0014] a. Based on the publicly available statistical information on tailings ponds, the location information of known tailings ponds was labeled, and high-resolution remote sensing image samples of tailings ponds with RGB three channels were collected from remote sensing images. The target recognition feature dataset was created using the LabelImg tool on images of the study area.

[0015] b. Use the Faster R-CNN model to identify tailings ponds. Use the model trained with features to identify tailings ponds that may exist in the image. Based on the confidence level of tailings pond identification given by the model, count and compare the real tailings ponds with the identification target objects retained after setting a threshold. Determine the threshold when all real tailings ponds are retained in the identification target. Use the tailings pond identification results under this threshold as the secondary screening sample dataset.

[0016] (2) Feature extraction based on U-Net and Res-Net models:

[0017] a. For the image dataset of the recognition results, obtain the image width, height, and aspect ratio as image features of the tailings dam dataset;

[0018] b. Implement image segmentation of tailings ponds based on the U-Net model to obtain the area features of the segmented objects;

[0019] c. Improve the U-Net model to achieve semantic segmentation of water, sand and dam in tailings dams, and obtain the geometric features and spatial distribution features of each component in the identification dataset;

[0020] d. Apply the Res-Net model to classify the tailings dam dataset, and use the overall probability of determining whether it is a tailings dam as the probability feature value;

[0021] (3) Tailings dam identification based on random forest:

[0022] a. Based on the principle of random forest algorithm, a tailings dam identification feature sample set is created. The training samples are divided into training set and validation set in an 8:2 ratio. The training set is used to train the random forest classification model.

[0023] b. Select the optimal parameter combination and verify the model classification accuracy to achieve secondary screening of tailings ponds.

[0024] This method selects a 2m resolution Google Level 16 remote sensing image from LocalspaceViewer, uses the fishing net tool in ArcGIS to slice the image, uses the LabelImg tool to create samples, and divides the dataset into training and test sets in an 8:2 ratio.

[0025] This method uses the Faster R-CNN model for coarse screening of tailings ponds. The steps are as follows: The Faster R-CNN object detection model is trained using a training set. This process mainly consists of three parts: feature detection and matching, candidate region generation, target region regression, and target classification.

[0026] (1) The input data is an image with length L and width W. The image is resampled to length M and width N. Then, a multi-scale convolutional feature layer is extracted through a deep convolutional feature extraction network.

[0027] (2) The RPN network first uses a 3×3 convolution kernel to convolve the feature layer through a sliding window, then generates 9 scale proposal regions and makes judgments, and then corrects the candidate regions that may contain the target by calculating the offset through region bounding regression.

[0028] (3) Use the interest pooling layer to extract the corresponding features from the shared convolutional layer and feed them into the fully connected layer to calculate the probability that the candidate region contains the target and classify it to ensure that the typical tailings pond target is identified in the recognition result.

[0029] Finally, based on the target identification results of the tailings dam, a raw image dataset containing features extracted from false alarm targets was created.

[0030] This method uses the length, width, and aspect ratio of an image to reflect the shape features of the detected target object. The shape features of the target image are calculated from the image pixels. First, the number of pixels in length and width of each sample is counted, and the aspect ratio is calculated using the following formula:

[0031] R=max{tan θ , cot θ}

[0032] in, θ This represents the angle between the base and the diagonal.

[0033] This method uses the U-Net model to perform image segmentation of the tailings dam and semantic segmentation of the water, sand, and dam components, respectively, to obtain and identify the area features of the tailings dam and the geometric and spatial distribution features of each component in the dataset. The steps are as follows:

[0034] First, five preliminary effective feature layers are obtained using the backbone and feature fusion is performed. The backbone feature extraction part of U-Net is a stack of convolution and max pooling.

[0035] Then, feature extraction is enhanced by upsampling the five preliminary effective feature layers obtained from the backbone and performing feature fusion to obtain a final effective feature layer that fuses all features.

[0036] Finally, prediction and classification are performed, using the last effective feature layer obtained to classify each feature point, that is, to classify each pixel.

[0037] This method applies a classification model to the classification of tailings dam datasets, using the overall probability of determining whether a data point is a tailings dam as a probability feature value. The steps are as follows:

[0038] First, input the image and preprocess it by halving the feature size according to the model requirements.

[0039] Secondly, the ResNet50 network model was used to train the classification model based on the Conv Block and Identity Block modules of ResNet50. The Conv Block was used to change the network dimension, and the Identity Block was used to deepen the network. The entire method was fully connected to obtain the final classification result.

[0040] Finally, the cross-entropy loss function is used to verify the network accuracy, as shown in the following formula:

[0041]

[0042] Where M represents the number of categories. Indicates the true label as 0 or 1, when the first... i The value is 1 if the sample belongs to class c, otherwise it is 0. This represents the predicted probability that the i-th sample belongs to class c, where i indicates which sample.

[0043] This method, based on multi-dimensional features, uses the random forest algorithm to combine features and train a classification model to extract the true tailings pond from the target identification results. The steps are as follows:

[0044] First, the characteristic values ​​of the tailings dam are arranged and combined with different quantitative characteristics;

[0045] Then, data with different feature combinations are input into a decision tree classifier to predict and classify the samples;

[0046] Finally, the precision and recall of the tailings dam classification results for each feature combination are calculated using the following formulas:

[0047]

[0048] In the above formulas: TP stands for True Positives, which refers to samples correctly assigned as positive samples; TN stands for True Negatives, which refers to samples correctly assigned as negative samples; all detections represent all detected target results; FP stands for False Positives, which refers to samples incorrectly assigned as positive samples; FN stands for False Negatives, which refers to samples incorrectly assigned as negative samples; all groundtruths represent the total number of real tailings ponds. Based on precision and recall, the optimal feature combination in this method system is selected, and a random forest model is used to extract tailings ponds.

[0049] Compared with the prior art, the present invention has the following main advantages:

[0050] 1. High-resolution remote sensing imagery (such as Google Earth 16-level three-channel RGB remote sensing imagery) was used as the raw image data for the identification and extraction of typical tailings ponds. By extracting multi-dimensional features from the identification results and eliminating false alarm targets based on the random forest algorithm, the typical tailings ponds in the region were successfully extracted. High-resolution remote sensing imagery data is characterized by its large data volume, high update frequency, and free access. The method for identifying and extracting typical tailings ponds based on this data has a wide range of applications, rapid information updates, and low usage costs, and can provide certain technical support for the scientific management of tailings ponds.

[0051] 2. The Faster R-CNN target detection algorithm was used to identify typical tailings ponds of different scales, shapes, and distributions in large-scale remote sensing images, effectively solving the problems of underreporting and under-detection in traditional tailings pond statistical management. This method effectively combines current remote sensing technology and target recognition technology to achieve more scientific detection and statistics of typical tailings ponds in the region, which can effectively contribute to the standardized management of tailings ponds.

[0052] 3. The present invention uses the U-Net network to design a semantic segmentation model for tailings ponds and a scene segmentation model for tailings pond water, sand, and dam to obtain the area features of tailings ponds, the geometric features of scene elements, and the spatial relationship features. The improved Res-Net image classification model is used to obtain the probability features of tailings ponds. The geometric features of the recognition result images are statistically analyzed. The random forest algorithm is used to screen the optimal feature combination of typical tailings ponds and classify samples based on multi-dimensional features, eliminating false alarm targets and achieving high-precision extraction of typical tailings ponds.

[0053] Taking Hubei, Hunan, and Jiangxi provinces as examples, the identification and extraction of tailings ponds can be explained in detail as follows:

[0054] The first innovation is based on Faster R-CNN for target detection: Known tailings dam locations are labeled using publicly available statistical information, and high-resolution RGB three-channel remote sensing image samples of tailings dams are collected from remote sensing imagery. Imagery of the study area is selected, and the LabelImg tool is used to create a target recognition feature dataset. The Faster R-CNN model is used to identify tailings dams, and the model trained with these features identifies potential tailings dams in the images. Google Earth Level 16 remote sensing imagery with three-channel RGB is used as the original image data, with an image size of 1200m*1200m and a resolution of approximately 2m. The original retrieval images are downloaded using LocaSpaceViewer4, and 225 images containing 259 tailings dam targets are extracted from Hubei, Hunan, and Jiangxi provinces. According to the target recognition data format requirements, the LabelImg tool is used for target detection dataset labeling, and the LabelMe tool is used for semantic segmentation dataset labeling. Target detection in tailings ponds was performed using the Faster R-CNN model. The dataset containing 259 real tailings ponds was divided into training and test sets in an 8:2 ratio.

[0055] The second innovation involves using the Faster R-CNN model to determine the confidence level of tailings pond identification. It statistically compares real tailings ponds with the target objects retained after setting a threshold, determining the threshold for retaining all real tailings ponds to achieve complete tailings pond detection. The tailings pond identification results at this threshold are then used as a secondary screening sample dataset. After training using these samples, the model is validated on the test set data. The detected target results are classified according to the model's confidence level, yielding the accuracy and recall results for target identification under different confidence thresholds, as shown below. Figure 2 As shown.

[0056] The third innovation involves feature extraction of the identified targets and classification of the results using a random forest based on multiple features, removing false alarm targets. A tailings pond semantic segmentation model is trained on a test dataset using the U-Net model. This model performs semantic segmentation on the image, extracting the tailings pond region and calculating the ratio of tailings pond pixels to the total number of pixels in the image. Based on the compositional characteristics of tailings ponds, the U-Net model is improved for training a multi-object semantic segmentation model. Through training, water, sand, dams, and background are segmented in the image, and the ratio of each object's pixels to the total number of pixels in the image is calculated. Figure 5To obtain image classification probabilistic features, a Res-Net model was used to directly classify images, with one-hot labels, to obtain the probability of whether an image is a tailings pond. After training, the model's loss on the training set was 0.001, the loss on the validation set was 0.04, the precision on the training set was 0.98, and the precision on the validation set was 0.75. The output probability of classifying an image as a tailings pond or not was used as the feature value, and the sample probability distribution is as follows. Figure 6 As shown in Table 2, pixel statistics were performed on the experimental samples to obtain image geometric features, and the aspect ratios were calculated. The 12 extracted features were combined using a traversal approach. A random forest classification model was used for training and testing to further determine the feature set that distinguishes tailings ponds from other land features. Table 3 shows the feature set. Among the four categories of features—sample geometry, sample classification probability, scene element area, and scene element topology—the number of random forest classification evaluators was compared based on different feature combinations. The model was then sorted according to its classification accuracy as shown in Table 4, and the most advantageous feature combinations were selected. A comprehensive analysis of the number of random forest classification model evaluators and the model classification accuracy under each feature combination showed that the model performed optimally when the selected features were image height, image width, probability of being a tailings pond, percentage of water pixels in object segmentation, percentage of dam pixels in object segmentation, adjacency relationship between water and dam, and adjacency relationship between sand and dam, and the number of evaluators was 115. After removing false alarm targets, the tailings pond identification accuracy was 0.8295. The classification results are used as the overall method for the identification and extraction of typical tailings ponds, achieving high-precision extraction of typical tailings ponds. Attached Figure Description

[0057] Figure 1 This is a flowchart of the method of the present invention.

[0058] Figure 2 This is a graph showing the precision and recall results of the Faster R-CNN model.

[0059] Figure 3 This is a partial recognition result image from Faster R-CNN, and the data in this image is from real remote sensing images.

[0060] Figure 4 This is a schematic diagram comparing a typical tailings dam with similar land features. The data in this diagram is from real remote sensing images.

[0061] Figure 5 This is a schematic diagram of the semantic segmentation results of the U-Net model. The data in this diagram is real remote sensing imagery.

[0062] Figure 6 This is a distribution chart of sample classification probability values. Detailed Implementation

[0063] The present invention provides a method for remote sensing target identification of typical tailings ponds based on deep learning and random forest, belonging to the technical field of remote sensing image processing and environmental protection measures. This method is based on large-scale high-resolution remote sensing images and eliminates false alarm targets through target detection, multi-dimensional feature extraction and classification based on random forest, thereby realizing the extraction of typical tailings ponds in the region. This method includes the following steps: 1. Acquire a regional RGB three-channel color remote sensing image dataset and create a tailings pond recognition training set based on the distribution of tailings ponds; 2. Use the Faster R-CNN model to perform tailings pond target detection based on remote sensing images, set a recognition probability threshold to retain images within the detection box, and create a tailings pond recognition result dataset; 3. Design a tailings pond semantic segmentation model and a tailings pond water, sand, and dam scene segmentation model based on the U-Net network to obtain tailings pond area features, scene element geometric features, and spatial relationship features. Obtain the tailings pond probability features through an improved Res-Net image classification model, statistically analyze the geometric features of the recognition result images, and create a feature label sample set; 4. Train and test the model based on a random forest classification model for single features and multi-feature combinations to obtain the optimal feature parameter combination, thereby achieving high-precision recognition of typical tailings ponds. This recognition method has the advantages of low implementation cost, high detection accuracy, and wide range, and can provide certain technical support for the scientific management of tailings ponds.

[0064] The remote sensing target recognition method for typical tailings ponds based on deep learning and random forest provided by this invention is mainly implemented by a system consisting of original image dataset production, target detection based on Faster R-CNN, multi-dimensional feature extraction based on U-Net and other models, and feature selection and classification based on random forest. Specifically: the original image dataset is produced to mark the target in the original image of the recognition area for training the target detection model; the Faster R-CNN model is used to first perform coarse extraction of tailings ponds in a large-scale remote sensing image; the U-Net model is used to extract multi-dimensional features such as the area of ​​the tailings pond, scene topology, and image aspect ratio from the recognition results for classification of false alarm targets and real targets; and random forest is used for typical feature selection and real target extraction and classification.

[0065] The present invention will be further described below with reference to embodiments and accompanying drawings, but is not limited to the contents described below.

[0066] The present invention provides a method for remote sensing target identification of typical tailings ponds based on deep learning and random forest. Specifically, the method involves: first, segmenting remote sensing images and labeling tailings ponds according to model requirements to create a target detection model and training the remote sensing dataset; then, training the target detection model based on samples, inputting data into the model to perform tailings pond target detection, and obtaining tailings pond identification results containing real targets and false alarm targets; extracting and quantifying features such as sample geometry, sample classification probability, scene element area, and scene element topological relationship features from the identification results; and using the random forest algorithm for combined feature selection and sample classification to obtain the optimal feature combination and the best extraction result for removing false alarm targets in tailings ponds.

[0067] The above methods can be used to achieve coarse target detection, feature extraction, and correct target classification for typical tailings ponds, specifically as follows:

[0068] (1) Object detection based on Faster R-CNN:

[0069] a. Based on the publicly available statistical information on tailings ponds, the location information of known tailings ponds was labeled, and high-resolution remote sensing image samples of tailings ponds with RGB three channels were collected from remote sensing images. The target recognition feature dataset was created using the LabelImg tool on images of the study area.

[0070] b. Use the Faster R-CNN model to identify tailings ponds. Use the model trained with features to identify tailings ponds that may exist in the image. Based on the confidence level of tailings pond identification given by the model, count and compare the real tailings ponds with the identification target objects retained after setting a threshold. Determine the threshold when all real tailings ponds are retained in the identification target. Use the tailings pond identification results under this threshold as the secondary screening sample dataset.

[0071] (2) Feature extraction based on U-Net and Res-Net models:

[0072] a. For the image dataset of the recognition results, obtain the image width, height, and aspect ratio as image features of the tailings dam dataset;

[0073] b. Implement image segmentation of tailings ponds based on the U-Net model to obtain the area features of the segmented objects;

[0074] c. Improve the U-Net model to achieve semantic segmentation of water, sand and dam in tailings dams, and obtain the geometric features and spatial distribution features of each component in the identification dataset;

[0075] d. Apply the Res-Net model to classify the tailings dam dataset, and use the overall probability of determining whether it is a tailings dam as the probability feature value.

[0076] (3) Tailings dam identification based on random forest:

[0077] a. Based on the principle of random forest algorithm, a tailings dam identification feature sample set is created. The training samples are divided into training set and validation set in an 8:2 ratio. The training set is used to train the random forest classification model.

[0078] b. Select the optimal parameter combination and verify the model classification accuracy to achieve secondary screening of tailings ponds.

[0079] This invention can use the following method to achieve preliminary screening of typical tailings ponds in large-scale remote sensing images:

[0080] (1) Select a Google Level 16 remote sensing image with a resolution of 2m in LocalspaceViewer, extract images from the image to create the original image dataset, and divide the dataset into training set and test set in a ratio of 8:2.

[0081] (2) Training the Faster R-CNN object detection model using the training set. This process mainly consists of three parts: feature detection and matching, candidate region generation, target region regression, and target classification.

[0082] a. The input data is an image with length L and width W, and the image is resampled to length M and width N. Then, a deep convolutional feature extraction network is used to extract multi-scale convolutional feature layers.

[0083] b. The RPN network first uses a 3×3 convolution kernel to convolve the feature layer through a sliding window, then generates 9 scale proposal regions and makes judgments, and then corrects the candidate regions that may contain the target by calculating the offset through region bounding regression.

[0084] c. Use an interest pooling layer to extract the corresponding features from the shared convolutional layer, and feed them into a fully connected layer to calculate the probability that the candidate region contains the target and classify it, so as to preserve the typical tailings pond targets in the identification results.

[0085] (3) Based on the target identification results of the tailings dam, create a raw image dataset containing false alarm targets with feature extraction.

[0086] This invention can use the following method to obtain image features of the recognition result image dataset: using the image length, width, and aspect ratio to reflect the shape features of the detected target object. The shape features of the target image are calculated through image pixels. First, the number of pixels in length and width of each sample is counted, and the aspect ratio is calculated using the following formula:

[0087] ,

[0088] Where R is the aspect ratio. θ This represents the angle between the base and the diagonal.

[0089] This invention can use the following method to obtain scene features of the image dataset of the recognition results: Based on the U-Net model, image segmentation of the tailings pond and semantic segmentation of the water, sand, and dam of the tailings pond are performed respectively, so as to obtain the area features of the tailings pond and the geometric features and spatial distribution features of each component in the recognition dataset. The steps are as follows:

[0090] (1) Five preliminary effective feature layers are obtained using the backbone and feature fusion is performed. The backbone feature extraction part of U-Net is a stack of convolution and max pooling.

[0091] (2) Enhance feature extraction, use the five preliminary effective feature layers obtained from the main part for upsampling, and perform feature fusion to obtain a final effective feature layer that integrates all features;

[0092] (3) Perform prediction classification, and use the last effective feature layer obtained to classify each feature point, that is, classify each pixel point.

[0093] This invention can obtain the classification probability features of the image dataset of the recognition results using the following method: applying the ResNet50 model to the classification of the tailings pond dataset, and using the overall probability of determining whether it is a tailings pond as the probability feature value. The steps are as follows:

[0094] (1) Input image, and preprocess the image by halving the feature size according to the model requirements;

[0095] (2) The ResNet50 network model is used to train the classification model based on the two modules of Conv Block and Identity Block in ResNet50. The Conv Block is used to change the network dimension, and the Identity Block is used to deepen the network. The whole method is fully connected to obtain the final classification result.

[0096] (3) The cross-entropy loss function is used to verify the network accuracy, and the formula is as follows:

[0097]

[0098] in: L This represents the model loss value. Li This represents the loss function value for a single sample. N Indicates the number of samples. M Indicates the number of categories, y ic Represents the true label (0 or 1), when the first... i Each sample belongs to cThe value is 1 if the class is active, otherwise it is 0. Indicates the prediction result of the first i The predicted probability that a sample belongs to class c. i Indicates the nth sample.

[0099] The present invention can also employ the following method to obtain the optimal feature set through multi-dimensional feature combination, and to remove false alarm targets from the preliminary detection results of typical tailings ponds based on the random forest algorithm, thereby obtaining the true targets. The steps are as follows:

[0100] (1) Arrange and combine the characteristic values ​​of the tailings dam with different quantitative characteristics;

[0101] (2) Input data with different feature combinations into a decision tree classifier to predict and classify the samples;

[0102] (3) Calculate the precision and recall of the tailings dam classification results for each feature combination according to the following formulas:

[0103] , ,

[0104] In the above formulas: TP stands for True Positives, which refers to samples correctly assigned as positive samples; TN stands for True Negatives, which refers to samples correctly assigned as negative samples; all detections represent all detected target results; FP stands for False Positives, which refers to samples incorrectly assigned as positive samples; FN stands for False Negatives, which refers to samples incorrectly assigned as negative samples; and all groundtruths represent the total number of real tailings ponds. Based on precision and recall, the optimal feature combination in this method is selected, and a random forest model is used to extract tailings ponds.

[0105] The method provided by this invention is used for the identification of remote sensing targets in typical tailings ponds.

[0106] Application examples:

[0107] Taking the three provinces of Hubei, Hunan and Jiangxi in the middle reaches of the Yangtze River urban agglomeration as the study area, and using Google Earth Level 16 remote sensing imagery with three-channel RGB images as the original image data, this invention will be further explained in conjunction with the accompanying drawings.

[0108] The present invention provides a method for remote sensing target identification of typical tailings ponds based on deep learning and random forest, such as... Figure 1 This is a flowchart of the research technology of this invention.

[0109] The first step used Google Earth Level 16 remote sensing imagery with three channels (RGB) as the raw image data. The image size was 1200m*1200m, with a resolution of approximately 2m. The original search images were downloaded using LocaSpaceViewer4, extracting 225 images containing 259 tailings pond targets from Hubei, Hunan, and Jiangxi provinces. Based on the target recognition data format requirements, the LabelImg tool was used for target detection dataset annotation, and the same tool was used for semantic segmentation dataset annotation.

[0110] The second step involves target detection in tailings ponds using the Faster R-CNN model. The dataset, containing 259 real tailings ponds, is divided into training and test sets in an 8:2 ratio. After training using samples, the model is validated on the test set data. The detected targets are classified according to the model's confidence level, and the accuracy and recall results for target recognition under different confidence thresholds are obtained, as shown below. Figure 2 As shown, the Faster R-CNN model was validated using a dataset containing 45 real tailings pond images. To maximize the retention of real target detection results, a confidence level of 0.02 corresponding to the maximum recall was selected as the threshold for retaining target detection results. In the detection results, 81 targets were detected, of which 43 were correctly identified as real tailings ponds, and 38 were incorrectly identified. The model accuracy was 0.53, and the tailings pond recall was 0.96. Figure 3 Based on the Faster R-CNN model, it is possible to identify tailings ponds of different levels in images. The lower the level of the tailings pond and the more complex the image landscape, the more false alarm targets the model will identify.

[0111] To further screen for real tailings ponds and eliminate false alarm targets, such as Figure 4 A comparative analysis of the characteristics of typical tailings ponds and similar land features was conducted. Based on the differences in tailings pond types, they can be divided into four main categories: valley type, hillside type, flatland type, and river-blocking type. Valley-type tailings ponds are similar in shape to reservoirs, both being tailings ponds with relatively short dams and long reservoir depths. Based on RGB three-channel imagery, hillside farmland and hillside dried-up reservoirs share similar scene structures. Flatland tailings ponds and their surrounding environment have complex land feature structures, making them difficult to distinguish from bare land and ponds in plain areas. River-blocking tailings ponds are generally distributed along rivers, similar to artificial ponds along rivers, and are usually surrounded by mountains and water bodies.

[0112] The third step is to extract features from the identified targets to eliminate false alarms. Based on the U-Net model, a semantic segmentation model for tailings ponds is trained using a test dataset. Semantic segmentation is performed on the image to extract the tailings pond region, and the ratio of tailings pond pixels to total image pixels is calculated, as follows: Figure 5 The semantic segmentation model, after testing, achieved an average intersection-over-union (IoU) ratio of 77.24% for target object segmentation and 89.52% for background segmentation; and an average IoU ratio of 70.33% and an average pixel accuracy of 70.21% for tailings dam target segmentation. Based on the characteristics of tailings dam composition, an improved U-Net model was trained for multi-object semantic segmentation of tailings dams. Through training, water, sand, dam bodies, and background were segmented in the image, and the ratio of each object's pixels to the total number of pixels in the image was calculated. This improved U-Net semantic segmentation model, validated by test samples, achieved an average IoU ratio of 54.23% and an average pixel accuracy of 64.19% for water background segmentation; an average IoU ratio of 4.33% and an average pixel accuracy of 8.72% for dam body background segmentation; an average IoU ratio of 46.94% and an average pixel accuracy of 53.28% for tailings sand background segmentation; and an average IoU ratio of 65.86% and an average pixel accuracy of 84.55% for other background areas. Based on the feature segmentation results, the area and spatial topological relationship of the tailings dam and its scene components are quantified as shown in Table 1 below.

[0113] Table 1 Scene segmentation feature quantization

[0114]

[0115] To obtain image classification probabilistic features, a ResNet model is used to directly classify images, with one-hot labels, to obtain the probability of whether an image is a tailings pond. After training, the model's loss on the training set is 0.001, and its loss on the validation set is 0.04. The training set precision is 0.98, and the validation set precision is 0.75. The output probability of classifying an image as a tailings pond or not is used as the feature value. The sample probability distribution is as follows: Figure 6 As shown.

[0116] To obtain the geometric features of the images, pixel statistics were performed on the experimental samples, and the aspect ratios were calculated, as shown in Table 2.

[0117] Table 2. Statistical analysis of sample geometric characteristics

[0118]

[0119] The fourth step involves combining multiple features using a traversal approach for the 12 features, and then training and testing the random forest classification model to further determine the feature set that distinguishes the tailings dam from other land features. As shown in Table 3, the number of random forest classification evaluators is compared based on different feature combinations among the four categories of features: sample geometry, sample classification probability, scene element area, and scene element topology.

[0120] Table 3. Random Forest Classification Characteristics

[0121]

[0122] The models were sorted according to their classification accuracy as shown in Table 4, and the optimal feature combinations were selected. A comprehensive analysis of the number of evaluators and classification accuracy of the random forest classification model under each feature combination revealed that the model performed optimally when the selected features were image height, image width, probability of being a tailings dam, proportion of water pixels in object segmentation, proportion of dam pixels in object segmentation, adjacency relationship between water and dam, and adjacency relationship between sand and dam, with a total of 115 evaluators. After removing false alarm targets, the tailings dam identification accuracy was 0.8295. In summary, the typical tailings dam identification method based on deep learning and random forest proposed in this study achieved an overall tailings dam identification accuracy of 0.83 and a recall rate of 0.96.

[0123] Table 4. Classification characteristics of random forests

[0124]

[0125] The core steps of this invention are: creating a remote sensing image target detection dataset; detecting typical tailings ponds based on Faster R-CNN; extracting target image features; extracting features based on U-Net and Res-Net models; extracting image category probability features; and selecting the optimal feature combination for typical tailings ponds based on random forest and extracting tailings pond classification. The order of these feature extraction steps is variable. This invention utilizes Google Earth Level 16 remote sensing image data to achieve high-precision detection and comprehensive survey of typical tailings ponds in a large research area, facilitating accurate and rapid acquisition of the number of tailings ponds in the region and providing effective technical support for tailings pond supervision and enforcement.

Claims

1. A remote sensing target identification method for typical tailings dams based on deep learning and random forest, characterized by: This method, implemented through a remote sensing target identification system for typical tailings ponds, consists of: original image dataset creation, target detection based on Faster R-CNN, multi-dimensional feature extraction based on the U-Net model, and feature selection and classification based on random forest. Specifically: the original image dataset creation is used to label the target in the original image of the identification area for training the target detection model; the Faster R-CNN model is used to first perform coarse extraction of tailings ponds in a large-scale remote sensing image; the U-Net model is used to extract multi-dimensional features of the identification results, such as the area of ​​the tailings pond, scene topology, and image aspect ratio, for the classification of false alarm targets and real targets; and random forest is used for typical feature selection and real target classification to achieve high-precision identification of typical tailings ponds. This method utilizes Faster R-CNN for typical tailings dam target detection, U-Net and Res-Net model-based feature extraction, and random forest-based tailings dam identification, specifically as follows: (1) Object detection based on Faster R-CNN: a. Based on the publicly available statistical information on tailings ponds, the location information of known tailings ponds was labeled, and high-resolution remote sensing image samples of tailings ponds with RGB three channels were collected from remote sensing images. The target recognition feature dataset was created using the LabelImg tool on images of the study area. b. Use the Faster R-CNN model to identify tailings ponds. Use the model trained with features to identify tailings ponds that may exist in the image. Based on the confidence level of tailings pond identification given by the model, count and compare the real tailings ponds with the identification target objects retained after setting a threshold. Determine the threshold when all real tailings ponds are retained in the identification target. Use the tailings pond identification results under this threshold as the secondary screening sample dataset. (2) Feature extraction based on U-Net and Res-Net models: a. For the image dataset of the recognition results, obtain the image width, height, and aspect ratio as image features of the tailings dam dataset; b. Implement image segmentation of tailings ponds based on the U-Net model to obtain the area features of the segmented objects; c. Improve the U-Net model to achieve semantic segmentation of water, sand and dam in tailings dams, and obtain the geometric features and spatial distribution features of each component in the identification dataset; d. Apply the Res-Net model to classify the tailings dam dataset, and use the overall probability of determining whether it is a tailings dam as the probability feature value; (3) Tailings dam identification based on random forest: a. Based on the principle of random forest algorithm, a tailings dam identification feature sample set is created. The training samples are divided into training set and validation set in an 8:2 ratio. The training set is used to train the random forest classification model. b. Select the optimal parameter combination and verify the model classification accuracy to achieve secondary screening of tailings ponds.

2. The method for remote sensing target identification of typical tailings dams based on deep learning and random forest according to claim 1, characterized in that: High-resolution remote sensing imagery was used to detect tailings dam targets over a large area. By extracting multidimensional features from the identification results and eliminating false alarm targets based on the random forest algorithm, typical tailings dams in the region were identified, providing technical support for the scientific management of tailings dams.

3. A remote sensing target identification method for typical tailings dams based on deep learning and random forest, characterized by: First, according to the model requirements, remote sensing images are segmented and tailings ponds are labeled to create a target detection model and train the remote sensing dataset. Then, the target detection model is trained based on the samples, and the data is input into the model to perform tailings pond target detection, resulting in tailings pond identification results containing real targets and false alarm targets. The identification results are subjected to feature extraction and quantification of sample geometry, sample classification probability, scene element area, and scene element topological relationship features. The random forest algorithm is used for combined feature screening and sample classification to obtain the optimal feature combination and the best classification result for removing false alarm targets in tailings ponds. This method utilizes Faster R-CNN for typical tailings dam target detection, U-Net and Res-Net model-based feature extraction, and random forest-based tailings dam identification, specifically as follows: (1) Object detection based on Faster R-CNN: a. Based on the publicly available statistical information on tailings ponds, the location information of known tailings ponds was labeled, and high-resolution remote sensing image samples of tailings ponds with RGB three channels were collected from remote sensing images. The target recognition feature dataset was created using the LabelImg tool on images of the study area. b. Use the Faster R-CNN model to identify tailings ponds. Use the model trained with features to identify tailings ponds that may exist in the image. Based on the confidence level of tailings pond identification given by the model, count and compare the real tailings ponds with the identification target objects retained after setting a threshold. Determine the threshold when all real tailings ponds are retained in the identification target. Use the tailings pond identification results under this threshold as the secondary screening sample dataset. (2) Feature extraction based on U-Net and Res-Net models: a. For the image dataset of the recognition results, obtain the image width, height, and aspect ratio as image features of the tailings dam dataset; b. Implement image segmentation of tailings ponds based on the U-Net model to obtain the area features of the segmented objects; c. Improve the U-Net model to achieve semantic segmentation of water, sand and dam in tailings dams, and obtain the geometric features and spatial distribution features of each component in the identification dataset; d. Apply the Res-Net model to classify the tailings dam dataset, and use the overall probability of determining whether it is a tailings dam as the probability feature value; (3) Tailings dam identification based on random forest: a. Based on the principle of random forest algorithm, a tailings dam identification feature sample set is created. The training samples are divided into training set and validation set in an 8:2 ratio. The training set is used to train the random forest classification model. b. Select the optimal parameter combination and verify the model classification accuracy to achieve secondary screening of tailings ponds.

4. The method according to claim 3, characterized in that: We selected a Google Level 16 remote sensing image with a resolution of 2m from LocalspaceViewer, used the fishing net tool in ArcGIS to slice the image, and used the LabelImg tool to create samples. The dataset was divided into training and test sets in an 8:2 ratio.

5. The method according to claim 3, characterized in that: The coarse screening of tailings ponds based on the Faster R-CNN model involves the following steps: training the Faster R-CNN object detection model using the training set. This process is divided into three parts: feature detection and matching, candidate region generation, target region regression, and target classification. (1) The input data is an image with length L and width W. The image is resampled to length M and width N. Then, a multi-scale convolutional feature layer is extracted through a deep convolutional feature extraction network. (2) The RPN network first uses a 3×3 convolution kernel to convolve the feature layer through a sliding window, then generates 9 scale proposal regions and makes judgments, and then corrects the candidate regions that may contain the target by calculating the offset through region bounding regression. (3) Use the interest pooling layer to extract the corresponding features from the shared convolutional layer and feed them into the fully connected layer to calculate the probability that the candidate region contains the target and classify it to ensure that the typical tailings pond target is identified in the recognition result. Finally, based on the target identification results of the tailings dam, a raw image dataset containing features extracted from false alarm targets was created.

6. The method according to claim 3, characterized in that: The shape features of the detected target object are reflected by the length, width, and aspect ratio of the image. The shape features of the target image are calculated from the image pixels. First, the number of pixels in length and width of each sample is counted, and then the values ​​are calculated using the following formula: R=max{tan θ ,cot θ }, in, θ This represents the angle between the base and the diagonal.

7. The method according to claim 3, characterized in that: Based on the U-Net model, image segmentation and semantic segmentation of water, sand, and dam in tailings ponds are performed respectively, realizing the acquisition and recognition of tailings pond area features and the acquisition of geometric features and spatial distribution features of each component element in the dataset. The steps are as follows: First, five preliminary effective feature layers are obtained using the backbone and feature fusion is performed. The backbone feature extraction part of U-Net is a stack of convolution and max pooling. Then, feature extraction is enhanced by upsampling the five preliminary effective feature layers obtained from the backbone and performing feature fusion to obtain a final effective feature layer that fuses all features. Finally, prediction and classification are performed, using the last effective feature layer obtained to classify each feature point, that is, to classify each pixel.

8. The method according to claim 3, characterized in that: The classification model is applied to the classification of the tailings dam dataset, and the probability of judging whether it is a tailings dam as a whole is used as the probability feature value. The steps are as follows: First, input the image and preprocess it by halving the feature size according to the model requirements. Secondly, the ResNet50 network model was used to train the classification model based on the Conv Block and Identity Block modules of ResNet50. The Conv Block was used to change the network dimension, and the Identity Block was used to deepen the network. The entire method was fully connected to obtain the final classification result. Finally, the cross-entropy loss function is used to verify the network accuracy, as shown in the following formula: Where M represents the number of categories. Indicates the true label as 0 or 1, when the first... i The value is 1 if the sample belongs to class c, otherwise it is 0. p ic This represents the predicted probability that the i-th sample belongs to class c, where i indicates which sample.

9. The method according to claim 3, characterized in that: Based on multidimensional features, a random forest algorithm is used to combine features and train a classification model to achieve the extraction of real tailings ponds from the target recognition results. The steps are as follows: First, the characteristic values ​​of the tailings dam are arranged and combined with different quantitative characteristics; Then, data with different feature combinations are input into a decision tree classifier to predict and classify the samples; Finally, the precision and recall of the tailings dam classification results for each feature combination are calculated using the following formulas: , In the above formulas: TP stands for True Positives, which refers to samples correctly assigned as positive samples; TN stands for True Negatives, which refers to samples correctly assigned as negative samples; all detections represent all detected target results; FP stands for False Positives, which refers to samples incorrectly assigned as positive samples; FN stands for False Negatives, which refers to samples incorrectly assigned as negative samples; all groundtruths represent the total number of real tailings ponds. Based on precision and recall, the optimal feature combination in this method system is selected, and a random forest model is used to extract tailings ponds.

Images