An adaptive remote sensing deep learning detection method for tailings ponds

Abstract

The adaptive tailings pond remote sensing deep learning detection method provided by the present invention is used for large-scale tailings pond identification and extraction. The present invention proposes a new tailings pond remote sensing extraction framework, namely: first, a feature pyramid network is used to perform deep feature learning of tailings ponds of different sizes based on a tailings pond remote sensing image data set; second, a cascade positioning optimization module is used to locate the tailings pond according to the characteristic value and use a multi-level detector to optimize the positioning coordinates; third, candidate detection objects are obtained according to the positioning optimization value, and multi-dimensional features are extracted and fused based on the deep learning algorithm and the traditional feature extraction algorithm to generate feature vectors; finally, an adaptive feature learning classifier is used to perform integrated learning and classification based on the feature vector, eliminate false alarm targets and extract tailings ponds. The present invention has the advantages of high positioning accuracy, low false alarm rate, and wide application range, and can provide an important basis for dynamic updating of tailings pond distribution information and safety management.

Description

Technical Field

[0001] The present invention belongs to the technical field of mine safety monitoring and remote sensing image processing, and in particular, is a remote sensing deep learning detection method for tailings ponds coupled with cascade optimization and feature adaptation. In the safety management of tailings ponds, the precise location information of tailings ponds in each region is extracted, which can be used to scientifically carry out safety monitoring and risk assessment of tailings ponds. Background Art

[0002] Tailings ponds are used to store tailings resources from industrial mining production. They are usually built with artificial dams and are a high potential energy mudslide hazard source. With the accumulation of tailings and water, the probability of tailings pond instability increases year by year, which will cause serious harm to the surrounding environment. Due to the need for mineral resources, tailings ponds of different levels are increasing year by year in various places. Large tailings ponds have the need for tailings dam safety assessment all year round due to their high potential energy. Small tailings ponds are small in scale and have low thresholds for opening. They are more likely to have low safety levels and tailings leakage accidents. Therefore, it is extremely important to accurately obtain the spatial distribution of tailings ponds and update information for scientific management of tailings ponds.

[0003] In the existing research, a variety of feature analysis and tailings pond detection methods have been proposed in the field of remote sensing image processing technology, successfully breaking through the technical barriers of tailings pond extraction. Existing methods are mainly divided into two branches: one is to build a tailings pond feature extraction model based on traditional machine learning methods, and the other is a target extraction method based on a deep learning framework. The traditional machine learning-based methods mainly focus on the scene composition characteristics of tailings ponds. Wang Jinhua et al. used the DT-tailings rule based on the decision tree to extract a single tailings pond from the GF-1 remote sensing image. BDMa et al. used principal component analysis and rapid atmospheric correction to process remote sensing images and extract spectral features of tailings ponds. Xiong Wencheng et al. analyzed the characteristics of manganese ore tailings ponds and qualitatively analyzed their scattering characteristics and the differences between synthetic aperture radar (SAR) and optical images. By using spatial and structural patterns, an object-oriented feature learning method was explored. Wang Chuangye et al. analyzed the spectral, texture and spatial geometric features of tailings and solid waste in the QuickBird dataset, and then determined the optimal segmentation threshold for tailings and solid waste. This type of research has made great achievements in analyzing the characteristics of tailings ponds, but these extraction methods can only be achieved by manually adjusting thresholds and image segmentation, which is not autonomous and flexible. In addition, although the above-mentioned machine learning classification methods have achieved the extraction of a single tailings pond, when applied to large-scale tailings pond detection, these methods perform poorly in the representation of tailings pond features. With the development of deep learning frameworks in the field of artificial intelligence and their powerful remote sensing image processing capabilities, some target detection frameworks have been initially tried to be applied to remote sensing detection of tailings ponds. Single-stage models such as YOLOv4 and SSD detection methods, and two-stage models such as Faster R-CNN and its improved models based on feature pyramids and attention mechanisms have all learned and extracted features of tailings ponds in remote sensing images. In addition, in order to solve the difficulties in site selection and boundary extraction of tailings ponds, a multi-task branch network (MTBNet) was proposed to optimize the coordinates of the detection candidate box, further improving the accuracy of the tailings pond detection model.

[0004] After analyzing the above methods and evaluating the existing research methods, high-precision large-scale remote sensing detection of tailings ponds is still a difficult task. Different from conventional homogeneous target detection objects, tailings ponds in different regions and grades are of different sizes and irregular shape boundaries, which poses a challenge to the feature learning of remote sensing images. The traditional framework used for the detection of fixed-shape objects (such as tanks, ships and airplanes) and performs well is directly transferred to the remote sensing image detection of large-scale tailings ponds. There are often problems of inaccurate positioning due to the diversity of object scales; in addition, due to the different scene compositions of tailings ponds at different stages, features cannot be effectively learned, and the false detection of many similar targets will cause the detection results to have low recognition accuracy. Combining the above methods, there are still two problems in the existing tailings pond detection that have not been effectively solved: 1) Low positioning accuracy. The existing detection method can output the location and category information of the object in the form of a region proposal box. When the positioning accuracy of the bounding box is low, there is a common misjudgment and loss of many real detection targets; 2) High false alarm rate. The tailings pond has a complex scene in the remote sensing image. In the detection results of the deep learning framework, they are usually accompanied by false alarm targets.

[0005] In recent years, the application of artificial intelligence technology has brought new development opportunities to all walks of life. The rapid development of artificial intelligence technology, especially deep learning technology, has also become an important driving force for the development of surveying and remote sensing disciplines. With the development of satellite commercialization at home and abroad and the popularization of drones, the amount of remote sensing data has increased rapidly, and the spatial resolution, temporal resolution, and spectral resolution have been continuously improved, providing basic data guarantee for remote sensing monitoring and other multi-field applications, and also bringing new directions for the rapid, accurate, and automatic identification of remote sensing targets, which is also a future development trend. The extraction of target ground object information in remote sensing data, such as attribute information, location range, and change information, is realized, but the limitations of cognitive level and technology affect the reliability of this information. The development of traditional remote sensing target recognition methods into intelligent remote sensing target recognition methods has the advantages of high data update frequency, time saving, and labor saving. On the basis of existing methods, it is worth making further method improvements based on the characteristics of tailings ponds to improve recognition accuracy and robustness of methods. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a tailings pond remote sensing deep learning detection method that couples cascade optimization and feature adaptation, so as to accurately and highly precisely realize large-scale tailings pond extraction, accurately reflect the distribution status of tailings ponds, and provide a scientific reference basis for dynamic updating of tailings pond area information and safety management.

[0007] The present invention solves the technical problem by adopting the following technical solutions:

[0008] The adaptive tailings pond remote sensing deep learning detection method provided by the present invention is mainly realized by a multi-scale feature extraction network, a cascade positioning optimization module, a multi-dimensional feature extraction and fusion module, and a feature adaptive learning classifier, wherein: the multi-scale feature extraction network first realizes the extraction and calculation of scene features of tailings ponds of different sizes in remote sensing images, and then the cascade positioning optimization module continuously optimizes the coordinates of the detection frame with lower precision through a regression detector based on the feature map calculated by the former to realize the coordinate positioning of the tailings pond, and then the multi-dimensional feature extraction and fusion module performs six major categories of features including geometry, probability, scene, topology, space and texture to obtain typical features of the tailings pond from multiple dimensions and fuses them into feature vectors for distinguishing from false alarm targets, and finally performs adaptive learning and binding of tailings pond features through an adaptive integrated learning classifier based on multi-dimensional features to realize the extraction of tailings ponds and the elimination of false alarm targets.

[0009] In the above method, a deep learning detection method for tailings ponds in remote sensing images with multi-stage cascade optimization can be adopted. Specifically, based on the existing high-resolution remote sensing image data, the positioning and classification of tailings ponds of different scales are optimized through multi-stage cascade positioning and classification, and high-precision extraction of tailings ponds in large-scale remote sensing images is achieved, which can provide technical support for the safety management and information update of large-scale tailings ponds.

[0010] The present invention also provides an adaptive tailings pond remote sensing deep learning detection method, which is a tailings pond remote sensing detection method that couples deep learning cascade optimization positioning and feature adaptive learning, specifically: first, multi-level scene deep learning feature extraction is performed on the remote sensing image, and then a cascade classifier is used to locate the position of the tailings pond and optimize the coordinates of the detection target to obtain a candidate area for tailings pond detection, and the detection object features are extracted and fused from six dimensions, and an adaptive integrated classifier is used to perform feature learning and cascade classification of the tailings pond to obtain high-precision tailings pond detection results.

[0011] In the method described, the method optimizes the location of tailings pond detection based on multi-scale features and cascade detectors and optimizes the classification of tailings ponds based on multi-dimensional feature extraction and adaptive feature learning, so that tailings ponds of different sizes can be detected, and false alarm targets in the detection results can be eliminated through multi-dimensional features. The method is specifically as follows:

[0012] (1) Position optimization based on multi-scale feature extraction and cascade positioning:

[0013] a. Input the original remote sensing image to be detected, perform image upsampling, and obtain a three-level image;

[0014] b. Downsample the extracted features to expand the field of view;

[0015] c. Combine the features on both sides through linking, integrate multi-scale features, and supplement the amount of information;

[0016] d. Input feature layer to locate candidate regions of features;

[0017] e. Design cascade detectors to optimize the location of tailings ponds;

[0018] f. Output the coordinates of the detected candidate boxes;

[0019] (2) Recognition optimization based on adaptive multi-dimensional feature ensemble learning classification:

[0020] a. Crop the identification results of the tailings pond proposed by the first step detection framework, and the cropped objects are used for further classification;

[0021] b. Based on RGB three-channel remote sensing images, extract the robust features of the candidate set objects: geometric, probability, scene, topology, spectrum, and texture features;

[0022] d. Normalize the features;

[0023] e. Unilateral classification and sampling;

[0024] f. Adaptive feature learning bundles strong and weak features to reduce the dimension of features and learn classification rules.

[0025] In the method described, the following steps can be adopted:

[0026] First, the original high-resolution remote sensing image is input, and sliding windows with scales of {340, 170, 84, 43} are used for upsampling to extract multi-scale features. The FPN network and Resnet152 are combined into a residual network, and convolutional layers {C2, C3, C4, C5} are used for feature extraction, where the step sizes are set to {4, 8, 16, 32} respectively.

[0027] Then, downsampling is used to retain the semantic features of the scene;

[0028] Finally, the features are fused using lateral connections to enhance the detail information.

[0029] In the method described above, the candidate target can be located based on the extraction of multi-scale features, and then the extraction of the location coordinates of the tailings pond can be continued. x , b y , b w , b h), and then use different thresholds to reclassify the regression frame for optimization of positioning, so that the four positioning coordinates of the tailings pond can be closer to the actual position of the tailings pond, where x is the detected candidate object of the tailings pond, g is the coordinate set of the tailings pond after regression positioning, and for each detected sample i input (g i , b i ) is used to train the optimization model, and the loss function of the detection box is minimized during the training process; the distance vector is also optimized during the regression process to ensure the stability of the regression positioning. The specific definition of this vector is as follows:

[0030]

[0031] Among them, (g x , g y , g w , g h ) represents the coordinates of the four vertices of the training sample g, (b x , b y , b w , b h ) is the coordinate of the target tailings pond extracted by the model in the image, (δ x , δ y , δ w , δ h ) represents the optimization parameter values ​​at the coordinates of the four corner points.

[0032] In the method described, a cascade detector containing a positioning optimization module can be used to optimize the position, so that the coordinates of the tailings pond located during the first detection process can be relocated according to the characteristics of the tailings pond.

[0033] In order to optimize the effective positioning, a single regression function is iteratively applied in the cascade detection process of the tailings pond, which is defined as follows:

[0034]

[0035] Among them, x represents the candidate object, b represents the suggestion box, It represents the connection between each iteration. In the proposed framework, the regression structure is designed based on these cascaded detectors to complete the optimization work:

[0036]

[0037] Where: T represents the number of connection layers, and the regression f at each level T It is optimized according to the sample distribution;

[0038] Among them, for each detection box, there is an IoU value u to evaluate and assess the detection box, and the label of each candidate box tailings pond is determined based on u:

[0039]

[0040] Among them, g y It represents the label of sample g. The IoU threshold is defined at each level to evaluate the effect of each level detector. It is difficult to achieve high-precision positioning based on a single detector in the tailings pond detection process. In the proposed model, different loss functions are defined to optimize positioning, where the value of u is {0.5, 0.6, 0.7}. In the original framework of R-CNN, the classifier h t and regressors in each level t, u t is set to be optimized according to the corresponding IoU threshold, when u t >u t-1 When , the loss function is defined as follows:

[0041] L(x t , g)=L cls (h t (x t ), y t )+λ[y t ≥1|L loc (f t (x t , b t )′g) (5)

[0042] Among them, b t =f t-1 (x t-1 , b t-1 ), g represents the real detection box x t ,λ=1balance parameter,y t It means x t Tags in u t under the circumstances.

[0043] The method can adopt a method including the following steps, so that multi-dimensional geographic features including geometry, texture, spectrum, topology, etc. can be extracted from the detected targets to screen the correctly identified tailings pond targets. The method steps are as follows:

[0044] First, the candidate area is cropped according to the optimized positioning coordinates to obtain the RGB sample set of the tailings pond candidate area.

[0045] Then, the geometric, probability, scene, topological, spectral and texture features of the candidate set objects are extracted based on the three-channel images respectively. The calculated eigenvalues ​​correspond to the eigenvectors respectively. For the i-th sample, the corresponding eigenvector is Fi = {Ge1, Ge2, Ge3, Pr1, Pr2, Se1, Se2, Se3, Se4, To1, To2, To3, Sp1, Sp2, Sp3, Te1, Te2, Te3, Te4, Te5, Te6, Te7, Te8}, where each element represents The height, width, aspect ratio of the sample in the table, the probability of the image being a tailings pond and the probability of not being a tailings pond obtained by image classification based on VGG16, the ratio of the area of ​​the tailings pond as a whole, the water body, the sand body, and the dam body corresponding to the tailings pond obtained by scene segmentation based on U-Net, the corresponding topological relationship between the water body, the sand body, and the dam body after segmentation, the mean of the spectral characteristic values ​​of the three bands R, G, and B between the samples, and the gray-level co-occurrence matrix are used to calculate the energy value, entropy, maximum probability, contrast, inverse difference moment, and correlation of the study area.

[0046] The method may include the following steps, so that the extracted features can all be used for the extraction of the tailings pond. The steps of the method are as follows:

[0047] Firstly, for the extracted data set of tailings pond features, the tailings pond data is sampled using a unilateral sampling method to reduce the amount of calculation of the tailings pond in the sample data set. When sampling the data, only the data with larger gradients are retained. However, in order to avoid discarding the data with smaller gradients and causing changes in the sample distribution, a constant is introduced for the samples with smaller gradients to balance them when calculating the gain. The GOSS algorithm first sorts all the values ​​of the features to be split in descending order according to their absolute values, and selects a% of the data with the largest absolute values. Then, b% of the data are randomly selected from the remaining data with smaller gradients. Then, these b% of the data are multiplied by a constant 1-a / b, paying more attention to the samples with insufficient training without changing the distribution of the original data set too much. Finally, (a+b)% is used to calculate the information gain.

[0048] Then, during classification, the estimated variance gain is used to split the tailings pond samples based on the data set, which is specifically expressed as follows:

[0049]

[0050] Among them, for the tailings pond training data set {x1,…,x n}, χ i It represents the feature χ in the multidimensional feature vector space S s , each instance is calculated based on the estimated variance gain To split the instance at feature j and node d, based on the data set A∪B. Where O represents the training data set of the tailings pond at the fixed node of the decision tree, , coefficient Used to normalize the sum of gradients on dataset B A c size;

[0051] Finally, for the extraction of features representing tailings ponds, in order to reduce the computational complexity, the exclusive features are saved in different datasets by adding the original values ​​with the offset. In order to avoid the loss of accuracy of the recognition results caused by reducing the number of features, an adaptive bundling strategy is used to select the features to be bundled. First, the feature relationship graph is constructed with reference to the conflicts between the proposed features, with weights calculated as edges. Second, the features are classified in descending order according to their degrees in the graph. Finally, each feature is checked in the ordered list and each feature is evaluated to determine whether it should be assigned to an existing bundle with small conflicts or created as a new bundle.

[0052] The method may include the following steps:

[0053] Firstly, the remote sensing image to be detected is input, and the tailings pond target detection is performed on it. The tailings pond samples are cropped according to the coordinates of the model candidate area to generate the candidate set of the model;

[0054] Then, the multi-dimensional features of the candidate set are calculated to produce the feature vectors and labels of the candidate objects.

[0055] Finally, an adaptive ensemble classifier is used to reclassify candidate set samples based on feature values, screen and extract the detected tailings pond targets, and map them.

[0056] Compared with the prior art, the present invention has the following main advantages:

[0057] (1) The present invention produces a tailings pond remote sensing identification dataset based on Google's 16-level remote sensing images. The multi-scale feature extraction network first realizes the extraction and calculation of scene features of tailings ponds of different sizes in remote sensing images. The cascade positioning optimization module continuously optimizes the coordinates of the detection frame with lower accuracy through different regression detectors based on the feature map calculated by the former to realize the coordinate positioning of the tailings pond, thereby completing the high-precision positioning task of the tailings pond.

[0058] (2) The present invention obtains the typical features of the tailings pond by performing multi-dimensional features of six categories, namely geometry, probability, scene, topology, space and texture, on the tailings pond candidate set generated by target monitoring, and fuses them into feature vectors to distinguish them from false alarm targets. Finally, the tailings pond features are analyzed and bound based on the multi-dimensional feature adaptive ensemble learning classifier to achieve reclassification of the tailings pond, thereby eliminating false alarm targets in the detection results and improving the category accuracy in the detection results.

[0059] (3) With the support of high-resolution remote sensing image tailings pond samples, the present invention can realize the training of tailings pond detection models and classification models. Based on large-scale high-resolution remote sensing image raw data, it can realize high-precision detection of regional tailings ponds. The wide distribution of samples makes the migration application of the model more regionally applicable and robust. The recognition results have high accuracy and precision and can be applied to large-scale tailings pond detection and mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flow chart of the method of the present invention.

[0061] Figure 2 This is a schematic diagram of different IoU threshold modifications.

[0062] Figure 3 This is a schematic diagram of sample distribution in the tailings pond area.

[0063] Figure 4 This is a schematic diagram of the tailings pond positioning results.

[0064] Figure 5 It is a diagram of feature importance.

[0065] Figure 6 This is a schematic diagram of the application in the demonstration area. DETAILED DESCRIPTION

[0066] The present invention provides a tailings pond remote sensing deep learning detection method coupled with cascade optimization and feature adaptation, which is used for large-scale tailings pond identification and extraction. The present invention proposes a brand-new tailings pond remote sensing extraction framework: first, a feature pyramid network is used to perform deep feature learning of tailings ponds of different scales based on a tailings pond remote sensing image data set; second, a cascade positioning optimization module is used to locate the tailings pond according to the characteristic value and a multi-level detector is used to optimize the positioning coordinates; third, candidate detection objects are obtained according to the positioning optimization value, and multi-dimensional features are extracted and fused based on the deep learning algorithm and the traditional feature extraction algorithm to generate feature vectors; finally, an adaptive feature learning classifier is used to perform integrated learning and classification based on the feature vector, eliminate false alarm targets and extract tailings ponds. The present invention has the advantages of high positioning accuracy, low false alarm rate, and wide application range, and can provide an important basis for dynamic updating of tailings pond distribution information and safety management.

[0067] The present invention is further described below in conjunction with embodiments and drawings, but is not limited to the following contents.

[0068] The tailings pond remote sensing deep learning detection method provided by the present invention with coupled cascade optimization and feature adaptation is used for large-scale tailings pond identification and extraction. Figure 1 As shown: it is mainly realized by a multi-scale feature extraction network, a cascade positioning optimization module, a multi-dimensional feature extraction and fusion module, and a feature adaptive learning classifier. The present invention has the advantages of high positioning accuracy, low false alarm rate, and a wide range of applications, and can provide an important basis for the dynamic update of tailings pond area information and safety management.

[0069] The multi-scale feature extraction network can use a feature pyramid to downsample the remote sensing image of the tailings pond to obtain tailings pond features of different scales, and obtain high-resolution and strong semantic feature information through upsampling and image fusion.

[0070] The cascade positioning optimization module can use three cascades with IoU threshold detectors that increase layer by layer to optimize the detection box positioning.

[0071] The multi-dimensional feature extraction and fusion module can use image processing algorithms to obtain geometric features, probability features, scene features, topological features, spatial features, and texture features of the detection target image, which are used to distinguish and characterize different types of tailings ponds.

[0072] The feature adaptive learning classifier can adopt a unilateral sampling method to sample the tailings pond data to reduce the amount of calculation of the tailings pond in the sample data set; for extracting features representing the tailings pond, an adaptive bundling strategy is used to select the features to be bound, which reduces the calculation complexity. By adding the original value and the offset, the proprietary features are saved in different sets, ensuring that various features bring improved accuracy to the recognition results.

[0073] The present invention provides a tailings pond remote sensing deep learning detection method coupled with cascade optimization and feature adaptation, and the method proposes a new tailings pond remote sensing extraction framework, which: firstly, inputs a remote sensing image data set to be detected into a multi-scale feature extraction network to calculate deep learning features of tailings ponds of different scales; secondly, inputs a feature layer into a cascade positioning optimization module, locates the tailings pond according to feature values, and uses a regression model to optimize the positioning coordinates; thirdly, outputs candidate detection objects according to the positioning optimization value, extracts and fuses multi-dimensional features based on a deep learning algorithm and a traditional feature extraction algorithm, and generates a feature vector; finally, inputs the feature vector into a feature adaptive learning classifier for integrated learning and classification, eliminates false alarm targets, and extracts the tailings pond.

[0074] In the above methods, the following methods can be used to realize deep learning detection of tailings ponds from remote sensing images with multi-stage cascade optimization, specifically:

[0075] (1) Position optimization based on multi-scale feature extraction and cascade positioning:

[0076] a. Input the original remote sensing image to be detected, perform image upsampling, and obtain a three-level image;

[0077] b. Downsample the extracted features to expand the field of view;

[0078] c. Combine the features on both sides through linking, integrate multi-scale features, and supplement the amount of information;

[0079] d. Input feature layer to locate candidate regions of features;

[0080] e. Design cascade detectors to optimize the location of tailings ponds;

[0081] f. Output the coordinates of the detected candidate boxes.

[0082] (2) Recognition optimization based on adaptive multi-dimensional feature ensemble learning classification:

[0083] a. The identification results of the tailings pond proposed by the first step detection framework are cropped, and the cropped objects are used for further classification.

[0084] b. Based on RGB three-channel remote sensing images, the robust features of the candidate set objects are extracted respectively: geometric, probability, scene, topological, spectral, and texture features.

[0085] d. Normalize the features;

[0086] e. Unilateral classification and sampling;

[0087] f. Adaptive features achieve feature dimensionality reduction by learning strong and weak features and bundling them, and learn classification rules;

[0088] The present invention can use the following method to obtain multi-level scene features of the tailings pond in remote sensing images:

[0089] First, the original high-resolution remote sensing image is input, and sliding windows with scales of {340, 170, 84, 43} are used for upsampling to extract multi-scale features. The FPN network and Resnet152 are combined into a residual network, and convolutional layers {C2, C3, C4, C5} are used for feature extraction, where the step sizes are set to {4, 8, 16, 32} respectively.

[0090] Then, downsampling is used to retain the semantic features of the scene;

[0091] Finally, the features are fused using lateral connections to enhance the detail information.

[0092] The present invention can also adopt the following method to obtain the candidate location area of ​​the tailings pond in the remote sensing image and perform location optimization:

[0093] First, based on the extraction of multi-scale features, the candidate target is located, and then the extraction of the location coordinates of the tailings pond is continued b = (b x , b y , b w , b h ), and then use different thresholds to optimize the positioning of the regression box reclassification, where x is the detected tailings pond candidate object, g is the tailings pond location coordinate set after regression positioning, and for each detected sample i input (g i , b i ) is used to train the optimization model, and the loss function of the detection box is minimized during the training process; the distance vector is also optimized during the regression process to ensure the stability of the regression positioning. The specific definition of this vector is as follows:

[0094]

[0095] Among them, (g x , g y , g w , g h ) represents the coordinates of the four vertices of the training sample g, (b x , b y , b w , b h ) is the coordinate of the target tailings pond extracted by the model in the image, (δ x , δ y , δ w , δ h ) represents the optimization parameter values ​​at the coordinates of the four corner points.

[0096] Then, in order to optimize the effective positioning, a single regression function is iterated in the process of cascade detection of tailings ponds, which is defined as follows:

[0097]

[0098] Among them, x represents the candidate object, b represents the suggestion box, It represents the connection between each iteration. In the proposed framework, the regression structure is designed based on these cascaded detectors to complete the optimization work:

[0099]

[0100] Where: T represents the number of connection layers, the regression of each level, f T It is optimized according to the sample distribution.

[0101] Among them, for each detection box, there is an IoU value u to evaluate and assess the detection box, and the label of each candidate box tailings pond is determined based on u:

[0102]

[0103] Among them, g y It represents the label of sample g, and the IoU threshold is defined at each level to evaluate the effect of each level detector.

[0104] It is difficult to achieve high-precision positioning based on a single detector during tailings pond detection. In the proposed model, different loss functions are defined to optimize positioning, where the value of u is {0.5, 0.6, 0.7}. In the original framework of R-CNN, the classifier h t and regressors in each level t, u t is set to be optimized according to the corresponding IoU threshold, when u t >u t-1 When , the loss function is defined as follows:

[0105] L(x t , g)=L cls (h t (x t ), y t )+λ[y t ≥1]L loc (f t (x t , b t ), g) (5)

[0106] Among them, b t =f t-1 (x t-1 , b t-1 ), g represents the real detection box x t ,λ=1balance parameter,y t It means x t Tags in u t under the circumstances.

[0107] The present invention can also adopt the following method to obtain the multi-dimensional features of the candidate location area of ​​the tailings pond:

[0108] First, the candidate area is cropped according to the optimized positioning coordinates to obtain the RGB sample set of the tailings pond candidate area.

[0109] Then, the geometric, probability, scene, topological, spectral and texture features of the candidate set objects are extracted based on the three-channel images respectively. The calculated eigenvalues ​​correspond to the eigenvectors respectively. For the i-th sample, the corresponding eigenvector is Fi = {Ge1, Ge2, Ge3, Pr1, Pr2, Se1, Se2, Se3, Se4, To1, To2, To3, Sp1, Sp2, Sp3, Te1, Te2, Te3, Te4, Te5, Te6, Te7, Te8}, where each element represents The height, width, aspect ratio of the sample in the table, the probability of the image being a tailings pond and the probability of not being a tailings pond obtained by image classification based on VGG16, the ratio of the area of ​​the tailings pond as a whole, the water body, the sand body, and the dam body corresponding to the tailings pond obtained by scene segmentation based on U-Net, the corresponding topological relationship between the water body, the sand body, and the dam body after segmentation, the mean of the spectral characteristic values ​​of the three bands R, G, and B between the samples, and the gray-level co-occurrence matrix are used to calculate the energy value, entropy, maximum probability, contrast, inverse difference moment, and correlation of the study area.

[0110] The present invention can also adopt the following method to eliminate false alarms and obtain accurate tailings pond targets based on multi-dimensional features:

[0111] First, for the extracted data set of tailings pond features, the tailings pond data is sampled using a unilateral sampling method to reduce the amount of calculation of the tailings pond in the sample data set. When sampling the data, only the data with larger gradients are retained. However, in order to avoid discarding data with small gradients and causing changes in sample distribution, a constant is introduced for samples with small gradients to balance when calculating the gain. The GOSS algorithm first sorts all the values ​​of the features to be split in descending order according to their absolute values, and selects a% of the data with the largest absolute value. Then randomly select b% of the data from the remaining smaller gradient data. Then multiply these b% of the data by a constant 1-a / b, pay more attention to samples with insufficient training without changing the distribution of the original data set too much, and finally use (a+b)% to calculate the information gain. In this way, the algorithm will pay more attention to samples with insufficient training without changing the distribution of the original data set too much.

[0112] Then, during classification, the estimated variance gain is used to split the tailings pond samples based on the data set, which is specifically expressed as follows:

[0113]

[0114] Among them, for the tailings pond training data set {x1,…,x n}, x i It represents the feature χ in the multidimensional feature vector space S s , each instance is calculated based on the estimated variance gain To split the instance at feature j and node d, based on the data set A∪B. Where O represents the training data set of the tailings pond at the fixed node of the decision tree, , coefficient Used to normalize the sum of gradients on dataset B A c size.

[0115] Finally, for extracting features representing tailings ponds, in order to reduce computational complexity, the exclusive features are saved in different datasets by adding the original values ​​with offsets. In order to avoid the loss of accuracy of the recognition results caused by reducing the number of features, an adaptive bundling strategy is used to select the features to be bundled. First, the feature relationship graph is constructed with reference to the conflicts between the proposed features, with weights calculated as edges; second, the features are classified in descending order according to their degrees in the graph; finally, each feature is checked in an ordered list and each feature is evaluated to determine whether it is assigned to an existing bundle with a small conflict or created as a new bundle.

[0116] Application examples:

[0117] The present invention is applied to three main typical tailings pond distribution areas in Hebei Province: Chicheng, Zhangbei, and Chongli. First, the high-resolution remote sensing image download platform Tuxin Earth (http: / / www.locaspace.cn / ) is used to download high-resolution remote sensing images of the sample collection area and the demonstration application area. Figure 2 As shown, the tailings ponds in nine provinces, namely Hebei Province, Shaanxi Province, Sichuan Province, Guizhou Province, Guangxi Province, Hunan Province, Hubei Province, Jiangxi Province and Anhui Province, are searched by manual visual interpretation. Google 16-level high-resolution remote sensing images are cropped and LabelImg software is used to mark tailings pond detection samples to produce tailings pond target detection samples. A total of 996 images are marked, including 1,260 tailings pond samples. The present invention is further explained in conjunction with the accompanying drawings.

[0118] In the first step, the original high-resolution remote sensing images of the training set samples are input, and sliding windows with scales of {340, 170, 84, 43} are used for upsampling to extract multi-scale features. The FPN network and Resnet152 are combined for residual network, and convolutional layers {C2, C3, C4, C5} are used for feature extraction, where the step sizes are set to {4, 8, 16, 32} respectively; then, downsampling is used to retain the semantic features of the scene; finally, in order to fully retain the scene features retained in the upsampling process, lateral connections are used to fuse the features to enhance the detail information.

[0119] In the second step, based on the extraction of multi-scale features, the candidate targets are located, and then the extraction of the positioning coordinates of the tailings pond is continued to optimize the positioning, where x is the detected tailings pond candidate set, and g is the tailings pond position coordinate set after regression positioning. Figure 3 For each detection box, the first-level detector sets the IoU value to 0.5, the second-level detector to 0.6, and the third-level detector to 0.7. The candidate detection boxes are compared with the true value boxes, and different confidence detection boxes are used to gradually retain high-precision detection boxes to optimize the positioning of areas with low recognition accuracy.

[0120] The fourth step is to crop the candidate area according to the optimized positioning coordinates to obtain the RGB sample set of the tailings pond candidate area. Then, the geometry, probability, scene characteristics, topology, spectrum and texture characteristics of the candidate set objects are extracted based on the three-channel images respectively. The calculated eigenvalues ​​correspond to the component feature vectors respectively. For the i-th sample, the corresponding feature vector is Fi={Ge1, Ge2, Ge3, Pr1, Pr2, Se1, Se2, Se3, Se4, To1, To2, To3, Sp1, Sp2, Sp3, Te1, Te2, Te3, Te4, Te5, Te 6, Te7, Te8}, where each element represents the height, width, aspect ratio of the sample, the probability of the image being a tailings pond and the probability of not being a tailings pond defined by VGG16 image classification, the ratio of the area of ​​the tailings pond corresponding to the water body, sand body, and dam body obtained by scene segmentation based on U-Net, the corresponding topological relationship between the water body, sand body, and dam body after segmentation, the mean of the spectral feature values ​​of the three bands of R, G, and B between the samples, and the energy value, entropy, maximum probability, contrast, inverse difference moment, and correlation of the study area are calculated using the gray level co-occurrence matrix. As shown in Table 1, the candidate object features (part) are detected for the training set, and as shown in Table 2, the candidate object features (part) are detected for the test set.

[0121] In the fifth step, for the extracted tailings pond feature data set, the tailings pond data is sampled using a unilateral sampling method to reduce the amount of calculation of the tailings pond in the sample data set. When sampling the data, only the data with a larger gradient is retained. However, in order to avoid discarding the data with a small gradient and causing changes in the sample distribution, a constant is introduced for the sample with a small gradient to balance it when calculating the gain. The GOSS algorithm first sorts all the values ​​of the features to be split in descending order according to the absolute value, and selects the 20% data with the largest absolute value. Then 30% of the data is randomly selected from the remaining smaller gradient data. Then, this 10% of the data is multiplied by a constant 1-2 / 3, paying more attention to the samples that are not well trained without changing the distribution of the original data set too much, and finally using 50% to calculate the information gain. In this way, the algorithm will pay more attention to the samples that are not well trained without changing the distribution of the original data set too much.

[0122] Finally, for the extraction of features representing tailings ponds, in order to reduce the computational complexity, the exclusive features are saved in different sets by adding the original values ​​with offsets. In order to avoid the loss of accuracy of the recognition results caused by reducing the number of features, an adaptive bundling strategy is used to select the features to be bundled. First, the feature relationship graph is constructed with reference to the conflicts between the proposed features, with weight calculations as edges; second, the features are classified in descending order according to their degrees in the graph; finally, each feature is checked in an ordered list and each feature is evaluated to determine whether it should be assigned to an existing bundle with a small conflict or created as a new bundle. Figure 4 As shown in the figure, the processed features can balance their contribution in the classification process. The recognition module represents the contribution ratio of the features in this method, which is compared with the traditional methods RF, GBDT, CatBoost, and AdaBoost.

[0123] The sixth step is to call the trained target detection and positioning optimization model and classification model, and input the remote sensing image to be extracted in the tailings pond demonstration area, such as Figure 5 As shown, Figure 5 (a) represents the location result of large-scale tailings pond, Figure 5 (b) represents the location result of the medium tailings pond, Figure 5 (c) represents the location result of small tailings pond, Figure 5 (d) shows the detection result after reclassification of the false alarm targets in the detection process. Figure 6 Mapping of tailings pond test results and demonstration areas in Chicheng County, Zhangbei District, and Chongli District of Hebei Province.

[0124] Table 1. Characteristics of candidate objects detected in the training set

[0125]

[0126] Table 1 Characteristics of candidate objects detected in the training set (continued)

[0127]

[0128] Table 2 Test set detection candidate object features

[0129]

[0130] Table 2 Test set detection candidate object characteristics (continued)

[0131]

Claims

1. An adaptive tailings pond remote sensing deep learning detection method, characterized by It is realized through a multi-scale feature extraction network, a cascade positioning optimization module, a multi-dimensional feature extraction and fusion module, and a feature adaptive learning classifier, wherein: the multi-scale feature extraction network first realizes the extraction and calculation of scene features of tailings ponds of different sizes in remote sensing images, and then the cascade positioning optimization module continuously optimizes the coordinates of the detection frame with lower accuracy through the regression detector based on the feature map calculated by the former to realize the coordinate positioning of the tailings pond, and then the multi-dimensional feature extraction and fusion module uses six major categories of features such as geometry, probability, scene, topology, space and texture to obtain typical features of the tailings pond from multiple dimensions and fuse them into feature vectors to distinguish them from false alarm targets, and finally, based on the multi-dimensional features, the adaptive integrated learning classifier performs adaptive learning and binding of tailings pond features to realize the extraction of tailings ponds and the elimination of false alarm targets; The method is: Firstly, for the extracted data set of tailings pond features, the tailings pond data is sampled using a one-sided sampling method to reduce the amount of calculation of the tailings pond in the sample data set. When sampling the data, only the data with larger gradients are retained. However, in order to avoid discarding the data with small gradients and bringing about changes in the sample distribution, a constant is introduced for the samples with small gradients to balance them when calculating the gain. The GOSS algorithm first sorts all the values ​​of the features to be split in descending order according to their absolute values, and selects a% of the data with the largest absolute values. Then, b% of the data are randomly selected from the remaining data with smaller gradients. Then, these b% of the data are multiplied by a constant 1-a / b, paying more attention to the samples with insufficient training without changing the distribution of the original data set too much. Finally, (a+b)% is used to calculate the information gain. Then, the estimated variance gain is used to split the tailings pond samples based on the dataset during classification. Finally, for the extraction of features representing tailings ponds, in order to reduce the computational complexity, the exclusive features are saved in different datasets by adding the original values ​​with the offset. In order to avoid the loss of accuracy of the recognition results caused by reducing the number of features, an adaptive bundling strategy is used to select the features to be bundled. First, the feature relationship graph is constructed with reference to the conflicts between the proposed features, with weights calculated as edges. Second, the features are classified in descending order according to their degrees in the graph. Finally, each feature is checked in the ordered list and each feature is evaluated to determine whether it should be assigned to an existing bundle with small conflicts or created as a new bundle.

2. The adaptive tailings pond remote sensing deep learning detection method according to claim 1 is characterized by A deep learning detection method for tailings ponds in remote sensing images with multi-stage cascade optimization is adopted. Specifically, based on the existing high-resolution remote sensing image data, the positioning and classification of tailings ponds of different scales are optimized through multi-stage cascade positioning and classification, and high-precision extraction of tailings ponds in large-scale remote sensing images is achieved. It can provide technical support for the safety management and information update of large-scale tailings ponds.

3. An adaptive tailings pond remote sensing deep learning detection method, characterized by A tailings pond remote sensing detection method that couples deep learning cascade optimization positioning and feature adaptive learning: First, multi-level scene deep learning feature extraction is performed on remote sensing images, and then a cascade classifier is used to locate the location of the tailings pond and optimize the coordinates of the detection target to obtain the candidate area for tailings pond detection. The detection object features are extracted and fused from six dimensions, and the adaptive integrated classifier is used to perform feature learning and cascade classification of the tailings pond to obtain high-precision tailings pond detection results; The method is: Firstly, for the extracted data set of tailings pond features, the tailings pond data is sampled using a one-sided sampling method to reduce the amount of calculation of the tailings pond in the sample data set. When sampling the data, only the data with larger gradients are retained. However, in order to avoid discarding the data with small gradients and bringing about changes in the sample distribution, a constant is introduced for the samples with small gradients to balance them when calculating the gain. The GOSS algorithm first sorts all the values ​​of the features to be split in descending order according to their absolute values, and selects a% of the data with the largest absolute values. Then, b% of the data are randomly selected from the remaining data with smaller gradients. Then, these b% of the data are multiplied by a constant 1-a / b, paying more attention to the samples with insufficient training without changing the distribution of the original data set too much. Finally, (a+b)% is used to calculate the information gain. Then, the estimated variance gain is used to split the tailings pond samples based on the dataset during classification. Finally, for extracting features representing tailings ponds, in order to reduce the computational complexity, the proprietary features are saved in different datasets by adding the original values ​​with the offsets. In order to avoid the loss of accuracy of the recognition results caused by reducing the number of features, an adaptive bundling strategy is used to select the features to be bundled. First, the feature relationship graph is constructed with reference to the conflicts between the proposed features, with weight calculation as the edge. Second, the features are classified in descending order according to their degrees in the graph. Finally, each feature is checked in the ordered list and each is evaluated to determine whether to be assigned to an existing bundle with a minor conflict or created as a new bundle.

4. The method according to claim 3, characterized in that The location of tailings pond detection is optimized based on multi-scale features and cascade detectors, and the classification of tailings ponds is optimized based on multi-dimensional feature extraction and adaptive feature learning. Specifically: (1) Position optimization based on multi-scale feature extraction and cascade positioning: a. Input the original remote sensing image to be detected, perform image upsampling, and obtain a three-level image; b. Downsample the extracted features to expand the field of view; c. Combine the features on both sides through linking, integrate multi-scale features, and supplement the amount of information; d. Input feature layer to locate candidate regions of features; e. Design cascade detectors to optimize the location of tailings ponds; f. Output the coordinates of the detected candidate boxes; (2) Recognition optimization based on adaptive multi-dimensional feature ensemble learning classification: a. Crop the identification results of the tailings pond proposed by the first step detection framework, and the cropped objects are used for further classification; b. Based on RGB three-channel remote sensing images, extract the robust features of the candidate set objects: geometric, probability, scene, topology, spectrum, and texture features; d. Normalize the features; e. Unilateral classification and sampling; f. Adaptive feature learning bundles strong and weak features to reduce the dimension of features and learn classification rules.

5. The method according to claim 4, characterized in that: First, the original high-resolution remote sensing image is input, and sliding windows with scales of {340, 170, 84, 43} are used for upsampling to extract multi-scale features. The FPN network and Resnet152 are combined into a residual network, and convolutional layers {C2, C3, C4, C5} are used for feature extraction, where the step sizes are set to {4, 8, 16, 32} respectively. Then, downsampling is used to retain the semantic features of the scene; Finally, the features are fused using lateral connections to enhance the detail information.

6. The method according to claim 4, characterized in that: Based on the extraction of multi-scale features, the candidate target is located, and then the extraction of the location coordinates of the tailings pond is continued. b=(b x , b y , b w , b h ), and then use different thresholds to reclassify the regression box for optimization of positioning, where x is the detected tailings pond candidate object, g is the tailings pond location coordinate set after regression positioning, and for each detected sample i input (g i , b i ) is used to train the optimization model, and the loss function of the detection box is minimized during the training process; the distance vector is also optimized during the regression process to ensure the stability of the regression positioning. The specific definition of this vector is as follows: Among them, (g x , g y , g w , g h ) represents the coordinates of the four vertices of the training sample g, (b x , b y , b w , b h ) is the coordinate of the target tailings pond extracted by the model in the image, (δ x , δ y , δ w , δ h ) represents the optimization parameter values ​​at the coordinates of the four corner points.

7. The method according to claim 4, characterized in that: Use a cascade detector with a positioning optimization module to optimize the position; In order to optimize the effective positioning, a single regression function is iteratively applied in the cascade detection process of the tailings pond, which is defined as follows: Among them, x represents the candidate object, b represents the suggestion box, It represents the connection between each iteration. In the proposed framework, the regression structure is designed based on these cascaded detectors to complete the optimization work: Where: T represents the number of connection layers, and the regression f at each level T It is optimized according to the sample distribution; Among them, for each detection box, there is an IoU value u to evaluate and assess the detection box, and the label of each candidate box tailings pond is determined based on u: Among them, g y It represents the label of sample g. The IoU threshold is defined at each level to evaluate the effect of each level detector. It is difficult to achieve high-precision positioning based on a single detector in the tailings pond detection process. In the proposed model, different loss functions are defined to optimize positioning, where the value of u is {0.5, 0.6, 0.7}. In the original framework of R-CNN, the classifier h t and regressors in each level t, u t is set to be optimized according to the corresponding IoU threshold, when u t >u t -1 When , the loss function is defined as follows: L(x t ,g)=L cls (h t (x t ),y t )+λ[y t ≥1]L loc (f t (x t ,b t ),g) (5) Among them, b t =f t-1 (x t-1 , b t-1 ), g represents the real detection box x t ,λ=1balance parameter,y t It means x t Tags in u t under the circumstances.

8. The method according to claim 4, characterized in that: First, the candidate area is cropped according to the optimized positioning coordinates to obtain the RGB sample set of the tailings pond candidate area. Then, the geometric, probability, scene, topological, spectral and texture features of the candidate set objects are extracted based on the three-channel images respectively. The calculated eigenvalues ​​correspond to the eigenvectors respectively. For the i-th sample, the corresponding eigenvector is Fi = {Ge1, Ge2, Ge3, Pr1, Pr2, Se1, Se2, Se3, Se4, To1, To2, To3, Sp1, Sp2, Sp3, Te1, Te2, Te3, Te4, Te5, Te6, Te7, Te8}, where each element represents The height, width, aspect ratio of the sample in the table, the probability of the image being a tailings pond and the probability of not being a tailings pond obtained by image classification based on VGG16, the ratio of the area of ​​the tailings pond as a whole, the water body, the sand body, and the dam body corresponding to the tailings pond obtained by scene segmentation based on U-Net, the corresponding topological relationship between the water body, the sand body, and the dam body after segmentation, the mean of the spectral characteristic values ​​of the three bands R, G, and B between the samples, and the gray-level co-occurrence matrix are used to calculate the energy value, entropy, maximum probability, contrast, inverse difference moment, and correlation of the study area.

9. The method according to claim 4, characterized in that: During classification, the estimated variance gain is used based on the data set to split the tailings pond samples. The specific expression is: Among them, for the tailings pond training data set {x1,…,x n }, Represents the features in the multidimensional feature vector space S Each instance is calculated based on the estimated variance gain To split the instance at feature j and node d, based on the data set A∪B. Where O represents the training data set of the tailings pond at the fixed node of the decision tree, , A l ={x i ∈A:x ij ≤d},A r ={x i ∈A:x ij >d},B l ={x i ∈B:x ij ≤d},B l ={x i ∈B:x ij >d}, coefficient Used to normalize the sum of gradients on dataset B A c size.

10. The method according to claim 4, characterized in that: Firstly, the remote sensing image to be detected is input, and the tailings pond target detection is performed on it. The tailings pond samples are cropped according to the coordinates of the model candidate area to generate the candidate set of the model; Then, the multi-dimensional features of the candidate set are calculated to produce the feature vectors and labels of the candidate objects. Finally, an adaptive ensemble classifier is used to reclassify candidate set samples based on feature values, screen and extract the detected tailings pond targets, and map them.

Images