A satellite remote sensing-based intelligent identification method for open-pit mining areas of a mine

Abstract

The application provides a satellite remote sensing-based intelligent identification method for a mine open-pit mining area, which is an effective method for intelligently extracting a mine area quickly and accurately through a convolutional neural network. First, original images are subjected to radiation correction and atmospheric correction. Second, the corrected images are preprocessed, PCA dimensionality reduction is performed, and the images are cropped, while the cropped coordinates are recorded. Third, the preprocessed images are detected for mine areas using a trained YOLOv5. Fourth, the images detected for mine areas by the YOLOv5 are segmented using a U-net++, and a refined mine contour is extracted. Finally, the extraction result is fused into the original images, and a whole mine identification result is obtained. The YOLOv5 is used to detect the approximate position of a mine, and the U-net++ is used for fine division, which ensures the speed and improves the identification accuracy, and can effectively solve the problem that the proportion of a mine area is small and the gap between positive and negative samples is too large. The CSP module in the YOLOv5 can effectively reduce the calculation amount and improve the identification accuracy, and the multi-head design can make the identification effect free from the limitation of object size, and can efficiently extract mine areas of various sizes.

Description

TECHNICAL FIELD

[0001] The present application relates to multispectral image target detection and image segmentation technology, and in particular to a method for realizing multispectral image target detection and image segmentation by using a convolutional neural network. BACKGROUND

[0002] Mine monitoring is of great significance to the rational development and sustainable utilization of mineral resources and land resources, and supports ecological protection and restoration. Detecting a mine area is needed for monitoring a mine. The earliest mine area detection is mostly based on the combination of manual visual remote sensing image and field investigation, which is costly, labor-intensive and low in efficiency. Machine learning provides a new way for mine area extraction. Using decision trees and random forests to extract mine areas can quickly obtain recognition results, but has the shortcomings of low accuracy and scattered extraction areas, because tree models are essentially based on single pixel judgment and recognition, rather than a connected whole. Convolutional neural networks are relatively mature in target detection and recognition, which can automatically extract features through convolution kernels without human design. Well-trained convolutional neural networks have high accuracy and fast speed in target extraction. However, the extraction of mine areas from remote sensing images faces the problems of extremely small proportion of mine areas in the total area and extremely unbalanced positive and negative samples. Single target detection or image segmentation cannot effectively design a loss function, and a small amount of positive samples is easily submerged by negative samples, resulting in poor network learning ability and low detection accuracy.

[0003] The present application designs a new neural network architecture that can directly use multispectral image target images as input, without manual delineation of mine areas or feature extraction. Based on remote sensing images, the present application can realize full-automatic recognition based on mine area images through convolutional neural networks, effectively improve the precision of intelligent automatic feature extraction of mine areas, and facilitate subsequent management and provide real-time monitoring and control for the management and planning of mineral resources. SUMMARY

[0004] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the present application to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the shortcomings of the existing single convolutional neural network method for extracting a small amount of target areas, the present application proposes a method combining target detection and image segmentation. The technical solution adopted by the present application is as follows:

[0006] A satellite remote sensing-based intelligent identification method for open-pit mining areas of a mine, the overall process is shown in Figure 1 , comprising the following steps:

[0007] Step 1: Obtain the original remote sensing image, and perform radiation correction and atmospheric correction on the image.

[0008] Step 2: Perform image preprocessing on the corrected remote sensing image, including band dimension reduction, image enhancement, and image cropping.

[0009] Step 3: Use YOLOv5 with a Transformer block to perform target detection on the processed image, and extract the mine area, and adaptively output the target.

[0010] Step 4: Use U-net++ to perform image segmentation on the part of the mine area detected by YOLOv5, and extract the fine contour of the mine. In the structure of U-net++, the encoder is reduced in dimension by 2x2 max pooling, while the number of convolution kernels is increased, the decoder is raised in dimension by 2x2 deconvolution, while the number of convolution kernels is reduced, the network integrates multiple sub-networks, each layer performs feature fusion, each layer has a loss function, adopts deep supervision, and records the intermediate loss. In the process of identifying the mine, the classification is only whether it is a mine or not a mine, so the output channel number is 2.

[0011] Step 5: Fuse the extraction result into the large-size image before cropping to obtain the final result.

[0012] Further, for step 1, for the image that has been corrected, the correction can be omitted, and for the original uncorrected image, use ENVI to perform radiation correction and atmospheric correction.

[0013] Further, the specific steps for image processing in step 2 are as follows:

[0014] Step 2.1: Use principal component analysis (PCA) to reduce the number of bands, so that the number of channels is reduced to 3, which is beneficial to subsequent use of YOLOv5 to extract targets;

[0015] Step 2.2: Use image enhancement to increase image information and reduce noise interference. The image enhancement methods applied are: image random rotation, flipping, translation, shearing, histogram equalization, and mosaic enhancement. In this method, the random rotation range is -45 degrees to 45 degrees, the random shearing maximum ratio is 0.2, and the mosaic enhancement adopts the method of enhancing 4 images to 1 mosaic image.

[0016] Step 2.3: Image cropping is to crop the corrected large-size image into a small-size image suitable for deep learning input, with a size of 256, 416 or 448 pixels, to facilitate convolutional neural network recognition. Before restoring the large-size image in step 5, the image size needs to be cropped to a fixed size. When cropping, the relative coordinates of the cropped image, i.e. the row and column numbers of its top-left corner and bottom-right corner in the original image, need to be recorded. The cropped image is stored in a different folder according to the original large-size image.

[0017] Further, for step 3, the trained YOLOv5 is used to detect targets in the processed image. The improved YOLOv5 consists of three parts: backbone, neck and head. The backbone network contains multiple CBL and CSP (Cross Stage Partial) modules, where CBL is a combination of convolution (conv), batch normalization (bn) and activation function (leakyrelu). The purpose of the CSP module is to reduce the amount of calculation and enhance the performance of gradient descent. It divides the input into two parts, one of which is calculated through the block, and the other is directly passed through the shortcut. Then the two parts are concatenated. The neck uses the PAN (Path Aggregation Network) network. PANet fuses deep semantic information and shallow texture information through bottom-up and top-down fusion paths. It performs multiple concatenation operations. Before concatenation, the dimensions are adjusted using up-sampling to keep the dimensions consistent. After concatenation, the CSP module is used to conduct the head. YOLOv5 contains a multi-head mechanism, with a total of 3 heads, which takes into account the recognition of large, medium and small objects. Head1 divides the original input into 76x76 grids, which is biased towards detecting small objects. Head2 is 38x38, which is biased towards recognizing medium objects. Head3 is 19x19, which is biased towards recognizing larger objects. In particular, a Transformer block is added in the head to improve the perception of environmental information and increase the accuracy of small target recognition. It contains: layer normalization, multi-head attention mechanism, Dropout and MLP (Multi-Layer Perceptron).

[0018] Further, the steps for training YOLOv5 are as follows:

[0019] Step 3.1:

[0020] (1) Create a dataset, such as Figure 2As shown, first, the ENVI is used to mark the ROI of the mine, and the output is an XML file; second, the geographic coordinates in the XML file are extracted using Python, and the geographic coordinates are converted to pixel coordinates using GDAL; then, PCA dimensionality reduction and image cropping are performed on the large-scale image (step 2), and the row and column coordinates of the cropped image are recorded, and the relative pixel coordinates of the cropped image are calculated based on the cropped coordinates of the large-scale image and the pixel coordinates of the mine (the row and column maximum values are within the cropped size), and each ROI region is assigned a separate ID; finally, the rectangular box label in YOLO format is extracted for each ID, i.e. the row and column range is calculated, converted to (x, y, w, h) form, and output as a txt format;

[0021] (2) The loss of using YOLOv5 to detect the target is calculated using Focal loss to balance the class imbalance problem of the mine area and the background. The formula of Focal loss is as follows:

[0022]

[0023] wherein, and represent the true values of positive and negative samples in binary classification, i.e. the mine area and background label value, and represent the predicted values output by the YOLOv5 network, is an adjustable factor, when it is 0, Focal loss degenerates into binary cross-entropy loss function; using SGD as the optimizer, the initial learning rate is set to 0.01, the weight decay coefficient is 0.001, and the momentum is 0.93;

[0024] Further, the steps of using YOLOv5 to detect the mine area are as follows:

[0025] Step 3.2:

[0026] (1) The image in step 2.2 in the detect recognition folder of YOLOv5 is detected, and adaptive cropping is performed on the image with mine area detected and saved. The confidence threshold for detection is 0.5, and adaptive cropping is used to determine the cropping requirement according to the target size, and small targets are adaptively cropped to a given minimum range to avoid too few target areas after cropping, which affects the feature extraction of U-net++ and affects the realization of the next step of segmentation;

[0027] Further, Figure 5 shows the detection results of YOLOv5 on the mine area, and it can be seen that the mine area can be well detected;

[0028] Further, for step 4, based on the image of the mine area detected in step 3, further image segmentation processing is performed using U-net++ to identify the fine contour of the mine. As shown in Figure 4 , the encoder uses 2x2 max pooling to reduce the dimension, while increasing the number of convolution kernels. The decoder uses 2x2 deconvolution to increase the dimension, while reducing the number of convolution kernels. The network integrates multiple subnetworks, each layer performs feature fusion, and each layer has a loss function. Deep supervision is adopted, and the intermediate loss is recorded. In the process of identifying the mine, the classification is only yes (mine) or no (not mine), so the output channel number is 2;

[0029] Further, the steps for training U-net++ are as follows:

[0030] Step 4.1:

[0031] (1) As shown in Figure 2 , the pixel coordinates of the mine sample obtained in step 3.1 are filled to obtain a mask label, which is made into a U-net++ label format, and all cropped images containing mine samples are used as training samples;

[0032] (2) Use Dice loss to train U-net++ network, and the expression of Dice coefficient is:

[0033]

[0034] wherein, respectively represent the ground truth and the network prediction value, and the value of Dice loss is ;

[0035] Use SGD as the optimizer of U-net++, the initial learning rate is 0.001, the learning rate decay coefficient is 0.1, the weight decay coefficient is 0.0001, and the momentum is 0.90;

[0036] Further, the steps for detecting the mine area using U-net++ are as follows:

[0037] Step 4.2:

[0038] The detection result of YOLOv5 in step 3 is obtained, the target output by YOLOv5 is used as prediction data, and the U-net++ network trained in step 4.1 is used for prediction. U-net++ can be pruned, so the prediction effect of different levels of network is tested respectively, and the best 4-layer integrated network is selected for final prediction, which can divide the fine boundary of the mine area, that is, it can identify the mine position, and also can extract the shape and coverage area of the mine;

[0039] Further, for step 5, the recognition result of step 4 is fused into the original image, a tensor with the same dimension as the original large-size remote sensing image and a value of 0 is first created using Python, the image position is determined using the clipping coordinates recorded in step 2, the specific values of the tensor are filled according to the extraction result of step 4, visualization output is performed using opencv, and the refined mine area recognition result of the whole large-size remote sensing image is obtained.

[0040] Compared with the prior art, the significant advantage of the present application is that by combining target detection and image segmentation of the convolutional neural network, the problem of inaccurate recognition caused by the small proportion of the mine area in the total area, which makes it difficult to design the loss function and train the network, is solved. For the recognition result, fusion processing is performed to restore the original size of the image, so that visual recognition of the result is more convenient and the position of the mine is easier to locate. The method of combining target detection and image segmentation adopted in the present application can effectively improve the recognition accuracy while ensuring the recognition speed, and endows the intelligent recognition of the mine with practical significance. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the present application will be described in detail below in combination with the drawings and detailed embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor. Among them:

[0042] Figure 1 is a flowchart of a mine open-pit mining area intelligent recognition method based on satellite remote sensing of the present application;

[0043] Figure 2 is a flowchart of data set making in the present application;

[0044] Figure 3 is a network structure schematic diagram of the improved YOLOv5 in the present application;

[0045] Figure 4 is a network structure schematic diagram of U-net++ in the present application;

[0046] Figure 5 is a schematic diagram of the detection result of the mine area using YOLOv5 in the present application;

[0047] Figure 6 is a schematic diagram of the segmentation result of the mine target using U-net++ in the present application;

[0048] Figure 7 is the result of splicing the extracted result of the clipped mine to the original remote sensing image size. EMBODIMENT

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] Please refer to Figures 1-7 :

[0051] A satellite remote sensing-based intelligent identification method for a mine open-pit mining area, the total process is shown in Figure 1 , comprising the following steps:

[0052] Step 1: Obtain the original remote sensing image, and perform radiation correction and atmospheric correction on the image.

[0053] Step 2: Perform image preprocessing on the corrected remote sensing image, including band dimension reduction, image enhancement, and image cropping.

[0054] Step 3: Use the improved YOLOv5 to perform target detection on the processed image, extract the approximate mine area, and adaptively output the target range.

[0055] Step 4: Use U-Ne++t to perform image segmentation on the part of the mine area detected by YOLOv5, and extract the fine contour of the mine.

[0056] Step 5: Fuse the extraction result into the large-size image before cropping to obtain the final result.

[0057] For step 1, for the image that has been corrected, the correction can be omitted, and for the original uncorrected image, ENVI is used for radiation correction and atmospheric correction.

[0058] The specific steps for image processing in step 2 are as follows:

[0059] Step 2.1: Use principal component analysis (PCA) to reduce the dimension of multiple bands, so that the number of channels is reduced to 3, which is beneficial to subsequent use of YOLOv5 to extract targets;

[0060] Step 2.2: Use image enhancement to increase image information and reduce noise interference. The image enhancement methods applied are: image random rotation, flipping, translation, shearing, histogram equalization, and mosaic enhancement. In this embodiment, the range of random rotation is -45 degrees to 45 degrees, the maximum shear ratio is 0.2, and the mosaic enhancement adopts the method of enhancing 4 images to 1 mosaic image.

[0061] Step 2.3: Image cropping is to crop the large size image after correction into a smaller size image suitable for deep learning input, the recommended cropping size is 256, 416, 448, which is convenient for convolutional neural network recognition, and other arbitrary suitable sizes can also be cropped, but the cropping size needs to be fixed to one, which is convenient for step 5 to restore the large size image. When cropping, the relative coordinates of the cropped image need to be recorded, that is, the row and column numbers of its upper left corner and lower right corner in the original image. The cropped image is stored in different folders according to the original large size image.

[0062] For step 3, the trained YOLOv5 is used to detect the processed image. As shown in Figure 3 The improved YOLOv5 is composed of three parts: backbone, neck and head. The backbone network contains multiple CBL and CSP (Cross Stage Partial) modules, where CBL is a combination of convolution (conv), batch normalization (bn) and activation function (leakyrelu). The purpose of the CSP module is to reduce the amount of calculation and enhance the performance of gradient descent. The main idea is to divide the input into two parts, one part is calculated through the block, and the other part is directly through the shortcut, and then the two parts are spliced (concatenated). The neck uses PAN (Path Aggregation Network) network. PANet fuses deep semantic information and shallow texture information through bottom-up and top-down fusion paths, and performs multiple splicing operations. Before splicing, the dimension is adjusted by upsampling to keep the dimension consistent. After splicing, it is transmitted to the head through the CSP module. YOLOv5 contains multiple head mechanisms, a total of 3 heads, which take into account the recognition of large, medium and small objects. head1 divides the original input into 76×76 grids, which is biased towards detecting small objects, head2 is 38×38, which is biased towards recognizing medium objects, and head3 is 19×19, which is biased towards recognizing larger objects. In particular, a Transformer block is added to the backbone network to improve the perception of environmental information and increase the accuracy of small target recognition. It contains: layer normalization, multi-head attention mechanism, Dropout and MLP (Multi-Layer Perceptron).

[0063] The steps to train YOLOv5 are as follows:

[0064] Step 3.1:

[0065] (1) Make dataset: as Figure 2As shown, first, the ENVI is used to mark the ROI of the mine, and the output is an XML file; second, the geographic coordinates in the XML file are extracted using Python, and the geographic coordinates are converted to pixel coordinates using GDAL; then, PCA dimensionality reduction and image cropping are performed on the large-scale image (step 2), and the row and column coordinates of the cropped image are recorded, and the relative pixel coordinates of the cropped image are calculated based on the cropped coordinates of the large-scale image and the pixel coordinates of the mine (the row and column maximum values are within the cropped size), and each ROI region is assigned a separate ID; finally, the rectangular box label in YOLO format is extracted for each ID, i.e. the row and column ranges are calculated and converted to (x, y, w, h) form, and output in txt format;

[0066] (2) The loss of using YOLOv5 to detect the target is calculated using Focal loss to balance the class imbalance problem of mine area and background. The formula of Focal loss is as follows:

[0067]

[0068] wherein, and represent the true values of positive and negative samples in binary classification, i.e. the mine area and background label value, and represent the predicted values output by the YOLOv5 network, is an adjustable factor, when it is 0, Focal loss degenerates into binary cross-entropy loss function; using SGD as the optimizer, the initial learning rate is set to 0.01, the weight decay coefficient is 0.001, and the momentum is 0.93;

[0069] The steps for detecting the mine area using YOLOv5 are as follows:

[0070] Step 3.2:

[0071] (1) The image in step 2.2 is detected in the detect folder of YOLOv5, and adaptive cropping is performed on the image with mine area detected and saved. The confidence threshold for detection is 0.5, and adaptive cropping is used to determine the cropping requirement according to the target size, and small targets are adaptively cropped to a given minimum range to avoid too few target areas after cropping, which affects the feature extraction of U-net++ and affects the implementation of the next step of segmentation;

[0072] Figure 5 The detection results of YOLOv5 on the mine area are shown, and it can be seen that the mine area can be well detected;

[0073] For step 4, based on the images of the mining areas detected in step 3, U-net++ is used for further image segmentation processing to identify the fine contours of the mines. The structure of U-net++ is as follows: Figure 4 As shown, the left side uses 2×2 max pooling to reduce dimensionality while increasing the number of convolutional kernels, while the right side uses 2×2 deconvolution to increase dimensionality while decreasing the number of convolutional kernels. In the process of identifying mines, the classification is only either "mine" or "not a mine," therefore the number of output channels is 2.

[0074] The steps for training U-net++ are as follows:

[0075] Step 4.1:

[0076] (1) Fill the pixel coordinates of the mine sample obtained in step 3.1 to obtain the mask mark, make it into U-net++ tag format, and use all the silhouette images containing the mine sample as training samples.

[0077] (2) The U-net++ network is trained using Dice loss. The expression for the Dice coefficients is:

[0078]

[0079] in, Let represent the ground truth and the network prediction, respectively. The value of Dice loss is... ;

[0080] SGD was used as the optimizer for U-net++ with an initial learning rate of 0.001, a learning rate decay factor of 0.1, a weight decay factor of 0.0001, and a momentum of 0.90.

[0081] The steps for using U-net++ to detect a mining area are as follows:

[0082] Step 4.2:

[0083] In step 3, the detection results of YOLOv5 were obtained. The image of the target detected by YOLOv5 was used as the prediction data. The U-net++ network trained in step 4.1 was used to make predictions, which can delineate the fine boundary of the mining area. That is, the location of the mine can be identified, and the shape and coverage area of ​​the mine can also be extracted.

[0084] For step 5, the recognition results of step 4 are fused into the original image. First, a tensor with the same dimension as the original large-size remote sensing image and a value of 0 is created using Python. The image location is determined using the cropping coordinates recorded in step 2. The specific values ​​of the tensor are filled in according to the extraction results of step 4. OpenCV is used for visualization output to obtain the refined mining area recognition results of the whole large-size remote sensing image.

[0085] It has to be noted that, in the present document, the terms "first", "second", etc. merely serve to identify a subject or action, without necessarily requiring or implying any such actual relationship or order between such subjects or actions. Moreover, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusions, such that a process, method, article, or apparatus that comprises a list of elements does not include those elements solely, but can include other elements not expressly listed, or can include elements inherent in such process, method, article, or apparatus.

[0086] While embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, combinations, and variations can be undertaken by those skilled in the art without departing from the spirit and scope of the present application, which is defined by the appended claims and their equivalents.

Claims

1. A satellite remote sensing-based intelligent identification method for a mine open-pit mining area, characterized in that, The method comprises the following steps: step 1: obtaining an original remote sensing image, and correcting the image; step 2: performing image preprocessing on the corrected remote sensing image, including band dimension reduction, image enhancement and image cropping; step 3: using YOLOv5 with a Transformer block to detect the processed image, extract the mine area and adaptively output the target range; step 4: using U-net++ to perform image segmentation on the part of the mine area detected by YOLOv5, and extract the fine contour of the mine; and step 5: fusing the extraction result into the large-size image before cropping to obtain the final result. In step 2, the specific steps of image preprocessing are as follows: step 2.1: using principal component analysis (PCA) to reduce the number of channels of the multi-band to 3, which is beneficial to subsequent extraction of the target by YOLOv5; step 2.2: using image enhancement to increase image information and reduce noise interference, and the image enhancement methods applied are: image random rotation, flipping, translation, shearing, histogram equalization and mosaic enhancement; wherein, the range of random rotation is-45 degrees to 45 degrees, the maximum proportion of random shearing is 0.2, and the mosaic enhancement adopts the mode of enhancing 4 images into 1 mosaic image; step 2.3: image cropping, the cropping size is any pixel of 256, 416 or 448, which is convenient for convolutional neural network recognition, and before restoring the large-size image in step 5, the image size is cropped to a fixed size, and the relative coordinates of the cropping are recorded, that is, the row and column numbers of the upper left corner and the lower right corner of the image in the original image, and the cropped image is stored in different folders according to the original large-size image; The steps of training YOLOv5 are as follows: Step 3.1, making a data set: first, use ENVI to mark the region of interest ROI for the mine, and output as an XML file; second, use Python to extract the geographic coordinates in the XML file, and use GDAL to convert the geographic coordinates to pixel coordinates; then, do PCA dimension reduction and image cropping on large-scale images, record the row and column coordinates of the cropped image, and calculate the relative pixel coordinates of the cropped image based on the cropping coordinates of the large-scale image and the pixel coordinates for the mine, the maximum values of the row and column are within the cropping size, and each ROI region is assigned a separate ID; finally, extract the rectangular box labels in YOLO format for each ID, and output as a txt file; use Focal loss to calculate the loss of using YOLOv5 to detect targets, to balance the class imbalance problem of mine area and background; the formula of Focal loss is as follows: Where y0 and y1 represent the true values of positive and negative samples in binary classification, i.e. mine area and background label value, p0 and p1 represent the predicted values of YOLOv5 network output, is an adjustable factor, when it is 0, Focal loss degenerates into binary cross-entropy loss function; use SGD as the optimizer, set the initial learning rate to 0.01, the weight decay coefficient to 0.001, and the momentum to 0.93; Step 3.2, make YOLOv5's detect recognize folder in step 2.2, crop and save the image with mine area detected; where the confidence threshold for detection is 0.5, adaptive cropping is to judge the cropping demand according to the target size, and small targets are adaptively cropped to a given minimum range to avoid too few cropped target areas, which affects the effective extraction of features by U-net++ and affects the realization of the next step of segmentation. 2.The satellite remote sensing based intelligent identification method for mine open-pit mining area according to claim 1, characterized in that, In step 1, ENVI is used for radiation correction and atmospheric correction. 3.The satellite remote sensing based intelligent identification method for mine open-pit mining area according to claim 1, characterized in that, In step 3, the improved YOLOv5 is composed of three parts: the main grid backbone, the neck and the head. The main network contains multiple CBL and CSP modules. CBL is the combination of convolution, batch normalization and activation function. The purpose of CSP module is to reduce the amount of calculation and enhance the performance of gradient descent. The input is divided into two parts, one part is calculated through block, and the other part is directly through shortcut, then the two parts are spliced. The neck uses PAN network. PAN network fuses deep semantic information and shallow texture information through bottom-up and top-down fusion paths, and performs multiple splicing operations to realize feature fusion. Before splicing, the dimension is adjusted by upsampling to keep the dimension consistent. After splicing, the CSP module is used to conduct to the head. YOLOv5 contains multi-head mechanism, which has three heads, which can recognize large, medium and small objects. Head1 divides the original input into 76x76 grids, which is biased to detect small objects. Head2 is 38x38, which is biased to identify medium objects. Head3 is 19x19, which is biased to identify larger objects. In the main network, the Transformer block is added to improve the perception of environmental information and increase the accuracy of small target recognition. It contains layer normalization, multi-head attention mechanism, Dropout and multi-layer perceptron MLP.

4. The satellite remote sensing-based intelligent identification method for the open-pit mining area of a mine according to claim 2, characterized in that, For step 4, based on the image of the mine area detected in step 3, further image segmentation processing is performed using U-net++ to identify the fine profile of the mine. Training U-net++ includes the following steps: Step 4.1, training U-net++: fill in the pixel coordinates of the mine samples obtained in step 3.1 to obtain a mask label, and make it into a U-net++ label format. All cropped images containing mine samples are used as training samples; use Dice loss to train U-net++ network, the expression of Dice coefficient is: Wherein, X, Y represent ground truth and network prediction value respectively, the value of Dice loss is 1-D; use SGD as the optimizer of U-net++, the initial learning rate is 0.001, the learning rate decay coefficient is 0.1, the weight decay coefficient is 0.0001, and the momentum is 0.90; Step 4.2, use U-net++ to detect the mine area: the detection result of YOLOv5 in step 3 is obtained, the image adaptively output by YOLOv5 detection target is used as prediction data, and the U-net++ network trained in step 4.1 is used for prediction.

5. The satellite remote sensing-based intelligent identification method for the open-pit mining area of a mine according to claim 2, characterized in that, For step 5, the recognition result of step 4 is fused into the original image. First, use Python to create a tensor with the same dimension as the original large size remote sensing image and the value is 0. Use the clipping coordinates recorded in step 2 to determine the image position. Fill in the specific values of the tensor according to the extraction results of step 4. Use opencv to do visualization output to get the fine mine area recognition result of the whole large size remote sensing image.

Images