Strip mine slope deformation identification method based on image data
By collecting image data using drones, combining the DeeplabV3+ algorithm and MobileNetV2 network with BlackMask image enhancement technology, high-precision identification and monitoring of open-pit mine slope deformation was achieved. This solved the problems of high cost and low accuracy of traditional methods, and improved the accuracy and efficiency of slope safety monitoring.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-03
AI Technical Summary
Traditional open-pit mine slope deformation monitoring technology is costly and difficult to accurately identify deformation areas, and cannot achieve high-precision pixel-level contour delineation.
An image-based method for identifying slope deformation in open-pit mines was adopted. Image data was collected by UAVs, and image segmentation and change detection were combined. The DeeplabV3+ algorithm and MobileNetV2 network were used for slope segmentation, and BlackMask image enhancement technology was used to identify the deformation areas of the slope.
It achieves high-precision slope deformation identification, reduces costs, decreases the workload of staff, enables continuous operation for long periods of time, and improves the accuracy and efficiency of slope safety monitoring.
Smart Images
Figure CN121789094A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer machine vision, and in particular to a method for identifying deformation of open-pit mine slopes based on image data. Background Technology
[0002] In open-pit coal mining, the stability of open-pit mine slopes is a core element in ensuring safe production. Slope instability, landslides, and other deformation disasters not only cause enormous economic losses and seriously threaten the safety of personnel and equipment, but can also trigger significant ecological and environmental problems. Therefore, high-precision and high-efficiency deformation monitoring and identification of open-pit mine slopes has always been a key technological challenge in the field of mine safety. Traditional contact-based slope deformation monitoring technologies, such as total stations, slope radar, and displacement gauges, estimate the overall displacement of the slope by periodically or continuously collecting data from discrete, predetermined monitoring points, resulting in high deployment costs. With the development of computer vision technology, methods have emerged that use drones or fixed cameras to collect slope images and perform deformation analysis through image processing. Early methods relied heavily on target detection or feature point matching techniques. Deep learning neural networks, after proper training, can recognize different types of images and exhibit excellent performance. However, they typically only provide bounding boxes containing the deformed area, failing to accurately delineate the pixel-level contours of the deformed region.
[0003] Therefore, a method for identifying deformation of open-pit mine slopes based on image data is provided to solve the above problems. Summary of the Invention
[0004] The purpose of this invention is to provide a method for identifying slope deformation in open-pit mines based on image data. In the actual operating environment of open-pit coal mines, this method can effectively improve the accuracy of slope identification and deformation monitoring. It is also low in cost, simple to install, and can work continuously for a long time, reducing the workload of workers and providing ideas for subsequent slope safety monitoring and practical engineering applications.
[0005] To achieve the above objectives, this invention provides a method for identifying deformation of open-pit mine slopes based on image data. The main structure includes image segmentation and change detection, and specifically includes the following steps: S1: Data acquisition. The drone acquires image data of the slope on site. Taking into account lighting, angle, height and drone performance, the drone flight route is planned to "surround the slope and ascend step by step" to acquire complete slope image data. S2: Data augmentation processing, combining traditional data augmentation with BlackMask image enhancement techniques, inserts black pixel blocks of a specific size as masks between pixels in the original image, forcing the visual model to extract features from the unmasked image areas. The density of the randomly added mask pixel blocks in the original image is controlled by Noise_ratio, and the size of the mask pixel blocks themselves is changed by the Block_size value, calculated as follows: ; In the formula, N represents a pixel block; W H and H represent the width and height of the black pixel block, respectively; S3: Create the dataset. During the dataset construction process, the labelme platform is used in Anaconda software to label and name the data augmented slope image dataset, and obtain the corresponding JSON file, including the slope name, coordinates and length and width information, to obtain the training dataset. The obtained training dataset is divided into training set and test set according to 8:2. S4: Build a slope image segmentation network, use the DeeplabV3+ algorithm to achieve intelligent slope segmentation, and input the training set into the slope image segmentation network for training; S5: Extract the slope mask from the segmentation results, and use the optimal training weight file obtained from the image segmentation model to segment two slope images from different time phases (P... t and P t+1 Slope segmentation prediction is performed. S6: Change detection network identifies the correspondence between image pixels in different time phases, identifies the difference map of two time phase slope mask files by image feature matching, eliminates pseudo-change noise and maps it to the original image, and realizes the output of slope deformation area identification results.
[0006] Preferably, the coal seam outcrops or gravel areas in S1 are switched to manual mode for shooting.
[0007] Preferably, in S2, image preprocessing techniques such as image denoising and image enhancement are used. Image enhancement preprocessing of training images is performed by rotating, changing image brightness and color saturation, and adjusting size to balance the number of images, highlight the main features of foreign objects, and expand the existing image dataset.
[0008] Preferably, under the influence of different production scales, mining methods, geological conditions, surrounding topography and rainfall and snowfall factors in S3, the pattern of open-pit mine slopes varies in actual site conditions. From the perspective of drones, the core difference in slope characteristics of different mining areas is reflected in the slope shape. Three slope shapes are set as straight, broken, and staggered.
[0009] Preferably, the main structure of the DeeplabV3+ model in S4 consists of two parts: an encoding module (Encode) and a decoding module (Decode). In the encoding module (Encode), the following steps are performed: S41: The Xception network extracts features from the slope area in the image; S42: Perform a 4x upsampling on the input high-level semantic features, then stack them with the low-level semantic features to achieve feature fusion, resulting in a resolution equal to the original. Figure 1 / 4 feature map; S43: Spatial Pyramid Pooling (ASPP) expands the receptive field and captures richer contextual information by integrating multiple dilated convolutions with different dilation rates, while simultaneously performing convolution and pooling operations on the feature maps. In the Decode module, perform the following operations: The feature map output from the encoder S42 is restored to the original image size using bilinear interpolation. Feature compression is then performed using 3×3 convolution to ensure that the output size is consistent with the input image size, thus completing the segmentation of the slope in the image. In the DeeplabV3+ model, the backbone network Xception is replaced with a lightweight MobileNetV2 network, and the CBAM attention mechanism is incorporated into the inverse residual structure of MobileNetV2.
[0010] Preferably, MobileNetV2 reduces the complexity of the model through depthwise separable convolution operations. The network structure includes depthwise separable convolutional layers (DSC), inverse residual structures (IR), and linear bottleneck structures (LB). DSC decomposes the standard convolution operation into pointwise convolution with depthwise kernels. Depthwise convolution extracts spatial features of the image through independent convolution kernels, while pointwise convolution achieves linear combination of channels through 1×1 convolution kernels. IR uses a "spread-convolution-compression" structure design for convolution. LB reduces the number of channels and computational cost while preserving useful image features to the maximum extent.
[0011] Preferably, intelligent segmentation in S4 can also employ YOLO-Seg or U-Net image segmentation models.
[0012] Preferably, the process of extracting the slope mask from the segmentation results in S5 is as follows: S51: Extract the original image dimensions and create a full black canvas of the same size to hold the merge mask; S52: Traverse the segmentation mask in the result, convert the segmentation mask from the tensor data output by the model into a NumPy array, adjust the size to match the original image, and then convert it into 255 levels of grayscale values. Use np.maximum to achieve the superposition and merging of multiple masks. S53: After thresholding, a pure binary mask is obtained. The pure binary mask only contains 0 and 255. The single-channel and three-channel black and white binary mask images (white foreground and black background) are saved respectively and named mask1 and mask2.
[0013] Preferably, in S6, the dual-time contrast network performs calculations based on the segmented binary masks, loads the two mask images mask1 and mask2 from S53, converts mask1 and mask2 into grayscale images and binarizes them, then distinguishes the change type through bitwise operations, and uses morphological opening operations to denoise and filter small areas to optimize the accuracy of the changed areas; cv2.bitwise_and is used to calculate the overlapping area of the mask1 and mask2, the mask area that mask1 has but mask2 does not have is the disappeared slope area, and the mask area that mask2 has but mask1 does not have is the newly added slope area. Then the newly added slope area and the disappeared slope area are superimposed on the original image in the form of "semi-transparent filling" to obtain the area of slope deformation.
[0014] Therefore, the present invention employs the above-mentioned method for identifying deformation of open-pit mine slopes based on image data, and the technical effects are as follows: (1) This invention is beneficial for monitoring the deformation of open-pit mine slopes, reducing the occurrence of slope accidents and improving the safety level of open-pit mines.
[0015] (2) This invention can effectively improve the accuracy of slope identification and deformation monitoring, and is low in cost, simple to install, and can work continuously for a long time, reducing the workload of staff and providing ideas for subsequent slope safety monitoring and engineering applications.
[0016] (3) The DeeplabV3+ framework of this invention, with its unique “Encoder-Decoder” network design, can effectively alleviate the resolution loss problem caused by downsampling of traditional convolutional networks. Through the dilated spatial pyramid pooling (ASPP) module, it can extract multi-scale contextual information by using dilated convolution with different dilation rates. It can simultaneously take into account the overall shape and local details of the slope, and can achieve accurate segmentation of open-pit mine slope images. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the slope deformation detection algorithm of an open-pit mine slope deformation identification method based on image data according to the present invention. Figure 2 This is a schematic diagram of the route for acquiring slope images by a drone in an embodiment of the present invention; Figure 3 This is a schematic diagram of the open-pit mine slope image data enhancement process in an embodiment of the present invention; Figure 4This is a network structure diagram of MobileNetV2 in an embodiment of the present invention; Figure 5 This is a diagram of the IR structure incorporating CBAM in an embodiment of the present invention. Detailed Implementation
[0018] The method of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0019] Unless otherwise defined, the methodological or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.
[0020] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this invention, unless otherwise explicitly specified and limited, the term "attached" and similar terms should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0021] Example like Figures 1-3 As shown, a method for identifying deformation of open-pit mine slopes based on image data, the main structure of which includes image segmentation and change detection, specifically includes the following steps: S1: Data Acquisition. Image data of the slope on site is obtained through drones. Taking into account lighting, angle, height and drone performance, a drone flight route of "surrounding all sides and gradually ascending" is planned to obtain complete slope image data. Coal seam outcrops or gravel areas in S1 are switched to manual mode for shooting.
[0022] S2: Data Augmentation Processing. Traditional data augmentation methods utilize geometric techniques such as image rotation, scaling, and cropping. However, engineering practice has shown that these methods are insufficient for improving the performance of slope segmentation models. This is because open-pit coal mine slopes have complex backgrounds and relatively regular geometric shapes, making it difficult for traditional data augmentation strategies to effectively expand the diversity of image samples, thus resulting in poor performance improvement for the visual model. To address this issue, this method combines traditional data augmentation with BlackMask image augmentation techniques. Black pixel blocks of a specific size are inserted as masks between pixels in the original image, forcing the visual model to extract features from the unmasked image areas. The density of the randomly added mask pixel blocks in the original image is controlled by Noise_ratio, and the size of the mask pixel blocks is changed by the Block_size value, calculated as follows: ; In the formula, N represents a pixel block; W H and H represent the width and height of the black pixel block, respectively; In S2, image preprocessing techniques such as image denoising and image enhancement are used. Rotation, changes in image brightness and color saturation, and resizing are used to enhance the training images, balance the number of images, highlight the main features of foreign objects, and expand the existing image dataset.
[0023] S3: Dataset Creation. During the dataset construction process, the labelme platform was used in Anaconda software to label and name the augmented slope image dataset (Slope1, Slope2, and Slope3), obtaining the corresponding JSON files, including slope names, coordinates, and length and width information, to obtain the training dataset. The obtained training dataset was divided into training and test sets in an 8:2 ratio. In S3, under the influence of different production scales, mining methods, geological conditions, surrounding terrain, and rainfall and snowfall factors, the pattern of open-pit mine slopes varies in actual field. From the perspective of drones, the core difference in slope characteristics of different mining areas is reflected in the slope shape. Three slope shapes were set as straight, broken, and staggered.
[0024] S4: A slope image segmentation network is built, and the DeeplabV3+ algorithm is used to achieve intelligent slope segmentation. The training set is input into the slope image segmentation network for training. With its unique "Encoder-Decoder" network design, it effectively alleviates the resolution loss problem caused by downsampling in traditional convolutional networks. Through the dilated spatial pyramid pooling (ASPP) module, multi-scale contextual information is extracted using dilated convolutions with different dilation rates, simultaneously taking into account the overall shape and local details of the slope, enabling accurate segmentation of open-pit mine slope images. Intelligent segmentation in S4 can also employ YOLO-Seg or U-Net image segmentation models.
[0025] The main structure of the DeeplabV3+ model in S4 consists of two parts: the encoding module (Encode) and the decoding module (Decode). The encoding module (Encode) performs the following steps: S41: The Xception network extracts features from the slope area in the image; S42: Perform a 4x upsampling on the input high-level semantic features, then stack them with the low-level semantic features to achieve feature fusion, resulting in a resolution equal to the original. Figure 1 / 4 feature map; S43: Spatial Pyramid Pooling (ASPP) expands the receptive field and captures richer contextual information by integrating multiple dilated convolutions with different dilation rates, while simultaneously performing convolution and pooling operations on the feature maps. In the Decode module, perform the following operations: The feature map output from the encoder S42 is restored to the original image size using bilinear interpolation. Feature compression is then performed using 3×3 convolution to ensure that the output size is consistent with the input image size, thus completing the segmentation of the slope in the image. In the DeeplabV3+ model, the backbone network Xception is replaced with the lightweight MobileNetV2 network, such as... Figure 5 As shown, the CBAM attention mechanism is incorporated into the inverse residual structure of MobileNetV2 to improve the DeeplabV3+ model's ability to identify and segment slopes.
[0026] MobileNetV2 reduces model complexity through depthwise separable convolution operations, thereby extracting feature information from slope images more efficiently and accurately. It also achieves accurate global information recognition with a smaller model size and computational cost, and is easily portable and optimizable. The network structure is as follows: Figure 4 As shown, it includes depth-separable convolutional layers (DSC), inverse residual structures (IR), and linear bottleneck structures (LB). DSC decomposes the standard convolution operation into pointwise convolution with depthwise kernels. Depthwise convolution extracts spatial features of the image through independent convolution kernels, while pointwise convolution achieves linear combination of channels through 1×1 convolution kernels. IR uses a "spread-convolution-compression" structure design for convolution. LB reduces the number of channels and computational cost while preserving useful image features to the maximum extent.
[0027] S5: Extract the slope mask from the segmentation results, and use the optimal training weight file obtained from the image segmentation model to segment two slope images from different time phases (P... t and P t+1 Slope segmentation prediction is performed; the process of extracting the slope mask from the segmentation results in S5 is as follows: S51: Extract the original image dimensions and create a full black canvas of the same size to hold the merge mask; S52: Traverse the segmentation mask in the result, convert the segmentation mask from the tensor data output by the model into a NumPy array, adjust the size to match the original image, and then convert it into 255 levels of grayscale values. Use np.maximum to achieve the superposition and merging of multiple masks. S53: After thresholding, a pure binary mask is obtained. The pure binary mask only contains 0 and 255. The single-channel and three-channel black and white binary mask images (white foreground and black background) are saved respectively and named mask1 and mask2.
[0028] S6: Change detection network identifies the correspondence between image pixels in different time phases, identifies the difference map of two time phase slope mask files by image feature matching, eliminates pseudo-change noise and maps it to the original image, and realizes the output of slope deformation area identification results.
[0029] In S6, the dual-time contrast network does not directly perform difference analysis on the original RGB image. Instead, it calculates based on the segmented binary mask. It loads two mask images, mask1 and mask2, from S53, converts them to grayscale and binarizes them, then distinguishes the type of change through bitwise operations, and uses morphological opening operations to denoise and filter small areas to optimize the accuracy of the changed areas. It uses cv2.bitwise_and to calculate the overlapping area of the mask1 and mask2. The mask areas that mask1 has but mask2 does not have are the disappeared slope areas, and the mask areas that mask2 has but mask1 does not have are the newly added slope areas. Then, the newly added slope areas and the disappeared slope areas are superimposed on the original image in the form of "semi-transparent filling" to obtain the area of slope deformation.
[0030] Therefore, this invention employs the aforementioned open-pit mine slope deformation recognition method based on image data. It uses an image segmentation framework to identify slope regions in images, extracts segmentation masks representing these regions into a change detection framework, and identifies slope changes by comparing differences in the mask files. Specifically, the image segmentation algorithm for open-pit mine slope regions in UAV images uses two independent image segmentation frameworks. The change detection network for the mask files primarily uses feature alignment, performing dual spatial and semantic feature alignment on the slope segmentation masks corresponding to UAV images from different time periods. Finally, it combines adaptive threshold segmentation with original image mapping to output the areas of slope change, effectively improving the accuracy of slope recognition and deformation monitoring.
[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for identifying deformation of open-pit mine slopes based on image data, characterized in that, The main structure includes image segmentation and change detection, specifically including the following steps: S1: Data acquisition. The drone acquires image data of the slope on site. Taking into account lighting, angle, height and drone performance, the drone flight route is planned to "surround the slope and ascend step by step" to acquire complete slope image data. S2: Data augmentation processing, combining traditional data augmentation with BlackMask image enhancement techniques, inserts black pixel blocks of a specific size as masks between pixels in the original image, forcing the visual model to extract features from the unmasked image areas. The density of the randomly added mask pixel blocks in the original image is controlled by Noise_ratio, and the size of the mask pixel blocks themselves is changed by the Block_size value, calculated as follows: ； In the formula, N represents a pixel block; W H and H represent the width and height of the black pixel block, respectively; S3: Create the dataset. During the dataset construction process, the labelme platform is used in Anaconda software to label and name the data augmented slope image dataset, and obtain the corresponding JSON file, including the slope name, coordinates and length and width information, to obtain the training dataset. The obtained training dataset is divided into training set and test set according to 8:
2. S4: Build a slope image segmentation network, use the DeeplabV3+ algorithm to achieve intelligent slope segmentation, and input the training set into the slope image segmentation network for training; S5: Extract the slope mask from the segmentation results, and use the optimal training weight file obtained from the image segmentation model to segment two slope images from different time phases (P... t and P t+1 Slope segmentation prediction is performed. S6: Change detection network identifies the correspondence between image pixels in different time phases, identifies the difference map of two time phase slope mask files by image feature matching, eliminates pseudo-change noise and maps it to the original image, and realizes the output of slope deformation area identification results.
2. The method for identifying open-pit mine slope deformation based on image data according to claim 1 is characterized in that, In S1, switch to manual mode for shooting coal seam outcrops or gravel areas.
3. The method for identifying open-pit mine slope deformation based on image data according to claim 1, characterized in that, In S2, image preprocessing techniques such as image denoising and image enhancement are used. Rotation, changes in image brightness and color saturation, and resizing are used to enhance the training images, balance the number of images, highlight the main features of foreign objects, and expand the existing image dataset.
4. The method for identifying open-pit mine slope deformation based on image data according to claim 1, characterized in that, Under the influence of different production scales, mining methods, geological conditions, surrounding topography, and rainfall and snowfall factors, the pattern of open-pit mine slopes varies in actual field. From the perspective of drones, the core difference in slope characteristics of different mining areas is reflected in the slope shape. Three slope shapes are set as straight, broken, and staggered.
5. The method for identifying open-pit mine slope deformation based on image data according to claim 1, characterized in that, The main structure of the DeeplabV3+ model in S4 consists of two parts: the encoding module (Encode) and the decoding module (Decode). The encoding module (Encode) performs the following steps: S41: The Xception network extracts features from the slope area in the image; S42: Perform a 4x upsampling on the input high-level semantic features, and then stack them with the low-level semantic features to achieve feature fusion, resulting in a feature map with a resolution of 1 / 4 of the original image; S43: Spatial Pyramid Pooling (ASPP) expands the receptive field and captures richer contextual information by integrating multiple dilated convolutions with different dilation rates, while simultaneously performing convolution and pooling operations on the feature maps. In the Decode module, perform the following operations: The feature map output from the encoder S42 is restored to the original image size using bilinear interpolation. Feature compression is then performed using 3×3 convolution to ensure that the output size is consistent with the input image size, thus completing the segmentation of the slope in the image. In the DeeplabV3+ model, the backbone network Xception is replaced with a lightweight MobileNetV2 network, and the CBAM attention mechanism is incorporated into the inverse residual structure of MobileNetV2.
6. The method for identifying open-pit mine slope deformation based on image data according to claim 5, characterized in that, MobileNetV2 reduces model complexity through depthwise separable convolution operations. The network structure includes depthwise separable convolutional layers (DSC), inverse residual structures (IR), and linear bottleneck structures (LB). DSC decomposes the standard convolution operation into pointwise convolution with depthwise kernels. Depthwise convolution extracts spatial features of the image through independent convolution kernels, while pointwise convolution achieves linear combination of channels through 1×1 convolution kernels. IR uses a "spread-convolution-compression" structure design for convolution. LB reduces the number of channels and computational cost while preserving useful image features to the maximum extent.
7. The method for identifying open-pit mine slope deformation based on image data according to claim 1, characterized in that, In S4, intelligent segmentation can also employ YOLO-Seg or U-Net image segmentation models.
8. The method for identifying open-pit mine slope deformation based on image data according to claim 1, characterized in that, The process of extracting the slope mask from the segmentation results in S5 is as follows: S51: Extract the original image dimensions and create a full black canvas of the same size to hold the merge mask; S52: Traverse the segmentation mask in the result, convert the segmentation mask from the tensor data output by the model into a NumPy array, adjust the size to match the original image, and then convert it into 255 levels of grayscale values. Use np.maximum to achieve the superposition and merging of multiple masks. S53: After thresholding, a pure binary mask is obtained. The pure binary mask only contains 0 and 255. The single-channel and three-channel black and white binary mask images (white foreground and black background) are saved respectively and named mask1 and mask2.
9. The method for identifying open-pit mine slope deformation based on image data according to claim 8, characterized in that, In S6, the dual-time contrast network performs calculations based on the segmented binary masks. It loads two mask images, mask1 and mask2, from S53, converts them to grayscale and binarizes them, then distinguishes the type of change through bitwise operations, and uses morphological opening operations to denoise and filter small areas to optimize the accuracy of the changed areas. It uses cv2.bitwise_and to calculate the overlapping area of the mask1 and mask2. The mask areas that mask1 has but mask2 does not have are the disappeared slope areas, and the mask areas that mask2 has but mask1 does not have are the newly added slope areas. Then, the newly added slope areas and the disappeared slope areas are superimposed on the original image in the form of "semi-transparent filling" to obtain the area of slope deformation.