Method for generating strip mine live-action three-dimensional step model
By extracting semantic information of open-pit mine benches through multi-scale segmentation and improved D-LinkNet and Mask R-CNN methods, a real-world 3D model of the open-pit mine is generated, solving the problems of bench uplift and crack detection, and improving the efficiency of mine safety monitoring and mining efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 甘肃省矿业权管理服务中心
- Filing Date
- 2023-12-14
- Publication Date
- 2026-04-24
AI Technical Summary
Existing methods based on point cloud clustering and DSM processing cannot effectively detect safety hazards such as bench heave, cracks and collapses in open-pit mines, resulting in low efficiency of mine safety monitoring.
We employ multi-scale segmentation of UAV oblique image processing, improved D-LinkNet and Mask R-CNN methods to extract semantic information of steps and reconstruct a real-world 3D model of an open-pit mine, integrating information on steps, cracks and uplifted soil.
It improves the efficiency of safety monitoring in open-pit mines, enabling timely detection and monitoring of bench changes, ensuring safe mine operation, optimizing mining plans, and reducing environmental impact.
Smart Images
Figure CN121921453A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of three-dimensional feature identification and reconstruction in intelligent mine monitoring, and particularly relates to a method for generating a three-dimensional step model of an open-pit mine. Background Technology
[0002] As a key component of open-pit mines, benches are unit bodies formed by horizontally stratifying rocks and minerals according to design rules during the mining process. They are crucial for mine planning and design, resource management, safety assessment, and environmental protection. Because open-pit mining activities cause surface changes, bench variations can impact the stability and safety of the mining area. Parameters such as bench location, height, and slope help determine reasonable mining plans, optimize mine layout and structure, and improve mining efficiency. Simultaneously, timely monitoring of bench deformation helps prevent geological disasters and accidents, ensuring the safety of miners and equipment. Understanding bench distribution and characteristics contributes to rational land reclamation and vegetation restoration, reducing the mine's impact on the surrounding environment and achieving sustainable development.
[0003] As mining operations deepen, the stability and management of benches and slopes are receiving increasing attention. Unmanned aerial vehicle (UAV) oblique photography provides strong support for the acquisition of 3D model data for open-pit mines. How to generate open-pit mine bench models based on UAV oblique imagery is a key and challenging issue in intelligent production and management of mining areas, and is of great significance to digital mine construction and intelligent mine monitoring. From the perspective of mine production and safety early warning, the establishment and updating of open-pit mine bench models plays a crucial role in the processes of open-pit mine mining, blasting, and operation management. Guided by the "Two Mountains" theory (referring to the concept of "lucid waters and lush mountains are invaluable assets"), constructing a real-world 3D bench model of an open-pit mine using semantically meaningful UAV oblique imagery enables timely detection and monitoring of bench changes, facilitating the implementation of corresponding measures to ensure the safe operation of the mining area. Therefore, timely and accurate detection and monitoring of bench surface deformation is of great significance for ensuring safe mine production and improving mining efficiency. However, methods such as point cloud clustering and DSM processing for bench line extraction cannot meet the needs of detecting safety hazards such as bench heave, cracks, and collapses. Summary of the Invention
[0004] The present invention addresses the technical problem that existing methods based on point cloud clustering and DSM processing for extracting step lines cannot meet the needs of detecting safety hazards such as step uplift, cracks, and collapse. It provides a method for generating a three-dimensional step model of an open-pit mine. This method utilizes oblique UAV images to extract targets such as steps, cracks, and soil uplift, and then reconstructs the three-dimensional model of the open-pit mine using semantic methods. This can effectively improve the efficiency of safety monitoring in open-pit mines and provide a basis for decision-making in preventing and curbing major accidents in open-pit mines.
[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0006] A method for generating a 3D staircase model of an open-pit mine, comprising:
[0007] (1) UAV oblique image processing based on multi-scale segmentation
[0008] Multi-scale segmentation of high-precision UAV oblique images;
[0009] (2) Step extraction based on improved D-LinkNet
[0010] Extracting step semantic information from UAV tilted images using an improved D-LinkNet;
[0011] (3) Detection of step cracks and heaving soil based on Mask R-CNN
[0012] Mask R-CNN was used to detect step cracks and heaving soil in oblique images from UAVs and the detection results were extracted.
[0013] (4) Reconstruction of 3D model of open-pit mine based on step semantic information
[0014] By integrating the semantic information of the steps, the detection information of step cracks and bulges into the 3D modeling process of the open-pit mine, a 3D model of the open-pit mine with semantic information is generated.
[0015] Furthermore, in step (1), the step of multi-scale segmentation of the high-precision UAV oblique image includes:
[0016] ① Divide the high-precision UAV tilt image into 512-pixel × 512-pixel images with a step size of 256 pixels;
[0017] ② Divide each 512-pixel × 512-pixel image into five segments: top left, bottom left, top right, bottom right, and center. Then flip the resulting image set to obtain multiple segmented images.
[0018] ③ Use bicubic interpolation with a 4-pixel × 4-pixel neighborhood to enlarge all images to 512 pixels × 512 pixels.
[0019] Preferably, in step ②, each 512-pixel × 512-pixel image is divided according to the multi-scale factor s = [0.875, 0.75, 0.625], and images of size 448-pixel × 448-pixel, 384-pixel × 384-pixel, and 320-pixel × 320-pixel are output respectively. Then, the resulting image set is horizontally and vertically flipped to obtain a total of 20 divided images.
[0020] Further, in step (2), the step of extracting step semantic information from UAV tilted images using the improved D-LinkNet includes:
[0021] ① Encoder feature extraction: The ResNet convolutional neural network is used to extract features from the input 512-pixel × 512-pixel UAV tilted image. The encoder gradually reduces the spatial resolution of the image while increasing the depth of the feature channels.
[0022] ② Feature Pyramid Attention Module: The feature pyramid attention module is embedded in the intermediate layer of the convolutional neural network ResNet and combined with the pooling strategy to better learn feature representations. The spatial pyramid attention module dynamically focuses on the step features according to different spatial scales, thereby achieving accurate segmentation of open-pit mine step information.
[0023] ③ Decoding and upsampling: The decoder combines the features extracted by the encoder and gradually restores the spatial resolution of the image through the linking module to generate a segmentation map that matches the size of the original image;
[0024] ④ Step generation and post-processing: At the model output, a pixel-level classifier assigns each pixel to the "step" or "non-step" category, generating a binary segmentation map of the steps, and then performs post-processing on the segmentation results.
[0025] Preferably, the post-processing includes morphological operations, small region removal, and smoothing.
[0026] Furthermore, the step of using Mask R-CNN to detect step cracks and bulges in UAV tilted images and extracting the detection results includes:
[0027] ①Feature extraction: A pre-trained convolutional neural network ResNet is used to extract feature representations from the input 512-pixel × 512-pixel UAV tilted image;
[0028] ② Candidate region extraction: A series of candidate target regions are generated in the 512-pixel × 512-pixel UAV oblique image by using a region candidate network;
[0029] ③ Target classification and bounding box regression: For each candidate region, Mask R-CNN classifies it and regresses its bounding box to determine the category and location of the target;
[0030] ④ Instance segmentation: Based on object classification and bounding box regression, Mask R-CNN introduces an additional branch to generate a precise pixel-level segmentation mask for each object;
[0031] Furthermore, in step (4), the semantic information of the steps, the detection information of the step cracks and bulges are fused with the original UAV tilted image with geographical location information to generate a Mesh model with step element information that can be used for real-world 3D modeling of open-pit mines.
[0032] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are as follows:
[0033] (1) To address the problem of limited data volume of oblique images from mining drones, this invention designs a multi-scale segmentation image processing method, which can effectively improve the accuracy of mining element extraction without model overfitting.
[0034] (2) To address the problem of refined extraction of benches in open-pit mines, an improved D-LinkNet method was used to extract the semantic information of the benches, while the Mask R-CNN method was used to extract the information of bench cracks and heaving soil.
[0035] (3) Integrating the semantic information of step lines, step cracks and uplifted soil into the open-pit mine real scene 3D modeling process can intuitively reflect the semantic information of open-pit mine steps, step cracks and uplifted soil on the 3D model, which is convenient for decision-makers to perform parameter calculation and safety supervision of the mine model. Attached Figure Description
[0036] Figure 1 This is a schematic diagram of a platform model in an open-pit mine.
[0037] Figure 2 This is a flowchart illustrating the generation process of the open-pit mine real-world three-dimensional staircase model according to the present invention;
[0038] Figure 3 This is a schematic diagram of the multi-scale segmentation UAV tilted image processing in this invention;
[0039] Figure 4 This is a schematic diagram of the step extraction model based on the improved D-LinkNet in this invention;
[0040] Figure 5 This is a schematic diagram of the feature pyramid attention structure in this invention;
[0041] Figure 6 This is a schematic diagram of the extraction of step cracks and heaving soil based on Mask R-CNN in this invention. Detailed Implementation
[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0043] like Figure 1 and 2 As shown in this embodiment, a method for generating a three-dimensional staircase model of an open-pit mine includes:
[0044] (1) UAV oblique image processing based on multi-scale segmentation
[0045] Multi-scale segmentation of high-precision UAV oblique images;
[0046] (2) Step extraction based on improved D-LinkNet
[0047] Extracting step semantic information from UAV tilted images using an improved D-LinkNet;
[0048] (3) Detection of step cracks and heaving soil based on Mask R-CNN
[0049] Mask R-CNN was used to detect step cracks and heaving soil in oblique images from UAVs and the detection results were extracted.
[0050] (4) Reconstruction of 3D model of open-pit mine based on step semantic information
[0051] By integrating the semantic information of the steps, the detection information of step cracks and bulges into the 3D modeling process of the open-pit mine, a 3D model of the open-pit mine with semantic information is generated.
[0052] Example
[0053] 1. UAV oblique image processing based on multi-scale segmentation
[0054] The number of oblique drone images of open-pit mines is relatively limited, and how to detect and identify steps, cracks, and uplifts based on this limited data is a problem that needs to be solved. Figure 3As shown, to effectively extract image features while avoiding overfitting during model training, the UAV tilted image is first segmented into 512 pixel × 512 pixel images with a step size of 256 pixels. Then, each 512 pixel × 512 pixel image is further segmented according to a multi-scale factor s = [0.875, 0.75, 0.625], outputting images of sizes 448 pixel × 448 pixels, 384 pixel × 384 pixels, and 320 pixel × 320 pixels. Specifically, each 512 pixel × 512 pixel image is segmented in five ways: top left, bottom left, top right, bottom right, and center. The resulting image set is then horizontally and vertically flipped, yielding a total of 20 segmented images. Finally, a bicubic interpolation method with a 4 pixel × 4 pixel neighborhood is used to enlarge all images to 512 pixel × 512 pixels. This multi-scale segmentation of UAV tilted images effectively improves feature extraction from open-pit mine UAV tilted images and avoids overfitting during training.
[0055] 2. Step Extraction Based on Improved D-LinkNet
[0056] Step information extraction is a crucial part of slope safety monitoring, providing fundamental calculation conditions for the automated acquisition of parameters such as slope angle and step height. D-LinkNet is a deep learning architecture specifically designed for image segmentation tasks. By combining the advantages of deep convolutional networks and LinkNet, it improves the ability to identify specific features in images. Although D-LinkNet enhances the extraction capability of step information, this network does not fully utilize the multi-level features of open-pit mines, and the extraction results have significant room for optimization.
[0057] In open-pit mine applications, to extract more accurate step information, a feature pyramid attention module is embedded in the intermediate layer to accurately extract semantic information of steps from UAV oblique imagery. Steps in open-pit mine images have high saliency, thus requiring a large receptive field to fully capture their global features. To address this issue, a feature pyramid network and an attention mechanism are combined to fully leverage the advantages of extracting global features from feature maps. This approach better captures the feature information of the step structure, improving the accuracy of step extraction in open-pit mine environments.
[0058] like Figure 4 and 5 As shown, the main process of step extraction in this embodiment includes the following steps:
[0059] (1) Encoder feature extraction: The input 512-pixel × 512-pixel image is extracted using a convolutional neural network ResNet. The encoder gradually reduces the spatial resolution of the image while increasing the depth of the feature channels.
[0060] (2) Feature pyramid attention module: The feature pyramid attention module is embedded in the middle layer of the convolutional neural network ResNet and combined with the pooling strategy to learn the feature representation better. The feature pyramid attention mechanism can dynamically focus on the step features according to different spatial scales, thereby achieving accurate segmentation of open-pit mine step information.
[0061] (3) Decoding and upsampling: The decoder combines the features extracted by the encoder and gradually restores the spatial resolution of the image through the linking module to generate a segmentation map that matches the size of the original image.
[0062] (4) Step generation and post-processing: At the model output, each pixel is assigned to the "step" or "non-step" category by a pixel-level classifier to generate a binary segmentation map of the step. The segmentation results are then post-processed, such as morphological operations, small region removal and smoothing, to improve the continuity of the step surface and the overall accuracy.
[0063] 3. Extraction of step cracks and heaving soil based on Mask R-CNN
[0064] Mask R-CNN is a widely used method in computer vision and remote sensing interpretation, which combines the capabilities of object detection and semantic segmentation.
[0065] like Figure 6 As shown, for an input 512-pixel × 512-pixel open-pit mine image, the process of extracting step cracks and uplifted earthwork based on Mask R-CNN includes the following steps:
[0066] (1) Feature extraction: Use a pre-trained convolutional neural network ResNet to extract feature representations of the input image;
[0067] (2) Candidate region extraction: A series of candidate target regions are generated in the image by using a Region Proposal Network (RPN);
[0068] (3) Target classification and bounding box regression: For each candidate region, Mask R-CNN classifies it and performs bounding box regression to determine the category and location of the target;
[0069] (4) Instance segmentation: Based on object classification and bounding box regression, Mask R-CNN introduces an additional branch to generate a precise pixel-level segmentation mask for each object.
[0070] 4. Reconstruction of 3D models of open-pit mines based on the semantic information of steps
[0071] By fusing UAV oblique images containing semantic information about steps, step cracks, and uplifted earthwork with original UAV oblique images containing geographic location information such as GPS, a Mesh model with step feature information can be generated for 3D modeling.
[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for generating a three-dimensional staircase model of an open-pit mine, characterized in that, include: (1) UAV oblique image processing based on multi-scale segmentation Multi-scale segmentation of high-precision UAV oblique images; (2) Step extraction based on improved D-LinkNet Extracting step semantic information from UAV tilted images using an improved D-LinkNet; (3) Detection of step cracks and heaving soil based on Mask R-CNN Mask R-CNN was used to detect step cracks and heaving soil in oblique images from UAVs and the detection results were extracted. (4) Reconstruction of 3D model of open-pit mine based on step semantic information By integrating the semantic information of the steps, the detection information of step cracks and bulges into the 3D modeling process of the open-pit mine, a 3D model of the open-pit mine with semantic information is generated.
2. The generation method according to claim 1, characterized in that, Step (1) involves multi-scale segmentation of the high-precision UAV oblique imagery, including: ① Divide the high-precision UAV tilt image into 512-pixel × 512-pixel images with a step size of 256 pixels; ② Divide each 512-pixel × 512-pixel image into five segments: top left, bottom left, top right, bottom right, and center. Then flip the resulting image set to obtain multiple segmented images. ③ Use bicubic interpolation with a 4-pixel × 4-pixel neighborhood to enlarge all images to 512 pixels × 512 pixels.
3. The generation method according to claim 2, characterized in that: In step ②, each 512-pixel × 512-pixel image is divided according to the multi-scale factor s = [0.875, 0.75, 0.625], and images of size 448-pixel × 448-pixel, 384-pixel × 384-pixel, and 320-pixel × 320-pixel are output respectively. Then, the resulting image set is horizontally and vertically flipped to obtain a total of 20 divided images.
4. The generation method according to claim 3, characterized in that, In step (2), the step of extracting step semantic information from UAV tilted images using the improved D-LinkNet includes: ① Encoder feature extraction: The ResNet convolutional neural network is used to extract features from the input 512-pixel × 512-pixel UAV tilted image. The encoder gradually reduces the spatial resolution of the image while increasing the depth of the feature channels. ② Feature Pyramid Attention Module: The feature pyramid attention module is embedded in the intermediate layer of the convolutional neural network ResNet and combined with the pooling strategy to better learn feature representations. The spatial pyramid attention module dynamically focuses on the step features according to different spatial scales, thereby achieving accurate segmentation of open-pit mine step information. ③ Decoding and upsampling: The decoder combines the features extracted by the encoder and gradually restores the spatial resolution of the image through the linking module to generate a segmentation map that matches the size of the original image; ④ Step generation and post-processing: At the model output, a pixel-level classifier assigns each pixel to the "step" or "non-step" category, generating a binary segmentation map of the steps, and then performs post-processing on the segmentation results.
5. The generation method according to claim 4, characterized in that, The post-processing includes morphological operations, small region removal, and smoothing.
6. The generation method according to claim 4, characterized in that, The steps for detecting step cracks and bulges in UAV tilted images using Mask R-CNN and extracting the detection results include: ①Feature extraction: A pre-trained convolutional neural network ResNet is used to extract feature representations from the input 512-pixel × 512-pixel UAV tilted image; ② Candidate region extraction: A series of candidate target regions are generated in the 512-pixel × 512-pixel UAV oblique image by using a region candidate network; ③ Target classification and bounding box regression: For each candidate region, Mask R-CNN classifies it and regresses its bounding box to determine the category and location of the target; ④ Instance segmentation: Building upon object classification and bounding box regression, Mask R-CNN introduces an additional branch to generate a precise pixel-level segmentation mask for each object.
7. The generation method according to claim 1, characterized in that, In step (4), the semantic information of the steps, the detection information of step cracks and bulges are fused with the original UAV tilted image with geographical location information to generate a Mesh model with step element information that can be used for real-world 3D modeling of open-pit mines.