A method, system, device and storage medium for constructing a three-dimensional model of an open-pit coal mine

Abstract

The present application relates to the technical field of open coal mine three-dimensional model construction, in particular to an open coal mine three-dimensional model construction method, system, device and storage medium, first, the images of each area of the open coal mine are mixed based on the pixel of dust concentration, splicing is realized, the open coal mine splicing graph is obtained, after dust removal treatment, the open coal mine dust-free graph is obtained; then, combined with the topography and dust concentration distribution information of the open coal mine to be measured, the open coal mine dust-free graph is subjected to semantic segmentation, different coal mine scenes are identified, and combined with the corresponding dust concentration distribution information, the unmanned aerial vehicle route is planned to obtain point cloud data, the initial point cloud model is constructed, and the surface is optimized to obtain the coal mine point cloud model; finally, the open coal mine dust-free graph and the coal mine point cloud model are unified in coordinates and registered to obtain a texture map, and after fusion, unnecessary point clouds are removed to obtain an open coal mine three-dimensional model, the accuracy and authenticity of the open coal mine three-dimensional model are improved, and the multi-scene application requirements are met.

Description

Technical Field

[0001] This invention relates to the field of open-pit coal mine 3D model construction technology, specifically to a method, system, equipment, and storage medium for constructing open-pit coal mine 3D models. Background Technology

[0002] Open-pit coal mines possess abundant resources and can serve as important national strategic energy reserve bases. Their economic role is reflected in ensuring energy supply, driving the extension of regional industrial chains, and promoting employment. Constructing a three-dimensional model of an open-pit coal mine can optimize resource recovery rates and reduce equipment operation and maintenance costs through digital mining planning. It also facilitates real-time monitoring of slope stability, dynamic prevention and control of equipment collision risks, and provides a digital technological paradigm for mine development under extremely arid and wind-eroded geological conditions. This will promote the transformation of open-pit mining from "experience-driven" to "data-driven," and the technological achievements can be extended to similar mines in Central Asia, forming an international mining technology export capability.

[0003] The complex terrain of open-pit coal mines (with many uneven steps, large surface undulations, and diverse lithology), coupled with the high dust concentration in the working environment, causes deviations between the collected point cloud data and the actual situation. Ultimately, this results in geometric distortion in the constructed 3D point cloud model, leading to poor accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, equipment, and storage medium for constructing three-dimensional models of open-pit coal mines.

[0005] The technical solution of this invention is as follows:

[0006] A method for constructing a 3D model of an open-pit coal mine includes the following operations:

[0007] S1. Acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, stitch the images of several regions together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, perform dust removal processing on the stitched image of the open-pit coal mine to obtain a dust-free image of the open-pit coal mine.

[0008] S2. Combining the terrain information and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation processing is performed on the dust-free map of the open-pit coal mine to obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, the flight path planning of each UAV is performed; according to the UAV flight path planning, the point cloud data of each scene is obtained; the point cloud data of each coal mine scene is preprocessed and then registered to obtain an initial point cloud model; the initial coal mine point cloud model is then optimized by surface processing to obtain the coal mine point cloud model.

[0009] S3. After registering the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system one, obtain the open-pit coal mine texture map; merge the open-pit coal mine texture map with the coal mine point cloud model, delete the non-ground point cloud, and supplement the missing area point cloud to obtain the open-pit coal mine 3D model.

[0010] In S1, the method for obtaining dust concentration distribution information is as follows: after each region image is denoised, its own region dark channel image is obtained; based on the dark channel image of each region and the atmospheric light value of the corresponding region, its own region transmittance map is obtained; based on its own region transmittance map, its own pixel dust concentration map is obtained; based on the image acquisition time, its own pixel dust concentration map is converted to the coal mine global coordinate system, and after spatial interpolation and outlier removal, the dust concentration distribution information is obtained by summarizing.

[0011] In S1, the method for obtaining the open-pit coal mine mosaic image is as follows: the pixel dust concentration of each region image is normalized to obtain the respective region dust weight image; feature points of several region images are obtained, and after coarse registration and removal of mismatched points, accurate feature point matching pairs are obtained; based on the accurate feature point matching pairs, the geometric transformation matrix between adjacent region images is obtained, and the region images are geometrically corrected to obtain several region corrected images; based on the region dust weight image, the pixels of the overlapping areas of all region corrected images are fused to obtain the open-pit coal mine mosaic image.

[0012] In S1, the method for obtaining the dust-free map of the open-pit coal mine is as follows: based on the dust concentration distribution information and the dust concentration level classification standard, the dust area in the spliced ​​map of the open-pit coal mine is divided into a first-level dust concentration area, a second-level dust concentration area, and a third-level dust concentration area. These areas are then processed by dust removal based on a GAN network, atmospheric scattering model processing, adaptive bilateral filtering, and LAB color space restoration processing to obtain the dust-free map of the open-pit coal mine. The average dust concentration of the first-level dust concentration area, the second-level dust concentration area, and the third-level dust concentration area decreases sequentially.

[0013] In S2, the surface optimization process is as follows: obtain all point clouds within the neighborhood of each point cloud in the initial coal mine point cloud model to obtain the neighborhood point cloud set of each point cloud; based on their respective neighborhood point cloud sets, construct their respective fitting surface functions, solve the surface function parameters by minimizing the sum of squared errors, calculate their respective new coordinates, replace the corresponding original coordinates, and obtain the coal mine point cloud model.

[0014] In S3, the method for obtaining the open-pit coal mine texture map is as follows: the pixel coordinates of the dust-free open-pit coal mine map are converted into two-dimensional projected coordinates in the world coordinate system to obtain a coordinate-normalized image; after point cloud resampling, the coal mine point cloud model is matched with the feature points of the coordinate-normalized image, the spatial transformation matrix is ​​calculated, and the coal mine point cloud model is spatially aligned to obtain a coordinate-aligned point cloud model; the coordinate-normalized image and the coordinate-aligned point cloud model are processed by texture mapping to obtain the open-pit coal mine texture map.

[0015] In S2, the method for acquiring coal mine scenes is as follows: after aligning the dust-free map of the open-pit coal mine with the terrain information of the open-pit coal mine to be tested in terms of spatial coordinates, the map is fused with the terrain information and dust concentration distribution information based on the attention mechanism to obtain a fused feature map; the fused feature map is processed by the trained U-Net semantic segmentation model to obtain several coal mine scenes.

[0016] A three-dimensional model construction system for open-pit coal mines, used to implement the aforementioned three-dimensional model construction method for open-pit coal mines, includes:

[0017] The open-pit coal mine dust-free map generation module is used to acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, the images of several regions are stitched together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, the stitched image of the open-pit coal mine is processed to remove dust to obtain a dust-free map of the open-pit coal mine.

[0018] The coal mine point cloud model generation module is used to combine the terrain information and dust concentration distribution information of the open-pit coal mine under test, perform semantic segmentation processing on the dust-free map of the open-pit coal mine, and obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, the module performs its own UAV flight path planning; according to the UAV flight path planning, the module obtains its own point cloud data; the point cloud data of each coal mine scene is preprocessed and then registered to obtain an initial point cloud model; the initial coal mine point cloud model is then optimized by surface processing to obtain the final coal mine point cloud model.

[0019] The open-pit coal mine 3D model generation module is used to register the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system I, and then obtain the open-pit coal mine texture map. The open-pit coal mine texture map is then fused with the coal mine point cloud model, and after deleting non-ground point clouds, the missing area point clouds are supplemented to obtain the open-pit coal mine 3D model.

[0020] A three-dimensional model construction device for an open-pit coal mine includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-mentioned three-dimensional model construction method for an open-pit coal mine.

[0021] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for constructing a three-dimensional model of an open-pit coal mine.

[0022] The beneficial effects of this invention are as follows:

[0023] This invention provides a method for constructing a 3D model of an open-pit coal mine. First, the pixel dust concentration of images in various areas of the open-pit coal mine to be tested is obtained and summarized into dust concentration distribution information. Based on this, pixel mixing is performed to stitch the regional images together, resulting in a stitched image of the open-pit coal mine. Then, dust removal processing is performed to obtain a dust-free image of the open-pit coal mine. Next, combining the terrain and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation is performed on the dust-free image to identify different coal mine scenes. Combined with the corresponding dust concentration distribution information, a drone flight path is planned to acquire point cloud data. An initial point cloud model is constructed and then surface optimization is performed to obtain a coal mine point cloud model. Finally, the dust-free image and the coal mine point cloud model are unified in coordinate and registered to obtain texture maps. After fusion, unnecessary point clouds are removed to obtain a 3D model of the open-pit coal mine. This gives the model clear, dust-free texture information that accurately corresponds to the point cloud, thereby improving the accuracy and realism of the 3D model of the open-pit coal mine and meeting the needs of multiple application scenarios. Detailed Implementation

[0024] This embodiment provides a method for constructing a three-dimensional model of an open-pit coal mine, including the following operations:

[0025] S1. Acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, stitch the images of several regions together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, perform dust removal processing on the stitched image of the open-pit coal mine to obtain a dust-free image of the open-pit coal mine.

[0026] S2. Combining the terrain information and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation processing is performed on the dust-free map of the open-pit coal mine to obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, the flight path planning of each UAV is performed; according to the UAV flight path planning, the point cloud data of each scene is obtained; the point cloud data of each coal mine scene is preprocessed and then registered to obtain an initial point cloud model; the initial coal mine point cloud model is then optimized by surface processing to obtain the coal mine point cloud model.

[0027] S3. After registering the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system one, obtain the open-pit coal mine texture map; merge the open-pit coal mine texture map with the coal mine point cloud model, delete the non-ground point cloud, and supplement the missing area point cloud to obtain the open-pit coal mine 3D model.

[0028] The specific operating steps are detailed below.

[0029] S1. Acquire several regional images of the open-pit coal mine to be tested, extract the pixel dust concentration of each regional image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, stitch the several regional images together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, perform dust removal processing on the stitched image of the open-pit coal mine to obtain a dust-free image of the open-pit coal mine.

[0030] The pixel dust concentration of images in various areas of the open-pit coal mine under test is obtained and summarized into dust concentration distribution information. Based on this, pixel mixing based on dust concentration is performed to stitch the regional images together to obtain a stitched image of the open-pit coal mine. Then, dust removal processing is performed to obtain a dust-free image of the open-pit coal mine. This provides a clear image foundation without dust interference for the subsequent generation of 3D models, ensuring more accurate texture mapping, improving the accuracy and integrity of the 3D model, and making the model more in line with the actual scene.

[0031] First, the RGB camera mounted on the drone takes pictures along a preset flight path (covering several areas of the open-pit coal mine to be tested, with overlapping areas between adjacent areas), and obtains RGB images of several areas of the open-pit coal mine to be tested, which are used as images of several areas of the open-pit coal mine to be tested.

[0032] Then, the pixel dust concentration of each region image is extracted, and combined with the image acquisition time and corresponding spatial location, the dust concentration distribution information is obtained by summarizing.

[0033] Specifically, after denoising each region's image, its respective dark channel image is obtained. Based on the atmospheric light values ​​of each region's dark channel image and the corresponding region's image, its respective region's transmittance map is obtained. Based on its respective region's transmittance map, its respective pixel dust concentration map is obtained. Based on the image acquisition time, its respective pixel dust concentration map is transformed to the coal mine's global coordinate system, ensuring that the dust concentration value of each pixel corresponds to the actual spatial location of the mine, eliminating positional deviations between regions. After spatial interpolation (interpolating blank areas) and outlier removal (outliers can be defined within a range according to actual needs), the dust concentration distribution information is obtained by summarizing the data. The above spatial interpolation can be achieved by performing Kriging interpolation on blank areas based on the concentration values ​​of adjacent areas.

[0034] The pixel dust concentration value in the pixel dust concentration map can be obtained using the following formula: P i =k 1 (1-t i )+k 2 (1-ti ) 2 , P i Location points in the pixel dust concentration map i The corresponding pixel dust concentration value, t i Location points in the regional transmittance map i transmittance, k 1 , k 2 These are the first and second fitting coefficients, respectively, which are coefficients obtained by fitting actual dust sampling data.

[0035] Next, based on the dust concentration distribution information, the images of several regions are stitched together to obtain a stitched image of the open-pit coal mine.

[0036] The stitching operation is as follows: The pixel dust concentration of each region image is normalized to obtain its respective region dust weight image; feature points of several region images are obtained, and after coarse registration and removal of mismatched points, accurate feature point matching pairs are obtained; based on the accurate feature point matching pairs, the geometric transformation matrix between adjacent region images is obtained, and the region images are geometrically corrected to obtain several region corrected images; based on the region dust weight images, pixel fusion is performed on the overlapping areas of all region corrected images to obtain the open-pit coal mine stitched image; the specific steps are detailed below.

[0037] Step 1: Normalize the pixel dust concentration of each region image and convert the pixel dust concentration into dust weight. The higher the pixel dust concentration, the lower the dust weight, so that pixels in low dust areas get higher weight in the stitching, reducing dust interference, and obtaining the dust weight image of each region.

[0038] Step 2: Extract feature points from each region image (including stable features such as coal seam edges and coal mine equipment outlines, which can be achieved through the SIFT algorithm), forming a region image feature point set; perform coarse matching on the region image feature point set (including but not limited to using the FLANN matcher) to obtain initial feature point matching pairs; based on the region dust weight image, filter the initial matching pairs, remove (low) weight feature point matching from regions with high dust concentration (mean dust concentration greater than the mean dust concentration threshold), and retain (high) weight feature point matching from low dust regions, reducing mismatches caused by texture blurring in high dust regions, thus achieving mismatch point removal and obtaining accurate feature point matching pairs.

[0039] Step 3: Based on the precise feature point matching pairs, calculate the homography matrix between adjacent region images (which describes the geometric transformation relationship between images and can be calculated by the RANSAC algorithm). Use this matrix as the geometric transformation matrix to perform geometric correction on the region images, resulting in several region-corrected images.

[0040] Step 4: Based on the regional dust weight image, perform weighted average pixel fusion processing on all regional correction images, making the pixel transition in the overlapping areas smooth and effectively weakening the splicing marks caused by dust to obtain the open-pit coal mine splicing image.

[0041] Finally, based on the dust concentration distribution information, the dust removal process was performed on the spliced ​​image of the open-pit coal mine to obtain a dust-free image of the open-pit coal mine.

[0042] Specifically, based on dust concentration distribution information and dust concentration level classification standards (the dust concentration threshold in the classification standards can be customized according to actual needs), the dust area in the open-pit coal mine mosaic image is divided into different dust concentration levels (Level 1 dust concentration area, Level 2 dust concentration area, and Level 3 dust concentration area, with the average dust concentration decreasing sequentially). These are then processed by GAN-based dust removal (for images corresponding to Level 1 dust concentration areas with high dust concentration), atmospheric scattering model processing (for images corresponding to Level 2 dust concentration areas with medium dust concentration), and adaptive bilateral filtering and LAB color space restoration (for Level 3 dust concentration areas with low dust concentration) to obtain a dust-free image of the open-pit coal mine.

[0043] The specific method for obtaining different dust concentration regions is as follows: Dust concentration distribution information is spatially overlaid with a mosaic image of an open-pit coal mine to mark dust concentrations in the image, resulting in a coal mine dust concentration marking map. This map is then processed through image segmentation (extracting the dust-affected areas) and boundary optimization (smoothing and correcting the boundaries of the dust-affected areas). The optimized segmented areas containing dust concentrations are designated as regions of interest (ROIs). For each ROI, data statistics and spatial analysis are used to determine that dust concentration decreases sequentially from the center outwards, thus identifying the concentration center (usually the maximum dust concentration point at the center of each ROI). Starting from the concentration center, dust concentration gradients in different directions are calculated. Based on the concentration gradients and concentration level classification standards, each ROI is progressively divided into graded regions from the concentration center outwards, and adjacent dust concentration regions of the same grade are connected to obtain different dust concentration regions with progressively increasing dust concentration levels from the center outwards.

[0044] The specific operations for dust removal based on the GAN network described above are as follows.

[0045] Step 1: Multimodal feature extraction is performed on the primary dust concentration area (image) to obtain texture features (image texture features of the primary dust concentration area, which can be realized by extracting local texture features through the SIFT algorithm), contour features (image edge texture features of the primary dust concentration area, which can be realized by capturing residual edge contours through Canny edge detection), and semantic features (location and regional information of coal mine equipment and / or slope steps and other features with significant characteristics), forming a set of feature fragments partially obscured by dust.

[0046] Step 2: Perform 3D constraint modeling on the feature fragment set and the 3D prior database of the open-pit coal mine scene (the database contains prior parameters such as the shape, structure and size parameters of coal mine equipment, coal mine steps, etc.). The preferred method is to fit the spatial mapping relationship between the fragments in the feature fragment set and the prior parameters to obtain the initial skeleton of the regional scene.

[0047] Step 3: The initial scene skeleton is processed by training a GAN model (using the feature fragment set as supervised data, iterating the loss function, and learning the pixel radiation characteristics of the occluded area) to obtain a regional scene optimization skeleton that is adapted to the region.

[0048] Step 4: Optimize the scene skeleton by rendering and completing the occluded areas to generate pixel values ​​of the dust-occluded areas, resulting in the first dust-removed area image with a complete and clear area after dust removal.

[0049] S2. Combining the terrain information and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation processing is performed on the dust-free map of the open-pit coal mine to obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, the flight path planning of each UAV is carried out; according to the UAV flight path planning, the point cloud data of each scene is obtained; the point cloud data of each coal mine scene is preprocessed and then registered to obtain an initial point cloud model; the initial coal mine point cloud model is optimized by surface processing to obtain the coal mine point cloud model.

[0050] By combining the topography and dust concentration distribution information of the open-pit coal mine under test, semantic segmentation is performed on the dust-free map of the open-pit coal mine to identify different coal mine scenarios. Based on the corresponding dust concentration distribution information, UAV flight paths are planned to acquire point cloud data. After preprocessing, registration and surface optimization, a coal mine point cloud model is obtained, which provides an accurate, complete and optimized point cloud foundation for the subsequent generation of 3D models. This ensures that the model matches the actual situation of each scenario and helps to improve the accuracy, detail richness and reliability of the 3D model.

[0051] First, by combining the terrain information and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation processing is performed on the dust-free map of the open-pit coal mine to obtain several coal mine scenes.

[0052] Specifically, after aligning the dust-free map of the open-pit coal mine with the topographic information of the open-pit coal mine to be tested in terms of spatial coordinates, the map is then fused with topographic information (mainly elevation, slope, and aspect information) and dust concentration distribution information based on an attention mechanism to enhance the edge features of steep terrain areas and historically high dust coverage areas, resulting in a fused feature map. The fused feature map is then processed by a trained U-Net semantic segmentation model to obtain several semantically clear coal mine scenes.

[0053] The coal mine scenes include: steep slopes with steps; mining operations with excavators, mining trucks, and other coal mining equipment in operation, characterized by rapid changes and high dust density; coal and rock mixed accumulation areas; transportation roads connecting the mining area, storage area, and various roads outside the mining area; and industrial plaza scenes encompassing office areas, equipment maintenance areas, and material storage areas.

[0054] The specific steps for feature fusion based on the attention mechanism described above are as follows.

[0055] Step 1: The dust-free map of the open-pit coal mine, after spatial coordinate alignment, is processed by a convolutional neural network (such as a CNN network) to extract visual features such as texture and edges, resulting in a dust-free map feature matrix. The terrain information is encoded using features, converting values ​​such as elevation, slope, and aspect into a feature matrix, which is then processed by a fully connected layer and mapped to a dimension compatible with the dust-free map feature matrix, resulting in a terrain feature matrix. The dust concentration distribution information is processed by feature encoding, matrix transformation, and a fully connected layer to obtain a dust distribution feature matrix.

[0056] Step 2: Concatenate the dust-free map feature matrix with the terrain feature matrix and the dust distribution feature matrix along the channel dimension to obtain the concatenated feature matrix; perform convolution and non-linear processing (sigmoid activation function) on the concatenated feature matrix to obtain the spatial weight matrix; perform global average pooling, fully connected processing and non-linear processing (sigmoid activation function) on the concatenated feature matrix to obtain the channel weight matrix.

[0057] Step 3: The feature matrix and spatial weight matrix are spliced ​​together and multiplied element-wise to enhance the spatial features of key regions, resulting in a spatial feature matrix. The spatial feature matrix and channel weight matrix are multiplied by channel to highlight the contribution of important feature channels. After convolution processing, the feature dimensions are integrated to obtain a fused feature map.

[0058] Then, based on each coal mine scenario and the corresponding dust concentration distribution information, the drone flight path is planned accordingly.

[0059] The terrain of the slope scene is relatively steep. In the drone flight path planning of the slope scene, the flight path goes up and down in a spiral around the slope, and gets close to the slope surface from different angles to acquire point cloud data. The drone flies at an altitude of 5-15m away from the slope to ensure that clear information of various parts of the slope is obtained.

[0060] In mining operation areas, dust concentrations are high. Therefore, the drone flight path planning in these areas combines a circular and zoned approach: first, the drone flies around the entire mining operation area at an altitude of 100-150m above the ground to obtain point clouds showing the overall layout and equipment distribution; then, it flies at low altitudes of 10-30m above the ground according to functional zones such as excavation areas, loading areas, and transportation channels. If the area is in a high-concentration dust zone, the drone flies at an altitude of 20-30m above the ground to avoid the impact of high-concentration dust clouds on the lidar's operation.

[0061] In the planning of UAV flight paths in the scenario of coal and rock mixed accumulation area, the flight path adopts grid-like coverage flight, parallel flight path, and uniform spacing to ensure full coverage of the coal storage area. The flight path height is 10-20m above the coal pile, which can not only clearly obtain the surface information of the coal pile, but also avoid the dust layer.

[0062] In the drone flight path planning for transportation road scenarios, the flight route flies along the transportation road and sets multiple altitude levels. First, at an altitude of 50-100m above the ground, the overall road layout, traffic flow and the approximate range of dust diffusion are obtained. Then, at an altitude of 10-30m above the ground, detailed shooting is taken of key road sections such as curves, intersections and severely damaged road surfaces.

[0063] In the drone flight path planning for the industrial plaza scenario, the flight path is a zigzag flight at an altitude of 10-20 m above the ground.

[0064] Next, following the drone's flight path planning, the lidar onboard the drone is used to acquire point cloud data for each coal mine scene.

[0065] Subsequently, the point cloud data for each coal mine scene were preprocessed and then registered to obtain an initial point cloud model. The preprocessing included point cloud denoising, filtering, point cloud density unification, and point cloud coordinate system unification.

[0066] The specific steps to obtain the initial point cloud model are as follows: Point cloud data for each coal mine scene are preprocessed to obtain their respective preprocessed point cloud data. Using the preprocessed point cloud data of any coal mine scene as the target point cloud and the preprocessed point cloud data of other coal mine scenes as the source point clouds, the nearest neighbor of each point in the source point cloud in the target point cloud is calculated, constructing corresponding point pairs. Based on the corresponding point pairs, a transformation matrix is ​​calculated. Through iterative transformation, the distance error between the source and target point clouds is minimized to a set threshold, completing the registration of the two point clouds. Point clouds from all coal mine scenes are sequentially registered with the reference point cloud (point cloud data from any coal mine scene) to eliminate residual errors after spatial alignment, enabling high-precision fusion of the point clouds in each coal mine scene in detail. All registered point cloud data are merged, and duplicate points are removed (based on a distance threshold; if the distance between two points is less than the threshold, one point is retained), forming an initial point cloud model covering the entire coal mine scene.

[0067] Finally, the initial coal mine point cloud model is optimized using surface processing to obtain the final coal mine point cloud model. Specifically, all point clouds within the neighborhood of each point cloud in the initial coal mine point cloud model are obtained, resulting in a neighborhood point cloud set for each point cloud. Based on their respective neighborhood point cloud sets, their respective fitting surface functions are constructed. By minimizing the sum of squared errors, the parameters of the surface function are solved, and their new coordinates are calculated. These new coordinates are then used to replace the corresponding original coordinates, thereby eliminating high-frequency noise and local fluctuations in the point cloud, making the surface of the initial coal mine point cloud model smoother and the model more realistic, thus obtaining the final coal mine point cloud model.

[0068] S3. After registering the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system one, obtain the open-pit coal mine texture map; merge the open-pit coal mine texture map with the coal mine point cloud model, delete the non-ground point cloud, and supplement the missing area point cloud to obtain the open-pit coal mine 3D model.

[0069] After unifying the coordinates and registering the dust-free open-pit coal mine map with the coal mine point cloud model, texture maps are obtained. After fusion, unnecessary point clouds are removed to obtain the 3D model of the open-pit coal mine. The model is given clear, dust-free texture information that accurately corresponds to the point cloud, making the 3D model more realistic and detailed on the basis of accurate geometric structure. This improves the accuracy and realism of the model and meets the needs of subsequent multi-scenario applications.

[0070] First, the open-pit coal mine dust-free map and the coal mine point cloud model are registered using coordinate system one to obtain the open-pit coal mine texture map.

[0071] Using a dust-free open-pit coal mine image as the texture data source, the pixel coordinates of the dust-free open-pit coal mine image are converted into two-dimensional projected coordinates in the world coordinate system through camera parameter analysis (including camera intrinsic and extrinsic parameter matrices), resulting in a coordinate-normalized image. The coal mine point cloud model serves as the geometric carrier. After point cloud resampling, key geometric feature points, such as step edge points and equipment vertices, are retained and matched with feature points of the coordinate-normalized image (which can be achieved by extracting texture corner points using the SIFT algorithm). The spatial transformation matrix is ​​calculated, and the coal mine point cloud model is spatially aligned (by multiplying the spatial transformation matrix with the coal mine point cloud model) to obtain a coordinate-aligned point cloud model. The coordinate-normalized image and the coordinate-aligned point cloud model are then processed through texture mapping to obtain an open-pit coal mine texture map that fits the geometric structure of the point cloud.

[0072] The texture mapping process described above can be achieved by determining the texture projection direction based on the surface normal vector of the point cloud and allocating pixel values ​​through an inverse distance weighting algorithm.

[0073] Finally, the open-pit coal mine texture map is fused with the coal mine point cloud model (this can be achieved through spatial registration), and the non-ground point cloud (such as houses, coal mine equipment, etc., which can be obtained by performing target detection on the dust-free map of the open-pit coal mine and obtaining the ground feature labels) is deleted. Then, the missing area point cloud is supplemented to obtain the 3D model of the open-pit coal mine.

[0074] Due to the influence of open-pit coal mine dust, especially when the dust concentration is high, point cloud data may be sparse, resulting in missing areas in the coal mine point cloud model, as well as missing areas after deleting non-ground point clouds. In order to complete the point cloud in the 3D model, the specific steps for point cloud completion in this embodiment are as follows.

[0075] Step 1: Traverse the coal mine point cloud model after integrating the open-pit coal mine texture map and deleting non-ground point clouds, identify missing regions (this can be achieved by calculating the point cloud density within the sliding window), and extract the neighborhood ground texture information (including texture color channel values ​​and color gradient direction) of the missing region's edge as the reference texture feature for each missing region.

[0076] Step 2: Select seed points from the edge points of each missing region (select according to the actual situation, preferably points with clear texture features and uniform distribution of surrounding point clouds). Starting from the seed points, gradually expand and supplement the new point cloud into the missing region according to the growth rules to complete the point cloud of the missing region and obtain the initial complete model.

[0077] In the process of gradually expanding and supplementing the missing region with new point clouds according to the growth rules, for each point cloud to be generated, the position value is predicted based on the surface function and reference texture features corresponding to the neighboring point cloud to obtain a new point cloud. This new point cloud is added to the missing region, and the neighboring region is updated. The above process is repeated until the entire missing region is filled.

[0078] The surface growth rules described above are as follows: the spatial position of the new point cloud is consistent with the normal vector direction of the neighboring point clouds to ensure surface continuity, and the similarity between the texture features of the new point cloud and the texture features of the neighboring point clouds is higher than a similarity threshold. Through texture similarity constraints, the completed point cloud can fully preserve these details, making the 3D model closer to the real scene.

[0079] Step 3: Smooth the initial completed model. It is preferable to adjust the coordinates of the new point cloud using the moving average method to make the transition between the completed area and the original edge point cloud more natural and avoid obvious elevation changes, thus obtaining the three-dimensional model of the open-pit coal mine.

[0080] If dynamic updates of the open-pit coal mine 3D model are subsequently achieved, the similarity between different coal mine scenes and the corresponding initial scene can be detected to determine whether the coal mine scene has changed. If it has changed, image acquisition, dust removal, point cloud acquisition, and point cloud and dust removal image fusion can be performed on the corresponding coal mine scene to replace the original coal mine scene's corresponding point cloud model, thereby achieving dynamic updates of the open-pit coal mine 3D model.

[0081] This embodiment also provides a three-dimensional model construction system for open-pit coal mines, used to implement the above-mentioned three-dimensional model construction method for open-pit coal mines, including:

[0082] The open-pit coal mine dust-free map generation module is used to acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, the images of several regions are stitched together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, the stitched image of the open-pit coal mine is processed to remove dust to obtain a dust-free map of the open-pit coal mine.

[0083] The coal mine point cloud model generation module is used to combine the terrain information and dust concentration distribution information of the open-pit coal mine under test, perform semantic segmentation processing on the dust-free map of the open-pit coal mine, and obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, the module performs its own UAV flight path planning; according to the UAV flight path planning, the module obtains its own point cloud data; the point cloud data of each coal mine scene is preprocessed and then registered to obtain an initial point cloud model; the initial coal mine point cloud model is then optimized by surface processing to obtain the final coal mine point cloud model.

[0084] The open-pit coal mine 3D model generation module is used to register the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system I, and then obtain the open-pit coal mine texture map. The open-pit coal mine texture map is then fused with the coal mine point cloud model, and after deleting non-ground point clouds, the missing area point clouds are supplemented to obtain the open-pit coal mine 3D model.

[0085] This embodiment also provides a three-dimensional model construction device for open-pit coal mines, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-described three-dimensional model construction method for open-pit coal mines.

[0086] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for constructing a three-dimensional model of an open-pit coal mine.

[0087] This embodiment provides a method for constructing a 3D model of an open-pit coal mine. First, the pixel dust concentration of images in various areas of the open-pit coal mine to be tested is obtained and summarized into dust concentration distribution information. Based on this, pixel mixing based on dust concentration is performed to stitch the regional images together to obtain a stitched image of the open-pit coal mine. Then, dust removal processing is performed to obtain a dust-free image of the open-pit coal mine. Next, semantic segmentation is performed on the dust-free image of the open-pit coal mine, combining the terrain and dust concentration distribution information of the open-pit coal mine to identify different coal mine scenes. Based on the corresponding dust concentration distribution information, a drone flight path is planned to obtain point cloud data. After constructing an initial point cloud model, surface optimization is performed to obtain a coal mine point cloud model. Finally, the dust-free image of the open-pit coal mine and the coal mine point cloud model are unified in coordinate and registered to obtain texture maps. After fusion, unnecessary point clouds are removed to obtain a 3D model of the open-pit coal mine. This gives the model clear, dust-free texture information that accurately corresponds to the point cloud, thereby improving the accuracy and realism of the 3D model of the open-pit coal mine and meeting the needs of multi-scenario applications.

Claims

1. A method for constructing a three-dimensional model of an open-pit coal mine, characterized in that, This includes the following operations: S1. Acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize to obtain dust concentration distribution information; specifically, after noise reduction processing, each region image is used to obtain its own region dark channel image; based on the dark channel image of each region and the atmospheric light value of the corresponding region image, the region transmittance map is obtained; based on the region transmittance map, the pixel dust concentration map is obtained; based on the image acquisition time, the pixel dust concentration map is converted to the global coordinate system of the coal mine, and after spatial interpolation and outlier removal, the dust concentration distribution information is obtained; spatial interpolation is achieved by performing Kriging interpolation on the blank area based on the concentration values ​​of adjacent areas; Based on dust concentration distribution information, images of several regions are stitched together to obtain a stitched image of an open-pit coal mine. Based on dust concentration distribution information, dust removal processing is performed on the spliced ​​image of the open-pit coal mine to obtain a dust-free image of the open-pit coal mine. S2. Combining the terrain information and dust concentration distribution information of the open-pit coal mine to be tested, semantic segmentation processing is performed on the dust-free map of the open-pit coal mine to obtain several coal mine scenes. Based on each coal mine scenario and corresponding dust concentration distribution information, the drone flight path is planned accordingly; and point cloud data is acquired according to the drone flight path plan. The point cloud data for each coal mine scene were preprocessed and then registered to obtain an initial point cloud model. The initial coal mine point cloud model was optimized by surface processing to obtain the coal mine point cloud model; S3. After registering the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system one, obtain the open-pit coal mine texture map; merge the open-pit coal mine texture map with the coal mine point cloud model, delete the non-ground point cloud, and supplement the missing area point cloud to obtain the open-pit coal mine 3D model.

2. The method for constructing a three-dimensional model of an open-pit coal mine according to claim 1, characterized in that, In S1, the method for obtaining the open-pit coal mine mosaic image is as follows: The pixel dust concentration of each region image is normalized to obtain the respective region dust weight image; the feature points of several region images are obtained, and after coarse registration and removal of mismatched points, accurate feature point matching pairs are obtained. Based on precise feature point matching pairs, the geometric transformation matrix between adjacent region images is obtained, and the region images are geometrically corrected to obtain several region corrected images. Based on the region dust weight image, the pixels of the overlapping areas of all region corrected images are fused to obtain an open-pit coal mine mosaic image.

3. The method for constructing a three-dimensional model of an open-pit coal mine according to claim 1, characterized in that, In S1, the method for obtaining the dust-free map of an open-pit coal mine is as follows: Based on dust concentration distribution information and dust concentration level classification standards, the dust area in the open-pit coal mine mosaic image is divided into first-level dust concentration area, second-level dust concentration area, and third-level dust concentration area. After dust removal processing based on GAN network, atmospheric scattering model processing, adaptive bilateral filtering and LAB color space restoration processing, a dust-free image of the open-pit coal mine is obtained. The average dust concentration decreases sequentially from the first-level dust concentration area to the second-level dust concentration area and the third-level dust concentration area.

4. The method for constructing a three-dimensional model of an open-pit coal mine according to claim 1, characterized in that, In S2, the surface optimization process is as follows: Obtain all point clouds within the neighborhood of each point cloud in the initial coal mine point cloud model to obtain the neighborhood point cloud set of each point cloud; based on their respective neighborhood point cloud sets, construct their respective fitting surface functions, solve the surface function parameters by minimizing the sum of squared errors, calculate their respective new coordinates, replace the corresponding original coordinates, and obtain the coal mine point cloud model.

5. The method for constructing a three-dimensional model of an open-pit coal mine according to claim 1, characterized in that, In S3, the method for obtaining open-pit coal mine texture maps is as follows: The pixel coordinates of the dust-free open-pit coal mine image are converted into two-dimensional projected coordinates in the world coordinate system to obtain a coordinate-normalized image. After point cloud resampling, the coal mine point cloud model is matched with the feature points of the coordinate-normalized image, and the spatial transformation matrix is ​​calculated to spatially align the coal mine point cloud model to obtain a coordinate-aligned point cloud model. The coordinate-normalized image and the coordinate-aligned point cloud model are then processed by texture mapping to obtain an open-pit coal mine texture map.

6. The method for constructing a three-dimensional model of an open-pit coal mine according to claim 1, characterized in that, In S2, the method for obtaining the coal mine scene is as follows: After aligning the dust-free map of the open-pit coal mine with the topographic information of the open-pit coal mine to be tested in terms of spatial coordinates, the map is then fused with the topographic information and dust concentration distribution information based on an attention mechanism to obtain a fused feature map. The fused feature map is then processed by a trained U-Net semantic segmentation model to obtain several coal mine scenes.

7. A three-dimensional model construction system for open-pit coal mines, used to implement the three-dimensional model construction method for open-pit coal mines as described in claim 1, characterized in that, include: The open-pit coal mine dust-free map generation module is used to acquire images of several regions of the open-pit coal mine to be tested, extract the pixel dust concentration of each region image, and summarize them to obtain dust concentration distribution information; based on the dust concentration distribution information, the images of several regions are stitched together to obtain a stitched image of the open-pit coal mine; based on the dust concentration distribution information, the stitched image of the open-pit coal mine is processed to remove dust to obtain a dust-free map of the open-pit coal mine. The coal mine point cloud model generation module is used to combine the terrain information and dust concentration distribution information of the open-pit coal mine under test, perform semantic segmentation processing on the dust-free map of the open-pit coal mine, and obtain several coal mine scenes; according to each coal mine scene and the corresponding dust concentration distribution information, perform their respective UAV flight path planning; and obtain their respective point cloud data according to the UAV flight path planning. The point cloud data for each coal mine scene were preprocessed and then registered to obtain an initial point cloud model. The initial coal mine point cloud model was optimized by surface processing to obtain the coal mine point cloud model; The open-pit coal mine 3D model generation module is used to register the dust-free open-pit coal mine map with the coal mine point cloud model using coordinate system I, and then obtain the open-pit coal mine texture map. The open-pit coal mine texture map is then fused with the coal mine point cloud model, and after deleting non-ground point clouds, the missing area point clouds are supplemented to obtain the open-pit coal mine 3D model.

8. A three-dimensional model construction device for open-pit coal mines, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the method for constructing a three-dimensional model of an open-pit coal mine as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer programs, wherein the computer programs, when executed by a processor, implement the method for constructing a three-dimensional model of an open-pit coal mine as described in any one of claims 1-6.