Mine three-dimensional reconstruction and compression method based on neural network

Abstract

The invention relates to the technical field of mine modeling, in particular to a mine three-dimensional reconstruction and compression method based on a neural network, which comprises the following steps: Step 1: a multi-source data acquisition stage: acquiring multi-source data through a heterogeneous sensing network consisting of a laser radar array, a binocular vision sensor array and an inertial measurement unit (IMU) deployed on a mine working surface; synchronously acquiring high-density point cloud data, a multi-view RGB-D image and equipment movement track data; step 2: a data preprocessing stage: performing space-time alignment, noise filtering and feature fusion processing on the original data to generate a standardized three-dimensional data cube with semantic annotation; according to the mine three-dimensional reconstruction and compression method based on the neural network, through the feature extraction and fusion capability of the neural network, the geometric accuracy and the detail reduction effect of the three-dimensional model are remarkably improved. Particularly, the reconstruction integrity of key parts such as roadway supporting structures, equipment and facilities is qualitatively improved, and meanwhile, texture details are more completely reserved.

Description

Technical Field

[0001] The present invention relates to the technical field of mine modeling, and in particular to a three-dimensional mine reconstruction and compression method based on a neural network. Background Art

[0002] With the rapid development of digital and intelligent mine construction, high-precision 3D reconstruction technology has significant application value in fields such as mine surveying and mapping, safety monitoring, and emergency rescue. Traditional mine 3D modeling relies primarily on a combination of laser scanning, photogrammetry, and inertial navigation technologies, but faces numerous technical bottlenecks in complex underground environments.

[0003] Existing methods typically use a single sensor (such as lidar or structured light cameras) for data acquisition, which struggles to overcome the challenges posed by dust interference, insufficient lighting, and complex geometric structures in mining environments. Multi-sensor fusion solutions often employ traditional extended Kalman filters or particle filters, which are prone to cumulative errors over long periods of time, leading to misaligned point cloud registration. The lack of an effective multimodal data alignment mechanism results in significant geometric fractures at model joints.

[0004] In order to solve the above problems, we made improvements and proposed a three-dimensional reconstruction and compression method of mines based on neural networks. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] The present invention provides a method for 3D reconstruction and compression of a mine based on a neural network, comprising the following steps:

[0007] Step 1: Multi-source data acquisition: A heterogeneous sensor network consisting of a lidar array, a binocular vision sensor array, and an inertial measurement unit (IMU) deployed on the mine working surface simultaneously acquires high-density point cloud data, multi-view RGB-D images, and equipment motion trajectory data.

[0008] Step 2: Data preprocessing: Spatiotemporal alignment, noise filtering, and feature fusion are performed on the raw data to generate a standardized 3D data cube with semantic annotations.

[0009] Step 3: In the neural network modeling phase, a multimodal neural network architecture is constructed, which includes a geometric feature encoder, a texture feature extractor, and a latent space projector. This architecture is trained end-to-end to achieve efficient representation of 3D features.

[0010] Step 4: 3D reconstruction: Based on the implicit feature field output by the neural network, adaptive octree segmentation and layered rendering technology are used to generate an editable lightweight 3D mine model.

[0011] Step 5: In the data compression stage, the low-dimensional potential representation output by the intermediate layer of the neural network is quantized and encoded, and the improved entropy coding algorithm is combined to achieve high-ratio compression of three-dimensional data.

[0012] As a preferred technical solution of the present invention, the laser radar array in Step 1 is distributed in a honeycomb topology, and the distance between adjacent nodes does not exceed 1 / 10 of the width of the working surface; the binocular vision sensor array includes at least 3 groups of camera pairs with cross-baseline configurations; and the sampling frequency of the inertial measurement unit is not less than 5 times the scanning frequency of the laser radar.

[0013] As a preferred technical solution of the present invention, Step 2 uses a statistical outlier removal algorithm to eliminate noise points and reduces the point cloud density to 5 cm resolution through voxel grid downsampling; bilateral filtering is used to eliminate image noise, and distortion correction is performed in combination with camera intrinsic parameters; and IMU and lidar data are fused based on extended Kalman filtering, and multi-frame point cloud registration is completed through an improved iterative closest point algorithm to generate a globally consistent point cloud map.

[0014] As a preferred technical solution of the present invention, the geometric feature encoder in Step 3 is composed of a 3D sparse convolution layer, a graph attention layer and a residual connection module; the texture feature extractor includes a multi-branch feature pyramid and a cross-modal attention mechanism; the latent space projector adopts a variational autoencoder structure and includes a KL divergence constrained regularization term.

[0015] As a preferred technical solution of the present invention, the geometric feature encoder adopts a hierarchical feature extraction strategy. The first level processes the original point cloud data to extract local surface curvature and normal vector features; the second level performs 3D convolution in voxelized space to capture mesoscale structural features; and the third level models global topological relationships through graph neural networks.

[0016] As a preferred technical solution of the present invention, the three-dimensional reconstruction process in Step 4 includes: discretizing the implicit feature field output by the neural network into a multi-layer octree structure, calculating the spatial gradient of the eigenvalue at each octree node, generating an isosurface mesh based on the Marching Cubes algorithm, applying a feature-aware mesh simplification algorithm to optimize the model complexity, and performing UV mapping on the simplified model and the texture atlas.

[0017] As a preferred technical solution of the present invention, the data compression method in Step 5 is to extract the top k feature channels with the maximum information entropy in the neural network latent space, perform non-uniform quantization on the selected features, adopt an adaptive quantization step based on statistical characteristics, apply an improved context-adaptive binary arithmetic encoder for entropy encoding, attach a metadata header file to record sensor parameters and compression configuration information, and optionally use the AES-256 algorithm to encrypt the compressed stream.

[0018] As a preferred technical solution of the present invention, the non-uniform quantization process adopts an optimization strategy of dynamically adjusting the quantization interval according to the probability distribution of the eigenvalue, retaining the sign bit and exponent bit of important features, performing lossy truncation on secondary features, and establishing a quantization error compensation mechanism.

[0019] The beneficial effects of the present invention are:

[0020] First, this neural network-based 3D mine reconstruction and compression method significantly improves the geometric accuracy and detail restoration of 3D models through the neural network's feature extraction and fusion capabilities. In particular, the reconstruction integrity of key areas such as tunnel support structures and equipment facilities has seen a qualitative leap, while also preserving more complete texture details.

[0021] Second, this neural network-based 3D mine reconstruction and compression method significantly shortens the conversion time from raw data to 3D models by optimizing the neural network architecture, while reducing system resource usage, making it possible to quickly reconstruct large-scale mine scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0023] Figure 1 This is a flow chart of a neural network-based mine 3D reconstruction and compression method of the present invention;

[0024] Figure 2 This is a flow chart of the three-dimensional reconstruction phase of a mine three-dimensional reconstruction and compression method based on a neural network according to the present invention;

[0025] Figure 3 This is a flow chart of the data compression phase of a neural network-based mine 3D reconstruction and compression method of the present invention; DETAILED DESCRIPTION

[0026] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0027] Example: Figure 1-Figure 3 As shown, a method for 3D reconstruction and compression of a mine based on a neural network includes the following steps:

[0028] Step 1: Multi-source data acquisition: A heterogeneous sensor network consisting of a lidar array, a binocular vision sensor array, and an inertial measurement unit (IMU) deployed on the mine working surface simultaneously acquires high-density point cloud data, multi-view RGB-D images, and equipment motion trajectory data.

[0029] Step 2: Data preprocessing: Spatiotemporal alignment, noise filtering, and feature fusion are performed on the raw data to generate a standardized 3D data cube with semantic annotations.

[0030] Step 3: In the neural network modeling phase, a multimodal neural network architecture is constructed, which includes a geometric feature encoder, a texture feature extractor, and a latent space projector. This architecture is trained end-to-end to achieve efficient representation of 3D features.

[0031] Step 4: In the 3D reconstruction phase, based on the implicit feature field output by the neural network, adaptive octree segmentation and layered rendering technology are used to generate an editable lightweight 3D mine model.

[0032] Step 5: In the data compression stage, the low-dimensional potential representation output by the intermediate layer of the neural network is quantized and encoded, and the improved entropy coding algorithm is combined to achieve high-ratio compression of three-dimensional data.

[0033] The lidar array in Step 1 is distributed in a honeycomb topology, and the distance between adjacent nodes does not exceed 1 / 10 of the width of the working surface; the binocular vision sensor array contains at least 3 sets of camera pairs with cross-baseline configurations; the sampling frequency of the inertial measurement unit is not less than 5 times the lidar scanning frequency.

[0034] Step 2 uses a statistical outlier removal algorithm to remove noise points and reduces the point cloud density to 5cm resolution through voxel grid downsampling. It uses bilateral filtering to eliminate image noise and combines camera intrinsic parameters for distortion correction. It also fuses IMU and lidar data based on extended Kalman filtering, completes multi-frame point cloud registration through an improved iterative closest point algorithm, and generates a globally consistent point cloud map.

[0035] The geometric feature encoder in Step 3 consists of a 3D sparse convolutional layer, a graph attention layer, and a residual connection module; the texture feature extractor includes a multi-branch feature pyramid and a cross-modal attention mechanism; the latent space projector adopts a variational autoencoder structure and includes a KL divergence-constrained regularization term.

[0036] The geometric feature encoder adopts a hierarchical feature extraction strategy. The first level processes the original point cloud data and extracts local surface curvature and normal vector features; the second level performs 3D convolution in voxelized space to capture mesoscale structural features; the third level models global topological relationships through graph neural networks.

[0037] The 3D reconstruction process in Step 4 includes: discretizing the implicit feature field output by the neural network into a multi-layer octree structure, calculating the spatial gradient of the eigenvalues ​​at each octree node, generating an isosurface mesh based on the Marching Cubes algorithm, applying a feature-aware mesh simplification algorithm to optimize the model complexity, and UV mapping the simplified model to the texture atlas.

[0038] The data compression method in Step 5 is to extract the top k feature channels with the largest information entropy in the neural network latent space, perform non-uniform quantization on the selected features, adopt an adaptive quantization step based on statistical characteristics, apply an improved context-adaptive binary arithmetic encoder for entropy coding, attach a metadata header file to record sensor parameters and compression configuration information, and optionally use the AES-256 algorithm to encrypt the compressed stream.

[0039] The non-uniform quantization process adopts an optimization strategy that dynamically adjusts the quantization interval according to the probability distribution of the eigenvalues, retains the sign and exponent bits of important features, performs lossy truncation on minor features, and establishes a quantization error compensation mechanism.

[0040] This neural network-based 3D mine reconstruction and compression method significantly improves the geometric accuracy and detail restoration of 3D models through the neural network's feature extraction and fusion capabilities. In particular, the reconstruction integrity of key areas such as tunnel support structures and equipment facilities has been significantly improved, while also preserving more complete texture details.

[0041] By optimizing the neural network architecture, the conversion time from raw data to three-dimensional models is greatly shortened, while system resource usage is reduced, making the rapid reconstruction of large-scale mine scenes possible.

[0042] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A mine 3D reconstruction and compression method based on neural network, characterized in that: The following steps are involved: Step 1: Multi-source data acquisition: A heterogeneous sensor network consisting of a lidar array, a binocular vision sensor array, and an inertial measurement unit (IMU) deployed on the mine working surface simultaneously acquires high-density point cloud data, multi-view RGB-D images, and equipment motion trajectory data. Step 2: Data preprocessing: Spatiotemporal alignment, noise filtering, and feature fusion are performed on the raw data to generate a standardized 3D data cube with semantic annotations. Step 3: In the neural network modeling phase, a multimodal neural network architecture is constructed, which includes a geometric feature encoder, a texture feature extractor, and a latent space projector. This architecture is trained end-to-end to achieve efficient representation of 3D features. Step 4: In the 3D reconstruction phase, based on the implicit feature field output by the neural network, adaptive octree segmentation and layered rendering technology are used to generate an editable lightweight 3D mine model. Step 5: In the data compression stage, the low-dimensional potential representation output by the intermediate layer of the neural network is quantized and encoded, and the improved entropy coding algorithm is combined to achieve high-ratio compression of three-dimensional data.

2. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 1, characterized in that: The lidar array in Step 1 is distributed in a honeycomb topology, and the distance between adjacent nodes does not exceed 1 / 10 of the width of the working surface; the binocular vision sensor array includes at least 3 groups of camera pairs with cross-baseline configurations; the sampling frequency of the inertial measurement unit is not less than 5 times the scanning frequency of the lidar.

3. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 1, characterized in that: The Step 2 uses a statistical outlier removal algorithm to remove noise points and reduces the point cloud density to 5 cm resolution through voxel grid downsampling. It uses bilateral filtering to eliminate image noise and combines camera intrinsic parameters for distortion correction. It also fuses IMU and lidar data based on an extended Kalman filter, completes multi-frame point cloud registration through an improved iterative closest point algorithm, and generates a globally consistent point cloud map.

4. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 1, characterized in that: The geometric feature encoder in Step 3 consists of a 3D sparse convolutional layer, a graph attention layer, and a residual connection module; the texture feature extractor includes a multi-branch feature pyramid and a cross-modal attention mechanism; the latent space projector adopts a variational autoencoder structure and includes a KL divergence constrained regularization term.

5. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 4, characterized in that: The geometric feature encoder adopts a hierarchical feature extraction strategy. The first level processes the original point cloud data to extract local surface curvature and normal vector features; The second level performs 3D convolution in voxelized space to capture mid-scale structural features; The third level models global topological relationships through graph neural networks.

6. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 1, characterized in that: The 3D reconstruction process in Step 4 includes: discretizing the implicit feature field output by the neural network into a multi-layer octree structure, calculating the spatial gradient of the eigenvalues ​​at each octree node, generating an isosurface mesh based on the Marching Cubes algorithm, applying a feature-aware mesh simplification algorithm to optimize the model complexity, and UV mapping the simplified model to the texture atlas.

7. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 1, characterized in that: The data compression method in Step 5 is to extract the top k feature channels with the maximum information entropy in the neural network latent space, perform non-uniform quantization on the selected features, adopt an adaptive quantization step based on statistical characteristics, apply an improved context-adaptive binary arithmetic encoder for entropy coding, attach a metadata header file to record sensor parameters and compression configuration information, and encrypt the compressed stream using the AES-256 algorithm.

8. The method for 3D reconstruction and compression of a mine based on a neural network according to claim 7, characterized in that: The non-uniform quantization process adopts an optimization strategy of dynamically adjusting the quantization interval according to the probability distribution of the eigenvalues, retaining the sign bit and exponent bit of important features, performing lossy truncation on minor features, and establishing a quantization error compensation mechanism.

Images