A method for three-dimensional reconstruction of mining material based on multi-scale feature fusion

Abstract

This invention discloses a method for 3D reconstruction of coal piles based on multi-scale feature fusion, relating to the field of machine vision technology. The method includes: extracting structured features from the residual point cloud of a coal pile; enhancing these features through a three-level multi-scale pyramid and dual-path global-local fusion; then fusing semantic and geometric information using a dual-branch approach of linear global attention and local geometric attention; introducing roughness perception and density adaptive FFN to generate robust global features; and finally reconstructing a high-fidelity point cloud using a progressive coarse-fine generator combined with a multi-resolution output mechanism. This process integrates multi-scale features, attention mechanisms, and progressive generation strategies, significantly improving the accuracy, efficiency, and robustness of the reconstruction, effectively solving the problem of inaccurate 3D modeling of coal piles due to severe missing features.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion. Background Technology

[0002] As one of the world's major energy sources, coal is seeing increasing attention from modern coal enterprises as the concept of smart mines becomes more widespread. Intelligent mine management is crucial for improving mine operational efficiency. Among the various approaches, 3D mine models constructed using 3D geological modeling technology represent the mainstream direction in mine modeling. 3D overall mine models, underground roadway models, and large coal pile models are key components in the development of digital and intelligent mines. The widespread application of multi-sensor technology provides strong support for acquiring 3D information of real-world scenes and lays a solid data foundation for 3D mine modeling. High-precision point cloud models are essential in the construction and analysis of 3D mine models, especially in large coal storage areas such as indoor and open-air stockpiles, where accurate 3D point cloud data is required for guidance and decision-making in stockpiling management.

[0003] The main shortcomings of existing technologies are as follows: 1) Coal pile point cloud data often suffers from missing data due to equipment resolution and its own error limitations, occlusion, and surface reflection, which seriously affects the accuracy of 3D modeling; 2) Traditional point cloud completion methods such as interpolation algorithms, symmetry analysis, or database matching have problems such as high requirements for structural integrity, noise sensitivity, and low computational efficiency; 3) Although deep learning-based methods such as PCN and AdaPoinTr can achieve end-to-end completion, they are insufficient in modeling local geometric features when dealing with non-rigid coal pile surfaces, making it difficult to take into account both global structure and microscopic details. Summary of the Invention

[0004] The purpose of this invention is to provide a method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion, which aims to solve or improve at least one of the above-mentioned technical problems.

[0005] To achieve the above objectives, the present invention provides the following solution: A method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion includes: Obtain the defect cloud of the coal pile and extract the structured feature set of the defect cloud; The structured feature set is extracted using a three-level multi-scale feature pyramid structure. Each level uses a dual-path structure to perform global-local fusion enhancement, resulting in multi-scale enhanced features. Multi-scale enhanced features are generated by fusing global semantic features and local geometric features through a two-branch parallel mechanism of linear global attention branch and local geometric attention branch. A roughness-aware and density-adaptive feedforward network (FFN) is introduced, and branch fusion features are combined to generate global features; Based on global features, a progressive coarse-fine generator is used to generate fine point clouds. Combined with a multi-resolution output mechanism, a multi-resolution point cloud set is generated to complete the reconstruction of the three-dimensional point cloud of the mineral material.

[0006] Furthermore, the defect cloud of the coal pile is obtained, and the structured feature set of the defect cloud is extracted, including: The raw point cloud data of the coal pile was collected using a 3D sensor as the residual point cloud. The residual point cloud is filtered and denoised. Based on the ground reference surface of the coal pile area, the Z coordinate of the remaining points is subtracted from the reference height and scaled to a preset normalization interval to obtain the normalized point cloud. K center points are selected from the normalized point cloud by sampling from the farthest point to obtain the set of center point coordinates; for each center point, neighboring points are searched in the Euclidean space neighborhood to generate K local point cloud groups to obtain the neighboring point set; Each local point cloud is grouped and input into a shared multilayer perceptron (MLP) to extract fixed-dimensional local geometric feature vectors; the center point is then encoded to generate a position embedding vector. The initial encoded features are obtained by adding the local geometric feature vector to the position embedding vector, and a structured feature set is constructed.

[0007] Furthermore, feature extraction is performed on the structured feature set using a three-level multi-scale feature pyramid structure. Each level employs a dual-path structure for global-local fusion enhancement, resulting in multi-scale enhanced features, including: The structured feature set is extracted at three levels using a three-level multi-scale feature pyramid structure, retaining the original resolution, 1 / 2 center point and 1 / 4 center point respectively; The features extracted at each level are processed in parallel through two pathways to extract global semantic features and local geometric features respectively, and then fused to obtain the fused features. The fused features at each level are linearly projected and residually connected to generate multi-scale enhanced features.

[0008] Furthermore, the multi-scale enhanced features are fused with global semantic features and local geometric features through a two-branch parallel mechanism of linear global attention branch and local geometric attention branch to generate branch fusion features, including: Multi-scale enhancement features Input the linear projection layer to generate query, key, and value matrices respectively, and perform linear attention calculation to obtain global semantic features; The local geometric attention branch takes the 3D coordinates of each center point and its k nearest neighbors in the point cloud as input, encodes the relative coordinate difference between point pairs through MLP, generates geometric weights, and fuses them with the feature vectors of neighboring points to generate local geometric features. By fusing global semantic features and local geometric features, branch fusion features are obtained.

[0009] Furthermore, a roughness-aware and density-adaptive feedforward network (FFN) is introduced, and combined with branch fusion features, global features are generated, including: Calculate roughness features and local density features; The branch fusion feature, roughness feature and local density feature are concatenated point by point to generate a concatenated feature vector; The spliced ​​feature vectors are nonlinearly enhanced by two layers of linear transformation and residual connection to generate density-enhanced features; LayerNorm normalization is applied to the density-enhanced features to obtain the global features.

[0010] Further, the roughness characteristics are calculated, including: The neighborhood mean coordinates are expressed as: In the formula, For the set of neighborhood points; Let be the three-dimensional coordinates of the j-th point in the neighborhood; The covariance matrix of the neighborhood point set is constructed based on the mean coordinates of the neighborhood, and its expression is: Solve the covariance matrix Maximum eigenvalue and minimum eigenvalue Calculate each center point Roughness characteristics The expression is: In the formula, Let be the roughness feature of the i-th point.

[0011] Furthermore, the local density features are calculated, including: Set the nearest neighbor number k, and obtain the center point. distance to the k-th nearest neighbor ,in Let Euclidean distance be the distance to the k-th nearest neighbor, and let Euclidean distance be the radius of the neighborhood. Based on nearest neighbor distance The local density feature is calculated using the following expression: In the formula, This represents a local density feature.

[0012] Furthermore, progressive coarse-fine generators include: In the coarse generation stage, based on global features Using MLP, a coarse point cloud is generated, expressed as: In the formula, For coarse point clouds; M represents the batch size; M represents the number of coarse points. In the fine generation stage, the coarse point cloud is upsampled and fused with encoder features. RefineNet is then used to learn the local geometric offset, expressed as: In the formula, This is an upsampling operation; To perform k-fold nearest-neighbor repetition interpolation along the neighborhood for each coarse point; The zero-mean Gaussian noise is of the same dimension as the upsampled point cloud, where The standard deviation of the disturbance. The identity matrix is ​​represented by ⊕; ⊕ represents the feature concatenation operation. Output the 3D coordinate offset; For encoder features; Conv1D is one-dimensional convolution; GroupNorm is group normalization; GELU is Gaussian error linear unit activation; The update and output process involves adding the upsampled points to the predicted displacements to generate a refined point cloud. The expression is as follows: In the formula, For fine point clouds.

[0013] Furthermore, a multi-resolution point cloud set is generated, including: Using a fine point cloud as a baseline, the total number of points for the original density is set; the size of the reference voxel is preset according to the accuracy requirements of the mineral reconstruction. By using a non-interpolation method, high, medium, and low resolution point clouds are generated by adjusting the voxel size and sampling point ratio, resulting in a multi-resolution point cloud set.

[0014] A method for 3D reconstruction of coal pile points based on multi-scale feature fusion further includes: constructing a dynamic weighted total loss function based on multi-scale geometry perception by fusing chamfer distance, ground movement distance, normal consistency, and curvature smoothing loss; and performing quality assessment and optimization on the completed coal pile point cloud. Specifically: Select the coal pile point cloud P from the multi-resolution point cloud set according to the application scenario; Obtain the reference point cloud Q of the coal pile, and calculate the chamfer distance and ground movement distance between the coal pile point cloud P and the reference point cloud Q. The expressions are as follows: In the formula, This is the chamfer distance; y is the distance traveled on the ground; N is the number of points in the coal pile point cloud P; M is the number of points in the reference point cloud Q; x is any point in the coal pile point cloud P; y is any point in the reference point cloud Q. It is a double shot; For point x, find the unique point in the reference point cloud Q; The normal consistency loss is calculated using the following expression: In the formula, This is due to the loss of normal uniformity. Let be the normal direction vector of the coal pile point cloud P; The normal direction vector of the reference point cloud Q; This is the dot product operation between vectors; Calculate the curvature k and the corresponding curvature gradient for each point in the coal pile point cloud P. ; The curvature smoothing loss is calculated using the following expression: In the formula, For curvature smoothing loss; For curvature gradient; Construct the total loss function to guide model training and optimization; its expression is: In the formula, λ1 represents the total loss; λ2, λ3, and λ4 are the weighting coefficients.

[0015] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects: This invention discloses a method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion. The method improves the accuracy, efficiency, and robustness of point cloud completion through multi-scale feature fusion strategy, attention mechanism, and progressive generation process, and solves the problem of inaccurate three-dimensional modeling caused by severe missing points in coal piles. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the workflow of the three-level multi-scale feature pyramid in this embodiment; Figure 3 This is a schematic diagram of the hierarchical structure of the multi-scale feature fusion pyramid in this embodiment; Figure 4 This is a schematic diagram of the hybrid high-efficiency attention mechanism in this embodiment; Figure 5 This is a schematic diagram of the progressive generator process in this embodiment; Figures 6-8 This diagram illustrates the differences in coal pile completion indices between the AdaPoinTr and MSF-3DRM network models at different defect ratios in this embodiment. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The purpose of this invention is to provide a method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion, which aims to solve or improve at least one of the above-mentioned technical problems.

[0020] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, this invention provides a method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion, comprising: Step 1: Obtain the defect cloud of the coal pile and extract the structured feature set of the defect cloud, including: Step 11: Use a 3D sensor to acquire raw point cloud data of the coal pile. ,in, Represents the spatial coordinates of the i-th point cloud; the original point cloud data as a whole suffers from missing data, noise, and non-uniform distribution, and is thus considered as a residual point cloud. Step 12: Using a statistical outlier removal algorithm or radius neighborhood filtering, isolated noise points in the residual point cloud are removed. Based on the estimated ground reference surface of the coal pile area, the Z coordinates of the remaining points are subtracted from the reference height, and then scaled to a preset normalization interval to obtain the normalized point cloud. ,in The height of each point has a uniform scale; Step 13: Obtain the normalized point cloud by sampling from the farthest point. Select K center points to obtain the set of center point coordinates. , Let k be the center point; for each center point, search for neighboring points in the Euclidean space neighborhood, generating K local point cloud groups, and obtain the neighboring point set. ;in, , The k-th local point cloud group is defined; r is the search radius. Center point The set of neighboring points; Step 14: Group each local point cloud. Input a shared multilayer perceptron (MLP) to extract fixed-dimensional local geometric feature vectors. ; for the center point Perform positional encoding to generate position embedding vectors. ; local geometric feature vectors With position embedding vector Adding them together yields the initial encoded features. Construct a structured feature set .

[0022] like Figure 2 As shown, in step 2, the structured feature set is extracted using a three-level multi-scale feature pyramid structure. Each level employs a dual-path structure for global-local fusion enhancement, resulting in multi-scale enhanced features. ,include: like Figure 3 As shown, the global pathway enhances the overall semantic expression through linear mapping and feature aggregation; the local pathway extracts neighborhood geometric features using a K-NN-based local aggregation method. When both have limited data, they are fused using a residual method to output fixed-dimensional enhanced features.

[0023] Step 21 involves extracting features from the structured feature set at three levels using a three-level multi-scale feature pyramid structure, including: The first level consists of three independent Feature Pyramid Levels in parallel. Level 1 is a 512-dimensional feature that focuses on local geometric details, maintains the original resolution, includes all K center points, and outputs the features. ; The second level, Level 2, enhances global semantic information with 1024-dimensional features. It employs the farthest point sampling (FPS) algorithm to downsample the point cloud from the first level by 1 / 2, retaining K / 2 center points, and outputs the features. ; The third level, Level 3, provides a higher-level abstract representation of the 2048-dimensional features. It uses the farthest point sampling algorithm (FPS) to downsample the point cloud of the first level by 1 / 4, retaining K / 4 center points, and outputs the features. ; Step 22: Perform dual-path parallel processing on the features extracted at each level, extracting global semantic features and local geometric features respectively, and then fuse them to obtain fused features, including: Step 221, for the input features Perform full-path processing to obtain global semantic features, including: For input features Perform a linear transformation to increase the dimensionality to And by introducing nonlinearity using the GELU activation function, we obtain the features. ; Features The input is fed into a Global Multilayer Perceptron (Global MLP) to process the features. Aggregate each point in the vector and output the global context vector. ; Will Expand to Broadcast vectors of the same length, and with Alignment yields global semantic features. , This represents the number of center points in the current level. Step 222, for the input features Local path processing is performed to obtain local geometric features, including: For input features Each center point in Query the corresponding neighbor set ; Perform local aggregation within the neighborhood to generate each center point. Local features , to obtain input features Local geometric features ; Step 223, global semantic features and local geometric features By adding them point by point, the fusion feature is obtained. .

[0024] Step 23: Perform linear projection and residual connections on the fused features of each level to generate multi-scale enhanced features, including: Step 231: Map the fused features of each level to a unified dimension through a linear projection layer. ,get ; Step 232: Upsample and align the three-level fused features; Step 233: Merge the aligned and fused features to generate a multi-scale feature vector. ; Step 234, will By performing a residual connection with the structured feature set Z, we obtain multi-scale enhanced features, expressed as: In the formula, This is a multi-scale enhancement feature.

[0025] like Figure 4 As shown, step 3 involves enhancing the multi-scale features. By employing a dual-branch parallel mechanism of linear global attention branch and local geometric attention branch, global semantic features and local geometric features are fused to generate branch fusion features. ,include: The global attention branch includes: Multi-scale enhancement features The input is a linear projection layer, which generates query, key, and value matrices, respectively. Linear attention is then performed to obtain the global semantic features, expressed as follows: In the formula, H represents the input features; Q represents the query generated in the AdaPoinTr network structure; K represents the key and query matching; and V represents the value obtained by weighted summation based on the similarity scores of the query and the key. This is a linear attention feature.

[0026] The linear attention branch described above achieves global context aggregation through linearized computation of Q, K, and V, with a computational complexity of [missing information - likely OCR error]. While maintaining global modeling capabilities, the computational complexity is reduced from O(N) 2 The efficiency is reduced to O(N), making it more suitable for efficient processing of real-scale point clouds such as coal piles.

[0027] The local geometric attention branch takes the 3D coordinates of each center point and its k nearest neighbors in the point cloud as input. It encodes the relative coordinate differences between point pairs using an MLP to generate geometric weights, which are then weighted and fused with the feature vectors of neighboring points to generate local geometric features. The expression is as follows: Where, p i Let p be the coordinates of the i-th point. j Let N be the coordinates of the neighboring point j of the i-th point. k (i) is the set of neighboring points of the i-th point, F rel A represents the relative positional feature between the i-th point and its neighbor j; local The local geometric features of the i-th point; It is the feature vector of neighboring point j; By fusing global semantic features and local geometric features, we obtain the branch fusion feature, expressed as: In the formula, This is a branch fusion feature; The outputs of the two branches are finally fused together to obtain the branch fusion feature H. out This approach enables the collaborative optimization of global semantic information and local geometric information within the same representation space, thereby ensuring both overall morphological consistency and enhanced edge and detail preservation during completion. It significantly reduces computational load while maintaining global modeling capabilities, making it suitable for processing large-scale point cloud data such as coal piles.

[0028] Step 4: Introduce a roughness-aware and density-adaptive feedforward network (FFN), and combine it with branch fusion features. Generate global features, including: Step 41, calculate roughness features and local density features, including: Calculating roughness features includes: The neighborhood mean coordinates are expressed as: In the formula, For the set of neighborhood points; Let be the three-dimensional coordinates of the j-th point in the neighborhood; The covariance matrix of the neighborhood point set is constructed based on the mean coordinates of the neighborhood, and its expression is: Solve the covariance matrix Maximum eigenvalue and minimum eigenvalue ; Calculate each center point Roughness characteristics And expand into dimensions Roughness feature vector; Roughness characteristics, expressed as: In the formula, Let be the roughness feature of the i-th point, used to describe the roughness of the point cloud surface.

[0029] Local density feature calculation includes: Set the nearest neighbor number k, and obtain the center point. distance to the k-th nearest neighbor ,in Let Euclidean distance be the distance to the k-th nearest neighbor, and let Euclidean distance be the radius of the neighborhood. Based on nearest neighbor distance Calculate local density features and expand them to dimensionality. The local density feature vector; Local density features, expressed as: In the formula, This represents a local density feature.

[0030] Step 42, merge branch features Roughness characteristics Local density features Perform point-by-point concatenation to generate a concatenated feature vector, expressed as: In the formula, To concatenate feature vectors; Step 43, concatenate the feature vectors Through two layers of linear transformation and residual connection, nonlinear enhancement is applied to the concatenated feature vectors to generate density-enhanced features, including: Perform a first-level linear transformation on the concatenated feature vectors to generate the first-transformed features, expressed as: In the formula, This is the first transformation feature; This is used as an activation function to alleviate gradient vanishing and improve the ability to mine features in sparse regions; This is the weight matrix; To learn the bias term; Perform a second-level linear transformation on the first change feature to generate the second transformation feature, expressed as: In the formula, This is the second transformation feature; This is the weight matrix; To learn the bias term; The second transformation feature is fused with the branch feature. Perform residual connections to generate density-enhanced features, expressed as follows: In the formula, This is a density-enhancing feature.

[0031] Step 44: Perform LayerNorm normalization on the density-enhanced features to obtain the global features, expressed as: In the formula, This is a global feature.

[0032] The above steps are adapted to the characteristics of coal piles and enhance the ability to perceive the micro-geometric properties of the coal pile surface. This paper introduces a surface roughness feature enhancement model and a density adaptive factor into geometric attention. Roughness features are used to characterize local surface undulations and irregularities, while the density adaptive mechanism enables the network to maintain stable aggregation in dense regions and improve the structural recovery ability in sparse regions, thereby improving the completion quality and robustness of complex coal pile surfaces.

[0033] like Figure 5 As shown, step 5, based on global features, employs a progressive coarse-fine generator and a multi-resolution output mechanism to generate a multi-resolution point cloud set, including: Progressive coarse-fine generators include: In the coarse generation stage, based on global features Using MLP, a coarse point cloud is generated, expressed as: In the formula, For coarse point clouds; M represents the batch size; M represents the number of coarse points. In the fine generation stage, the coarse point cloud is upsampled and fused with encoder features. RefineNet is then used to learn the local geometric offset, expressed as: In the formula, This is an upsampling operation; To perform k-fold nearest-neighbor repetition interpolation along the neighborhood for each coarse point; The zero-mean Gaussian noise is of the same dimension as the upsampled point cloud, where The standard deviation of the disturbance. The identity matrix is ​​represented by ⊕; ⊕ represents the feature concatenation operation. Output the 3D coordinate offset; For encoder features; Conv1D is a one-dimensional convolution to extract local structural features of point clouds; GroupNorm is group normalization to stabilize training; GELU is Gaussian error linear unit activation to enhance the ability to fit details. Encoder features are the collective term for the multi-scale integrated feature set output by the encoder module in multiple steps. They are the core feature set obtained after completing the full-process feature extraction, multi-scale enhancement, attention fusion and physical attribute perception of the coal pile residual point cloud. They include key information such as the local geometry, global semantics, surface roughness, point cloud density and cross-scale context of the coal pile point cloud.

[0034] The feature data comes from the entire process of the encoder module in the patent processing the defect cloud of the coal pile residue.

[0035] , The structured feature set extracted in step 1 consists of local geometric features and initial encoded features of the position embedding vector. The multi-scale enhanced features are obtained by linear projection and residual connection after dual-path fusion of a three-level multi-scale pyramid. The branch fusion feature is the result of fusing two branches: linear global attention and local geometric attention. This is a density-enhanced feature obtained by splicing branch fusion features and roughness / density features, followed by nonlinear enhancement using FFN.

[0036] The update and output process involves adding the upsampled points to the predicted displacements to generate a refined point cloud. The expression is as follows: In the formula, For fine point clouds.

[0037] Based on fine point cloud A multi-resolution output mechanism is used to generate point cloud results of different densities, resulting in a multi-resolution point cloud set, including: S1, with fine point cloud As a baseline, let the total number of points M of the original density be 1×10 in this embodiment. 5 Up to 5×10 5 Point; preset the reference voxel size according to the accuracy requirements of the ore reconstruction. In this embodiment, the value is 0.05m; S2, using a non-interpolation method, generates high, medium, and low resolution point clouds by adjusting voxel size and sampling point ratio, resulting in a multi-resolution point cloud set, including: Generate high-resolution point clouds, including: Set voxel size For fine point clouds Voxel downsampling is performed so that each voxel retains one center point, and duplicate and overlapping points are removed; For the downsampling results, check the areas with high roughness. If the point density in the area is lower than... ( ,in (where the point cloud coverage area is), then the original fine point cloud is sampled from the farthest point (FPS). To supplement the corresponding regions with a small number of points (no more than 5% of the total number of points, ranging from 4×10³ to 2×10⁴), excessive loss of details on rough surfaces is avoided, and the final output is a high-resolution point cloud. The points are approximately 0.8. .

[0038] Generate medium-resolution point clouds, including: Set voxel size ,right Voxel downsampling is performed so that one center point is retained within each voxel; Calculate the average density of the mineral. For density greater than average In high-density areas, FPS downsampling is used (retaining 60% of the points in the area), and for areas with lower density than the average density... In low-density regions, all downsampling results are retained to avoid global morphological distortion, and the final output is a medium-resolution point cloud. The points are approximately 0.4. .

[0039] Generating low-resolution point clouds includes: From fine point clouds Uniform sampling Each point ensures uniform coverage of the overall shape; Extract the image contour point set of the mineral material. If the contour points in the FPS result... If the percentage is less than 10%, calculate the number of contour points that need to be added. ,from Remove from non-contour points Internal redundant points (prioritizing the removal of duplicate points that are farthest from the contour and have the highest density), and from the original fine point cloud. The contour points are then filled with an equal number of convex hull contour points. The number of replacement points does not exceed 5000, ensuring the total number of points remains constant at 100,000. The final contour point percentage is maintained within the range of [10%, 15%] to ensure no missing contour points. The final output is a low-resolution point cloud. The number of points is approximately 0.1M.

[0040] As shown in Table 1, in practical applications, point clouds of different resolutions are selected according to the usage scenario, following the strategy of prioritizing scenario accuracy requirements and adapting to the underlying computing power of the equipment. This is combined with the actual application scenario of mineral material 3D reconstruction and the capabilities of the terminal equipment to achieve accurate matching.

[0041] Table 1. Adaptable Scenarios for Point Clouds at Different Resolutions

[0042] Step 6: By fusing chamfer distance, ground movement distance, normal consistency, and curvature smoothing loss, a multi-scale geometric perception dynamic weighted total loss function is constructed to assess and optimize the quality of the completed coal pile point cloud, including: Step 61: Select the coal pile point cloud P from the multi-resolution point cloud set according to the application scenario; Step 62: Obtain the reference point cloud Q of the coal pile, and calculate the chamfer distance (CD) and earthmover's distance (EMD) between the coal pile point cloud P and the reference point cloud Q. The expressions are as follows: In the formula, This is the chamfer distance; y is the distance traveled on the ground; N is the number of points in the coal pile point cloud P; M is the number of points in the reference point cloud Q; x is any point in the coal pile point cloud P; y is any point in the reference point cloud Q. It is a double shot; For point x, find the unique point in the reference point cloud Q; Among them, the reference point cloud is obtained through high-precision 3D scanning or multi-view fusion, and has high geometric integrity and low noise characteristics, which is used to supervise model training and reconstruction quality assessment.

[0043] Calculate the normal consistency loss to ensure that the normal direction of the coal pile point cloud P is consistent with the reference point cloud Q, thereby improving the surface detail accuracy of the coal pile. The expression is: In the formula, This is due to the loss of normal uniformity. Let be the normal direction vector of the coal pile point cloud P; The normal direction vector of the reference point cloud Q; This is the dot product operation between vectors; Calculate the curvature smoothing loss to constrain the curvature variations of the point cloud surface. Ensure the generated surface has good smoothness, including: Calculate the curvature k and the corresponding curvature gradient for each point in the coal pile point cloud P. ; The curvature smoothing loss is calculated using the following expression: In the formula, For curvature smoothing loss; For curvature gradient; Construct the total loss function to guide model training and optimization; its expression is: In the formula, λ1 represents the total loss; λ2, λ3, and λ4 are weighting coefficients that control the relative importance of each loss term.

[0044] The method of this patent is analyzed using data obtained from virtual construction, laboratory setup, and on-site scanning, including: All experiments were conducted using PyTorch 1.20.1 + CUDA 11.8, with the following hardware: CPU i9-11900KF and GPU NVIDIA 3090 24G. The virtual coal pile point cloud was generated using Python. Random parameters were used to control the pile's height, radius, and ellipticity. Gaussian noise was added to simulate surface irregularities, and sinusoidal distortion was introduced to achieve morphological changes.

[0045] The performance evaluation of coal pile point cloud completion was conducted using three numerical values: chamfer distance (CD), F1 score, and ground movement distance (EMD). The evaluation indicators are as follows: a smaller CD distance is better; a smaller EMD distance is also better; and a higher F1 score is better. This method (MSF-3DRM) was compared with representative point cloud completion networks such as PCN and AdaPoinTr. The experiment used 800 coal pile point clouds as training samples and 200 coal pile point clouds as test data, with the hyperparameters of each network model maintained at their original optimal settings. This method systematically tested the model's completion performance on coal pile point cloud data with missing rates ranging from 5% to 30%.

[0046] This paper uses 800 coal pile point clouds as training samples and 200 coal pile point clouds as test data, keeping the hyperparameters of each network model at their original optimal settings. A quantitative comparative analysis of the completion results of these methods on the coal pile point cloud test data is presented, and the results are shown in Table 2.

[0047] Table 2. Quantitative analysis of the coal pile point cloud completion effect of different completion network models.

[0048] The comparison results show that, compared with the AdaPoinTr method, the CD of the present invention is reduced by 14.5%, thus verifying the effectiveness of the present invention. Although the number of parameters only increases by 8.7%, the F1-score is improved by 2.3%, indicating that the gain brought by the efficient attention mechanism outweighs the computational redundancy.

[0049] To verify the stability and robustness of the MSF-3DRM network proposed in this invention, the model's completion performance was tested on coal pile point cloud data with missing rates ranging from 5% to 30%.

[0050] As shown in Table 3 and Figures 6-8 As shown, as the point cloud missing rate increases from 5% to 30%, the CD and EMD values ​​of all comparative methods show an upward trend, while the F1-score shows a downward trend. This indicates that the increased degree of point cloud missingness poses a challenge to the completion performance of all methods. However, the MSF-3DRM model significantly outperforms the comparative methods at all missing rates. Its performance advantage is particularly pronounced under high missing rate conditions. To address the feature insufficiency problem caused by a single attention mechanism, and considering that for smaller missing rates, the input features of MSF-3DRM and AdaPoinTr are essentially the same, their metrics are similar here.

[0051] Table 3 Comparison of coal pile completion indices for different defect ratios using AdaPoinTr and MSF-3DRM network models

[0052] The effectiveness of the multi-scale feature fusion pyramid and hybrid efficient attention mechanism employed by MSF-3DRM was verified through comparison. The multi-scale pyramid ensures that even with severely insufficient input information (high incompleteness rate), the model can still perform more reliable geometric inference by fusing contextual information from different scales. The hybrid efficient attention mechanism (especially the local geometric attention branch) enhances the modeling ability of the micro-geometry of the coal pile surface. The progressive generation strategy also effectively avoids global structural distortion caused by generating high-density point clouds all at once, even with high incompleteness rates. Notably, the performance degradation of MSF-3DRM with increasing incompleteness rate is significantly smaller than that of the comparative methods, which fully demonstrates that this method has stronger robustness and stability when facing different degrees of data incompleteness.

[0053] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0054] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion, characterized in that, include: Obtain the defect cloud of the coal pile and extract the structured feature set of the defect cloud; The structured feature set is extracted using a three-level multi-scale feature pyramid structure. Each level uses a dual-path structure to perform global-local fusion enhancement, resulting in multi-scale enhanced features. Multi-scale enhanced features are generated by fusing global semantic features and local geometric features through a two-branch parallel mechanism of linear global attention branch and local geometric attention branch. A roughness-aware and density-adaptive feedforward network (FFN) is introduced, and branch fusion features are combined to generate global features; Based on global features, a progressive coarse-fine generator is used to generate fine point clouds. Combined with a multi-resolution output mechanism, a multi-resolution point cloud set is generated to complete the reconstruction of the three-dimensional point cloud of the mineral material.

2. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The process of obtaining the residual defect cloud of the coal pile and extracting the structured feature set of the residual defect cloud includes: The raw point cloud data of the coal pile was collected using a 3D sensor as the residual point cloud. The residual point cloud is filtered and denoised. Based on the ground reference surface of the coal pile area, the Z coordinate of the remaining points is subtracted from the reference height and scaled to a preset normalization interval to obtain the normalized point cloud. K center points are selected from the normalized point cloud by sampling from the farthest point to obtain the set of center point coordinates; for each center point, neighboring points are searched in the Euclidean space neighborhood to generate K local point cloud groups to obtain the neighboring point set; Each local point cloud is grouped and input into a shared multilayer perceptron (MLP) to extract fixed-dimensional local geometric feature vectors; the center point is then encoded to generate a position embedding vector. The initial encoded features are obtained by adding the local geometric feature vector to the position embedding vector, and a structured feature set is constructed.

3. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The structured feature set is extracted using a three-level multi-scale feature pyramid structure. Each level employs a dual-path structure for global-local fusion enhancement, resulting in multi-scale enhanced features, including: The structured feature set is extracted at three levels using a three-level multi-scale feature pyramid structure, retaining the original resolution, 1 / 2 center point and 1 / 4 center point respectively; The features extracted at each level are processed in parallel through two pathways to extract global semantic features and local geometric features respectively, and then fused to obtain the fused features. The fused features at each level are linearly projected and residually connected to generate multi-scale enhanced features.

4. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The process of fusing global semantic features and local geometric features through a two-branch parallel mechanism of linear global attention branch and local geometric attention branch to generate branch fusion features includes: Multi-scale enhancement features Input the linear projection layer to generate query, key, and value matrices respectively, and perform linear attention calculation to obtain global semantic features; The local geometric attention branch takes the 3D coordinates of each center point and its k nearest neighbors in the point cloud as input, encodes the relative coordinate difference between point pairs through MLP, generates geometric weights, and fuses them with the feature vectors of neighboring points to generate local geometric features. By fusing global semantic features and local geometric features, branch fusion features are obtained.

5. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The roughness-aware and density-adaptive feedforward network (FFN) is introduced, and combined with branch fusion features, to generate global features, including: Calculate roughness features and local density features; The branch fusion feature, roughness feature and local density feature are concatenated point by point to generate a concatenated feature vector; The spliced ​​feature vectors are nonlinearly enhanced by two layers of linear transformation and residual connection to generate density-enhanced features; LayerNorm normalization is applied to the density-enhanced features to obtain the global features.

6. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 5, characterized in that, Calculating roughness features includes: The neighborhood mean coordinates are expressed as: In the formula, For the set of neighborhood points; Let be the three-dimensional coordinates of the j-th point in the neighborhood; The covariance matrix of the neighborhood point set is constructed based on the mean coordinates of the neighborhood, and its expression is: Solve the covariance matrix Maximum eigenvalue and minimum eigenvalue Calculate each center point Roughness characteristics The expression is: In the formula, Let be the roughness feature of the i-th point.

7. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 5, characterized in that, Calculating local density features includes: Set the nearest neighbor number k, and obtain the center point. distance to the k-th nearest neighbor ,in Let Euclidean distance be the distance to the k-th nearest neighbor, and let Euclidean distance be the radius of the neighborhood. Based on nearest neighbor distance The local density feature is calculated using the following expression: In the formula, This represents a local density feature.

8. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The progressive coarse-fine generator includes: In the coarse generation stage, based on global features Using MLP, a coarse point cloud is generated, expressed as: In the formula, For coarse point clouds; M represents the batch size; M represents the number of coarse points. In the fine generation stage, the coarse point cloud is upsampled and fused with encoder features. RefineNet is then used to learn the local geometric offset, expressed as: In the formula, This is an upsampling operation; To perform k-fold nearest-neighbor repetition interpolation along the neighborhood for each coarse point; The zero-mean Gaussian noise is of the same dimension as the upsampled point cloud, where The standard deviation of the disturbance. The identity matrix is ​​represented by ⊕; ⊕ represents the feature concatenation operation. Output the 3D coordinate offset; For encoder features; Conv1D is one-dimensional convolution; GroupNorm is group normalization; GELU is Gaussian error linear unit activation; The update and output process involves adding the upsampled points to the predicted displacements to generate a refined point cloud. The expression is as follows: In the formula, For fine point clouds.

9. The method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, The generation of the multi-resolution point cloud set includes: Using a fine point cloud as a baseline, the total number of points for the original density is set; the size of the reference voxel is preset according to the accuracy requirements of the mineral reconstruction. By using a non-interpolation method, high, medium, and low resolution point clouds are generated by adjusting the voxel size and sampling point ratio, resulting in a multi-resolution point cloud set.

10. A method for three-dimensional reconstruction of mineral materials based on multi-scale feature fusion according to claim 1, characterized in that, Also includes: By fusing chamfer distance, ground movement distance, normal consistency, and curvature smoothing loss, a multi-scale geometric perception dynamic weighted total loss function is constructed to assess and optimize the quality of the completed coal pile point cloud. Specifically: Select the coal pile point cloud P from the multi-resolution point cloud set according to the application scenario; Obtain the reference point cloud Q of the coal pile, and calculate the chamfer distance and ground movement distance between the coal pile point cloud P and the reference point cloud Q. The expressions are as follows: In the formula, This is the chamfer distance; y is the distance traveled on the ground; N is the number of points in the coal pile point cloud P; M is the number of points in the reference point cloud Q; x is any point in the coal pile point cloud P; y is any point in the reference point cloud Q. It is a double shot; To find a unique point in the reference point cloud Q for point x; The normal consistency loss is calculated using the following expression: In the formula, This is due to the loss of normal uniformity. Let be the normal direction vector of the coal pile point cloud P; The normal direction vector of the reference point cloud Q; This refers to the dot product operation between vectors. Calculate the curvature k and the corresponding curvature gradient for each point in the coal pile point cloud P. ; The curvature smoothing loss is calculated using the following expression: In the formula, For curvature smoothing loss; For curvature gradient; Construct the total loss function to guide model training and optimization; its expression is: In the formula, Total loss; λ1, λ2, λ3, and λ4 are weighting coefficients.

Images