An automated measurement method for the volume of a bulk material pile

Abstract

The present invention discloses an automated measurement method for the volume of a bulk material pile. The method includes: obtaining the original point cloud data of the bulk material pile, using the AFF-GrowSP network to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile. In the AFF-GrowSP network, the features of the original point cloud data of the bulk material pile are dynamically weighted and adjusted through the AFF module, and the volume of the three-dimensional model of the bulk material pile is calculated using the point cloud data of the bulk material pile. By adding the AFF module, the present invention dynamically adjusts features of different scales during the feature extraction process of the original point cloud data of the bulk material pile, thereby improving the accuracy of point cloud segmentation and solving the problem of low recognition accuracy of complex point cloud semantics based on deep learning. The calculation accuracy and calculation efficiency of the volume of the bulk material pile are improved, and it has strong generalization ability.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional point cloud volume automatic measurement, and particularly relates to an automatic measurement method for the volume of a bulk material pile. Background Art

[0002] At present, the volume measurement of bulk material piles is mainly divided into the following three categories: photogrammetry method, laser and photography combined measurement method, and laser scanning measurement method.

[0003] The main principle of the laser scanning measurement method is to emit laser and use the pulse method for scanning and ranging. The scanning device achieves the effect of range scanning by means of rotation or swinging, etc., so as to complete the three-dimensional drawing of surrounding objects. The point cloud data of the bulk material pile is obtained by using a laser scanning device, and then the deep learning method is used to perform semantic recognition on the point cloud data. Finally, the three-dimensional point cloud model of the bulk material pile is reconstructed and the volume is calculated. In the process of performing semantic recognition on the point cloud data, the self-supervised three-dimensional point cloud semantic segmentation method does not require manual annotation or any pre-training, and can successfully identify the complex semantic classes of each point in the scene. However, the laser scanning device has a large collection area, and it is very easy to obtain the point cloud of objects near the bulk material pile, such as pedestrians, vehicles, etc., when collecting the bulk material pile, which affects the recognition accuracy of the complex semantics of the point cloud, thus affecting the calculation of the volume of the bulk material pile. Summary of the Invention

[0004] The purpose of the present invention is to provide an automatic measurement method for the volume of a bulk material pile to improve the recognition accuracy of the complex semantics of the point cloud data of the bulk material pile.

[0005] The present invention adopts the following technical solutions:

[0006] An automatic measurement method for the volume of a bulk material pile, comprising the following steps:

[0007] Obtain the original point cloud data of the bulk material pile;

[0008] Use the AFF-GrowSP network to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; in the AFF-GrowSP network, the features of the original point cloud data of the bulk material pile are dynamically weighted and adjusted through the AFF module;

[0009] Calculate the volume of the three-dimensional model of the bulk material pile by using the point cloud data of the bulk material pile.

[0010] Further, the dynamic weight adjustment of the original point cloud data of the bulk material pile through the AFF module includes:

[0011] Add and fuse the small-scale features and large-scale features of the original point cloud data of the bulk material pile element by element to obtain the original features;

[0012] Extract the global features and local features of the original features;

[0013] Add the global features and local features element by element to obtain the weight coefficients;

[0014] Use the weight coefficients to adjust the weights of the small-scale features and large-scale features respectively, and fuse the small-scale features and large-scale features with adjusted weights to obtain the fused features.

[0015] Furthermore, before obtaining the bulk material pile point cloud data, it also includes:

[0016] Use a hybrid filtering method based on the optimization idea to denoise the original point cloud data of the bulk material pile.

[0017] Furthermore, using a hybrid filtering method based on the optimization idea to denoise the original point cloud data of the bulk material pile includes:

[0018] Use an improved radius filtering method to perform large-scale denoising on the original point cloud data of the bulk material pile; among them, use the average value of the number of neighborhood points to replace the threshold of the number of neighborhood points, and reduce the parameter of the radius filtering algorithm from 2 to 1.

[0019] Furthermore, after performing large-scale denoising on the original point cloud data of the bulk material pile, it also includes:

[0020] Perform small-scale denoising on the original point cloud data of the bulk material pile.

[0021] Furthermore, use the DBSCAN density clustering algorithm to perform small-scale denoising on the original point cloud data of the bulk material pile.

[0022] Furthermore, calculating the volume of the three-dimensional model of the bulk material pile using the bulk material pile point cloud data includes:

[0023] Use the greedy projection triangulation algorithm based on MLS to reconstruct the bulk material pile.

[0024] Furthermore, using the greedy projection triangulation algorithm based on MLS to reconstruct the bulk material pile includes:

[0025] Use MLS to estimate and reconstruct the normal vectors of the bulk material pile point cloud data.

[0026] Furthermore, using the greedy projection triangulation algorithm based on MLS to reconstruct the bulk material pile also includes:

[0027] Adopt the octree algorithm to perform neighborhood search on the bulk material pile point cloud data.

[0028] Furthermore, obtaining the original point cloud data of the bulk material pile is to perform non-contact scanning on the bulk material pile using a three-dimensional laser scanner.

[0029] The beneficial effects of the present invention are as follows: The present invention uses the AFF-GrowSP network as a point cloud semantic segmentation model. By adding an attention mechanism-based feature fusion module (Attention Feature Fusion, AFF) to the GrowSP network, this module is used to fuse the global features and local features of the original point cloud data of the bulk material pile, and dynamically adjust the weights of the fused features, thereby improving the recognition accuracy of complex semantics of the point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flowchart of the method of the present invention;

[0031] Figure 2 is a schematic structural diagram of the AFF-GrowSP network of the present invention;

[0032] Figure 3 is a schematic structural diagram of the feature extractor in the AFF-GrowSP network structure of the present invention;

[0033] Figure 4 is a schematic diagram of the attention mechanism-based feature fusion module in the feature extractor of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The present invention proposes an automatic measurement method for the volume of bulk material piles based on a self-supervised AFF-GrowSP semantic segmentation network, specifically for the automatic measurement of the volume of multiple bulk material piles with complex curved surfaces. The present invention first performs non-contact scanning on the bulk material pile through a three-dimensional laser scanner to obtain the original point cloud data. The noise is reduced by using a hybrid filtering method based on the optimization idea; secondly, the preprocessed point cloud data is passed through the AFF-GrowSP semantic segmentation network to segment the target object, remove other objects in the scene, and extract the point cloud data of the bulk material pile; finally, the three-dimensional reconstruction and volume calculation of the coal pile data are completed using the greedy triangulation algorithm based on MLS.

[0036] The specific process of an automatic measurement method for the volume of a bulk material pile of the present invention is as follows, as Figure 1 shown:

[0037] S110, obtain the original point cloud data of the bulk material pile.

[0038] For some bulk material pile measurements, they may exist in high-pollution and high-risk industries, and manual participation may cause diseases. Therefore, the present invention uses a non-contact measurement method with lidar to obtain the original point cloud data of the bulk material pile.

[0039] S120, denoise the original point cloud data of the bulk material pile by using a hybrid filtering method based on the optimization idea.

[0040] The noise points in the original point cloud data of the bulk material pile will change the shape of the material pile, resulting in errors in volume measurement. Moreover, the noise points will waste storage space, increase the running time, and reduce work efficiency. The present invention uses a hybrid filtering method based on the optimization idea to denoise the noise points step by step. First, use the improved radius filtering method to perform large-scale denoising on the original point cloud data, removing most of the far noise points and a small part of the near noise points with lower density. Then, use the DBSCAN density clustering algorithm to apply to the small-scale noise in the point cloud to complete the second-stage denoising. The accurate denoising of the noisy point cloud data is achieved. Compared with the classical spatial domain filtering algorithms, the filtering method based on the optimization idea is convenient for subsequent segmentation operations, reduces the final volume measurement error, and improves the algorithm processing efficiency. It has good self-adaptability and retains the details and edge information of the original point cloud while denoising.

[0041] The present invention uses a hybrid filtering method based on the optimization idea to denoise the noise points step by step. The distribution range of the points in the point cloud is quantified by the mean and standard deviation. If the mean Euclidean distance of a certain point to other points is much lower than that of other domain points and is lower than 2 to 3 times the standard deviation, it is regarded as a large-scale point, otherwise it is a small-scale point.

[0042] Denoising the original point cloud data of the bulk material pile by using a hybrid filtering method based on the optimization idea includes:

[0043] Use the improved radius filtering method to perform large-scale denoising on the original point cloud data of the bulk material pile;

[0044] Use the DBSCAN density clustering algorithm to perform small-scale denoising on the original point cloud data of the bulk material pile.

[0045] Among them, using the improved radius filtering method to perform large-scale denoising on the original point cloud data of the bulk material pile includes:

[0046] Set the search radius and the threshold of the number of domain points;

[0047] Calculate the number of domain points within the search radius;

[0048] Calculate the average value of the number of domain points;

[0049] Use the average value of the number of domain points to replace the threshold of the number of domain points, and reduce the parameter of the radius filtering algorithm from 2 to 1;

[0050] Delete the points outside the range of the average value of the number of domain points.

[0051] The specific denoising process is as follows:

[0052] S1201, input the original point cloud data of the bulk material pile;

[0053] S1202, perform the first-stage denoising, calculate the number of neighboring points N based on the radius r i ;

[0054] S1203, calculate the average value of N i ;

[0055] S1204, use to replace the threshold m, and reduce the radius filtering algorithm parameter from 2 to 1;

[0056] S1205, delete the points outside the threshold range to obtain the original point cloud data P1 of the bulk material pile after large-scale denoising;

[0057] S1206, enter the second-stage denoising, and establish the KDTree data structure of the point cloud in the previous step;

[0058] S1207, initialize the parameters EPS and N eps,min ;

[0059] S1208, randomly select a point p from the original point cloud data P1 of the bulk material pile i , and determine whether it is a core point. If not, it is a noise point, and delete this noise point;

[0060] S1209, traverse the point cloud data P1 to find all density-reachable points of the point p i , and construct a point cluster C;

[0061] S1210, output the point cluster C to obtain the original point cloud data P of the bulk material pile after hybrid filtering denoising o , and the point cluster C is the original point cloud data P of the bulk material pile o .

[0062] S130, use the AFF-GrowSP network to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; in the AFF-GrowSP network, the features of the original point cloud data of the bulk material pile are dynamically weighted and adjusted through the AFF module.

[0063] The stacking environment of the actual pile body is relatively complex, and the acquisition area of the lidar is large. When collecting the bulk material pile, it is easy to obtain the point cloud of the objects near the bulk material pile, which affects the subsequent volume calculation. To solve the above problems, the present invention uses a self-supervised AFF-GrowSP point cloud semantic segmentation network to separately segment the point cloud of the bulk material pile, as Figure 2As shown in the figure. For the 3D semantic segmentation problem, existing methods mainly rely on a large number of manual annotations to train neural networks, such as PointNet, PointNet++, and 3D point cloud tasks based on 2D vision processing. GrowSP is a completely self-supervised method that requires no manual annotation or any pre-training. It can successfully identify the complex semantic classes of each point in the scene. And it uses an attention mechanism-based feature fusion module (Attention Feature Fusion, AFF) to effectively fuse and complement features at different scales. As Figure 2 shown, this network can automatically process point cloud data without manual intervention, effectively improving the efficiency and accuracy of coal pile volume measurement.

[0064] GrowSP discovers 3D semantic information through the progressive growth of superpoints. This method consists of three parts: ① Feature extractor, which learns the feature information of each point. GrowSP follows the SparseConv architecture. The encoder uses the ResNet16 structure, and the decoder consists of 4 MLP layers. These MLP layers generate 128-dimensional features for interpolation and are finally combined together via the AFF module to obtain point cloud feature information. ② Superpoint constructor, which gradually increases the size of superpoints. This module constructs initial superpoints through the VCCS and region growing algorithms before the network training starts, and then uses the K-means clustering algorithm to obtain larger superpoints. Repeating the above steps can achieve this. ③ Semantic primitive clustering module, GrowSP regards semantic segmentation as a problem of 3D feature learning and clustering. For each Epoch, each input point cloud will have several superpoints, and each superpoint represents an object and a part of the object. For the entire point cloud data, all superpoints can be regarded as a set of huge basic semantic elements. The network constructs initial and expanded superpoints and combines them into semantic elements to output point cloud semantic information.

[0065] Based on the attention mechanism-based feature fusion module, in common point cloud data of bulk material piles, there are various types of point cloud data such as pedestrians, forklifts, and bulk material piles. The scales of these data are different. For example, the scale of the bulk material pile is large, while the scales of pedestrians and forklifts are small. In the field of deep learning, small-scale features represent delicate local information, while large-scale features represent broad context knowledge and rough semantic information. Regarding the problem of how to efficiently fuse and complement these two types of features, as Figure 3 shown, the present invention introduces the AFF module. This module analyzes the mutual relationship of features at different scales and dynamically adjusts their fusion weights, aiming to simultaneously retain small-scale features with rich details and integrate the context semantics in large-scale features, thereby achieving more accurate semantic recognition. The architecture of the module is as Figure 4 shown.

[0066] The dynamic weight adjustment of the original point cloud data of the bulk material pile includes:

[0067] Add the small-scale features and large-scale features of the original point cloud data of the bulk material pile element by element to obtain the original features;

[0068] Extract the global features and local features of the original features;

[0069] Add the global features and local features element by element to obtain the weight coefficients;

[0070] Use the weight coefficients to adjust the weights of the small-scale features and large-scale features respectively, and fuse the small-scale features and large-scale features with adjusted weights to obtain the fused features.

[0071] Figure 4 where X, Y, X′ ∈ R C×N , X is the small-scale feature extracted by the encoding network, Y is the large-scale feature obtained by inverse distance interpolation through feature propagation, and N and C respectively represent the size and number of channels of the feature point cloud. As Figure 4 shown in (a) of, first fuse X and Y element by element to obtain the feature X′; then send X′ into Figure 4 (b) to extract the global feature g(X′) and the local feature L(X′). The calculation process of the global feature is:

[0072] g(X′) = B{W2δ{B{W1[Γ(X′)]}}} (1)

[0073] The calculation process of the local feature is:

[0074] L(X′) = B{W2δ{B[W1(X′)]}} (2)

[0075] In the formula, Γ(X′) is the global average pooling function, B is the batch normalization layer, δ is the ReLU activation function, and W1 and W2 are the fully connected layers. For Figure 4 (b), finally obtain the weight coefficient w ∈ R for feature fusion C×N , and the calculation formula is:

[0076]

[0077] In the formula, σ is the Sigmoid function, is element-wise addition. Finally, the features X and Y are weighted and fused according to w to obtain the fused feature Z, and the fusion process is:

[0078]

[0079] In the formula, I represents the all-1 matrix, which is used for the weighting operation in feature fusion.

[0080] Specifically:

[0081] S1301. Input the original point cloud data P of the bulk material pile that has undergone preprocessing of hybrid filtering and denoising o into the AFF-GrowSP network, and regard the input point cloud as a set of H points;

[0082] S1302. Pass the set through a feature extractor network based on the SpareConv architecture, through an encoder with a ResNet16 structure and a decoder with 4 layers of MLP, and finally generate 128-dimensional features for interpolation to obtain per-point features;

[0083] S1303. Use the 4 layers of MLP as the X and Y inputs of the AFF module in sequence. Combine these features through 3 cascaded AFF modules to finally obtain the feature F of the point h ;

[0084] S1304. Input F h into the Superpoint Constructor to gradually generate larger and larger superpoints, which are used to assist the network in automatically discovering the semantic information of the point cloud;

[0085] S1305. Input the generated superpoints into the semantic clustering module to generate pseudo-labels for all superpoints, and the pseudo-labels will be used to optimize the feature extractor during the training process;

[0086] S1306. Combine the superpoints into semantic elements, thereby outputting the semantic information of the point cloud.

[0087] S140. Calculate the volume of the 3D model of the bulk material pile using the point cloud data of the bulk material pile.

[0088] After segmentation by the AFF-GorwSP network, the separate pile body point cloud of the bulk material pile is obtained. The segmented pile body point cloud data has the characteristics of high density and high precision. Direct 3D reconstruction takes a long time, and the surface of the reconstructed model is not smooth enough, with problems such as small holes. The present invention proposes a greedy projection triangulation algorithm based on MLS. Compared with the traditional greedy projection algorithm that relies on PCA, the present invention performs normal vector calculation and reconstruction based on MLS normal vector estimation, solving problems such as the low accuracy rate of the PCA algorithm and the resulting incorrect topological structure during reconstruction. In addition, the present invention considers optimizing the domain search process to improve the reconstruction rate, and uses an octree instead of the less efficient Kdtree. Finally, the sum of the volumes of several triangular prisms is calculated through the orthographic projection method to obtain the volume of the bulk material pile.

[0089] Calculating the volume of the 3D model of the bulk material pile using the point cloud data of the bulk material pile includes:

[0090] Reconstruct the bulk material pile using the greedy projection triangulation algorithm based on MLS.

[0091] Reconstructing the bulk material pile using the MLS-based greedy projection triangulation algorithm includes:

[0092] Estimating and reconstructing the normal vectors of the bulk material pile point cloud data using MLS.

[0093] It also includes:

[0094] Adopting the octree algorithm to perform neighborhood search on the bulk material pile point cloud data.

[0095] Specific implementation process:

[0096] S1401, performing downsampling on the segmented pile body point cloud using the Voxel Grid filtering method to simplify the point cloud data while ensuring the uniformity of its density distribution;

[0097] S1402, performing smoothing processing and resampling on the downsampled point cloud using the MLS algorithm to obtain point cloud data with a smooth surface;

[0098] S1403, for the point cloud in step 2, reconstructing the point cloud using MLS-based normal estimation and performing neighborhood search using the octree to finally obtain a better surface model;

[0099] S1404, calculating the volume of the constructed surface model using the orthographic projection method.

Claims

1. An automated method for measuring the volume of a bulk material pile, characterized in that: The following steps are involved: Obtaining the original point cloud data of bulk material pile; The AFF-GrowSP network is used to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; the AFF-GrowSP network uses the AFF module to dynamically adjust the weight of the features of the original point cloud data of the bulk material pile; including: S1301, inputting the raw point cloud data of the bulk material pile that has been preprocessed by hybrid filtering and denoising into the AFF-GrowSP network, and treating the input point cloud as a set of H points; S1302, passing the set through a feature extractor network based on the SpareConv architecture, an encoder with a ResNet16 structure and a decoder with a 4-layer MLP structure, and finally generating a 128-dimensional feature for interpolation to obtain a feature for each point; S1303, the four-layer MLP is used as the X and Y inputs of the AFF module in sequence, and the three AFF modules are connected in series to combine these features and finally obtain the point features; S1304, inputting to Superpoint Constructor to gradually generate larger and larger superpoints, which are used to assist the network to automatically discover the semantic information of the point cloud; S1305, inputting the generated super points into a semantic clustering module to generate pseudo labels for all super points, and the pseudo labels will be used to optimize the feature extractor during the training process; S1306, combining super points into semantic elements, thereby outputting point cloud semantic information; The volume of the bulk material pile 3D model is calculated using the bulk material pile point cloud data.

2. The method for automatically measuring the volume of bulk material according to claim 1, characterized in that: Dynamic weight adjustment of the original point cloud data of bulk material pile by AFF module includes: The small-scale features and large-scale features of the original point cloud data of the bulk material pile are added and fused element by element to obtain the original features; Extract global features and local features of original features; Add the global features and local features element by element to get the weight coefficient; The weight coefficients are used to adjust the weights of small-scale features and large-scale features respectively, and the small-scale features and large-scale features after the adjusted weights are fused to obtain fused features.

3. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 2, characterized in that: Before obtaining the bulk material pile point cloud data, it also includes: The raw point cloud data of bulk material pile is denoised using a hybrid filtering method based on optimization idea.

4. The method for automatically measuring the volume of bulk material according to claim 3, characterized in that: The method of using a hybrid filtering method based on optimization idea to perform noise reduction processing on the original point cloud data of bulk material pile includes: The improved radius filtering method is used to perform large-scale denoising on the original point cloud data of bulk piles. The average value of the number of domain points is used instead of the threshold of the number of domain points, and the parameter of the radius filtering algorithm is reduced from 2 to 1.

5. The method for automatically measuring the volume of bulk material according to claim 4, characterized in that: After large-scale denoising of the original point cloud data of bulk material pile, it also includes: Perform small-scale denoising on the original point cloud data of bulk material pile.

6. The method for automatically measuring the volume of bulk material according to claim 5, characterized in that: The DBSCAN density clustering algorithm is used to perform small-scale denoising on the original point cloud data of bulk material pile.

7. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 6, characterized in that: The method of calculating the volume of the three-dimensional model of the bulk material pile by using the bulk material pile point cloud data includes: The bulk material pile is reconstructed using the MLS-based greedy projection triangulation algorithm.

8. The method for automatically measuring the volume of bulk material according to claim 7, characterized in that: The method of reconstructing the bulk material pile by using the MLS-based greedy projection triangulation algorithm includes: MLS is used to estimate and reconstruct the normal vector of bulk material pile point cloud data.

9. The method for automatically measuring the volume of bulk material according to claim 8, characterized in that: The method of reconstructing the bulk material pile by using the MLS-based greedy projection triangulation algorithm also includes: The octree algorithm is used to perform domain search on bulk material pile point cloud data.

10. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 9, characterized in that: The method of obtaining the original point cloud data of the bulk material pile is to perform non-contact scanning on the bulk material pile using a three-dimensional laser scanner.

Images