Method for automatically measuring volume of bulk material pile

Abstract

The invention discloses an automatic measurement method for the volume of a bulk material pile, and the method comprises the steps: obtaining the original point cloud data of the bulk material pile, carrying out the point cloud segmentation of the original point cloud data of the bulk material pile through an AFF-GrowSP network, and obtaining the point cloud data of the bulk material pile, in the AFF-GrowSP network, dynamic weight adjustment is carried out on the features of the original point cloud data of the bulk material pile through an AFF module, and the volume of a bulk material pile three-dimensional model is calculated by using the point cloud data of the bulk material pile; according to the method, the AFF module is added, and the features of different scales are dynamically adjusted in the process of extracting the original point cloud data features of the bulk material pile, so that the accuracy of point cloud segmentation is improved, and the problem that point cloud segmentation based on deep learning is relatively low in recognition accuracy of point cloud complex semantics is solved; the calculation accuracy and calculation efficiency of the bulk material pile volume are improved, and the method has high generalization ability.

Description

Technical Field

[0001] The invention belongs to the field of automatic measurement of three-dimensional point cloud volume, and in particular relates to an automatic measurement method for bulk material pile volume. Background Art

[0002] At present, bulk material pile volume measurement is mainly divided into the following three categories: photogrammetry, laser and photography combined measurement, and laser scanning measurement.

[0003] The main principle of the laser scanning measurement method is to emit laser and use the pulse method to scan and measure the distance. The scanning device achieves the effect of range scanning by means of rotation or swinging, thereby completing the three-dimensional drawing of the surrounding objects. The point cloud data of the bulk pile is obtained by using a laser scanning device, and then the point cloud data is semantically recognized by using a deep learning method. Finally, the three-dimensional point cloud model of the bulk pile is reconstructed and the volume is calculated. In the process of semantic recognition of point cloud data, the three-dimensional point cloud semantic segmentation method based on self-supervision does not require manual annotation or any pre-training, and can successfully identify the complex semantic classes of each point in the scene. However, the laser scanning device has a large collection area. When collecting bulk piles, it is easy to obtain point clouds of objects near the bulk pile, such as pedestrians and vehicles, which affects the recognition accuracy of the complex semantics of the point cloud, thereby affecting the calculation of the volume of the bulk pile. Summary of the invention

[0004] The purpose of the present invention is to provide an automated method for measuring the volume of a bulk material pile, so as to improve the recognition accuracy of complex semantics of bulk material pile point cloud data.

[0005] The present invention adopts the following technical solutions:

[0006] An automatic method for measuring the volume of a bulk material pile comprises the following steps:

[0007] Obtaining the original point cloud data of bulk material pile;

[0008] The AFF-GrowSP network is used to segment the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; the AFF module in the AFF-GrowSP network dynamically adjusts the weight of the features of the original point cloud data of the bulk material pile;

[0009] The volume of the bulk material pile 3D model is calculated using the bulk material pile point cloud data.

[0010] Furthermore, the dynamic weight adjustment of the original point cloud data of the bulk pile by the AFF module includes:

[0011] The small-scale features and large-scale features of the original point cloud data of the bulk material pile are added and fused element by element to obtain the original features;

[0012] Extract global features and local features of original features;

[0013] Add the global features and local features element by element to get the weight coefficient;

[0014] The weight coefficients are used to adjust the weights of small-scale features and large-scale features respectively, and the small-scale features and large-scale features after the adjusted weights are fused to obtain fused features.

[0015] Furthermore, before obtaining the bulk material pile point cloud data, the following steps are also included:

[0016] The raw point cloud data of bulk material pile is denoised using a hybrid filtering method based on optimization idea.

[0017] Furthermore, the hybrid filtering method based on optimization idea is used to reduce the noise of the original point cloud data of bulk material pile, including:

[0018] The improved radius filtering method is used to perform large-scale denoising on the original point cloud data of bulk piles. The average value of the number of domain points is used instead of the threshold of the number of domain points, and the parameter of the radius filtering algorithm is reduced from 2 to 1.

[0019] Furthermore, after large-scale denoising of the original point cloud data of the bulk material pile, the following steps are also included:

[0020] Perform small-scale denoising on the original point cloud data of bulk material pile.

[0021] Furthermore, the DBSCAN density clustering algorithm is used to perform small-scale denoising on the original point cloud data of bulk pile.

[0022] Further, calculating the volume of the bulk material pile three-dimensional model using the bulk material pile point cloud data includes:

[0023] The bulk material pile is reconstructed using the MLS-based greedy projection triangulation algorithm.

[0024] Furthermore, the bulk material pile is reconstructed using the MLS-based greedy projection triangulation algorithm, including:

[0025] MLS is used to estimate and reconstruct the normal vector of bulk material pile point cloud data.

[0026] Furthermore, the reconstruction of the bulk material pile using the MLS-based greedy projection triangulation algorithm also includes:

[0027] The octree algorithm is used to perform domain search on bulk material pile point cloud data.

[0028] Furthermore, the original point cloud data of the bulk material pile is obtained by performing non-contact scanning on the bulk material pile using a three-dimensional laser scanner.

[0029] The beneficial effects of the present invention are as follows: the present invention uses the AFF-GrowSP network as a point cloud semantic segmentation model, by adding an Attention Feature Fusion (AFF) module of the attention mechanism in the GrowSP network, utilizing the module to fuse the global features and local features of the original point cloud data of the bulk pile, and dynamically adjusting the weights of the fused features, thereby improving the recognition accuracy of complex semantics of the point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the method of the present invention;

[0031] Figure 2 It is a schematic diagram of the structure of the AFF-GrowSP network of the present invention;

[0032] Figure 3 It is a structural schematic diagram of the feature extractor in the AFF-GrowSP network structure of the present invention;

[0033] Figure 4 Schematic diagram of the feature fusion module based on the attention mechanism in the feature extractor of the present invention. DETAILED DESCRIPTION

[0034] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The present invention proposes an automated volume measurement method for bulk material piles based on a self-supervised AFF-GrowSP semantic segmentation network, which is specifically designed for automated volume measurement of multiple bulk material piles and complex surfaces. The present invention first performs non-contact scanning of the bulk material pile using a three-dimensional laser scanner to obtain raw point cloud data. A hybrid filtering method based on optimization ideas is used to reduce noise; secondly, the pre-processed point cloud data is passed through the AFF-GrowSP semantic segmentation network to segment the target object, remove other objects in the scene, and extract the bulk material pile point cloud data; finally, a greedy triangulation algorithm based on MLS is used to complete the three-dimensional reconstruction and volume calculation of the coal pile data.

[0036] The specific process of the automatic measurement method of bulk material volume of the present invention is as follows: Figure 1 As shown:

[0037] S110, obtaining original point cloud data of the bulk material pile.

[0038] Some bulk material pile measurements may occur in high-pollution and high-risk industries, and human involvement may cause disease. Therefore, the present invention adopts a laser radar non-contact measurement method to obtain the original bulk material pile point cloud data.

[0039] S120, using a hybrid filtering method based on optimization idea to perform noise reduction processing on the original point cloud data of the bulk material pile.

[0040] The noise points in the original point cloud data of bulk material piles will change the shape of the piles and cause errors in volume measurement. In addition, noise points will waste storage space, increase running time, and reduce work efficiency. The present invention uses a hybrid filtering method based on optimization ideas to reduce the noise points in steps. First, the improved radius filtering method is used to perform the first stage of large-scale denoising on the original point cloud data. After removing most of the distant noise points and a small number of near noise points with low density, the DBSCAN density clustering algorithm is applied to the small-scale noise of the point cloud to complete the second stage of denoising. Accurate denoising of noisy point cloud data is achieved. Compared with classic spatial domain filtering algorithms, the filtering method based on optimization ideas facilitates subsequent segmentation operations, reduces the final volume measurement error and improves the algorithm processing efficiency. It has good adaptability and retains the details and edge information of the original point cloud while denoising.

[0041] The present invention uses a hybrid filtering method based on optimization ideas to reduce the noise points in steps, and quantifies the distribution range of points in the point cloud by the mean and standard deviation. If the mean of the Euclidean distance from a point to other points is much lower than that of other field points and is lower than 2 to 3 times the standard deviation, it is regarded as a large-scale point, otherwise it is a small-scale point.

[0042] The hybrid filtering method based on optimization idea is used to reduce the noise of the original point cloud data of bulk pile, including:

[0043] The improved radius filter method is used to perform large-scale denoising on the original point cloud data of bulk material piles;

[0044] The DBSCAN density clustering algorithm is used to perform small-scale denoising on the original point cloud data of bulk material pile.

[0045] Among them, the large-scale denoising of the original point cloud data of bulk material pile using the improved radius filtering method includes:

[0046] Set the search radius and field point number threshold;

[0047] Calculate the number of field points within the search radius;

[0048] Calculate the average number of field points;

[0049] The average value of the number of domain points is used instead of the threshold of the number of domain points, and the parameter of the radius filtering algorithm is reduced from 2 to 1;

[0050] Remove points that are not within the average number of points in the neighborhood.

[0051] The denoising process is as follows:

[0052] S1201, inputting the original point cloud data of the bulk material pile;

[0053] S1202, perform the first stage of denoising, and calculate the number of domain points N based on the radius r i ;

[0054] S1203, calculate N i The average

[0055] S1204, use Instead of the threshold m, the radius filter algorithm parameter is reduced from 2 to 1;

[0056] S1205, deleting points that are not within the threshold range, and obtaining the bulk pile original point cloud data P after large-scale denoising. 1 ;

[0057] S1206, entering the second stage of denoising, establishing the KDTree data structure of the point cloud in the previous step;

[0058] S1207, Initialize parameters EPS and N eps,min ;

[0059] S1208, randomly select the original point cloud data P from the bulk pile 1 Select point p i , and determine whether it is a core point, if not, it is a noise point, and delete the noise point;

[0060] S1209, traverse point cloud data P 1 , find point p i All density-reachable points of , construct a point cluster C;

[0061] S1210, output point cluster C, and obtain the original point cloud data P of bulk material pile after hybrid filtering and denoising. o , point cluster C is the original point cloud data P of bulk material pile o .

[0062] S130, using the AFF-GrowSP network to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; in the AFF-GrowSP network, the AFF module is used to dynamically adjust the weight of the features of the original point cloud data of the bulk material pile.

[0063] The actual stacking environment of the pile is relatively complex, and the laser radar collection area is large. When collecting the bulk material pile, it is easy to obtain the point cloud of objects near the bulk material pile, which affects the subsequent volume calculation. To address the above problems, the present invention uses the AFF-GrowSP point cloud semantic segmentation network based on self-supervision to separately segment the bulk material pile point cloud, such as Figure 2As shown. For the problem of 3D semantic segmentation, existing methods mainly rely on a large amount of manual annotation to train neural networks, such as pointnet, pointnet++, and 2D vision-based 3D point cloud processing tasks. GrowSP is a completely self-supervised method that does not require manual annotation or any pre-training. It can successfully identify the complex semantic classes of each point in the scene. And use the feature fusion module (Attention Feature Fusion, AFF) of the attention mechanism to effectively fuse and complement features of different scales. Figure 2 As shown in the figure, the network can automatically process point cloud data without human intervention, effectively improving the efficiency and accuracy of coal pile volume measurement.

[0064] GrowSP discovers 3D semantic information through the progressive growth of superpoints. The method consists of three parts: ① Feature extractor, which learns the feature information of each point. GrowSP follows the SparseConv architecture. The encoder uses the ResNet16 structure, and the decoder consists of 4 MLP layers. These MLP layers generate 128-dimensional features for interpolation, and are finally combined through the AFF module to obtain point cloud feature information. ② Superpoint constructor, which gradually increases the size of superpoints. Before the network training starts, this module first constructs the initial superpoints through VCCS and region growing algorithms, and then uses the K-means clustering algorithm to obtain larger superpoints, and repeats the above steps. ③ Semantic primitive clustering module, GrowSP regards semantic segmentation as a problem of 3D feature learning and clustering. For each Epoch, each input point cloud will have several superpoints, each superpoint represents an object and a part of an object. For the entire point cloud data, all superpoints can be regarded as a huge collection of basic semantic elements. The network constructs initial and expanded superpoints, combines them into semantic elements, and outputs point cloud semantic information.

[0065] Feature fusion module based on attention mechanism. In common bulk material pile point cloud data, there are various types of point cloud data such as pedestrians, forklifts, bulk material piles, etc. The scales of these data are different. For example, the scale of bulk material piles is large, while the scales of pedestrians and forklifts are small. In the field of deep learning, small-scale features represent delicate local information, while large-scale features represent broad contextual knowledge and rough semantic information. Regarding the problem of how to efficiently integrate and complement these two, such as Figure 3 As shown in FIG, the present invention introduces an AFF module. This module analyzes the relationship between features of different scales and dynamically adjusts their fusion weights, aiming to simultaneously retain small-scale features with rich details and integrate the contextual semantics in large-scale features, thereby achieving more accurate semantic recognition. The architecture of the module is shown in FIG. Figure 4 shown.

[0066] Dynamic weight adjustment of the original point cloud data of bulk material pile includes:

[0067] The small-scale features and large-scale features of the original point cloud data of the bulk material pile are added and fused element by element to obtain the original features;

[0068] Extract global features and local features of original features;

[0069] Add the global features and local features element by element to get the weight coefficient;

[0070] The weight coefficients are used to adjust the weights of small-scale features and large-scale features respectively, and the small-scale features and large-scale features after the adjusted weights are fused to obtain fused features.

[0071] Figure 4 In the above equation, X,Y,X′∈R C×N , X is the small-scale feature extracted by the encoding network, Y is the large-scale feature obtained by feature propagation through inverse distance interpolation, and N and C represent the size and number of channels of the feature point cloud, respectively. Figure 4 As shown in (a) of the figure, X and Y are first element-wise added and fused to obtain feature X′; then X′ is sent to Figure 4 In (b), the global feature g(X′) and local feature L(X′) are extracted. The calculation process of the global feature is:

[0072] g(X′)=B{W 2 δ{B{W 1 [Γ(X′)]}}} (1)

[0073] The calculation process of local features is:

[0074] L(X′)=B{W 2 δ{B[W 1 (X′)]}} (2)

[0075] In the formula, Γ(X′) is the global average pooling function, B is the batch normalization layer, δ is the ReLU activation function, and W 1 and W 2 is a fully connected layer. Figure 4 (b), finally get the weight coefficient w∈R of feature fusion C×N , the calculation formula is:

[0076]

[0077] Where σ is the Sigmoid function, is element-wise addition. Finally, the features X and Y are weighted fused according to w to obtain the fused feature Z. The fusion process is:

[0078]

[0079] Where I represents the all-1 matrix, which is used for weighted operations in feature fusion.

[0080] Specifically:

[0081] S1301, the original point cloud data P of the bulk material pile that has been pre-processed by hybrid filtering and denoising o Input the AFF-GrowSP network and regard the input point cloud as a set of H points;

[0082] S1302, passing the set through a feature extractor network based on the SpareConv architecture, an encoder with a ResNet16 structure and a decoder with a 4-layer MLP structure, and finally generating a 128-dimensional feature for interpolation to obtain a feature for each point;

[0083] S1303: The four-layer MLP is used as the X and Y inputs of the AFF module in sequence. The three AFF modules are connected in series to combine these features and finally obtain the feature F of the point. h ;

[0084] S1304, F h Input to Superpoint Constructor to gradually generate larger and larger superpoints. These superpoints are used to assist the network to automatically discover the semantic information of point clouds.

[0085] S1305, inputting the generated super points into a semantic clustering module to generate pseudo labels for all super points, and the pseudo labels will be used to optimize the feature extractor during the training process;

[0086] S1306, combining super points into semantic elements, thereby outputting point cloud semantic information.

[0087] S140, calculating the volume of the three-dimensional model of the bulk material pile using the bulk material pile point cloud data.

[0088] After the AFF-GorwSP network segmentation, a separate pile point cloud of the bulk material pile is obtained. The segmented pile point cloud data has the characteristics of high density and high precision. Direct three-dimensional reconstruction takes a long time, and the reconstructed model surface is not smooth enough and there are problems of small holes. The present invention proposes a greedy projection triangulation algorithm based on MLS. Compared with the traditional greedy projection algorithm that relies on PCA, the present invention calculates and reconstructs the normal vector based on MLS normal vector estimation, which solves the problem that the accuracy of the PCA algorithm is not high, resulting in incorrect topological structure during reconstruction. In addition, the present invention considers optimizing the field search process and improving the reconstruction rate, and uses an octree instead of the less efficient Kdtree. Finally, the volume of several triangular prisms is calculated by the orthographic projection method to obtain the volume of the bulk material pile.

[0089] Calculating the volume of the bulk material pile 3D model using bulk material pile point cloud data includes:

[0090] The bulk material pile is reconstructed using the MLS-based greedy projection triangulation algorithm.

[0091] The reconstruction of bulk material pile using MLS-based greedy projection triangulation algorithm includes:

[0092] MLS is used to estimate and reconstruct the normal vector of bulk material pile point cloud data.

[0093] Also includes:

[0094] The octree algorithm is used to perform domain search on bulk material pile point cloud data.

[0095] Specific implementation process:

[0096] S1401, down-sampling the segmented pile point cloud using a Voxel Grid filtering method to simplify the point cloud data while ensuring the uniformity of its density distribution;

[0097] S1402, smoothing and resampling the downsampled point cloud using the MLS algorithm to obtain point cloud data with smoothed surface;

[0098] S1403, reconstructing the point cloud in step 2 by using normal estimation based on MLS, and performing a neighborhood search using an octree, and finally obtaining a surface model with a better effect;

[0099] S1404, calculating the volume of the constructed surface model using an orthographic projection method.

Claims

1. An automated method for measuring the volume of a bulk material pile, characterized in that: The following steps are involved: Obtaining the original point cloud data of bulk material pile; The AFF-GrowSP network is used to perform point cloud segmentation on the original point cloud data of the bulk material pile to obtain the point cloud data of the bulk material pile; the AFF-GrowSP network uses the AFF module to dynamically adjust the weight of the features of the original point cloud data of the bulk material pile; The volume of the bulk material pile 3D model is calculated using the bulk material pile point cloud data.

2. The method for automatically measuring the volume of bulk material according to claim 1, characterized in that: Dynamic weight adjustment of the original point cloud data of bulk material pile by AFF module includes: The small-scale features and large-scale features of the original point cloud data of the bulk material pile are added and fused element by element to obtain the original features; Extract global features and local features of original features; Add the global features and local features element by element to get the weight coefficient; The weight coefficients are used to adjust the weights of small-scale features and large-scale features respectively, and the small-scale features and large-scale features after the adjusted weights are fused to obtain fused features.

3. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 2, characterized in that: Before obtaining the bulk material pile point cloud data, it also includes: The raw point cloud data of bulk material pile is denoised using a hybrid filtering method based on optimization idea.

4. The method for automatically measuring the volume of bulk material according to claim 3, characterized in that: The method of using a hybrid filtering method based on optimization idea to perform noise reduction processing on the original point cloud data of bulk material pile includes: The improved radius filtering method is used to perform large-scale denoising on the original point cloud data of bulk piles. The average value of the number of domain points is used instead of the threshold of the number of domain points, and the parameter of the radius filtering algorithm is reduced from 2 to 1.

5. The method for automatically measuring the volume of bulk material according to claim 4, characterized in that: After large-scale denoising of the original point cloud data of bulk material pile, it also includes: Perform small-scale denoising on the original point cloud data of bulk material pile.

6. The method for automatically measuring the volume of bulk material according to claim 5, characterized in that: The DBSCAN density clustering algorithm is used to perform small-scale denoising on the original point cloud data of bulk material pile.

7. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 6, characterized in that: The method of calculating the volume of the three-dimensional model of the bulk material pile by using the bulk material pile point cloud data includes: The bulk material pile is reconstructed using the MLS-based greedy projection triangulation algorithm.

8. The method for automatically measuring the volume of bulk material according to claim 7, characterized in that: The method of reconstructing the bulk material pile by using the MLS-based greedy projection triangulation algorithm includes: MLS is used to estimate and reconstruct the normal vector of bulk material pile point cloud data.

9. The method for automatically measuring the volume of bulk material according to claim 8, characterized in that: The method of reconstructing the bulk material pile by using the MLS-based greedy projection triangulation algorithm also includes: The octree algorithm is used to perform domain search on bulk material pile point cloud data.

10. The method for automatically measuring the volume of a bulk material pile according to claim 1 or 9, characterized in that: The method of obtaining the original point cloud data of the bulk material pile is to perform non-contact scanning on the bulk material pile using a three-dimensional laser scanner.

Images