Cement conveyor volume measurement method based on deep learning point cloud segmentation

Abstract

The invention provides a cement conveyor volume measurement method based on deep learning point cloud segmentation. The method comprises the following steps: obtaining 3D point cloud data by using a laser line scanning 3D camera, and determining RGB values of pixels corresponding to the 3D point cloud data to obtain 3D point cloud with color features; inputting the 3D point cloud with the color features into an improved Point-Net + + neural network for 3D point cloud segmentation, and obtaining 3D point cloud data only containing the cement surface in the conveyor after segmentation; constructing a triangulation network based on the cement irregular surface in the segmented 3D point cloud data by adopting a triangulation method, and performing volume calculation according to the constructed triangulation network; wherein the improved Point-Net + + neural network is constructed on the basis of a framework of an original Point-Net + + point cloud segmentation network, and comprises a core point convolution, a context sensing module and a Focal Loss loss function. Therefore, the performance of a point cloud segmentation algorithm is improved, and even under complex conditions, the surface point cloud data of the cement material in the conveyor can still be accurately segmented for volume measurement.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision volume measurement, and particularly relates to a method for measuring the volume of a cement conveyor by deep learning point cloud segmentation. Background Art

[0002] With the development of technology and the need for refined management of the production process, it is currently necessary to measure the volume of cement on a bucket conveyor to accurately grasp the daily production data of the factory.

[0003] Due to the uniform motion attribute of the bucket conveyor, the data collection of the cement on the bucket conveyor is often inaccurate, resulting in a more complex subsequent point cloud segmentation.

[0004] For the point cloud segmentation task in a complex environment, the traditional segmentation paradigm based on a single laser point cloud has the dual challenges of the lack of multi-modal data fusion and limited computational efficiency, making it difficult to meet the accuracy requirements of actual engineering scenarios.

[0005] Therefore, an improved technical solution is needed to address the above deficiencies in the prior art. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for measuring the volume of a cement conveyor by deep learning point cloud segmentation to solve or alleviate the problems existing in the above prior art.

[0007] To achieve the above purpose, the present invention provides the following technical solutions:

[0008] The present invention provides a method for measuring the volume of a cement conveyor by deep learning point cloud segmentation, and the measurement method includes:

[0009] Step S1: Obtain 3D point cloud data by using a laser line scan 3D camera, and determine the RGB value of the pixels corresponding to the 3D point cloud data to obtain a 3D point cloud with color features;

[0010] Step S2: Input the 3D point cloud with color features into an improved Point-Net++ neural network for 3D point cloud segmentation to obtain segmented 3D point cloud data; the segmented 3D point cloud data only contains the 3D point cloud data of the surface of the cement in the conveyor;

[0011] Step S3: Adopt the triangulation method to construct a triangular mesh based on the irregular surface of the cement in the segmented 3D point cloud data, and perform volume calculation according to the constructed triangular mesh;

[0012] Among them, the improved Point-Net++ neural network is constructed based on the framework of the original Point-Net++ point cloud segmentation network, including: core point convolution and context awareness module;

[0013] The core point convolution is used to replace the feature extraction structure in the SA abstraction layer of the original Point-Net++ point cloud segmentation network;

[0014] The context awareness module is set after the last FP decoding layer of the improved Point-Net++ point cloud segmentation network;

[0015] The improved Point-Net++ neural network further includes: using the Focal Loss function to replace the original loss function in the original Point-Net++ point cloud segmentation network.

[0016] Preferably, in step S1, obtaining 3D point cloud data by using a laser line-scanning 3D camera includes:

[0017] Adopting a line laser system, based on the principle of triangulation, three-dimensional reconstruction is performed on the original data collected by the laser line-scanning 3D camera to obtain 3D point cloud data.

[0018] Preferably, the improved Point-Net++ neural network specifically performs the following steps:

[0019] Perform downsampling and obtain global features of the space;

[0020] Then gradually restore to the original point cloud size through the upsampling layer;

[0021] Finally, the fully connected layer realizes the deep representation learning of the global spatial features of each point cloud by effectively fusing multi-scale context information, and then completes the accurate semantic classification task.

[0022] Preferably, before performing downsampling and obtaining local features of the space, the method further includes:

[0023] Using color features as auxiliary features, and using 3D point cloud data with color features to replace the 3D point cloud data only containing position information in the input end of the improved Point-Net++ neural network in the original Point-Net++ point cloud segmentation network;

[0024] The performing downsampling and obtaining local features of the space is specifically:

[0025] Hierarchical downsampling is achieved through core point convolution. While effectively reducing the amount of point cloud data, local geometric features are captured by core points with fixed preset positions, weights are calculated for each core point based on the distances and positions of neighboring points, and finally, convolution operations are performed on the neighborhood point set using the core points and their weights to obtain new features.

[0026] Preferably, the triangulation method is the Delaunay triangulation method.

[0027] Preferably, in step S3, a triangular mesh is constructed for the irregular surface of the cement, and the specific steps are as follows:

[0028] S31. Connect the points on the irregular surface of the cement to generate initial triangles;

[0029] S32. Expand the initial triangles into a triangular mesh;

[0030] S33. Repeat step S32 until all points in the point set are processed;

[0031] The point set is the set composed of all points in the segmented 3D point cloud data.

[0032] Preferably, the Delaunay triangulation method is specifically: the Delaunay triangulation method performs two-dimensional triangulation on the cement heap to obtain triangulation triangles.

[0033] Preferably, the volume calculation in step S3 is specifically:

[0034] The vertex of each triangulation triangle corresponds to a projection point in the point cloud model of the irregular surface of the cement. Each triangulation triangle has a one-to-one correspondence with a triangular patch on the irregular surface of the cement. The projection triangle and the triangular patch form a combined body; each combined body consists of two tetrahedrons and a triangular prism. The volume of each combined body is equal to the sum of the volumes of these three spatial bodies, and the calculation formula is:

[0035] V i = V ABC-A′B′C′ + V B-A′B′C + V A-A′BC′ ,

[0036] where, V i is the volume of the i-th combined body, V ABC-A′B′C′ represents the volume of the triangular prism, V B-A′B′C and V A-A′BC′ represent the volumes of the two tetrahedrons. A, B, and C are points in the triangulation triangle, and A', B', and C' are the corresponding projection points;

[0037] The volumes of all combined bodies are added together to obtain the total volume of the cement in the conveyor.

[0038] The embodiments of the present disclosure also propose a volume measurement system for a cement conveyor using deep learning point cloud segmentation, and the system uses the steps of the above-mentioned volume measurement method for a cement conveyor using deep learning point cloud segmentation.

[0039] Compared with the closest prior art, the technical solutions of the embodiments of the present invention have the following beneficial effects:

[0040] When collecting data on cement on a chain bucket conveyor, the present invention uses a laser line scan 3D camera for three-dimensional reconstruction to obtain point cloud data. This technical means is reliable and stable, and also has the advantages of being more easily visualizable and more accurate. This application uses an improved deep learning convolutional neural network to perform semantic segmentation on the point cloud data collected by the laser line scan 3D camera, addresses image information problems, improves the performance of point cloud segmentation, and obtains point cloud data that only contains the surface of the cement in the conveyor. It can also perform point cloud segmentation and volume measurement on the piled body in the cement conveyor under complex conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings constituting a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. Among them:

[0042] Figure 1 is a schematic structural diagram of the improved Point-Net++ neural network according to the embodiments of the present disclosure.

[0043] Figure 2 is a conveying diagram of a cement piled body according to the embodiments of the present disclosure.

[0044] Figure 3 is a schematic diagram of volume calculation according to the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] The following will refer to the accompanying drawings and describe the present application in detail in conjunction with embodiments. Each example is provided by way of explanation of the present application rather than limitation of the present application. In fact, those skilled in the art will clearly understand that modifications and variations can be made to the present application without departing from the scope or spirit of the present application. For example, features shown or described as part of one embodiment can be used in another embodiment to produce yet another embodiment. Therefore, it is desirable that the present application includes such modifications and variations that fall within the scope of the appended claims and their equivalents.

[0046] In the following description, the terms "first / second / third" involved are only used to distinguish similar objects and do not represent a specific order of the objects. Understandably, "first / second / third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this disclosure belongs. The terms used herein are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.

[0048] In the description of this application, the orientation or positional relationship indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing this application and does not require this application to be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to this application. The terms "connected", "connected to", and "disposed" used in this application should be understood in a broad sense. For example, it can be a fixed connection or a detachable connection; it can be directly connected or indirectly connected through an intermediate component; it can be a wired connection, a radio connection, or a wireless communication signal connection. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0049] The embodiments of this application mainly relate to the technical field of machine vision volume measurement. Specifically, a method for measuring the volume of a cement conveyor by deep learning point cloud segmentation is disclosed. The specific steps of this embodiment are as follows:

[0050] Step S1: Obtain 3D point cloud data using a laser line-scanning 3D camera. According to the 3D point cloud data, determine the RGB values of the pixels corresponding to the 3D point cloud data to obtain a 3D point cloud with color features.

[0051] Step S2: Input the 3D point cloud with color features into an improved Point-Net++ neural network for 3D point cloud segmentation to obtain segmented 3D point cloud data; the segmented 3D point cloud data only contains the 3D point cloud data of the surface of the cement in the conveyor.

[0052] Step S3: Adopt the triangulation method to construct a triangular mesh based on the irregular surface of the cement in the segmented 3D point cloud data, and perform volume calculation according to the constructed triangular mesh.

[0053] For step S1, a laser line-scanning 3D camera is used to collect data on the cement pile in a chain bucket conveyor moving at a constant speed. During the process of image acquisition by the camera, the internal and external parameters of the camera need to be calibrated first. After calibration, the pixel coordinates on the image collected by the camera can be converted into world coordinates for measurement. Due to the imaging characteristics of a monocular camera, the depth information of the target object from the optical axis cannot be directly obtained from a two-dimensional image alone. Therefore, in this embodiment, a line laser structured light system is introduced, and the depth information of the target surface points is calculated through the principle of laser triangulation. The light plane projected by the line laser system constrains the target object, enabling the depth information of the target object to be solved. Finally, through the solution of the depth information of the light plane and the calibration of the camera internal parameters, it is projected into the camera uv coordinate system, and the RGB values of the corresponding pixels are obtained, and 3D point cloud data with color characteristics can be obtained.

[0054] Specifically, using a laser line-scanning 3D camera to collect data on the cement pile in a chain bucket conveyor moving at a constant speed can better achieve high-precision reconstruction of the three-dimensional shape of the object surface compared with traditional methods. And since the target object being collected is moving at a constant speed, the constant-speed motion characteristics of the target object fully match the frame rate-motion speed synchronization constraint condition of the laser line-scanning camera, ensuring the spatial continuity and geometric accuracy of the collected data.

[0055] Figure 1 The structural schematic diagram of the improved Point-Net++ neural network according to the embodiment of the present invention is shown.

[0056] As Figure 1 shown in the process, the 3D point cloud data with color characteristics obtained from step S1 is downsampled layer by layer through a multi-level SA module and local geometric features are extracted, then upsampled step by step through an FP module to restore to the original point cloud size, and then sent to the context awareness module, and then the class imbalance problem is optimized through the Focal Loss function; the 3D point cloud data with color characteristics is finally segmented by the improved Point-Net++ neural network to obtain 3D point cloud data only containing the surface of the cement in the conveyor. Step S2 performs the processing process as Figure 1 shown, including:

[0057] Among them, the improved Point-Net++ neural network is constructed based on the framework of the original Point-Net++ point cloud segmentation network, including: core point convolution and context awareness module. The core point convolution technology adopted replaces the feature extraction structure in the SA abstraction layer of the original Point-Net++ point cloud segmentation network. The SA abstraction layer is specifically composed of three abstraction layers: SA1, SA2, and SA3. The context awareness module is deployed after the last FP decoding layer of the improved Point-Net++ point cloud segmentation network. And the Focal Loss function is used to replace the original loss function in the original Point-Net++ point cloud segmentation network.

[0058] For step S2, in one embodiment, the 3D point cloud segmentation process for the to-be-processed 3D point cloud data with color features includes:

[0059] Input the to-be-processed 3D point cloud data with color features ( Figure 1 ColorPoint Cloud in it) into the SA1 abstraction layer of the improved Point-Net++ neural network for downsampling. The SA1 abstraction layer includes: a sampling grouping module ( Figure 1 Sample Grouping in it) and core point convolution ( Figure 1 PAConv in it). The sampling grouping module performs sampling and grouping processing on the incoming 3D point cloud data with color features. The purpose of sampling is to reduce the amount of point cloud data, reduce the computational burden, and retain the important structural information of the point cloud data. The purpose of grouping is to construct a local neighborhood for extracting local features. The result achieved is to generate a set containing neighboring points for each center point for subsequent convolution or feature aggregation operations. The point cloud data output by the sampling grouping module is transmitted to the core point convolution for local feature extraction processing. The core point convolution captures local geometric features through core points with preset fixed positions, calculates a weight for each core point according to the distance and position of neighboring points, and finally uses the core points and their weights to perform a convolution operation on the neighboring point set to obtain new features. In this way, the effects of reducing the amount of point cloud data, removing redundant points, retaining key geometric features, and reducing the computational burden are achieved. Finally, a pooling operation is performed on the local features processed by the core point convolution to compress and abstract the local features, and the input features are aggregated through a sliding window to play the roles of dimensionality reduction (reducing the amount of data and computational complexity), feature stability (enhancing the robustness to small translations and rotations of the input), and hierarchical abstraction (extracting more global semantic information layer by layer).

[0060] In practical applications, although the traditional convolution method is simple and efficient, its ability to extract local features of point clouds is limited and the efficiency is low, making it inapplicable to complex environments. When the core point convolution method extracts features from point cloud data, it adds spatial information, enhancing the ability to capture local geometric features of point clouds, significantly improving the performance of point cloud processing. This enables the modified model to more accurately identify the geometric differences between the regular geometric shapes of conveyor components and the accumulation patterns of cement, reducing misclassification and enhancing the model's feature extraction ability for point cloud data of cement conveyors. Moreover, due to the sparse point clouds at the far end from the camera and the dense point clouds at the near end in the scanned point clouds, the fixed-radius neighborhood grouping of the original Point-Net++ is prone to failure in sparse regions. After replacing it with the core point convolution, the coverage range of its kernel points can be dynamically adjusted. In sparse regions, a larger kernel size is used to capture the context, effectively reducing the problem of missed detection of sparse cement point clouds at the far end of the conveyor, avoiding overfitting in high-density regions and underfitting in low-density regions. For example, at a distance of 5m, the recall rate of cement point clouds is significantly improved.

[0061] Further, the SA1 abstraction layer inputs the processed local features of the point cloud to the FP3 decoding layer through upsampling.

[0062] Based on this, in one embodiment, the method may further include:

[0063] The SA1 abstraction layer transmits the processed local feature data of the point cloud to the next SA2 abstraction layer, and the SA2 abstraction layer transmits the processed local feature data of the point cloud to the next SA3 abstraction layer. Through the downsampling process of the SA3 abstraction layer, it is transmitted to the FP1 decoding layer for upsampling. After each SA abstraction layer extracts local features through the core point convolution, higher-level features are generated through pooling operations, and the receptive field gradually expands from SA1 to SA3, covering the entire conveyor cross-section. The higher-level features aggregate global information at the highest layer to generate global features.

[0064] The FP decoding layer performs upsampling processing on the data from the SA abstraction layer. The main purpose of the improved spatial decoding part of Point-Net++ is to map the aggregated local spatial information to all the point clouds in the point cloud set to obtain the point features of each original point in the point cloud set. The FP decoding layer restores the point cloud density step by step through three-level FP mode by upsampling. The specific implementation process is to propagate the color-weighted features of the points obtained through the inverse distance weighted interpolation technique ( Figure 1 IDW in Figure 1 and then concatenate the obtained weighted features with the spatial local features obtained from the corresponding point set SA abstraction layer during the encoding process through cross-layer skip connection ( Figure 1 Concate in ). The combined features after concatenation are aggregated through a single Pointnet layer structure using the core point convolution method to obtain the final point features and gradually restore them to the size of the original point cloud.

[0065] Based on this, in one embodiment, the method may further include:

[0066] The FP decoding layer is specifically composed of three decoding layers, namely FP1, FP2, and FP3. Among them, in the FP1 decoding layer, the inverse distance weighted interpolation technology module processes the point cloud local features transmitted by the SA3 layer, and splices them with the point cloud local features transmitted by the SA2 layer through a cross-layer skip connection. The spliced combined features are aggregated by a single Pointnet layer structure using the kernel point convolution method to obtain new point cloud features; then the point cloud features obtained by the FP1 layer are used as the interpolation part in the next layer, the FP2 layer, to obtain features using the inverse distance weighted interpolation technology, and then spliced with the point cloud features transmitted in the SA1 layer to obtain a new combined feature. The spliced new combined feature is aggregated by a single Pointnet layer structure using the kernel point convolution method to obtain another new point cloud feature; finally, the point cloud features obtained by the FP2 layer are used as the interpolation part in the next layer, the FP3 layer, to obtain features using the inverse distance weighted interpolation technology, and then spliced with the original point cloud coordinates output from the laser line scan 3D camera, and finally the point cloud features of each point cloud are obtained using the kernel point convolution method.

[0067] In the process of feature propagation (FP3) to the fully connected layer (FC), a context awareness module is added to solve the following two complex environmental situations: misjudgment caused by insufficient local features when the conveyor support blocks part of the cement area; misjudgment caused by highly similar local geometric features when the cement contacts the conveyor chain or idler. The final point cloud features of each point cloud obtained by the FP3 layer are transmitted to the context awareness module for processing, and the Focal Loss function is used to optimize the class imbalance problem to balance the problem of excessive difference in the amount of point cloud data separated by the cement surface and the conveyor side in the point cloud data, so as to ensure the convergence speed and sufficient training of the category with a smaller point cloud occupancy.

[0068] In practical applications, the FP decoding layer in the original PointNet++ neural network relies on skip connections to recover the detailed information lost during hierarchical downsampling and lacks explicit modeling of global semantics, which will lead to misjudgment due to insufficient local features when the conveyor support blocks part of the cement area, and when the cement contacts the conveyor chain or idler, the local geometric features are highly similar and cannot be distinguished only by local information. Based on the above complex situations, a context awareness module is introduced at the stage of feature propagation to the fully connected layer, aiming to learn the correlation between features of the same category and the difference between features of different categories, so that the enhanced features are improved in both representation ability and robustness.

[0069] Furthermore, the calculation formula of the used Focal Loss function is:

[0070] L FL (y) = -α y log(p y )(1 - p y ) λ ,

[0071] where: α y is the class weight, which can suppress the imbalance in the number of positive and negative samples. The λ coefficient makes the model focus more on difficult-to-classify samples during training by reducing the weight of easy-to-classify samples. y represents the true class label of the sample, and p y is the probability value predicted as y. In this application, the Focal Loss function with parameters α = 1 and γ = 2 is used to train the model.

[0072] Finally, the processed point cloud features are input into the fully connected layer. The fully connected layer performs point-by-point feature mapping on all the input point cloud features through equivalent full connection and conducts classification processing to obtain the predicted semantic result for each point, that is, the point cloud data segmented to only contain the surface of the cement in the conveyor.

[0073] By performing segmentation processing on the improved Point-Net++ neural network, the point cloud data only containing the surface of the cement in the conveyor is finally obtained, thereby providing high-precision input for volume measurement. Since the surface shape of the cement in the conveyor is irregular, as Figure 2 shown, the bottom surface of the cement pile is rectangular, and the top area is an irregular cone. When calculating the irregular volume, since there is no overlapping part in the space of the cement stockpile, the Delaunay triangulation method is used to perform two-dimensional triangulation on the cement pile body, and the obtained triangulation triangles are used to construct a triangular mesh.

[0074] For the construction of the triangular mesh, first find the two closest points in the point set and connect them into an edge, and then find the other endpoint of the triangle containing this edge according to the Delaunay empty circle criterion of the triangular mesh. Process all newly generated edges in turn until all points are processed. The basic operation steps of the triangular mesh are as follows:

[0075] S31. Connect the points on the irregular surface of the cement to generate initial triangles;

[0076] S32. Expand the initial triangles into a triangular mesh;

[0077] S33. Repeat step S32 until all points in the point set are processed;

[0078] The point set is the set composed of all points in the segmented 3D point cloud data.

[0079] After the above-mentioned point cloud of the cement heap is dissected by the two-dimensional dissection method, a point cloud model of the cement heap is obtained. First, the point cloud is projected onto the XY plane. At this time, the projection points are scattered points in the two-dimensional plane. The Delaunay triangulation is performed on the projected scattered points. According to the dissection criteria of the Delaunay triangulation, it can be known that the dissected triangles are adjacent and do not contain each other. The vertices of each dissected triangle correspond to a projection point in the heap point cloud model. Then, each dissected triangle and a triangular patch on the heap surface have a one-to-one correspondence. The projected triangle and the triangular patch can form a combined body, as Figure 3 shown. A, B, and C are points in the original point cloud data, and A', B', and C' are the corresponding projection points. From Figure 3 it can be seen that each combined body is composed of two tetrahedrons and a triangular prism, and the volume of each combined body is equal to the sum of the volumes of these three spatial bodies. The calculation formula for a single combined body is:

[0080] V i =V ABC-A′B′C′ +V B-A′B′C +V A-A′BC′ ,

[0081] where V i is the volume of the i-th combined body, V ABC-A′B′C′ represents the volume of the triangular prism, and V B-A′B′C and V A-A′BC′ represent the volumes of the two tetrahedrons. A, B, and C are points in the dissected triangle, and A', B', and C' are the corresponding projection points. By adding up the volumes of all combined bodies, the total volume of the cement in the conveyor can be obtained.

[0082] In addition, the present application also discloses a volume measurement system for a cement conveyor with deep learning point cloud segmentation. This system calculates the total volume of the cement in the conveyor by using the steps of the above-mentioned embodiment.

[0083] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation, characterized in that, The measurement method includes: Step S1: Obtain 3D point cloud data using a laser line-scanning 3D camera. According to the 3D point cloud data, determine the RGB values of the pixels corresponding to the 3D point cloud data to obtain a 3D point cloud with color features. Step S2: Input the 3D point cloud with color features into an improved Point-Net++ neural network for 3D point cloud segmentation to obtain segmented 3D point cloud data; the segmented 3D point cloud data only contains the 3D point cloud data of the surface of the cement in the conveyor. Step S3: Adopt the triangulation method to construct a triangular mesh based on the irregular surface of the cement in the segmented 3D point cloud data, and perform volume calculation according to the constructed triangular mesh. Among them, the improved Point-Net++ neural network is constructed based on the framework of the original Point-Net++ point cloud segmentation network, including: keypoint convolution, context awareness module. The keypoint convolution is used to replace the feature extraction structure in the SA abstraction layer of the original Point-Net++ point cloud segmentation network. The context awareness module is set after the last FP decoding layer of the improved Point-Net++ point cloud segmentation network. The improved Point-Net++ neural network further includes: using the Focal Loss function to replace the original loss function in the original Point-Net++ point cloud segmentation network.

2. The method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 1, wherein In step S1, obtaining 3D point cloud data using a laser line-scanning 3D camera includes: Adopt a line laser system to perform three-dimensional reconstruction on the original data collected by the laser line-scanning 3D camera based on the principle of triangulation to obtain 3D point cloud data.

3. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 1, characterized in that, The improved Point-Net++ neural network specifically performs the following steps: Perform downsampling and obtain local features in space. Then gradually restore to the original point cloud size through the upsampling layer. Finally, the fully connected layer realizes deep representation learning of the global spatial features of each point cloud by effectively fusing multi-scale context information, and then completes the accurate semantic classification task.

4. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 3, wherein, Before performing downsampling and obtaining local features in space, the method further includes: Using color features as auxiliary features, using the 3D point cloud data with color features to replace the 3D point cloud data only containing position information in the input end of the improved Point-Net++ neural network in the original Point-Net++ point cloud segmentation network. The performing downsampling and obtaining local features in space is specifically: Realize hierarchical downsampling through keypoint convolution. While effectively reducing the amount of point cloud data, capture local geometric features through keypoints with preset fixed positions, calculate weights for each keypoint according to the distance and position of the neighboring points, and finally perform a convolution operation on the neighboring point set using the keypoints and their weights to obtain new features.

5. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 1, characterized in that, The triangulation method is the Delaunay triangulation method.

6. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 1, characterized in that, In step S3, constructing a triangular mesh for the irregular surface of the cement, the specific steps are as follows: S31. Connect the points on the irregular surface of the cement to generate initial triangles. S32. Expand the initial triangles into a triangular mesh. S33. Repeat step S32 until all points in the point set are processed; The point set is a set composed of all points in the segmented 3D point cloud data.

7. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 5, characterized in that, The Delaunay triangulation method specifically is: the Delaunay triangulation method performs two-dimensional triangulation on the cement heap to obtain triangulated triangles.

8. A method for measuring the volume of a cement conveyor by deep learning point cloud segmentation according to claim 7, characterized in that, The volume calculation in step S3 specifically is: The vertex of each triangulated triangle corresponds to a projection point in the point cloud model of the irregular surface of the cement. Each triangulated triangle has a one-to-one correspondence with a triangular patch on the irregular surface of the cement. The projected triangle and the triangular patch form a composite body; each composite body consists of two tetrahedrons and a triangular prism. The volume of each composite body is equal to the sum of the volumes of these three spatial bodies. The calculation formula is: V i = V ABC-A′B′C′ + V B-A′B′C + V A-A′BC′ , Among them, V i is the volume of the i-th composite body, V ABC-A′B′C′ represents the volume of the triangular prism, V B-A′B′C and V A-A′BC′ represent the volumes of two tetrahedrons. A, B, and C are points in the triangulated triangle, and A', B', and C' are the corresponding projection points; Add up the volumes of all composite bodies to obtain the total volume of the cement in the conveyor.

9. A volume measurement system for a cement conveyor with deep learning point cloud segmentation. The system is used to execute the steps of a volume measurement method for a cement conveyor with deep learning point cloud segmentation according to any one of claims 1 to 8.

Images