Drilling rig vehicle body point cloud elimination method and system based on point cloud segmentation network model

Abstract

The application belongs to the field of point cloud segmentation, and provides a drilling rig vehicle body point cloud elimination method and system based on a point cloud segmentation network model, which comprises the following steps: obtaining a first point cloud by normalizing and dimensionally upgrading mine point clouds; obtaining a second point cloud by dimensionally upgrading the first point cloud after feature transformation, and obtaining a third point cloud by dimensionally upgrading the second point cloud; obtaining a hierarchical down-sampling point cloud by grading and down-sampling the first point cloud to three sizes and then splicing, and obtaining point cloud local features by combining the hierarchical down-sampling point cloud and the second point cloud; obtaining a fourth point cloud by processing the third point cloud and the hierarchical down-sampling point cloud, splicing the fourth point cloud and the third point cloud, and then performing maximum pooling to obtain a fifth point cloud; obtaining the predicted category of the point cloud by predicting and classifying the fifth point cloud after being copied N times as point cloud global features and point cloud local features, and screening out drilling rig vehicle body point clouds for elimination. The application realizes all-around coverage of point cloud multi-scale features by hierarchical down-sampling different sizes, and realizes accurate segmentation of drilling rig vehicle body point clouds.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of point cloud segmentation, and particularly relates to a drilling rig vehicle body point cloud elimination method and system based on a point cloud segmentation network model. BACKGROUND

[0002] The statements in this section merely provide background information related to the application and do not necessarily constitute prior art.

[0003] With the promotion of the smart mine strategy, underground unmanned driving has become the core direction of mine intelligence, and an accurate mine roadway environment map is the basis for unmanned driving. At present, the existing underground environment mapping mainly relies on the laser radar scanning imaging carried by the drilling rig, and the three-dimensional environment is reconstructed through point cloud data, but the influence of the drilling rig vehicle body point cloud on the establishment of the environment map is not considered. The drilling rig vehicle body point cloud is mixed with the environment point cloud, and the traditional manual elimination method is low in efficiency and large in error. At the same time, deep learning promotes the rapid development of point cloud segmentation technology, and network such as PointNet, PointNet++, and RandLA-Net has been applied to various scenes, but there are few special segmentation schemes for the special scene of the mine drilling rig. On the one hand, the drilling rig vehicle body point cloud data will be mixed with the environment point cloud, resulting in that the final environment map does not match the actual scene. On the other hand, the existing point cloud segmentation network has problems such as insufficient multi-scale feature extraction, insufficient use of context information, and low segmentation precision of small targets (such as local components of the drilling rig) when dealing with complex underground scenes. The root cause of these problems is that the traditional mapping process does not specifically eliminate the drilling rig vehicle body point cloud, and lacks a special segmentation scheme. The downsampling strategy (such as the farthest point sampling) of the existing network is not adapted to the sparse and uneven distribution characteristics of the underground scene point cloud, resulting in the loss of features of large components and small components of the drilling rig. And in the feature extraction process, since the point cloud of small components such as drilling rig bolts and pipelines accounts for a low proportion, the existing network model does not effectively focus on the key area, making it difficult to accurately identify small components. The spatial relationship between the components of the drilling rig and the environment is not enough, which makes it difficult to use context information and distinguish the point cloud of the drilling rig and the environment with similar geometric features, and it is easy to misjudge similar structures in the environment as the drilling rig.

[0004] In summary, the technologies related to mine point cloud segmentation mainly focus on the differentiation between ore body and surrounding rock, and do not solve the problem of drilling rig vehicle body point cloud elimination. These schemes all face the same technical bottleneck: the adaptability of the general network architecture to the underground drilling scene is poor, the multi-scale feature extraction, context modeling and small target segmentation capability are insufficient, resulting in large environment mapping errors. SUMMARY

[0005] In order to solve the above problems, the present application provides a drilling rig vehicle body point cloud elimination method and system based on a point cloud segmentation network model, which identifies the drilling rig vehicle body point cloud by using a point cloud segmentation network first, and then eliminates the body point cloud based on the vehicle body point cloud features to realize the identification and elimination of the drilling rig vehicle body point cloud. The newly proposed SA-Pointnet point cloud segmentation network is optimized for the point cloud characteristics of the mine drilling rig, improves the utilization efficiency of contextual information, significantly improves the small target segmentation accuracy, and realizes the accurate identification of the drilling rig vehicle body point cloud; the optimized environment map has higher consistency with the actual scene, provides a reliable data basis for the mapping of unmanned driving in mines, and promotes the landing efficiency of mine intelligence.

[0006] According to some embodiments, the first aspect of the present application provides a drilling rig vehicle body point cloud elimination method based on a point cloud segmentation network model, which adopts the following technical scheme:

[0007] A drilling rig vehicle body point cloud elimination method based on a point cloud segmentation network model, comprising:

[0008] segmenting the drilling rig vehicle body point cloud from the mine point cloud by using a pre-trained point cloud segmentation network model;

[0009] eliminating the corresponding drilling rig vehicle body point cloud in the mine point cloud based on the coordinates of the drilling rig vehicle body point cloud;

[0010] wherein the drilling rig vehicle body point cloud is segmented from the mine point cloud by using a pre-trained point cloud segmentation network model, specifically:

[0011] standardizing and dimensioning the mine point cloud to obtain a first point cloud;

[0012] dimensioning the first point cloud after feature transformation to obtain a second point cloud, and dimensioning the second point cloud to obtain a third point cloud;

[0013] grading and downsampling the first point cloud into three sizes, and splicing the three size-processed point clouds into a graded downsampled point cloud, and combining the graded downsampled point cloud and the second point cloud to form a point cloud local feature;

[0014] processing the third point cloud and the graded downsampled point cloud based on a self-attention mechanism to obtain a fourth point cloud, and splicing the fourth point cloud and the third point cloud and then performing maximum pooling to obtain a fifth point cloud;

[0015] copying the fifth point cloud N times to serve as a point cloud global feature, and performing full connection dimension reduction on the point cloud global feature and the point cloud local feature to obtain the predicted categories of N point clouds and filter out the drilling rig vehicle body point cloud.

[0016] As a further technical limitation, the mine point cloud is standardized and dimensioned to obtain a first point cloud, specifically:

[0017] The 3D input transformation network is used for normalizing the mine point cloud;

[0018] The first point cloud is obtained by dimensionality increasing the normalized mine point cloud based on the first shared multi-layer perception machine.

[0019] As a further technical limitation, the first point cloud is subjected to feature transformation and then dimensionality increasing to obtain a second point cloud, and the second point cloud is subjected to dimensionality increasing to obtain a third point cloud, specifically:

[0020] The first point cloud is subjected to feature transformation by using the 64-dimensional second spatial transformation network;

[0021] The first point cloud subjected to feature transformation is subjected to dimensionality increasing to obtain the second point cloud based on the second shared multi-layer perception machine;

[0022] The third point cloud is obtained by dimensionality increasing the second point cloud according to the third shared multi-layer perception machine.

[0023] As a further technical limitation, the first point cloud is subjected to hierarchical downsampling into three sizes, and the point clouds subjected to processing of the three sizes are spliced into a hierarchical downsampling point cloud, specifically:

[0024] The first point cloud is subjected to downsampling to obtain a first-size point cloud, the first-size point cloud is subjected to neighborhood searching and dimensionality increasing to obtain a first downsampling point cloud;

[0025] The first point cloud is spliced with N replicated first downsampling point clouds, and then subjected to downsampling to obtain a second-size point cloud, the second-size point cloud is subjected to neighborhood searching and dimensionality increasing to obtain a second downsampling point cloud;

[0026] The first point cloud is spliced with N replicated second downsampling point clouds to obtain a third-size point cloud, the third-size point cloud is subjected to neighborhood searching and dimensionality increasing to obtain a third downsampling point cloud;

[0027] The first downsampling point cloud, the second downsampling point cloud and the third downsampling point cloud are respectively replicated N times and spliced to obtain a hierarchical downsampling point cloud.

[0028] As a further technical limitation, the first-size point cloud is subjected to neighborhood searching and dimensionality increasing to obtain the first downsampling point cloud, specifically:

[0029] K points around each point in the first-size point cloud are searched with the point as the origin;

[0030] The first-size point cloud and the K points searched for each point are subjected to dimensionality increasing by using the sixth shared multi-layer perception machine to obtain a first-size dimensionality-increased point cloud;

[0031] The first-size dimensionality-increased point cloud is subjected to twice maximum pooling operation to obtain the first downsampling point cloud.

[0032] As a further technical limitation, the neighborhood search and dimensionality increase of the second size point cloud are performed to obtain a second down-sampling point cloud, specifically:

[0033] Around each point in the second size point cloud, 2K points are searched as the origin;

[0034] The seventh shared multi-layer perception machine is used to increase the dimensionality of the second size point cloud and the 2K points searched for each point to obtain a second size dimensionality increased point cloud;

[0035] The second size dimensionality increased point cloud is subjected to two maximum pooling operations to obtain a second down-sampling point cloud.

[0036] As a further technical limitation, the neighborhood search and dimensionality increase of the third size point cloud are performed to obtain a third down-sampling point cloud, specifically:

[0037] Around each point in the third size point cloud, 4K points are searched as the origin;

[0038] The eighth shared multi-layer perception machine is used to increase the dimensionality of the third size point cloud and the 4K points searched for each point to obtain a third size dimensionality increased point cloud;

[0039] The third size dimensionality increased point cloud is subjected to two maximum pooling operations to obtain a third down-sampling point cloud.

[0040] As a further technical limitation, the fifth point cloud is replicated N times to serve as point cloud global features and point cloud local features for fully connected dimensionality reduction to obtain the predicted categories of N point clouds and screen out the drilling rig body point cloud, specifically:

[0041] The fifth point cloud is replicated N times to serve as point cloud global features and point cloud local features as input, and the fourth shared multi-layer perception machine and the fifth shared multi-layer perception machine are connected to perform fully connected dimensionality reduction to obtain the predicted categories of N point clouds;

[0042] The predicted category is screened out as the drilling rig body point cloud to complete the drilling rig body point cloud segmentation.

[0043] As a further technical limitation, based on the coordinates of the drilling rig body point cloud, the corresponding drilling rig body point cloud is removed from the mine point cloud, specifically:

[0044] The coordinates of the drilling rig body point cloud are calculated based on the classification label in the drilling rig body point cloud;

[0045] The corresponding drilling rig body point cloud is determined in the mine point cloud according to the coordinates of the drilling rig body point cloud;

[0046] The corresponding drilling rig body point cloud is removed to obtain the mine roadway environment point cloud.

[0047] According to some embodiments, a second aspect of the present application provides a drilling rig vehicle body point cloud culling system based on a point cloud segmentation network model, which adopts the following technical solutions:

[0048] A drilling rig vehicle body point cloud culling system based on a point cloud segmentation network model, comprising:

[0049] A point cloud segmentation module configured to segment drilling rig vehicle body point cloud from mine point cloud by using a pre-trained point cloud segmentation network model;

[0050] A point cloud culling module configured to cull the corresponding drilling rig vehicle body point cloud from the mine point cloud based on the coordinates of the drilling rig vehicle body point cloud;

[0051] Wherein, the drilling rig vehicle body point cloud is segmented from the mine point cloud by using the pre-trained point cloud segmentation network model, specifically:

[0052] The mine point cloud is standardized and dimensioned to obtain a first point cloud;

[0053] The first point cloud is dimensioned after feature transformation to obtain a second point cloud, and the second point cloud is dimensioned to obtain a third point cloud;

[0054] The first point cloud is down-sampled into three sizes, and the point clouds processed in the three sizes are spliced into a hierarchical down-sampled point cloud, and the hierarchical down-sampled point cloud and the second point cloud are combined to form point cloud local features;

[0055] The third point cloud and the hierarchical down-sampled point cloud are processed based on a self-attention mechanism to obtain a fourth point cloud, and the fourth point cloud and the third point cloud are spliced and then subjected to maximum pooling to obtain a fifth point cloud;

[0056] Based on the fifth point cloud, N copies are made to serve as point cloud global features and point cloud local features for full connection dimension reduction, to obtain the predicted categories of N point clouds and filter out the drilling rig vehicle body point cloud.

[0057] Compared with the prior art, the present application has the following advantages:

[0058] The present application is down-sampled by stages, so that the point cloud segmentation network model can obtain rich and accurate features covering multiple scales and dimensions (128 dimensions, 384 dimensions, 896 dimensions); small-scale 128-dimensional features grasp the global, medium-scale 384-dimensional features connect the local and part of the global, and original-scale 896-dimensional features extract fine local parts; after splicing, the features are comprehensive, providing sufficient "materials" for segmentation, enabling the point cloud segmentation network model to accurately identify the attributes of different regions of the point cloud, such as distinguishing roads, buildings, and vegetation, and improving the accuracy of the segmentation categories.

[0059] The application makes the point cloud segmentation network model more accurate in segmenting different scale targets by hierarchical downsampling and multi-scale feature fusion, for example, in segmenting complex scenes, small targets (such as traffic signs) are located by small-scale global features, and large targets (such as buildings) are outlined and structured by large-scale features, multi-scale cooperation makes the boundaries of various targets clearer and the categories more accurate, improves the overall segmentation accuracy, and in experiments, the mIoU (mean intersection over union) of the single scale model segmentation is improved by several percentage points, greatly improving the segmentation accuracy of the point cloud segmentation network model.

[0060] The application avoids high computation of directly processing full-quantity point clouds by hierarchical downsampling, quickly processes small scales first, and then gradually deepens, so that the overall computational complexity is controllable, while ensuring the improvement of segmentation accuracy, the inference speed of the point cloud segmentation network model is guaranteed, and the point cloud segmentation scene (such as real-time point cloud segmentation in autonomous driving) with real-time demand can be adapted, the balance between efficiency and performance is realized, the point cloud segmentation network model is both easy to use and practical, and thus the balance between efficiency and performance is realized. BRIEF DESCRIPTION OF DRAWINGS

[0061] The drawings accompanying the specification of this application form a part thereof, serve to provide further understanding of the application, and together with the description of the exemplary embodiments of the application and the explanation thereof serve to explain the application, and do not constitute an improper limitation of the application.

[0062] Figure 1 is a flow chart of a drilling rig vehicle body point cloud elimination method based on a point cloud segmentation network model in embodiment one of the application;

[0063] Figure 2 is an architecture diagram of a point cloud segmentation network model (SA-Pointnet) in embodiment one of the application;

[0064] Figure 3 is a hierarchical downsampling feature extraction flow chart in embodiment one of the application;

[0065] Figure 4 is a self-attention mechanism flow chart in embodiment one of the application;

[0066] Figure 5 is a system architecture diagram of a drilling rig vehicle body point cloud elimination system based on a point cloud segmentation network model in embodiment two of the application; DETAILED DESCRIPTION

[0067] The application will be further described below in conjunction with the drawings and embodiments.

[0068] It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the application belongs.

[0069] It is to be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, devices, components and / or combinations thereof, but do not preclude the presence or addition of one or more other features, steps, operations, devices, components and / or combinations thereof.

[0070] The embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0071] Embodiment One

[0072] As shown in the figure, the present embodiment provides a drilling rig vehicle body point cloud culling method based on a point cloud segmentation network model. The method in the present embodiment comprises the following steps: Figure 1 Step S1: segmenting the drilling rig vehicle body point cloud from the mine point cloud by using a pre-trained point cloud segmentation network model, specifically:

[0073] Step S1.1: normalizing and dimensioning the mine point cloud to obtain a first point cloud;

[0074] Step S1.2: performing feature transformation on the first point cloud and then dimensioning to obtain a second point cloud, and dimensioning the second point cloud to obtain a third point cloud;

[0075] Step S1.3: hierarchically downsampling the first point cloud into three sizes, and splicing the point clouds processed in the three sizes into a hierarchical downsampling point cloud, and composing point cloud local features from the hierarchical downsampling point cloud and the second point cloud;

[0076] Step S1.4: processing the third point cloud and the hierarchical downsampling point cloud based on a self-attention mechanism to obtain a fourth point cloud, and performing maximum pooling on the fourth point cloud and the third point cloud after splicing to obtain a fifth point cloud;

[0077] Step S1.5: based on the fifth point cloud, copying N copies as point cloud global features and point cloud local features for fully connected dimension reduction to obtain the predicted categories of N point clouds and filter out the drilling rig vehicle body point cloud.

[0078] Step S2: based on the coordinates of the drilling rig vehicle body point cloud, culling the corresponding drilling rig vehicle body point cloud in the mine point cloud.

[0079] In step S1, as shown in the figure,

[0080] Figure 2 ​As shown, this embodiment proposes a point cloud segmentation network model, which improves the Pointnet network by integrating hierarchical downsampling and self-attention mechanisms to obtain the final point cloud segmentation network model, namely SA-Pointnet. The SA-Pointnet network model proposed in this embodiment aims to solve the point cloud segmentation problem of mining drilling rigs. By integrating hierarchical downsampling and self-attention mechanisms for feature extraction, it achieves accurate identification and removal of the drilling rig's point cloud.

[0081] The drilling rig body point cloud is segmented from the mine point cloud using a pre-trained point cloud segmentation network model. The specific process is as follows:

[0082] When the pre-trained SA-Pointnet network model (point cloud segmentation network model) receives a mine point cloud with three-dimensional coordinate information;

[0083] First, a 3D input transform network is used to normalize the point cloud of the mine. Second, a first shared multilayer perceptron is used to upscale the features of the normalized point cloud to 64 dimensions to obtain the first point cloud. It can be understood that the 3D input transform network here is a 3×3 input transform model, with matrix multiplication and T-Net as the main architecture. The specific structure is based on the general model structure in the existing technology and will not be elaborated further.

[0084] The first point cloud of 64 dimensions has two transmission directions:

[0085] One is that the points are passed to the hierarchical downsampling feature extraction module for hierarchical downsampling to obtain hierarchical downsampling point clouds, thereby improving the accuracy of the model in extracting local features from the point clouds;

[0086] Secondly, the feature transformation process is performed by a 64-dimensional second spatial transformation network and then fed into a second shared multilayer perceptron for upsizing to 256 dimensions, resulting in a second point cloud. This second point cloud is further fed into a third shared multilayer perceptron for upsizing to 1024 dimensions, resulting in a third point cloud. It can be understood that the 64-dimensional second spatial transformation network here is a 64×64 feature transform model, which together with the input transform model forms a joint alignment network to ensure that the point cloud achieves spatial consistency at both the global and local levels. The 64×64 feature transform model uses matrix multiplication and T-Net as its main architecture. The specific structure follows the general model structure in existing technologies and will not be elaborated further.

[0087] Then, the 1024-dimensional third point cloud is input together with the hierarchical downsampled point cloud into the self-attention mechanism module to obtain the fourth point cloud;

[0088] Then, the fourth point cloud obtained after processing by the self-attention module is concatenated with the third point cloud of 1024 dimensions and then max pooling is performed to obtain the fifth point cloud. This completes the symmetric operation of the point cloud to achieve feature dimensionality reduction. The feature dimension of the fifth point cloud is 3456.

[0089] Finally, the 3456-dimensional fifth point cloud is copied N times and used as the global feature of the point cloud. Together with the local feature of the point cloud composed of the hierarchical downsampled point cloud and the 256-dimensional second point cloud, it is input into the fourth shared multilayer perceptron and the fifth shared multilayer perceptron for fully connected dimensionality reduction operation. The predicted categories of N point clouds are obtained and the segmentation result of the drilling rig vehicle body point cloud is output.

[0090] like Figure 3 As shown, in step S1.3, the first point cloud is downsampled into three sizes, and the point clouds processed by the three sizes are stitched together to form a downsampled point cloud. Specifically:

[0091] The first point cloud is downsampled to obtain a first-size point cloud. The first-size point cloud is then subjected to a neighborhood search and dimensionality upscaling to obtain the first downsampled point cloud.

[0092] The first point cloud is concatenated with N copies of the first downsampled point cloud and then downsampled to obtain the second-size point cloud. The second-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the second downsampled point cloud.

[0093] The first point cloud is concatenated with the second downsampled point cloud (copied N times) to obtain the third-size point cloud. The third-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the third downsampled point cloud.

[0094] The first downsampled point cloud, the second downsampled point cloud, and the third downsampled point cloud are copied N times and then spliced ​​together to obtain a hierarchical downsampled point cloud.

[0095] The main idea of ​​hierarchical downsampling in this embodiment is to transform each point in the target point cloud from low dimension to high dimension by sharing a multilayer perceptron, thereby obtaining richer local features of the point cloud; the 64-dimensional first point cloud is divided into three sizes, namely the first size -N / 512, the second size N / 64 and the third size -N, and the point clouds processed by the three sizes are stitched together as the final feature extraction result, that is, the hierarchical downsampled point cloud.

[0096] Since point cloud data has rich local and global features, features at different scales are crucial for segmentation tasks. This embodiment classifies point clouds into N / 512, N / 64, and N sizes, which can obtain global overview features at a small scale (such as N / 512), take into account both local and partial global features at a medium scale (N / 64), and focus on fine local details at the original scale (N), thus comprehensively covering the multi-scale features of point clouds and helping to achieve accurate segmentation.

[0097] Specifically, the first point cloud is down-sampled to obtain a first size point cloud, neighborhood search and dimensionality increase are performed on the first size point cloud to obtain a first down-sampled point cloud, specifically as follows:

[0098] The number N of the first point cloud is down-sampled to N / 512 as the first size point cloud;

[0099] Each point in the first size point cloud is taken as the origin, and K points around it are searched;

[0100] The first size point cloud and the K points searched for each point are subjected to feature dimensionality increase by the sixth shared multi-layer perception machine from 64 dimensions to 128 dimensions to obtain a first size dimensionality increased point cloud;

[0101] The N / 512 x (K+1) 128-dimensional first size dimensionality increased point cloud is subjected to twice maximum pooling operation to obtain a 128-dimensional first down-sampled point cloud.

[0102] Specifically, the first point cloud is concatenated with N replicated first down-sampled point clouds, and then down-sampled to obtain a second size point cloud, neighborhood search and dimensionality increase are performed on the second size point cloud to obtain a second down-sampled point cloud, specifically as follows:

[0103] The first point cloud is concatenated with N replicated first down-sampled point clouds, and then down-sampled so that the number N of the concatenated point cloud is down-sampled to N / 64 as the second size point cloud;

[0104] Each point in the second size point cloud is taken as the origin, and 2K points around it are searched;

[0105] The second size point cloud and the 2K points searched for each point are subjected to feature dimensionality increase by the seventh shared multi-layer perception machine from 64 dimensions to 384 dimensions to obtain a second size dimensionality increased point cloud;

[0106] The N / 64 x (2K+1) 384-dimensional second size dimensionality increased point cloud is subjected to twice maximum pooling operation to obtain a 384-dimensional second down-sampled point cloud.

[0107] Specifically, the first point cloud is concatenated with N replicated second down-sampled point clouds to obtain a third size point cloud, neighborhood search and dimensionality increase are performed on the third size point cloud to obtain a third down-sampled point cloud, specifically as follows:

[0108] The first point cloud is concatenated with N replicated second down-sampled point clouds to obtain a third size point cloud, and the size of the third size point cloud is N;

[0109] Each point in the third size point cloud is taken as the origin, and 4K points around it are searched;

[0110] The eighth shared multi-layer perception machine is used to find features of the third size point cloud and 4K points found for each point, and the dimension of the features is increased from 64 to 896, so as to obtain a third size increased dimension point cloud with 896 dimensions;

[0111] The N*(4K+1) third size increased dimension point clouds with 896 dimensions are subjected to twice maximum pooling operations, so as to obtain a third size reduced sampling point cloud with 896 dimensions.

[0112] The output of the hierarchical reduced sampling feature extraction module is the result of copying and splicing the above-mentioned maximum pooled 128-dimensional, 384-dimensional and 896-dimensional feature point clouds N times. The final result can be used as part of the local feature of the point cloud for subsequent operations. The reduced sampling, K value setting and pooling operation in the structure are described in detail below.

[0113] Reduced sampling refers to selecting representative points as center points from the input point cloud data. In this embodiment, FPS (reduced sampling) is used to select N / 512, N / 64 and N points from N points. The basic idea of FPS is to iteratively select the point farthest from the selected points until the required number K, 2K and 4K is reached. This can ensure that the selected points are uniformly distributed in the point cloud, so as to better represent the features of the entire point cloud.

[0114] The setting of the K value is a key link of hierarchical reduced sampling. Its role is to select the neighborhood points of each target point cloud after reduced sampling and reset the neighborhood point cloud coordinates with each target point as the center point, so as to obtain high-dimensional point cloud features after subsequent multi-layer perception machine processing. In view of the problem that if the K value is set to a small value when the number of target sampling points decreases, the spatial point cloud will be seriously missing, resulting in low feature extraction accuracy. In this embodiment, the number of field points of N / 512, N / 64 and N sampling points is set to K, 2K and 4K respectively.

[0115] The pooling operation of each size is divided into two types, namely neighborhood-based pooling and global-based pooling. The neighborhood-based pooling respectively realizes maximum pooling on the points in each neighborhood, and the above-mentioned three sizes generate N / 512, N / 64 and N feature vectors after pooling respectively. The global-based pooling aggregates the neighborhood pooling results of each size, and three sizes generate a high-dimensional feature vector respectively.

[0116] If the large-scale original point cloud (size N) is directly processed, the calculation complexity is high and the memory occupation is large. The hierarchical reduced sampling adopts the "simple first and complex later" strategy. The small size point cloud has small data amount, and the basic features are extracted with low calculation cost first. The medium size expands on the basis of the small size features, and the calculation amount is moderately increased. The original size is finally processed, and the previous features assist at this time. The most detailed features can be mined without excessive consumption of resources, so that the calculation resources are reasonably allocated.

[0117] This embodiment constructs a feature hierarchy of "low-dimensional to high-dimensional, global to local". The small-sized feature is upgraded from 64 dimensions to 128 dimensions through a shared multilayer perceptron, laying the foundation for subsequent higher-scale feature processing; the medium-sized and original-sized features are upgraded sequentially (384 dimensions and 896 dimensions), enabling features to gradually move from a preliminary global overview to a detailed local description, improving the ability to understand complex scenes during segmentation, and realizing hierarchical feature construction and transfer.

[0118] like Figure 4 As shown, in step S1.4, the third point cloud and the hierarchical downsampled point cloud are processed based on the self-attention mechanism to obtain the fourth point cloud. The fourth point cloud is then concatenated with the third point cloud and max pooled to obtain the fifth point cloud. Specifically:

[0119] The third point cloud and the hierarchical downsampled point cloud are jointly input into the self-attention mechanism, and more refined global and local features are obtained by configuring the point cloud weights;

[0120] Query Matrix Key vector matrix and value vector matrix As shown in formula (1), specifically:

[0121] (1);

[0122] in, , and These are matrix transformations trained on a multilayer perceptron, used to compute all query vector matrices, all key vector matrices, and all value vector matrices. The input to the self-attention mechanism after processing by the ninth shared multilayer perceptron is obtained by stitching together the third point cloud and the hierarchical downsampled point cloud as the input to the self-attention mechanism, and then processing it by the ninth shared multilayer perceptron. .

[0123] The attention weight formula is shown in formula (2), specifically:

[0124] (2);

[0125] in, This represents the attention weight value. This is the transpose of the key vector matrix.

[0126] The weight information normalization operation is shown in formula (3). Compared with directly calculating the ratio value, it utilizes... The function implements the normalization operation, which can highlight the large proportion of attention scores.

[0127] (3);

[0128] wherein, is the weight information normalization result; is the key vector dimension, which is 2432.

[0129] The above calculated is multiplied by the value vector matrix to output the final attention feature value, and after the tenth shared multi-layer perception processing, the final point cloud feature result, i.e., the fourth point cloud, is output. The feature result of the attention module can be used as part of the global feature of the point cloud, for subsequent global and local point cloud splicing and point cloud category prediction after full connection.

[0130] The fourth point cloud is spliced with the third point cloud to obtain the fifth point cloud through maximum pooling.

[0131] In a specific embodiment, in step S1.5, the fifth point cloud is copied N times as the global feature of the point cloud and the local feature of the point cloud for full connection dimension reduction, to obtain the predicted category of N point clouds and screen out the rig body point cloud, specifically:

[0132] The fifth point cloud is copied N times as the global feature of the point cloud and the local feature of the point cloud as input, and the fourth shared multi-layer perception and the fifth shared multi-layer perception are connected for full connection dimension reduction to obtain the predicted category of N point clouds;

[0133] The predicted category of the rig body point cloud is screened out to complete the rig body point cloud segmentation.

[0134] As shown in Figure 2 , after the full connection dimension reduction of the point cloud segmentation network model in step S1.5, an m-dimensional vector is predicted for each point in the N point clouds, and each element in the vector corresponds to the probability of the point belonging to a certain category (after softmax activation). Finally, the predicted category of the point is obtained by taking the index of the maximum value of the corresponding probability of each point; the predicted category includes the rig body point cloud and other mine tunnel environment point clouds of various categories, and the categories of the mine tunnel environment point clouds are not limited here, and the number of corresponding categories can be determined according to the actual scene.

[0135] Therefore, m in Figure 2 here represents the number of categories of the segmentation task (i.e., the total number of target categories to be distinguished), and it should be noted that in this embodiment, the rig body point cloud predicted as this category is mainly segmented out; and after step S1 processing, the segmented rig body point cloud is assigned a pre-set label value, i.e., the "rig body point cloud" classification label.

[0136] In a specific embodiment, in step S2, based on the coordinates of the rig body point cloud, the corresponding rig body point cloud in the mine point cloud is removed, specifically:

[0137] Calculate the coordinates of the drilling rig vehicle point cloud based on the classification label in the drilling rig vehicle point cloud;

[0138] Determine the corresponding drilling rig vehicle point cloud in the mine point cloud according to the coordinates of the drilling rig vehicle point cloud;

[0139] The corresponding drilling rig vehicle point cloud is removed to obtain the mine tunnel environment point cloud.

[0140] In a specific embodiment, the drilling rig vehicle point cloud segmented by the point cloud segmentation network model (SA-Pointnet) in step S1 is assigned a pre-set label value; this step identifies the corresponding classification label value in the drilling rig vehicle point cloud segmented in step S1, and obtains the three-dimensional coordinates of the drilling rig vehicle point cloud according to the corresponding classification label value features in the drilling rig vehicle point cloud. Then, the mine point cloud is filtered according to the three-dimensional coordinate values of the drilling rig vehicle point cloud, so as to effectively remove the drilling rig vehicle point cloud in the mine point cloud and generate a more pure mine tunnel environment point cloud. This processing procedure significantly improves the accuracy and reliability of subsequent point cloud processing tasks (such as obstacle detection, ground segmentation, high-precision map construction, etc.), and is suitable for various scenarios such as autonomous driving, high-precision map updating, and intelligent transportation.

[0141] A calculation method for obtaining the three-dimensional coordinates of the drilling rig vehicle point cloud based on the corresponding classification label value features in the drilling rig vehicle point cloud is given below, which is as follows:

[0142] Based on the corresponding classification label value features in the drilling rig vehicle point cloud, the clustering algorithm is used for separation, and finally the coordinates of each independent vehicle point cloud cluster are calculated.

[0143] It can be understood that, for the drilling rig vehicle point cloud, not a point is calculated, but the representative coordinates (such as the center point) of each independent drilling rig vehicle in the scene and its spatial range (such as the bounding box) are calculated. This is usually divided into two steps:

[0144] Instance segmentation: the point cloud belonging to the same drilling rig vehicle classification label is further separated into individual independent vehicles.

[0145] Coordinate calculation: the representative coordinates of each independent vehicle instance point cloud cluster are calculated.

[0146] Specifically, it is as follows:

[0147] Step 1: instance segmentation based on the segmented drilling rig vehicle point cloud, including:

[0148] The point cloud segmentation network model usually performs semantic segmentation first, that is, each point is labeled with a classification label such as "ground", "drilling rig vehicle body", "pedestrian", etc. In this embodiment, the classification label of "drilling rig vehicle body" is mainly selected, but it will not be distinguished in step S1 which "drilling rig vehicle body". All points labeled as "drilling rig vehicle body" are still mixed together. Therefore, a clustering algorithm is used to separate these points into independent clusters, and each cluster represents an independent vehicle.

[0149] The following commonly used clustering algorithms can be used, but are not limited to, Euclidean clustering, DBSCAN algorithm, etc. After this step, a list clusters = [cluster1, cluster2,..., clustern] is obtained,

[0150] Each drilling rig vehicle body point cloud cluster cluster_i (i∈1, 2, 3,..., n) contains the three-dimensional coordinates of all points belonging to the ith drilling rig vehicle body.

[0151] Step 2: Calculate the representative coordinates for each drilling rig vehicle body point cloud cluster, including:

[0152] For each independent drilling rig vehicle body point cloud cluster cluster_i obtained in the previous step, its "coordinates" can be defined in multiple ways. For example, the geometric center of the drilling rig vehicle body point cloud cluster can be calculated as the three-dimensional coordinates of the drilling rig vehicle body point cloud. This method is very simple and fast, but it is easily affected by uneven distribution of point clouds. For example, if only half of the vehicle is scanned, the center point will deviate seriously from the geometric center of the real object;

[0153] Alternatively, the center of the bounding box of the drilling rig vehicle body point cloud cluster can be calculated as the three-dimensional coordinates of the drilling rig vehicle body point cloud. This is a more commonly used and more reliable method, which can better represent the overall spatial position and size of the object and is relatively insensitive to the completeness of the drilling rig vehicle body point cloud.

[0154] For vehicles with inconsistent directions (such as vehicles parked diagonally on the roadside), a directional bounding box can be used. This is usually done by principal component analysis (PCA) to find the main direction of the point cloud, and then a bounding box is established along the main direction. The calculated center point and direction are more accurate.

[0155] It can be understood that the above method can be selected according to actual needs, or other disclosed methods in the prior art can be selected. Here, it is not limited, but only to explain the calculation of the three-dimensional coordinates of the drilling rig vehicle body point cloud.

[0156] Embodiment Two

[0157] As shown in Figure 5 , the embodiment provides a drilling rig vehicle body point cloud culling system based on a point cloud segmentation network model, comprising:

[0158] The point cloud segmentation module is configured to segment the drilling rig vehicle body point cloud from the mine point cloud by using a pre-trained point cloud segmentation network model.

[0159] The point cloud elimination module is configured to eliminate the corresponding drilling rig vehicle body point cloud in the mine point cloud based on the coordinates of the drilling rig vehicle body point cloud.

[0160] The point cloud segmentation module is configured to segment the drilling rig vehicle body point cloud from the mine point cloud by using a pre-trained point cloud segmentation network model.

[0161] The mine point cloud is standardized and dimensioned to obtain a first point cloud.

[0162] The first point cloud is dimensioned after feature transformation to obtain a second point cloud, and the second point cloud is dimensioned to obtain a third point cloud.

[0163] The first point cloud is down-sampled into three sizes, and the point clouds processed in the three sizes are spliced into a down-sampled point cloud. The down-sampled point cloud and the second point cloud form a point cloud local feature.

[0164] The third point cloud and the down-sampled point cloud are processed based on a self-attention mechanism to obtain a fourth point cloud. The fourth point cloud and the third point cloud are spliced and then subjected to maximum pooling to obtain a fifth point cloud.

[0165] Based on the fifth point cloud, N copies are made to serve as a point cloud global feature and a point cloud local feature for full connection dimension reduction. N point cloud prediction categories are obtained and a drilling rig vehicle body point cloud is selected.

[0166] It should be noted that the point cloud segmentation module and the point cloud elimination module correspond to steps S1 to S2 in Embodiment One. The above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment One. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0167] The descriptions of the various embodiments in the above embodiments have different focuses. The parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0168] The proposed system can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of the above modules is only a logical functional division. In actual implementation, another division method can be used, for example, multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0169] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like.

[0170] Although the specific embodiments of the present application are described above with reference to the drawings, the description is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A method for drilling rig vehicle body point cloud culling based on a point cloud segmentation network model, characterized in that, include: The drilling rig body point cloud is segmented from the mine point cloud using a pre-trained point cloud segmentation network model. Based on the coordinates of the drilling rig body point cloud, the corresponding drilling rig body point cloud is removed from the mine point cloud; Specifically, the step of segmenting the drilling rig vehicle point cloud from the mine point cloud using a pre-trained point cloud segmentation network model involves: The point cloud of the mine is standardized and upgraded to obtain the first point cloud; The first point cloud is transformed by features and then its dimension is increased to obtain the second point cloud. The second point cloud is then increased in dimension to obtain the third point cloud. The first point cloud is downsampled into three sizes, and the point clouds processed from the three sizes are then stitched together to form a hierarchically downsampled point cloud. Specifically: The first point cloud is downsampled to obtain a first-size point cloud. The first-size point cloud is then subjected to a neighborhood search and dimensionality upscaling to obtain the first downsampled point cloud. The first point cloud is concatenated with N copies of the first downsampled point cloud and then downsampled to obtain the second-size point cloud. The second-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the second downsampled point cloud. The first point cloud is concatenated with the second downsampled point cloud (copied N times) to obtain the third-size point cloud. The third-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the third downsampled point cloud. The first downsampled point cloud, the second downsampled point cloud, and the third downsampled point cloud are copied N times and then stitched together to obtain a hierarchical downsampled point cloud. The hierarchical downsampling point cloud and the second point cloud are combined to form the local features of the point cloud; The third point cloud and the hierarchical downsampling point cloud are processed based on the self-attention mechanism to obtain the fourth point cloud. The fourth point cloud is then concatenated with the third point cloud and max pooling is performed to obtain the fifth point cloud. After copying the fifth point cloud N times, the global and local features of the point cloud are used for full-connection dimensionality reduction to obtain N predicted categories of the point cloud and then the drilling rig vehicle body point cloud is selected.

2. The rig vehicle body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The standardization and dimensionality-upgrading of the mine point cloud yields the first point cloud, specifically as follows: A 3D input transformation network is used to standardize the point cloud of a mine. The first point cloud is obtained by upscaling the normalized mine point cloud using the first shared multilayer perceptron.

3. The rig body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The process of performing feature transformation on the first point cloud and then increasing its dimensionality to obtain the second point cloud, and then increasing the dimensionality of the second point cloud to obtain the third point cloud, specifically involves: The first point cloud is transformed using a 64-dimensional second spatial transformation network; The second point cloud is obtained by upsizing the first point cloud after feature transformation based on the second shared multilayer perceptron. The third point cloud is obtained by upscaling the second point cloud using the third shared multilayer perceptron.

4. The rig body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The process of performing a neighborhood search and dimensionality upscaling on the first-size point cloud to obtain the first downsampled point cloud specifically involves: Using each point in the first-size point cloud as the origin, find K points around it; The first-size point cloud and the K points found by each point are upgraded using the sixth shared multilayer perceptron to obtain the first-size upgraded point cloud; The first upsampled point cloud is obtained by performing two max pooling operations on the first-size upsampled point cloud.

5. The rig body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The process of performing a neighborhood search and dimensionality upscaling on the second-sized point cloud to obtain the second downsampled point cloud is as follows: Using each point in the second-size point cloud as the origin, find 2K points around it; The second-size point cloud and the 2K points found by each point are up-dimensional using the seventh shared multilayer perceptron to obtain the second-size up-dimensional point cloud; The second-dimensional upsampled point cloud is obtained by performing two max pooling operations.

6. The rig body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The process of performing a neighborhood search and dimensionality upscaling on the third-size point cloud to obtain the third downsampled point cloud is as follows: Using each point in the third-size point cloud as the origin, find 4K points around it; The third-dimensional point cloud and the 4K points found by each point are up-dimensioned using the eighth shared multilayer perceptron to obtain the third-dimensional up-dimension point cloud; The third-dimensional upsampled point cloud is obtained by performing two max pooling operations.

7. The rig body point cloud culling method based on a point cloud segmentation network model according to claim 1, wherein, The process involves replicating the fifth point cloud N times and using the resulting global and local features for fully connected dimensionality reduction to obtain N predicted point cloud categories. The drilling rig vehicle body point cloud is then selected from these categories. Specifically: The fifth point cloud is copied N times and used as the global and local features of the point cloud as input. The connected fourth and fifth shared multilayer perceptrons are used to perform fully connected dimensionality reduction to obtain the predicted categories of N point clouds. The point cloud of the drilling rig body is selected based on the predicted category, and the point cloud of the drilling rig body is segmented.

8. The drilling rig vehicle body point cloud removal method based on a point cloud segmentation network model as described in claim 1, characterized in that, The process of removing the corresponding drilling rig body point cloud from the mine point cloud based on the coordinates of the drilling rig body point cloud is as follows: The coordinates of the drilling rig body point cloud are calculated based on the classification labels in the drilling rig body point cloud; Determine the corresponding drilling rig body point cloud in the mine point cloud based on the coordinates of the drilling rig body point cloud; The corresponding drilling rig body point cloud is removed to obtain the mine roadway environment point cloud.

9. A drilling rig vehicle body point cloud removal system based on a point cloud segmentation network model, characterized in that, include: The point cloud segmentation module is configured to segment the drilling rig body point cloud from the mine point cloud using a pre-trained point cloud segmentation network model. The point cloud removal module is configured to remove the corresponding drilling rig body point cloud from the mine point cloud based on the coordinates of the drilling rig body point cloud. Specifically, the step of segmenting the drilling rig vehicle point cloud from the mine point cloud using a pre-trained point cloud segmentation network model involves: The point cloud of the mine is standardized and upgraded to obtain the first point cloud; The first point cloud is transformed by features and then its dimension is increased to obtain the second point cloud. The second point cloud is then increased in dimension to obtain the third point cloud. The first point cloud is downsampled into three sizes, and the point clouds processed from the three sizes are then stitched together to form a hierarchically downsampled point cloud. Specifically: The first point cloud is downsampled to obtain a first-size point cloud. The first-size point cloud is then subjected to a neighborhood search and dimensionality upscaling to obtain the first downsampled point cloud. The first point cloud is concatenated with N copies of the first downsampled point cloud and then downsampled to obtain the second-size point cloud. The second-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the second downsampled point cloud. The first point cloud is concatenated with the second downsampled point cloud (copied N times) to obtain the third-size point cloud. The third-size point cloud is then subjected to neighborhood search and dimensionality upscaling to obtain the third downsampled point cloud. The first downsampled point cloud, the second downsampled point cloud, and the third downsampled point cloud are copied N times and then stitched together to obtain a hierarchical downsampled point cloud. The hierarchical downsampling point cloud and the second point cloud are combined to form the local features of the point cloud; The third point cloud and the hierarchical downsampling point cloud are processed based on the self-attention mechanism to obtain the fourth point cloud. The fourth point cloud is then concatenated with the third point cloud and max pooling is performed to obtain the fifth point cloud. After copying the fifth point cloud N times, the global and local features of the point cloud are used for full-connection dimensionality reduction to obtain N predicted categories of the point cloud and then the drilling rig vehicle body point cloud is selected.

Images