A 3D measurement segmentation method based on deep learning

Abstract

This application discloses a three-dimensional measurement and segmentation method based on deep learning, which uses a deep model to automatically segment and identify workpiece point clouds. In the deep learning network model, in order to fuse and learn global features and local features, the deep learning network model is based on the idea of feature fusion, uses a U-Net network structure, and uses difference downsampling based on point cloud similarity in the sampling layer, aiming to solve the problem of selecting the most prominent receptive field of each layer of the encoding layer to learn point cloud features; the deep learning network model extracts and optimizes the local features of the receptive field through the self-attention mechanism, draws on the spatial mixing multi-layer perceptron in the multi-layer perceptron mixer to establish global connections between receptive fields and extract global features of the receptive field, and on this basis, performs feature fusion learning of local features and global features through a convolutional network to improve the accuracy of automatic recognition and segmentation of workpiece point clouds.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional measurement, and more specifically, to a three-dimensional measurement segmentation method based on deep learning. Background Art

[0002] In three-dimensional measurement, the parameters that usually need to be measured include the three-dimensional coordinates, size, shape, curvature, inclination, surface quality, etc. of the object or scene, and these parameters can be obtained through different measurement methods. However, contact mechanical measurement using equipment such as calipers, vernier calipers, and micrometers requires a lot of manpower and time costs; non-contact three-dimensional measurement methods based on structured light can simply, quickly, and measure information such as the three-dimensional coordinates and surface shape of objects in large quantities, making them very suitable for in-situ processing measurements. However, in addition to the workpiece point cloud we need, the point cloud obtained by non-contact three-dimensional measurement also scans the positioning fixture, base platform, etc. Therefore, point cloud segmentation is an important preprocessing process for three-dimensional point cloud measurement. The accuracy of point cloud segmentation affects the accuracy of subsequent splicing, reconstruction, and processing.

[0003] Point cloud segmentation refers to the process of classifying and segmenting a three-dimensional point cloud into multiple subsets according to preset criteria. Currently, common point cloud segmentation methods include traditional methods based on geometric shapes and topological structures and deep learning-based methods. Traditional point cloud segmentation offers good stability, can handle high-density point cloud data, and can achieve high segmentation accuracy for simple scenes. However, it requires numerous parameter adjustments and is sensitive to noise, has difficulty adapting to point clouds of varying shapes, is sensitive to changes in point density, and cannot handle complex relationships between point clouds, resulting in high labor costs. Recent advances in deep learning technology have, to a certain extent, addressed these issues. Deep learning point cloud segmentation involves inputting a point cloud into a deep learning network to predict the classification of each point. It offers high accuracy and fast processing speed, is suitable for complex scenes, can handle large amounts of data, and significantly reduces labor costs. However, most existing network models rely on spatial coordinate relationships to select receptive fields to learn point cloud features. As the dimensionality of the extracted features increases, considering only spatial coordinate relationships fails to select the most prominent receptive fields for learning, and often only emphasizes learning local features within the receptive fields. How to comprehensively learn point cloud features while taking into account both global and local features, the design of network models still needs to be improved. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the above-mentioned existing technologies and propose a three-dimensional measurement segmentation algorithm based on deep learning.

[0005] The deep learning-based three-dimensional measurement and segmentation algorithm proposed in this application uses a deep model to automatically segment and identify workpiece point clouds. In the deep learning network model, in order to fuse and learn global features and local features, the deep learning network model is based on the idea of feature fusion, uses a U-Net network structure, and uses difference downsampling based on point cloud similarity in the sampling layer, aiming to solve the problem of selecting the most prominent receptive field of each layer of the encoding layer to learn point cloud features; the deep learning network model extracts and optimizes the local features of the receptive field through the self-attention mechanism, draws on the spatial mixing multi-layer perceptron in the multi-layer perceptron mixer to establish global connections between receptive fields and extract global features of the receptive field, and on this basis, performs feature fusion learning of local features and global features through the convolutional network to improve the accuracy of automatic recognition and segmentation of workpiece point clouds.

[0006] The technical solution of this application is:

[0007] A three-dimensional measurement segmentation method based on deep learning includes the following steps:

[0008] Step 1: Use a structured light scanner to scan the workpiece scene point cloud, and use the input embedding layer to embed the input to obtain the original point cloud of the model input;

[0009] Step 2: First, use the sampling layer and grouping layer to extract the receptive field of the current input point cloud, then use the neighborhood self-attention layer to extract and optimize the local features of the receptive field, and then use the spatial hybrid multi-layer perceptron to extract the global features of the receptive field. Finally, use the 2D convolution layer to interactively extract the local features and global features of the receptive field to obtain a layer of encoding layer features;

[0010] Step 3: Take the features of the previous coding layer as input and repeat the coding operation of step 2 twice to obtain the coding features of the entire point cloud;

[0011] Step 4: First, the coding features of the entire point cloud are optimized through the neighborhood self-attention layer and spliced onto the coding features of the corresponding layer. Then, the spliced features are decoded through the feature propagation layer to obtain a decoding layer feature.

[0012] Step 5: Take the previous decoding layer features as input and repeat the decoding operation of step 4 twice until the decoding layer features are spliced to the original original point cloud for decoding, and obtain the decoding layer features of each point in the point cloud;

[0013] Step 6: Use the fully connected network to obtain the predicted classification of each point in the point cloud to achieve the segmentation task.

[0014] Further, the step 1 specifically includes:

[0015] Step 1.1, the set of point clouds obtained by scanning is P = (p0, p1, ..., p n-1 )∈R3 , each point contains (x,y,z,n x ,n y ,n z ,R,G,B) three-dimensional coordinates, normal vector information and color information;

[0016] In step 1.2, the input embedding layer is used to embed the scanned point cloud so that the feature dimension of the point cloud is expanded to 32 dimensions. The input embedding layer is a linear layer and a multi-head self-attention layer.

[0017] Further, the step 2 specifically includes:

[0018] Step 2.1: First, use the sampling layer to extract the center point of the receptive field, and then use the grouping layer to extract the neighboring points of the center point of the receptive field to form the receptive field. The sampling layer uses difference downsampling, and the grouping layer uses the K-means clustering algorithm.

[0019] Step 2.2: Use the neighborhood self-attention layer to extract and optimize the local features of the receptive field. The self-attention mechanism is used to establish connections between neighboring points in the receptive field and adjust the feature weights to extract and optimize the local features of the receptive field.

[0020] Step 2.3: Use spatial hybrid multilayer perceptron to extract global features of receptive fields, establish connections between all receptive fields through linear transformation, and extract global features of receptive fields;

[0021] In step 2.4, a 2D convolutional layer is used to interactively extract the local features and global features of the receptive field to obtain a layer of encoding layer features.

[0022] Further, the step 3 specifically includes:

[0023] Step 3.1, define the features embedded by input as the original point cloud features;

[0024] In step 3.2, following the U-Net structure, the encoding layer needs to be performed three times. The number of receptive fields sampled in the three encoding layers is 512, 128, and 1 respectively; the receptive field radius is 0.2, 0.4, and 1 respectively; the number of nearest neighbors is 32, 64, and 128 respectively; the feature dimensions extracted are 128, 256, and 1024 respectively; the receptive fields of the three layers are defined as receptive field 1, receptive field 2, and receptive field 3 respectively. The resulting encoded features are defined as encoded feature 1, encoded feature 2, and encoded feature 3 respectively.

[0025] In step 3.2.1, when searching for neighboring points, if the number of neighboring points in the receptive field is insufficient, the coordinates of the first point are repeated.

[0026] Further, the step 4 specifically includes:

[0027] Step 4.1: Use the K-means clustering algorithm to search for the three nearest receptive domains 3 of receptive domain 2, and optimize the encoding feature 3 of receptive domain 3 through the neighborhood self-attention layer, and then splice it with the encoding feature 2 of receptive domain 2;

[0028] In step 4.2, the feature propagation layer is used to decode the concatenated point cloud features and propagate them to the receptive field 2 to obtain the decoded feature 1.

[0029] Further, the step 5 specifically includes:

[0030] Step 5.1, perform second-layer decoding based on the first-layer decoding;

[0031] Step 5.1.1: Use the K-means clustering algorithm to search for the three nearest receptive domains 2 of receptive domain 1, optimize the decoding feature 1 of receptive domain 2 through the neighborhood self-attention layer, and then concatenate it with the encoding feature 1 of receptive domain 1;

[0032] Step 5.1.2: Use the feature propagation layer to decode the concatenated point cloud features and propagate them to receptive field 1 to obtain decoded features 2.

[0033] Step 5.2, perform third-layer decoding based on the second-layer decoding;

[0034] Step 5.2.1: Use the K-means clustering algorithm to search for the three nearest receptive fields 1 of the input point cloud, optimize the decoded features 2 of receptive field 1 through the neighborhood self-attention layer, and then splice them with the original point cloud features;

[0035] In step 5.2.2, the feature propagation layer is used to decode the spliced point cloud features and propagate them to the original point cloud to obtain decoded features 3.

[0036] Further, the step 6 specifically includes:

[0037] In step 6.1, the fully connected network is finally used to obtain the predicted classification of each point in the point cloud, and the points are segmented according to their categories to achieve the segmentation task.

[0038] A three-dimensional measurement segmentation method based on deep learning includes the following steps:

[0039] Step 1: Use a structured light scanner to scan the workpiece and obtain the original point cloud for model input;

[0040] Step 2: For the input point cloud data, a multi-head self-attention mechanism is used to embed the input and expand the input feature dimension to obtain 32-dimensional original input features;

[0041] Step 3: Based on the input point cloud data, use the sampling module to select the center point of the receptive field:

[0042] 1) Sampling layer: Use the difference downsampling method to downsample the original point cloud to obtain the center point of the receptive field. The difference downsampling steps are as follows:

[0043] 1. Set a sampling point set S to store the sampling points;

[0044] 2. Calculate the cosine similarity between each point in the point cloud and take the point with the largest average difference from other points as the initial sampling point;

[0045] Among them, cosine similarity and difference are calculated using the following formula:

[0046]

[0047] Difference=θ(1-Cosine(x,y))

[0048] x i ,y i Represents the coordinates of any i-th sampling point;

[0049] Where θ is the difference factor, which defaults to 0.5.

[0050] 3. Select the point with the largest average difference from the sampled sub-point cloud in the remaining point cloud;

[0051] 4. Add the point with the largest difference to the sampling point set S;

[0052] 5. Repeat steps 3 and 4 until the number of members in the sampling point set S meets the number of sampling points;

[0053] 2) Grouping layer: Use the K-means clustering algorithm to search for neighboring points within a sphere with a radius of R and the center of the receptive field to construct the receptive field;

[0054] Step 3: Based on the receptive field constructed by the sampling grouping module, the neighborhood self-attention mechanism is used to extract the features of the neighboring points in the receptive field as local features, and the connection between the receptive fields is established through the spatial hybrid multi-layer perceptron to extract the global features. Finally, the local features and global features are fused and extracted by three layers of 2D convolution to obtain the coding layer features.

[0055] Step 4: Following the U-Net structure, the encoding layer needs to be performed three times. The number of receptive fields sampled in the three encoding layers is 512, 128, and 1 respectively; the receptive field radius is 0.2, 0.4, and 1 respectively; the number of nearest neighbors is 32, 64, and 128 respectively; and the feature dimensions extracted are 128, 256, and 1024 respectively. The receptive fields of the three layers are defined as receptive field 1, receptive field 2, and receptive field 3 respectively. The resulting encoded features are defined as encoded feature 1, encoded feature 2, and encoded feature 3 respectively.

[0056] Step 5: Use the K-means clustering algorithm to search for the three nearest receptive domains 3 of receptive domain 2, and optimize the encoded feature 3 of receptive domain 3 through the neighborhood self-attention layer, and then splice it with the encoded feature 2 of receptive domain 2; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 2 to obtain the 256-dimensional decoding feature 1;

[0057] Step 6: Use the K-means clustering algorithm to search for the three nearest receptive domains 2 of receptive domain 1, and optimize the decoding feature 1 of receptive domain 2 through the neighborhood self-attention layer, and then splice it with the encoding feature 1 of receptive domain 1; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 1 to obtain the 128-dimensional decoding feature 2;

[0058] Step 7: Use the K-means clustering algorithm to search for the three nearest receptive fields 1 of the input point cloud, and optimize the decoded features 2 of the receptive field 1 through the neighborhood self-attention layer, and then splice them with the original point cloud features; use the feature propagation layer to decode and propagate the spliced point cloud features to the original point cloud to obtain 128-dimensional decoded features 3;

[0059] Step 8: Finally, use the fully connected network to obtain the predicted classification of each point in the point cloud, and segment it according to the category of each point to achieve the segmentation task.

[0060] The beneficial effects of this application are:

[0061] First, to address the problem of high time cost in manually identifying workpiece point clouds after three-dimensional measurement, the present invention uses a deep model to automatically segment and identify workpiece point clouds; in the deep learning network model, to address the lack of global feature learning in existing network models, the network model of the present invention is based on the idea of feature fusion to fuse global features and local features for learning; using the U-Net network structure, the sampling layer uses difference downsampling based on point cloud similarity to solve the problem of selecting the most prominent receptive field of each layer of the encoding layer to learn point cloud features; the network model extracts and optimizes the local features of the receptive field through the self-attention mechanism, draws on the spatial hybrid multi-layer perceptron in the multi-layer perceptron mixer to establish global connections between receptive fields to extract global features of the receptive field, and on this basis, uses a convolutional network to fuse local features and global features to improve the accuracy of automatic recognition and segmentation of workpiece point clouds.

[0062] Second, the method of this application is universal and can also be applied to other deep learning models. Furthermore, the multilayer perceptron mixer has not previously been applied to deep learning network models in the point cloud field. The spatially hybrid multilayer perceptron in the multilayer perceptron mixer performs linear transformations on the quantitative channels, fully considering the global characteristics of the point cloud and compensating for the lack of global features in the U-Net structure. The improved network model can achieve faster and more efficient segmentation and extraction of workpieces, and this invention is suitable for workpiece segmentation in in-situ measurement scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The present application will be further described in detail below with reference to the embodiments in the accompanying drawings, but this does not constitute any limitation to the present application.

[0064] Figure 1 This is a structural diagram of the three-dimensional measurement segmentation model.

[0065] Figure 2 This is the U-Net structure diagram.

[0066] Figure 3 A multi-head self-attention mechanism for the input embedding layer.

[0067] Figure 4 It is the neighborhood self-attention layer of the encoding layer and the decoding layer.

[0068] Figure 5 Schematic diagram of feature extraction by interactively fusing local features and global features in the coding layer.

[0069] Figure 6 This is the decoding layer structure diagram.

[0070] Figure 7 The effect diagram of the workpiece segmentation is displayed for in-situ measurement. DETAILED DESCRIPTION

[0071] The present invention will be further described below by way of examples in conjunction with the accompanying drawings. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that a person of ordinary skill in the art may make several variations and improvements without departing from the scope of the present invention. These all fall within the scope of protection of the present invention. The following detailed description of the embodiments of the present invention is provided in conjunction with the accompanying drawings, but the scope of protection of the present invention is not limited to the following examples.

[0072] Figure 1 and Figure 2 The structure diagram of the 3D measurement segmentation model and the U-Net structure diagram are respectively shown. The specific process of using this model to implement classification and segmentation tasks is as follows:

[0073] Step 1: For the input point cloud data, use the multi-head self-attention mechanism to embed the input and expand the input feature dimension to obtain 32-dimensional original input features.

[0074] Step 2: Based on the input point cloud data, use the sampling module to select the center point of the receptive field.

[0075] 1) Sampling layer: Use the difference downsampling method to downsample the original point cloud to obtain the center point of the receptive field. The difference downsampling steps are as follows:

[0076] 1. Set a sampling point set S to store the sampling points.

[0077] 2. Calculate the cosine similarity between each point in the point cloud and take the point with the largest average difference from other points as the initial sampling point;

[0078] 3. Select the point with the largest average difference from the sampled sub-point cloud in the remaining point cloud;

[0079] 4. Add the point with the largest difference to the sampling point set S;

[0080] 5. Repeat steps 3 and 4 until the number of members in the sampling point set S meets the number of sampling points.

[0081] The Cosine similarity operator and the difference operator are as follows:

[0082]

[0083] Difference=θ(1-Cosine(x,y))

[0084] Among them, x i ,y i Represents the coordinates of any i-th sampling point;

[0085] Where θ is the difference factor, which defaults to 0.5.

[0086] 2) Grouping layer: Use the K-means clustering algorithm to search for neighboring points in a sphere with a radius of R and the center of the receptive field to construct the receptive field.

[0087] Step 3: Based on the receptive field constructed by the sampling grouping module, the neighborhood self-attention mechanism is used to extract the features of the neighboring points in the receptive field as local features, and the connection between the receptive fields is established through the spatial hybrid multi-layer perceptron to extract the global features. Finally, the local features and global features are fused and extracted by three layers of 2D convolution to obtain the coding layer features.

[0088] Step 4: Following the U-Net structure, the encoding layer needs to be performed three times. The number of receptive fields sampled in the three encoding layers is 512, 128, and 1 respectively; the receptive field radius is 0.2, 0.4, and 1 respectively; the number of nearest neighbors is 32, 64, and 128 respectively; and the feature dimensions extracted are 128, 256, and 1024 respectively. The receptive fields of the three layers are defined as receptive field 1, receptive field 2, and receptive field 3 respectively. The resulting encoded features are defined as encoded feature 1, encoded feature 2, and encoded feature 3 respectively.

[0089] Step 5: Use the K-means clustering algorithm to search for the three nearest receptive domains 3 of receptive domain 2, and optimize the encoded feature 3 of receptive domain 3 through the neighborhood self-attention layer, and then splice it with the encoded feature 2 of receptive domain 2; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 2 to obtain the 256-dimensional decoding feature 1;

[0090] Step 6: Use the K-means clustering algorithm to search for the three nearest receptive domains 2 of receptive domain 1, and optimize the decoding feature 1 of receptive domain 2 through the neighborhood self-attention layer, and then splice it with the encoding feature 1 of receptive domain 1; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 1 to obtain the 128-dimensional decoding feature 2;

[0091] Step 7: Use the K-means clustering algorithm to search for the three nearest receptive fields 1 of the input point cloud, and optimize the decoded features 2 of the receptive field 1 through the neighborhood self-attention layer, and then splice them with the original point cloud features; use the feature propagation layer to decode and propagate the spliced point cloud features to the original point cloud to obtain 128-dimensional decoded features 3;

[0092] Step 8: Finally, use the fully connected network to obtain the predicted classification of each point in the point cloud, and segment it according to the category of each point to achieve the segmentation task.

[0093] Figure 3 The multi-head self-attention mechanism of the input embedding layer is used to embed input and increase and optimize input features.

[0094] The initial point information includes the absolute position (x, y, z) of each point, and some also include color information (R, G, B) and normal vector information (N x ,N y ,N z), so the input data is an N×C matrix formed by the initial information of N points. Due to its sparsity, point clouds cannot reflect the connections between points. To effectively extract the initial global features of the points, the input embedding layer processes the point cloud data through a linear layer and a multi-head self-attention mechanism. The linear layer expands the feature dimension, which this paper expands to 32 dimensions. The multi-head self-attention mechanism connects the entire point set and enhances the global features of the input point set. The multi-head self-attention operator is shown below:

[0095] Head i =Attention(X Q W Q ,X K W K ,X V W V )

[0096] MutiHead(Q,K,V)=Concat(Head1,Head2,…,Head h )W O

[0097] Where Q, K, and V are the query vector, key vector, and value vector, respectively, obtained by applying a linear layer to the input features. The query vector, key vector, and value vector are first fed into a scaled dot-product attention algorithm, performing h rounds of attention, also known as multi-head self-attention. Each round is computed as a single head, and parameters are not shared between heads. The parameter W used for the linear transformation of the query vector, key vector, and value vector is different each time. The results of these h rounds of scaled dot-product attention are then concatenated and linearly transformed again, resulting in the value used as the result of multi-head self-attention. As can be seen, multi-head self-attention differs in that it performs h rounds of calculations rather than just one, allowing the model to learn relevant information in different representation subspaces.

[0098] Figure 4 It is the neighborhood self-attention layer of the encoding layer and the decoding layer.

[0099] The self-attention mechanism can adaptively learn the correlation and importance of different features, and assign weights to different features through the back-propagation learning algorithm to enhance the extracted features. Our input is the sampled and grouped receptive field features. We adaptively learn the relationship between the query vector Q and the key vector K of the neighboring points in the receptive field through the subtraction relationship and the linear layer. The attention factor Adjust the inner product of the numerator, and then normalize the weight matrix by the normalized exponential function softmax to adjust the characteristics of the value vector V of the neighboring points:

[0100]

[0101] In neighborhood self-attention, Q is the feature of the center point of the receptive field, K and V are the key vector and value vector features of the neighboring points in the receptive field, and relative position encoding δ is added in between to calculate the association weight matrix between the center point and the neighboring points to enhance the local features of the point cloud. The relative position encoding δ operator is shown below:

[0102] δ=θ(p)

[0103] Where p is the relative coordinate of the neighboring points in the receptive field relative to the center point of the receptive field. The encoding function δ has two linear layers and an activation function.

[0104] Figure 5 Flowchart of feature extraction for interactive fusion of local features and global features at the encoding layer.

[0105] The encoder samples M points with the most prominent features from the input point cloud with a data format of N×C to form a sub-point cloud with a data format of M×C, and then searches for the K points closest to each point in the sub-point cloud to form M receptive fields, obtaining a receptive field point cloud with a data format of M×K×C; then the neighborhood self-attention layer is used to extract the local features of the receptive field in the dimension of K number of neighboring points; then, two linear layers and activation functions are used to magnify the number of receptive fields by four times and then restore it in the dimension of M number of receptive fields, and a layer of residual is added, that is, a spatial hybrid multilayer perceptron, to establish the connection between the receptive fields and extract the global features of the receptive fields; finally, three layers of 2D convolution layers are used to fuse the local features and global features in the feature dimension C to obtain the encoding features with a data format of M×K×D.

[0106] Figure 6 This is the decoding layer structure diagram.

[0107] The decoder uses the K-means clustering algorithm to search for the three current receptive fields closest to the corresponding receptive field, and optimizes the encoding features of the current receptive field through the neighborhood self-attention layer, and then splices them with the encoding features of the corresponding receptive field; finally, the feature propagation layer is used to decode the spliced point cloud features and propagate them to the corresponding receptive field to obtain the decoded features; the feature propagation layer is a 2D convolution layer containing a convolution kernel of 1×1, a normalization function, and an activation function.

[0108] The segmentation results of the workpiece scene measured by this segmentation model are as follows: Figure 7 As shown, it can be seen that the present invention can accurately segment the workpiece with a very low error segmentation rate, which proves the effectiveness of the present invention.

[0109] The present invention conducted experiments on the ShapeNetPart public component segmentation dataset and the scene segmentation dataset of actual in-situ workpiece measurements, respectively. The results were compared with PointNet and PointNet++ networks using the U-Net architecture. Tables 1 and 2 show that the proposed method significantly improves segmentation accuracy compared to PointNet and PointNet++.

[0110] Table 1: Segmentation results on the ShapeNetPart dataset

[0111]

[0112]

[0113] Table 2: Segmentation results on the in-situ measurement artifact scene segmentation dataset

[0114] method Overall accuracy (%) Average instance intersection-over-union ratio (%) PointNet 79.0 38.0 PointNet++ 99.7 96.3 Model of this application 99.8 96.7

[0115] In summary, the present invention uses a deep model to automatically segment and identify workpiece point clouds. Based on the idea of feature fusion in the deep learning network model, the U-Net network structure is used, and difference downsampling based on point cloud similarity is used in the sampling layer. The local features of the receptive domain are extracted and optimized through the self-attention mechanism, and the spatial hybrid multi-layer perceptron in the multi-layer perceptron mixer is used to establish global connections between receptive domains to extract global features of the receptive domain. On this basis, the local and global features are fused and learned through the convolutional network, which improves the accuracy of segmentation and obtains better segmentation effects in actual measurement applications.

[0116] The above embodiments are preferred implementation modes of the present application and are only used to facilitate the explanation of the present application. They are not intended to limit the present application in any form. Any person with ordinary knowledge in the relevant technical field, if they do not depart from the scope of the technical features proposed in the present application, can make equivalent embodiments by making partial changes or modifications to the technical content disclosed in the present application, and they do not depart from the technical features of the present application. They are still within the scope of the technical features of the present application.

Claims

1. A three-dimensional measurement and segmentation method based on deep learning, characterized in that: The following steps are involved: Step 1: Use the deep model to automatically segment and identify the workpiece point cloud; Step 2: First, use the sampling layer and grouping layer to extract the receptive field of the current input point cloud, then use the neighborhood self-attention layer to extract and optimize the local features of the receptive field, and then use the spatial hybrid multi-layer perceptron to extract the global features of the receptive field. Finally, use the 2D convolution layer to interactively extract the local features and global features of the receptive field to obtain a layer of encoding layer features; Step 3: Take the features of the previous coding layer as input and repeat the coding operation of step 2 twice to obtain the coding features of the entire point cloud; step 3 includes the following sub-steps: Step 3.1, define the features embedded by input as the original point cloud features; In step 3.2, following the U-Net structure, the encoding layer needs to be performed three times. The number of receptive fields sampled by the three encoding layers is 512, 128, and 1 respectively; the receptive field radius is 0.2, 0.4, and 1 respectively; the number of neighbor points is 32, 64, and 128 respectively; the extracted feature dimensions are 128, 256, and 1024 respectively; the three receptive fields are defined as receptive field 1, receptive field 2, and receptive field 3 respectively; the obtained encoding features are defined as encoding feature 1, encoding feature 2, and encoding feature 3 respectively; when searching for neighbor points, if the number of neighbor points in the receptive field is insufficient, the coordinates of the first point are repeated; In step 4, the coding features of the entire point cloud are first optimized through the neighborhood self-attention layer and spliced onto the coding features of the corresponding layer. The spliced features are then decoded through the feature propagation layer to obtain a layer of decoding layer features. Step 4 includes the following sub-steps: Step 4.1: Use the K-means clustering algorithm to search for the three nearest receptive domains 3 of receptive domain 2, and optimize the encoding feature 3 of receptive domain 3 through the neighborhood self-attention layer, and then splice it with the encoding feature 2 of receptive domain 2; Step 4.2: Use the feature propagation layer to decode the spliced point cloud features and propagate them to the receptive field 2 to obtain the decoded feature 1. In step 5, the decoding layer features of the previous layer are used as input, and the decoding operation of step 4 is repeated twice until the decoding layer features are spliced to the original original point cloud for decoding, and the decoding layer features of each point in the point cloud are obtained; step 5 specifically includes the following sub-steps: Step 5.1, perform second-layer decoding based on the first-layer decoding; Step 5.1.1: Use the K-means clustering algorithm to search for the three nearest receptive domains 2 of receptive domain 1, optimize the decoding feature 1 of receptive domain 2 through the neighborhood self-attention layer, and then concatenate it with the encoding feature 1 of receptive domain 1; Step 5.1.2: Use the feature propagation layer to decode the concatenated point cloud features and propagate them to receptive field 1 to obtain decoded features 2. Step 5.2, perform third-layer decoding based on the second-layer decoding; Step 5.2.1: Use the K-means clustering algorithm to search for the three nearest receptive fields 1 of the input point cloud, optimize the decoded features 2 of receptive field 1 through the neighborhood self-attention layer, and then splice them with the original point cloud features; Step 5.2.2: Use the feature propagation layer to decode the spliced point cloud features and propagate them to the original point cloud to obtain decoded features 3. Step 6: Use the fully connected network to obtain the predicted classification of each point in the point cloud to achieve the segmentation task.

2. The three-dimensional measurement and segmentation method based on deep learning according to claim 1, characterized in that: The step 1 comprises: using a structured light scanner to scan a workpiece scene point cloud, and using an input embedding layer to perform input embedding to obtain an original point cloud of the model input; The following sub-steps are included: Step 1.1, the set of point clouds obtained by scanning is P = (p0, p1, ..., p n-1 )∈R 3 , each point contains (x,y,z,n x ,n y ,n z ,R,G,B) three-dimensional coordinates, normal vector information and color information; In step 1.2, the input embedding layer is used to embed the scanned point cloud so that the feature dimension of the point cloud is expanded to 32 dimensions. The input embedding layer is a linear layer and a multi-head self-attention layer.

3. The three-dimensional measurement and segmentation method based on deep learning according to claim 1, characterized in that: Described step 2 comprises the following sub-steps: Step 2.1: First, use the sampling layer to extract the center point of the receptive field, and then use the grouping layer to extract the neighboring points of the center point of the receptive field to form the receptive field. The sampling layer uses difference downsampling, and the grouping layer uses the K-means clustering algorithm. Step 2.2: Use the neighborhood self-attention layer to extract and optimize the local features of the receptive field. The self-attention mechanism is used to establish connections between neighboring points in the receptive field and adjust the feature weights to extract and optimize the local features of the receptive field. Step 2.3: Use spatial hybrid multilayer perceptron to extract global features of receptive fields, establish connections between all receptive fields through linear transformation, and extract global features of receptive fields; In step 2.4, a 2D convolutional layer is used to interactively extract the local features and global features of the receptive field to obtain a layer of encoding layer features.

4. The three-dimensional measurement and segmentation method based on deep learning according to claim 1, characterized in that: The step 6 includes: finally using the fully connected network to obtain the predicted classification of each point in the point cloud, and segmenting according to the category of each point to achieve the segmentation task.

5. A three-dimensional measurement and segmentation method based on deep learning, the technical characteristics of which are: The following steps are involved: Step 1: Use a structured light scanner to scan the workpiece and obtain the original point cloud for model input; Step 2: For the input point cloud data, a multi-head self-attention mechanism is used to embed the input and expand the input feature dimension to obtain 32-dimensional original input features; Step 3: Based on the input point cloud data, use the sampling module to select the center point of the receptive field: 1) Sampling layer: Use the difference downsampling method to downsample the original point cloud to obtain the center point of the receptive field. The difference downsampling steps are as follows:

1. Set a sampling point set S to store the sampling points; 2. Calculate the cosine similarity between each point in the point cloud and take the point with the largest average difference from other points as the initial sampling point; Among them, cosine similarity and difference are calculated using the following formula: Difference=θ(1-Cosine(x,y)) Among them, x i ,y i Represents the coordinates of any i-th sampling point; Among them, θ is the difference factor, which is 0.5; 3. Select the point with the largest average difference from the sampled sub-point cloud in the remaining point cloud; 4. Add the point with the largest difference to the sampling point set S; 5. Repeat 3 and 4 until the number of members in the sampling point set S meets the number of sampling points; 2) Grouping layer: Use the K-means clustering algorithm to search for neighboring points within a sphere with a radius of R and the center of the receptive field to construct the receptive field; Step 3: Based on the receptive field constructed by the sampling grouping module, the neighborhood self-attention mechanism is used to extract the features of the neighboring points in the receptive field as local features, and the spatial hybrid multi-layer perceptron is used to establish the connection between the receptive fields to extract the global features. Finally, the local features and the global features are fused and extracted by three layers of 2D convolution to obtain the encoding layer features; Step 4: Following the U-Net structure, the encoding layer needs to be performed three times. The number of receptive fields sampled by the three encoding layers are 512, 128, and 1 respectively; the receptive field radius is 0.2, 0.4, and 1 respectively; the number of neighbor points is 32, 64, and 128 respectively; the extracted feature dimensions are 128, 256, and 1024 respectively; the three layers of receptive fields are defined as receptive field 1, receptive field 2, and receptive field 3 respectively; the obtained encoding features are defined as encoding feature 1, encoding feature 2, and encoding feature 3 respectively; Step 5: Use the K-means clustering algorithm to search for the three nearest receptive domains 3 of receptive domain 2, and optimize the encoded feature 3 of receptive domain 3 through the neighborhood self-attention layer, and then splice it with the encoded feature 2 of receptive domain 2; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 2 to obtain the 256-dimensional decoding feature 1; Step 6: Use the K-means clustering algorithm to search for the three nearest receptive domains 2 of receptive domain 1, and optimize the decoding feature 1 of receptive domain 2 through the neighborhood self-attention layer, and then splice it with the encoding feature 1 of receptive domain 1; use the feature propagation layer to decode the spliced point cloud features and propagate them to receptive domain 1 to obtain the 128-dimensional decoding feature 2; Step 7: Use the K-means clustering algorithm to search for the three nearest receptive fields 1 of the input point cloud, and optimize the decoded features 2 of the receptive field 1 through the neighborhood self-attention layer, and then splice them with the original point cloud features; use the feature propagation layer to decode and propagate the spliced point cloud features to the original point cloud to obtain 128-dimensional decoded features 3; Step 8: Finally, use the fully connected network to obtain the predicted classification of each point in the point cloud, and segment it according to the category of each point to achieve the segmentation task.

Images