A 3D point cloud classification algorithm based on global perception and local enhancement

Abstract

This invention relates to the field of 3D point cloud classification, and more specifically, to a 3D point cloud classification algorithm based on global perception and local enhancement. The key technical points are: S1, acquiring point cloud data; S2, fusing a multi-head self-attention mechanism and a multilayer perceptron to construct a global perception module, inputting the point cloud data into the global perception module, and outputting global features; S3, introducing a local geometric weight module into the sampling and grouping strategy to construct a local enhancement module, inputting the point cloud data into the local enhancement module, and outputting local features; S4, fusing the global features and the local features. The advantages of this invention are enhanced perception of the overall structure of the point cloud, improved model's ability to express detailed local regions, and effectively improved classification accuracy and robustness. It can be applied to fields such as 3D defect detection, object classification, and autonomous driving.

Description

Technical Field

[0001] This invention relates to the field of 3D point cloud classification, and more specifically, to a 3D point cloud classification algorithm based on global perception and local enhancement. Background Technology

[0002] 3D point cloud classification involves processing point cloud data, which is composed of a large number of 3D points. It identifies and classifies points by analyzing their distribution and characteristics in 3D space. This process covers data processing, feature learning, model training, and classification decision-making, and is widely used in fields such as autonomous driving, robot vision, and architectural and urban modeling.

[0003] PointNet, PointNet++, and other 3D point cloud classification algorithms have achieved significant results. They can directly process point cloud data, avoiding information loss and maintaining its integrity. They can also obtain global features through global max pooling operations, are insensitive to order, and have strong rotation and translation invariance. However, these algorithms mainly rely on local neighborhood information, making it difficult to effectively model large-scale contextual relationships, which can affect the understanding of complex scenes. Furthermore, relying on global max pooling to process local features may overlook small local structural details, leading to insufficient extraction of local structural information. Therefore, developing algorithms that can enhance global feature extraction from point clouds, strengthen the connections between local regions, and are highly accurate and robust will benefit the application and development of 3D point cloud data processing. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a 3D point cloud classification algorithm based on global perception and local enhancement to solve the above problems.

[0005] The above-mentioned technical objective of this invention is achieved through the following technical solution: a 3D point cloud classification algorithm based on global perception and local enhancement, comprising the following steps:

[0006] S1. Acquire point cloud data;

[0007] S2. Integrate multi-head self-attention mechanism and multilayer perceptron to construct global perception module, input the point cloud data into global perception module, and output global features;

[0008] S3. Introduce the local geometric weight module into the sampling and grouping strategy to construct a local enhancement module. Input the point cloud data into the local enhancement module and output local features.

[0009] S4. Perform feature fusion between the global features and the local features.

[0010] In one embodiment, step S2 specifically includes the following steps:

[0011] S21. Input the point cloud data into the multi-head self-attention mechanism and output the point cloud features;

[0012] S22. Input the point cloud features into the multilayer perceptron and output the global features.

[0013] In one embodiment, step S21 specifically includes the following steps:

[0014] S211, Transfer the point cloud data Input the multi-head self-attention mechanism, where B represents the batch size, N represents the number of points, and C represents the feature dimension;

[0015] The multi-head self-attention mechanism contains multiple self-attention modules, in which the trainable weight matrix w is... i Q w i K and w i V Perform linear transformations respectively to generate the query matrix Q. i Keyword matrix K i sum numerical matrix V i The specific formula is as follows:

[0016] (Q i ,K i V i )=(w i Q ,w i K ,w i V )×I

[0017] Where i = 1, 2, ..., n, represents the index of the self-attention module;

[0018] S212, regarding the query matrix Q i and the keyword matrix K i Perform matrix multiplication to obtain the query matrix Q. i and the keyword matrix K i The correlation r between them i The specific formula is as follows:

[0019] r i =Q i ×K i T

[0020] S213. Use the softmax function to measure the correlation r. i Transform into probability distribution R i The specific formula is as follows:

[0021] R i =softmax(r i )

[0022] S214, the probability distribution R i With the numerical matrix V i Perform weighted summation to calculate feature c. i The specific formula is as follows:

[0023] c i =R i ×V i

[0024] Where i = 1, 2, ..., n, represents the index of the self-attention module;

[0025] S215. The features c calculated by the multiple self-attention modules i The overall features are obtained by performing a splicing operation. The specific formula is as follows:

[0026] C = concat(c0, c1, ..., c h )

[0027] Where C represents the overall feature, and h = 1, 2, ..., n represents the number of self-attention modules;

[0028] S216. Multiply the overall feature C by the linear transformation matrix to perform a linear transformation, obtaining the point cloud feature X, with the specific formula as follows:

[0029] X = CW 0

[0030] Among them, W 0 Let represent the linear transformation matrix.

[0031] In one embodiment, step S22 specifically includes the following steps:

[0032] S221. Input the point cloud feature X into the multilayer perceptron;

[0033] S222. The point cloud feature X is first expanded in dimension through the first fully connected layer, then processed using the non-linear activation function ReLU, and finally output as result H1. The specific formula is as follows:

[0034] H1 = ReLU(X×W1 + b1)

[0035] Where W1 is the weight matrix of the first fully connected layer, and b1 is the bias term;

[0036] S223. The result H1 is compressed in dimension through a second fully connected layer, making the dimension of result H1 the same as that of the point cloud feature X in step S221. Finally, result H2 is output, with the specific formula as follows:

[0037] H2 = H1 × W2 + b2

[0038] Where W2 is the weight matrix of the second fully connected layer, and b2 is the bias term;

[0039] S224. The result H2 and the point cloud feature X in step S221 are first added together through residual connection, then normalized, and finally the result H is output. out :

[0040] H out =LayerNorm(H2+X)

[0041] Among them, H out This is a global feature.

[0042] In one embodiment, step S3 specifically includes the following steps:

[0043] S31. The local enhancement module first obtains the local region center point of the point cloud data and the neighboring points of the local region center point, and then obtains the relative coordinates and Euclidean distance between the local region center point and the neighboring points.

[0044] The relative coordinates and the Euclidean distance are concatenated to form a neighborhood feature weight, and the neighborhood feature weight is used to assign a weight value to each of the neighborhood points of the local region center point.

[0045] S32. The local feature extraction module performs geometric feature enhancement on the point cloud data based on the weight value, and finally outputs it as a local feature.

[0046] In one embodiment, the formula for calculating the relative coordinate γ is as follows:

[0047] γ=X f -X c

[0048] in, This represents the coordinate information of the center point of the local region. The coordinate information of the neighboring points is represented by B, the batch size is represented by M, the number of center points is represented by S, the number of neighboring points is represented by D, and the dimension of the feature is represented by D.

[0049] In one embodiment, the Euclidean distance λ is specifically calculated as follows:

[0050] λ=||X f-X c ||2

[0051] Where ||·||2 represents the L2 norm, X f With the X c Satisfy ||X f -X c The relationship is ||2≤δ.

[0052] In one embodiment, the formula for calculating the neighborhood feature weights is as follows:

[0053] ρ = cat(γ, λ)

[0054] Where ρ represents the neighborhood feature weight, and cat(·) is the abstract function for performing the splicing operation.

[0055] In one embodiment, the formula for geometric feature enhancement is specifically as follows:

[0056] Y = cat(ρ, X) d )

[0057] in, The features are obtained by sampling and grouping the point cloud data, where B represents the batch size, M represents the number of center points, S represents the number of neighboring points, D represents the dimension of the feature, and Y represents the local feature.

[0058] In one embodiment, the formula for feature fusion in step S4 is specifically as follows:

[0059] T = cat(Y, H) out )

[0060] Where T represents the final output after the feature fusion of the local features and the global features.

[0061] The aforementioned 3D point cloud classification algorithm based on global perception and local enhancement has the following beneficial effects:

[0062] It enhances the perception of the overall structure of point clouds, improves the model's ability to represent local regions in detail, and effectively improves the accuracy and robustness of classification. It can be applied to fields such as 3D defect detection, object classification, and autonomous driving. Attached Figure Description

[0063] Figure 1 This is a flowchart illustrating the classification method in this embodiment;

[0064] Figure 2 This is a schematic diagram of the architecture of the self-attention module in this embodiment;

[0065] Figure 3 This is a schematic diagram of the architecture of the global perception module in this embodiment;

[0066] Figure 4 This is a schematic diagram of the architecture of the local enhancement module in this embodiment;

[0067] Figure 5 This is a schematic diagram illustrating the application of the classification method in this embodiment.

[0068] In the picture: 1. Camera; 2. Object. Detailed Implementation

[0069] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0070] A 3D point cloud classification algorithm based on global perception and local enhancement, such as Figure 1 As shown, it includes the following steps:

[0071] S1. Acquire point cloud data;

[0072] S2. Integrate multi-head self-attention mechanism and multilayer perceptron to construct global perception module. Input point cloud data into global perception module and output global features.

[0073] S21. Input the point cloud data into the multi-head self-attention mechanism and output the point cloud features;

[0074] S211, Transfer point cloud data Input multi-head self-attention mechanism, where B represents batch size, N represents the number of points, and C represents feature dimension;

[0075] like Figure 2 As shown, the multi-head self-attention mechanism contains multiple self-attention modules. Within each self-attention module, the trainable weight matrix w... i Q w i K and w i V Perform linear transformations respectively to generate the query matrix Q. i Keyword matrix K i sum numerical matrix V i The specific formula is as follows:

[0076] (Q i ,K i V i )=(w i Q ,w i K ,w i V )×I

[0077] Where i = 1, 2, ..., n, represents the index of the self-attention module;

[0078] S212, Regarding the query matrix Q i and Keyword Matrix K i Perform matrix multiplication to obtain the query matrix Q. i and Keyword Matrix K i The correlation r between them i The specific formula is as follows:

[0079] r i =Q i ×K i T

[0080] S213. Use the softmax function to measure the correlation r i Transform into probability distribution R i The specific formula is as follows:

[0081] R i =softmax(r i )

[0082] S214, the probability distribution R i With numerical matrix V i Perform weighted summation to calculate feature c. i The specific formula is as follows:

[0083] c i =R i ×V i

[0084] Where i = 1, 2, ..., n, represents the index of the self-attention module;

[0085] S215, Features c calculated from multiple self-attention modules i The overall features are obtained by performing a splicing operation. The specific formula is as follows:

[0086] C = concat(c0, c1, ..., c h )

[0087] Where C represents the overall feature, and h = 1, 2, ..., n represents the number of self-attention modules;

[0088] S216. Multiply the overall feature C by the linear transformation matrix to perform a linear transformation, obtaining the point cloud feature X. The specific formula is as follows:

[0089] X = CW 0

[0090] Among them, W 0 This represents a linear transformation matrix.

[0091] In this embodiment, as Figure 2 and Figure 3 As shown, in order to improve the model's overall structural perception of point cloud data, a global perception module is constructed by integrating a multi-head self-attention mechanism and a multilayer perceptron. The multi-head self-attention mechanism calculates attention weights, assigns weights to other relevant points for each point, and performs a weighted summation on each pair of points, so that each point can be aggregated to relevant information in its context. Based on this, the multilayer perceptron further processes the point cloud features generated by the multi-head self-attention mechanism, and further improves the model's feature representation capability through nonlinear mapping and local complex feature transformation.

[0092] By obtaining the query matrix Q i and Keyword Matrix K i The correlation r between them i The correlation between each point in the point cloud data and other points was calculated, reflecting the dependencies between points in the point cloud data. Then, the correlation was converted into a probability distribution R using the softmax function. i This normalizes the correlation scores into a probability distribution, and the sum of all correlation scores is 1. These probability distributions are then compared with the numerical matrix V. i We perform a weighted summation to obtain the feature c calculated by each self-attention module. i .

[0093] In this way, the feature c is calculated i The points used in the weighted summation of the point cloud data focus on the information most relevant to the current point. This allows each point to effectively acquire feature information from other relevant points, thereby capturing the global context of the point cloud.

[0094] S22. Input the point cloud features into the multilayer perceptron and output the global features.

[0095] S221. Input the point cloud feature X into the multilayer perceptron;

[0096] S222. First, expand the dimension of the point cloud feature X through the first fully connected layer, then perform feature processing using the non-linear activation function ReLU, and finally output the result H1. The specific formula is as follows:

[0097] H1 = ReLU(X×W1 + b1)

[0098] Where W1 is the weight matrix of the first fully connected layer, and b1 is the bias term;

[0099] S223. Result H1 is compressed in dimension through a second fully connected layer, making the dimension of result H1 the same as that of point cloud feature X in step S221. Finally, result H2 is output, with the specific formula as follows:

[0100] H2 = H1 × W2 + b2

[0101] Where W2 is the weight matrix of the second fully connected layer, and b2 is the bias term;

[0102] S224. The result H2 and the point cloud feature X in step S221 are first added together through residual connection, then normalized, and finally the result H is output. out :

[0103] H out =LayerNorm(H2+X)

[0104] Among them, H out This is a global feature.

[0105] In this embodiment, as Figure 3 As shown, the point cloud feature X is used as the input feature, and its shape depends on the output of the previous layer. W1 is the weight matrix of the first fully connected layer, which is used to expand the feature dimension of the input feature. The result H1 is the output after passing through the ReLU activation function. By introducing the non-linear activation function ReLU, the expressive power of the model is enhanced, enabling it to capture complex patterns and relationships in the input features.

[0106] W2 is the weight matrix of the second fully connected layer, which is used to compress the expanded high-dimensional features back to the original feature dimension to ensure that the model can pass the same dimension of feature representation between each layer, while retaining the rich feature information after expansion and nonlinear transformation.

[0107] S3. Introduce the local geometric weight module into the sampling and grouping strategy to construct a local enhancement module. Input point cloud data into the local enhancement module and output local features.

[0108] S31. The local enhancement module first obtains the local region center point of the point cloud data and the neighboring points of the local region center point, and then obtains the relative coordinates and Euclidean distance between the local region center point and the neighboring points.

[0109] The relative coordinates and Euclidean distance are concatenated to form neighborhood feature weights. These neighborhood feature weights are then used to assign weight values ​​to each neighboring point of the local region's center point.

[0110] S32. The local feature extraction module enhances the geometric features of the point cloud data based on the weight values, and finally outputs them as local features.

[0111] The specific formula for calculating the relative coordinate γ is as follows:

[0112] γ=X f -X c

[0113] in, This represents the coordinate information of the center point of a local region. The coordinates of the neighboring points are represented by B, the batch size is represented by M, the number of center points is represented by S, and the dimension of the feature is represented by D.

[0114] The specific formula for calculating the Euclidean distance λ is as follows:

[0115] λ=||X f -X c ||2

[0116] Where ||·||2 represents the L2 norm, X f With X c Satisfy ||X f -X c The relationship is ||2≤δ.

[0117] The specific formula for calculating the neighborhood feature weights is as follows:

[0118] ρ = cat(γ, λ)

[0119] Where ρ represents the neighborhood feature weights, and cat(·) is an abstract function for performing the concatenation operation.

[0120] The formula for geometric feature enhancement is as follows:

[0121] Y = cat(ρ, X) d )

[0122] in, The features are obtained from point cloud data through sampling and grouping strategies. B represents the batch size, M represents the number of center points, S represents the number of neighboring points, D represents the dimension of the feature, and Y represents the local feature.

[0123] In this embodiment, as Figure 1 and Figure 4 As shown, to enable the model to express local regions in detail, adaptive local geometric weights are introduced and combined with sampling and grouping strategies to construct a local enhancement module. This module can extract features at a finer granular level, fully capturing the important features of point cloud data. This module, by introducing a local geometric weight module, assigns a corresponding weight value to each neighboring point based on the relative coordinates of the local region's center point and its neighboring points, as well as the Euclidean distance between them. The magnitude of the weight value quantifies the geometric relationship between the center point and its neighboring points; a larger weight value indicates a stronger connection between the neighboring points and the center point, thus requiring enhanced feature representation.

[0124] S4. Perform feature fusion between global and local features.

[0125] The specific formula for feature fusion in step S4 is as follows:

[0126] T = cat(Y, H)out )

[0127] Here, T represents the final output after feature fusion of local and global features. This fusion method ensures the complementarity of global and local information, effectively improving the model's point cloud representation capability.

[0128] like Figure 5 As shown, a binocular stereo vision method is used to acquire three-dimensional information about an object. The positional relationship between the two cameras 1 and the coordinates of the object 2 in the left and right images are obtained. Using machine vision, the two cameras 1 simultaneously and synchronously acquire images, and the pixel difference between the corresponding points of the left and right cameras 1 in an image is obtained (to obtain depth information), thereby obtaining three-dimensional information. The three-dimensional information (point cloud data) is then used as input to a classification algorithm to determine the quality of the object 2 corresponding to the image.

[0129] Therefore, this invention is applicable to enterprises with 3D defect detection needs. By using the classification algorithm of this invention in combination with binocular vision (to acquire the point cloud of the object), defect detection of 3D objects can be achieved.

[0130] This invention can also be applied to the field of autonomous driving. By combining the classification method of this invention with an autonomous driving system, it can be used to identify and classify obstacles on the road (such as pedestrians, vehicles, traffic signs, etc.) to help the autonomous driving system make real-time decisions.

[0131] This invention can also be applied to the field of artificial intelligence. By using the classification algorithm of this invention in combination with robot vision, robots can understand their surrounding environment, navigate, avoid obstacles, and identify targets to perform complex tasks.

[0132] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A 3D point cloud classification algorithm based on global perception and local enhancement, characterized in that, Includes the following steps: S1. Acquire point cloud data; S2. Integrate multi-head self-attention mechanism and multilayer perceptron to construct global perception module, input the point cloud data into global perception module, and output global features; S3. Introduce the local geometric weight module into the sampling and grouping strategy to construct a local enhancement module. Input the point cloud data into the local enhancement module and output local features. S4. Perform feature fusion between the global features and the local features; Step S3 specifically includes the following steps: S31. The local enhancement module first obtains the local region center point of the point cloud data and the neighboring points of the local region center point, and then obtains the relative coordinates and Euclidean distance between the local region center point and the neighboring points. The relative coordinates and the Euclidean distance are concatenated to form a neighborhood feature weight, and the neighborhood feature weight is used to assign a weight value to each of the neighborhood points of the local region center point. S32. The local feature extraction module performs geometric feature enhancement on the point cloud data based on the weight value, and finally outputs it as a local feature.

2. The 3D point cloud classification algorithm based on global perception and local enhancement according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21. Input the point cloud data into the multi-head self-attention mechanism and output the point cloud features; S22. Input the point cloud features into the multilayer perceptron and output the global features.

3. The 3D point cloud classification algorithm based on global perception and local enhancement according to claim 2, characterized in that, Step S21 specifically includes the following steps: S211, Transfer the point cloud data Input the multi-head self-attention mechanism, where B represents the batch size, N represents the number of points, and C represents the feature dimension; The multi-head self-attention mechanism contains multiple self-attention modules, in which the trainable weight matrix w is... i Q w i K and w i V Perform linear transformations respectively to generate the query matrix Q. i Keyword matrix K i sum numerical matrix V i The specific formula is as follows: (Q i ,K i ,V i )=(w i Q ,w i K ,w i V )×I Where i = 1, 2, ..., n, represents the index of the self-attention module; S212, regarding the query matrix Q i and the keyword matrix K i Perform matrix multiplication to obtain the query matrix Q. i and the keyword matrix K i The correlation r between them i The specific formula is as follows: r i =Q i ×K i T S213. Use the softmax function to measure the correlation r. i Transform into probability distribution R i The specific formula is as follows: R i =softmax(r i ) S214, the probability distribution R i With the numerical matrix V i Perform weighted summation to calculate feature c. i The specific formula is as follows: c i =R i ×V i Where i = 1, 2, ..., n, represents the index of the self-attention module; S215. The features c calculated by the multiple self-attention modules i The overall features are obtained by performing a splicing operation. The specific formula is as follows: C=concat(c0,c1,...,c h ) Where C represents the overall feature, and h = 1, 2, ..., n represents the number of self-attention modules; S216. Multiply the overall feature C by the linear transformation matrix to perform a linear transformation, obtaining the point cloud feature X, with the specific formula as follows: X=CW 0 Among them, W 0 Let represent the linear transformation matrix.

4. The 3D point cloud classification algorithm based on global perception and local enhancement according to claim 3, characterized in that, Step S22 specifically includes the following steps: S221. Input the point cloud feature X into the multilayer perceptron; S222. The point cloud feature X is first expanded in dimension through the first fully connected layer, then processed using the non-linear activation function ReLU, and finally output as result H1. The specific formula is as follows: H1 = ReLU(X×W1 + b1) Where W1 is the weight matrix of the first fully connected layer, and b1 is the bias term; S223. The result H1 is compressed in dimension through a second fully connected layer, making the dimension of result H1 the same as that of the point cloud feature X in step S221. Finally, result H2 is output, with the specific formula as follows: H2 = H1 × W2 + b2 Where W2 is the weight matrix of the second fully connected layer, and b2 is the bias term; S224. The result H2 and the point cloud feature X in step S221 are first added together through residual connection, then normalized, and finally the result H is output. out : H out =LayerNorm(H2+X) Among them, H out This is a global feature.

5. The 3D point cloud classification algorithm based on global perception and local enhancement according to claim 1, characterized in that, The specific formula for calculating the relative coordinate γ is as follows: γ=X f -X c in, This represents the coordinate information of the center point of the local region. The coordinate information of the neighboring points is represented by B, the batch size is represented by M, the number of center points is represented by S, the number of neighboring points is represented by D, and the dimension of the feature is represented by D.

6. The 3D point cloud classification algorithm based on global perception and local enhancement according to claim 5, characterized in that, The specific formula for calculating the Euclidean distance λ is as follows: λ=||X f -X c ||2 Where ||·||2 represents the L2 norm, X f With the X c Satisfy ||X f -X c The relationship is ||2≤δ.

7. A 3D point cloud classification algorithm based on global perception and local enhancement according to claim 6, characterized in that, The formula for calculating the neighborhood feature weights is as follows: ρ = cat(γ, λ) Where ρ represents the neighborhood feature weight, and cat(·) is the abstract function for performing the splicing operation.

8. A 3D point cloud classification algorithm based on global perception and local enhancement according to claim 7, characterized in that, The formula for enhancing the geometric features is as follows: Y=cat(ρ,X d ) in, The features are obtained by sampling and grouping the point cloud data, where B represents the batch size, M represents the number of center points, S represents the number of neighboring points, D represents the dimension of the feature, and Y represents the local feature.

9. A 3D point cloud classification algorithm based on global perception and local enhancement according to claim 8, characterized in that, The specific formula for feature fusion in step S4 is as follows: T=cat(Y,H out ) Where T represents the final output after the feature fusion of the local features and the global features.

Images