Three-dimensional point cloud classification algorithm based on global perception and local enhancement

Abstract

The invention relates to the field of three-dimensional point cloud classification, in particular to a three-dimensional point cloud classification algorithm based on global perception and local enhancement, which is technically characterized by comprising the following steps: S1, acquiring point cloud data; s2, fusing a multi-head self-attention mechanism and a multi-layer sensor, constructing a global sensing module, inputting the point cloud data into the global sensing module, and outputting global features; s3, introducing a local geometric weight module into a sampling and grouping strategy, constructing a local enhancement module, inputting the point cloud data into the local enhancement module, and outputting local features; and S4, carrying out feature fusion on the global features and the local features. The method has the advantages that the overall structure perception of the point cloud is enhanced, the meticulous expression ability of the model to the local area is improved, the classification accuracy and robustness are effectively improved, and the method can be applied to the fields of three-dimensional defect detection, article classification, automatic driving and the like.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional point cloud classification, and more specifically, to a three-dimensional point cloud classification algorithm based on global perception and local enhancement. Background Art

[0002] 3D point cloud classification is the processing of point cloud data, where point cloud data is composed of a large number of 3D points. It is identified and classified by analyzing the distribution and characteristics of 3D points in 3D space. It covers data processing, feature learning, model training and classification decision-making, and is widely used in autonomous driving, robot vision, architectural and urban modeling and other fields.

[0003] PointNet, PointNet++ and other 3D point cloud classification algorithms have achieved remarkable results. They can directly process point cloud data, avoid information loss, maintain its integrity, and obtain global features through global maximum pooling operations. They are insensitive to order and have strong rotation and translation invariance. However, these algorithms mainly rely on local neighborhood information, which makes it difficult to effectively model contextual relationships in a larger range, which will affect the understanding of complex scenes. They also rely on global maximum pooling to process local features, which may ignore smaller local structural details and lead to insufficient extraction of local structural information. Therefore, the development of algorithms that can enhance the extraction of global features of point clouds, strengthen local area connections, and have high accuracy and strong robustness is conducive to the application and development of 3D point cloud data processing. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a three-dimensional point cloud classification algorithm based on global perception and local enhancement to solve the above problems.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: A three-dimensional point cloud classification algorithm based on global perception and local enhancement, comprising the following steps:

[0006] S1. Obtain point cloud data;

[0007] S2, integrating the multi-head self-attention mechanism and the multi-layer perceptron, constructing a global perception module, inputting the point cloud data into the global perception module, and outputting global features;

[0008] S3, introducing the local geometric weight module into the sampling and grouping strategy, constructing a local enhancement module, inputting the point cloud data into the local enhancement module, and outputting local features;

[0009] S4: Fusing the global features with the local features.

[0010] In one embodiment, the step S2 specifically includes the following steps:

[0011] S21, inputting the point cloud data into the multi-head self-attention mechanism, and outputting point cloud features;

[0012] S22: Input the point cloud features into the multi-layer perceptron, and output the global features.

[0013] In one embodiment, the step S21 specifically includes the following steps:

[0014] S211, the point cloud data Input the multi-head self-attention mechanism, where B represents the batch size, N represents the number of points, and C represents the feature dimension;

[0015] The multi-head self-attention mechanism contains multiple self-attention modules. In the self-attention module, the trainable weight matrix w i Q 、w i K and w i V Perform linear transformations respectively to generate the query matrix Q i , keyword matrix K i and the numerical matrix V i , the specific formula is:

[0016] (Q i ,K i ,V i )=(w i Q ,w i K ,w i V )×I

[0017] Wherein, i=1,2,...,n, represents the index of the self-attention module;

[0018] S212: the query matrix Q i and the keyword matrix K i Perform matrix multiplication to obtain the query matrix Q i and the keyword matrix K i The correlation between i , the specific formula is:

[0019] r i =Q i ×K i T

[0020] S213, using the softmax function to convert the correlation r i Convert to probability distribution R i , the specific formula is:

[0021] R i =softmax(r i )

[0022] S214, the probability distribution R i With the numerical matrix V i Perform weighted summation to calculate feature c i , the specific formula is:

[0023] c i =R i ×V i

[0024] Wherein, i=1,2,...,n, represents the index of the self-attention module;

[0025] S215, the features c calculated by the multiple self-attention modules i Perform splicing operations to obtain the overall features. The specific formula is:

[0026] C=concat(c0,c1...,c h )

[0027] Wherein, C represents the overall feature, h=1,2,...,n, represents the number of the self-attention modules;

[0028] S216, multiplying the overall feature C by the linear transformation matrix, performing linear transformation, and obtaining the point cloud feature X. The specific formula is:

[0029] X=CW 0

[0030] Among them, W 0 represents the linear transformation matrix.

[0031] In one embodiment, the step S22 specifically includes the following steps:

[0032] S221, inputting the point cloud feature X into the multilayer perceptron;

[0033] S222, the point cloud feature X is first expanded in dimension through the first fully connected layer, and then the nonlinear activation function ReLU is used for feature processing, and finally the result H1 is output. The specific formula is:

[0034] H1=ReLU(X×W1+b1)

[0035] Wherein, W1 is the weight matrix of the first fully connected layer, and b1 is the bias term;

[0036] S223, the result H1 is compressed in dimension through the second fully connected layer so that the dimension of the result H1 is the same as that of the point cloud feature X in step S221, and finally the result H2 is output. The specific formula is:

[0037] H2=H1×W2+b2

[0038] Among them, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term;

[0039] S224: The result H2 is first added to the point cloud feature X in step S221 through a residual connection, and then normalized, and finally the result H is output. out :

[0040] H out =LayerNorm(H2+X)

[0041] Among them, H out It is a global feature.

[0042] In one embodiment, step S3 specifically includes the following steps:

[0043] S31, the local enhancement module first obtains the local area center point of the point cloud data and the neighborhood points of the local area center point, and then obtains the relative coordinates and Euclidean distances between the local area center point and the neighborhood points;

[0044] Performing a splicing operation on the relative coordinates and the Euclidean distance to form a neighborhood feature weight, and using the neighborhood feature weight to assign a weight value to each of the neighborhood points of the central point of the local area;

[0045] S32. The local feature extraction module performs geometric feature enhancement on the point cloud data based on the weight value, and finally outputs it as a local feature.

[0046] In one embodiment, the calculation formula of the relative coordinate γ is specifically:

[0047] γ=X f -X c

[0048] in, represents the coordinate information of the center point of the local area, represents the coordinate information of the neighborhood points, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, and D represents the dimension of the feature.

[0049] In one embodiment, the calculation formula of the Euclidean distance λ is specifically:

[0050] λ=||X f-X c ||2

[0051] Among them, ||·||2 represents the second norm, X f With the X c Satisfy||X f -X c ||2≤δ.

[0052] In one embodiment, the neighborhood feature weight calculation formula is specifically:

[0053] ρ=cat(γ,λ)

[0054] Wherein, ρ represents the neighborhood feature weight, and cat(·) is the abstract function for performing the concatenation operation.

[0055] In one embodiment, the formula for geometric feature enhancement is specifically:

[0056] Y=cat(ρ,X d )

[0057] in, is the feature obtained by sampling and grouping the point cloud data, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, D represents the dimension of the feature, and Y represents the local feature.

[0058] In one embodiment, the formula for performing the feature fusion in step S4 is specifically:

[0059] T=cat(Y,H out )

[0060] Wherein, T represents the final output after the feature fusion of the local feature and the global feature.

[0061] The above three-dimensional point cloud classification algorithm based on global perception and local enhancement has the following beneficial effects:

[0062] It enhances the overall structural perception of point clouds, improves the model's ability to express local areas in detail, and effectively improves the accuracy and robustness of classification. It can be applied to 3D defect detection, object classification, autonomous driving and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Schematic diagram of the classification method in this embodiment;

[0064] Figure 2 Schematic diagram of the architecture of the self-attention module in this embodiment;

[0065] Figure 3 Schematic diagram of the architecture of the global perception module in this embodiment;

[0066] Figure 4 is a schematic diagram of the architecture of the local enhancement module in this embodiment;

[0067] Figure 5 Schematic diagram of the application of the classification method in this embodiment.

[0068] In the picture: 1. Camera; 2. Object. DETAILED DESCRIPTION

[0069] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0070] A 3D point cloud classification algorithm based on global perception and local enhancement, such as Figure 1 As shown, the following steps are included:

[0071] S1. Obtain point cloud data;

[0072] S2, integrate the multi-head self-attention mechanism and multi-layer perceptron to build a global perception module, input the point cloud data into the global perception module, and output the global features;

[0073] S21, input the point cloud data into the multi-head self-attention mechanism and output the point cloud features;

[0074] S211, point cloud data Enter the multi-head self-attention mechanism, where B represents the batch size, N represents the number of points, and C represents the feature dimension;

[0075] like Figure 2 As shown in the figure, the multi-head self-attention mechanism contains multiple self-attention modules. In the self-attention module, the trainable weight matrix w i Q 、w i K and w i V Perform linear transformations respectively to generate the query matrix Q i , keyword matrix K i and the numerical matrix V i , the specific formula is:

[0076] (Q i ,K i ,V i )=(w i Q ,w i K ,w i V )×I

[0077] Where i=1,2,...,n represents the index of the self-attention module;

[0078] S212, query matrix Q i and keyword matrix K i Perform matrix multiplication to obtain the query matrix Q i and keyword matrix K i The correlation between i , the specific formula is:

[0079] r i =Q i ×K i T

[0080] S213, use the softmax function to convert the correlation r i Convert to probability distribution R i , the specific formula is:

[0081] R i =softmax(r i )

[0082] S214, the probability distribution R i With the numerical matrix V i Perform weighted summation to calculate feature c i , the specific formula is:

[0083] c i =R i ×V i

[0084] Where i=1,2,...,n represents the index of the self-attention module;

[0085] S215, features c calculated by multiple self-attention modules i Perform splicing operations to obtain the overall features. The specific formula is:

[0086] C=concat(c0,c1...,c h )

[0087] Where C represents the overall feature, h = 1, 2, ..., n, represents the number of self-attention modules;

[0088] S216, multiply the overall feature C by the linear transformation matrix, perform linear transformation, and obtain the point cloud feature X. The specific formula is:

[0089] X=CW 0

[0090] Among them, W 0 Represents a linear transformation matrix.

[0091] In this embodiment, if Figure 2 and Figure 3 As shown in the figure, in order to improve the model's overall structural perception of point cloud data, the multi-head self-attention mechanism and the multi-layer perceptron are integrated to construct a global perception module, wherein the multi-head self-attention mechanism calculates the attention weight, assigns weights of other related points to each point, and performs weighted summation on each pair of points, so that each point can be aggregated to the relevant information in its context; on this basis, the multi-layer perceptron further processes the point cloud features generated by the multi-head self-attention mechanism, and further improves the feature representation capability of the model through nonlinear mapping and local complex feature conversion.

[0092] By obtaining the query matrix Q i and keyword matrix K i The correlation between i , the correlation between each point in the point cloud data and other points is calculated, reflecting the dependency between points in the point cloud data. Then, the softmax function is used to convert the correlation into a probability distribution R i , so that the relevance scores are normalized to probability distributions and the sum of all relevance scores is 1, and these probability distributions are combined with the numerical matrix V i Perform weighted summation to obtain the feature c calculated by each self-attention module i .

[0093] In this way, the calculated feature c i It is used for each point in the weighted sum of the point cloud data, focusing on the information most relevant to the current point, so that each point can effectively obtain the feature information of other related points, thereby capturing the global context of the point cloud.

[0094] S22. Input the point cloud features into a multi-layer perceptron and output the global features.

[0095] S221, input the point cloud feature X into the multi-layer perceptron;

[0096] S222, the point cloud feature X is first expanded in dimension through the first fully connected layer, and then the nonlinear activation function ReLU is used for feature processing, and finally the result H1 is output. The specific formula is:

[0097] H1=ReLU(X×W1+b1)

[0098] Among them, W1 is the weight matrix of the first fully connected layer, and b1 is the bias term;

[0099] S223, the result H1 is compressed by the second fully connected layer to make the result H1 have the same dimension as the point cloud feature X in step S221, and finally the result H2 is output. The specific formula is:

[0100] H2=H1×W2+b2

[0101] Among them, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term;

[0102] S224: The result H2 is first added to the point cloud feature X in step S221 through a residual connection, and then normalized, and finally the result H is output. out :

[0103] H out =LayerNorm(H2+X)

[0104] Among them, H out It is a global feature.

[0105] In this embodiment, if Figure 3 As shown in the figure, the point cloud feature X is used as the input feature, and its shape depends on the output of the previous layer. W1 is the weight matrix of the first fully connected layer, which is used to expand the feature dimension of the input feature. As a result, H1 is the output after the ReLU activation function. By introducing the nonlinear activation function ReLU, the expressive ability of the model is enhanced, enabling it to capture the complex patterns and relationships in the input features.

[0106] Among them, W2 is the weight matrix of the second fully connected layer, which is used to compress the expanded high-dimensional features back to the original feature dimensions to ensure that the model can transmit feature representations of the same dimension between each layer while retaining the rich feature information after expansion and nonlinear transformation.

[0107] S3, introduce the local geometric weight module into the sampling and grouping strategy, build a local enhancement module, input the point cloud data into the local enhancement module, and output local features;

[0108] S31, the local enhancement module first obtains the local area center point of the point cloud data and the neighborhood points of the local area center point, and then obtains the relative coordinates and Euclidean distances between the local area center point and the neighborhood points;

[0109] The relative coordinates and the Euclidean distance are concatenated to form neighborhood feature weights, and the neighborhood feature weights are used to assign weight values ​​to each neighborhood point of the local area center point;

[0110] S32. The local feature extraction module performs geometric feature enhancement on the point cloud data based on the weight value, and finally outputs it as a local feature.

[0111] The calculation formula of the relative coordinate γ is as follows:

[0112] γ=X f -X c

[0113] in, Represents the coordinate information of the center point of the local area, Represents the coordinate information of the neighborhood points, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, and D represents the dimension of the feature.

[0114] The calculation formula of Euclidean distance λ is as follows:

[0115] λ=||X f -X c ||2

[0116] Among them, ||·||2 represents the second norm, X f With X c Satisfy||X f -X c ||2≤δ.

[0117] The specific calculation formula for neighborhood feature weight is:

[0118] ρ=cat(γ,λ)

[0119] Among them, ρ represents the neighborhood feature weight, and cat(·) is the abstract function for the concatenation operation.

[0120] The specific formula for geometric feature enhancement is:

[0121] Y=cat(ρ,X d )

[0122] in, It is the feature obtained by sampling and grouping strategy of point cloud data, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, D represents the dimension of the feature, and Y represents the local feature.

[0123] In this embodiment, if Figure 1 and Figure 4 As shown in the figure, in order to make the model have the ability to express local areas in detail, an adaptive local geometric weight is introduced, and a local enhancement module is constructed in combination with the sampling and grouping strategy, which can extract features in a more fine-grained manner and fully capture the important features of point cloud data. This module introduces a local geometric weight module, combines the relative coordinates of the center point and the neighborhood points in the local area and the Euclidean distance between the two, and assigns a corresponding weight value to each neighborhood point. The size of the weight value quantifies the geometric relationship between the center point and the neighborhood points. The larger the weight value, the closer the connection between the neighborhood point and the center point, and thus its feature expression needs to be enhanced.

[0124] S4. Fuse the global features with the local features.

[0125] The formula for feature fusion in step S4 is as follows:

[0126] T=cat(Y,Hout )

[0127] Among them, T represents the final output after the local features and global features are fused. This fusion method ensures the complementarity of global and local information and effectively improves the point cloud representation ability of the model.

[0128] like Figure 5 As shown, binocular stereo vision is used to obtain three-dimensional information of the object, the positional relationship between the two cameras 1 and the coordinates of the object 2 in the left and right images are obtained, and machine vision is used to simultaneously collect images through the two cameras 1 to obtain the pixel difference between the corresponding points of an image imaged by the left and right cameras 1 (obtain depth information), and then obtain three-dimensional information, and then send the three-dimensional information (point cloud data) as input to the classification algorithm to judge the quality of the object 2 corresponding to the picture.

[0129] Therefore, the present invention is applicable to enterprises with three-dimensional defect detection needs. The classification algorithm of the present invention combined with binocular vision (obtaining the point cloud of the object) can be used to perform defect detection on three-dimensional objects.

[0130] The present invention can also be applied to the field of autonomous driving. By combining the classification method of the present invention with the autonomous driving system, it can be used to identify and classify obstacles on the road (such as pedestrians, vehicles, traffic signs, etc.), helping the autonomous driving system to make real-time decisions.

[0131] The present invention can also be applied to the field of artificial intelligence. By utilizing the classification algorithm of the present invention in combination with robot vision, the robot can understand the surrounding environment, perform navigation, obstacle avoidance and target recognition, and perform complex tasks.

[0132] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A 3D point cloud classification algorithm based on global perception and local enhancement, characterized in that: The following steps are involved: S1. Obtain point cloud data; S2, integrating the multi-head self-attention mechanism and the multi-layer perceptron, constructing a global perception module, inputting the point cloud data into the global perception module, and outputting global features; S3, introducing a local geometric weight module into the sampling and grouping strategy, constructing a local enhancement module, inputting the point cloud data into the local enhancement module, and outputting local features; S4: Fusing the global features with the local features.

2. A three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21, inputting the point cloud data into the multi-head self-attention mechanism, and outputting point cloud features; S22: Input the point cloud features into the multi-layer perceptron and output the global features.

3. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 2, characterized in that: The step S21 specifically includes the following steps: S211, the point cloud data Input the multi-head self-attention mechanism, where B represents the batch size, N represents the number of points, and C represents the feature dimension; The multi-head self-attention mechanism contains multiple self-attention modules. In the self-attention module, the trainable weight matrix w i Q 、w i K and w i V Perform linear transformations respectively to generate the query matrix Q i , keyword matrix K i and the numerical matrix V i , the specific formula is: (Q i ,K i ,V i )=(w i Q ,w i K ,w i V )×I Wherein, i=1,2,...,n, represents the index of the self-attention module; S212: the query matrix Q i and the keyword matrix K i Perform matrix multiplication to obtain the query matrix Q i and the keyword matrix K i The correlation between i , the specific formula is: r i =Q i ×K i T S213, using the softmax function to convert the correlation r i Convert to probability distribution R i , the specific formula is: R i =softmax(r i ) S214, the probability distribution R i With the numerical matrix V i Perform weighted summation to calculate feature c i , the specific formula is: c i =R i ×V i Wherein, i=1,2,...,n, represents the index of the self-attention module; S215, the features c calculated by the multiple self-attention modules i Perform splicing operations to obtain the overall features. The specific formula is: C=concat(c0,c1...,c h ) Wherein, C represents the overall feature, h=1,2,...,n, represents the number of the self-attention modules; S216, multiplying the overall feature C by the linear transformation matrix, performing linear transformation, and obtaining the point cloud feature X. The specific formula is: X=CW 0 Among them, W 0 represents the linear transformation matrix.

4. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 3 is characterized in that: The step S22 specifically includes the following steps: S221, inputting the point cloud feature X into the multilayer perceptron; S222, the point cloud feature X is first expanded in dimension through the first fully connected layer, and then the nonlinear activation function ReLU is used for feature processing, and finally the result H1 is output. The specific formula is: H1=ReLU(X×W1+b1) Wherein, W1 is the weight matrix of the first fully connected layer, and b1 is the bias term; S223, the result H1 is compressed in dimension through the second fully connected layer so that the dimension of the result H1 is the same as that of the point cloud feature X in step S221, and finally the result H2 is output. The specific formula is: H2=H1×W2+b2 Among them, W2 is the weight matrix of the second fully connected layer, and b2 is the bias term; S224: The result H2 is first added to the point cloud feature X in step S221 through a residual connection, and then normalized, and finally the result H is output. out : H out =LayerNorm(H2+X) Among them, H out It is a global feature.

5. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 1, characterized in that: The step S3 specifically comprises the following steps: S31, the local enhancement module first obtains the local area center point of the point cloud data and the neighborhood points of the local area center point, and then obtains the relative coordinates and Euclidean distances between the local area center point and the neighborhood points; Performing a splicing operation on the relative coordinates and the Euclidean distance to form a neighborhood feature weight, and using the neighborhood feature weight to assign a weight value to each of the neighborhood points of the central point of the local area; S32. The local feature extraction module performs geometric feature enhancement on the point cloud data based on the weight value, and finally outputs it as a local feature.

6. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 5, characterized in that: The calculation formula of the relative coordinate γ is specifically: γ=X f -X c in, represents the coordinate information of the center point of the local area, represents the coordinate information of the neighborhood points, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, and D represents the dimension of the feature.

7. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 6, characterized in that: The calculation formula of the Euclidean distance λ is specifically: λ=||X f -X c ||2 Among them, ||·||2 represents the second norm, X f With the X c Satisfy||X f -X c ||2≤δ.

8. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 7, characterized in that: The neighborhood feature weight calculation formula is specifically: ρ=cat(γ,λ) Wherein, ρ represents the neighborhood feature weight, and cat(·) is the abstract function for performing the concatenation operation.

9. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 8, characterized in that: The formula for geometric feature enhancement is specifically: Y=cat(ρ,X d ) in, is the feature obtained by sampling and grouping the point cloud data, B represents the batch size, M represents the number of center points, S represents the number of neighborhood points, D represents the dimension of the feature, and Y represents the local feature.

10. The three-dimensional point cloud classification algorithm based on global perception and local enhancement according to claim 9, characterized in that: The formula for performing the feature fusion in step S4 is specifically: T=cat(Y,H out ) Wherein, T represents the final output after the feature fusion of the local feature and the global feature.

Images