A point cloud shape analysis method based on spatial geometry perception convolutional neural network

Abstract

The application relates to the field of 3D vision, in particular to a point cloud shape analysis method based on a space geometry perception convolutional neural network. According to a K nearest neighbor algorithm, a point cloud is constructed as graph structure data, and a domain adaptive convolution kernel with different geometric shapes is adaptively generated according to features in each neighborhood. Subsequently, unit direction vectors, elevation angles and azimuth angles of neighborhood nodes are taken as prior geometric information, and convolution operation is performed on the generated convolution kernel to realize space geometry perception convolution operation. Subsequently, graph attention pooling is designed to coarsen the point cloud, realize multi-scale analysis and reduce the calculation cost. Finally, space geometry perception convolution and graph attention pooling are taken as basic units to construct two networks to realize point cloud classification and component segmentation tasks. The application can effectively classify and component segment target point clouds, and ensures the invariance of point cloud translation and scaling transformation, so that the model has stronger robustness.

Description

Technical Field

[0001] This invention relates to the field of 3D vision, specifically to a point cloud shape analysis method based on a spatial geometry-aware convolutional neural network. Background Technology

[0002] 3D vision has become a hot topic and is widely used in many emerging technologies, such as autonomous driving, virtual reality, and positioning and navigation. 3D data, as the foundation of 3D vision research, typically takes the following forms: multi-view images, voxels, meshes, and point clouds. Multi-view images represent 3D objects as 2D images taken from multiple perspectives, facilitating processing and analysis; however, they lack spatial geometric information and cannot accurately describe the essential properties of 3D objects. Voxels can be considered the basic units constituting 3D targets, but they are often limited by resolution and come with high computational costs. Mesh data is often used to visualize 3D shapes, but the data itself cannot be directly obtained from 3D scanning equipment and requires further abstraction. Conversely, point clouds are collections of points on the surface of 3D objects, describing the geometric information and spatial distribution of the 3D object, and can be directly obtained from 3D scanning equipment. Therefore, point cloud data has attracted increasing research interest.

[0003] Due to the significant scientific importance and wide application value of point cloud shape analysis, numerous methods have been proposed, including projection methods, point-wise MLP-based methods, convolutional methods, and graph-based methods. Projection-based methods typically transform point clouds into regularized data such as multi-view images or voxel grids, and then process and analyze them using standard 2D and 3D convolutional neural networks. However, since data transformation often results in the loss of information due to the destruction of 3D structures, subsequent research often employs more direct methods for feature extraction from point clouds. Point-wise MLP-based methods use a combination of multiple perceptrons and symmetric functions to extract point cloud features, addressing the permutation invariance of point clouds. Convolutional methods attempt to extend standard 2D convolution to point clouds based on their properties. Graph-based methods typically construct point clouds as graph-structured data and define convolution and pooling operations on the graph in the spatial domain to achieve feature extraction. However, these methods often lack consideration for the translation and scaling invariance of point clouds, meaning the models are susceptible to the effects of translation and scaling disturbances. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to design a spatially geometrically aware convolution operator that ensures the model remains invariant to translation and scaling transformations, thereby enhancing the model's robustness.

[0005] Another objective of this invention is to provide a unified framework for the classification and component segmentation of point clouds.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] This invention discloses a point cloud shape analysis method based on a spatial geometric perception graph convolutional neural network. The method includes the following steps: Step S1: Extract initial geometric features with translation and scale invariance through an initial geometric feature description module, and find each point p in the point cloud using the K-nearest neighbor algorithm. i The two nearest neighbors p i1 ,p i2 This allows for the construction of a series of triangular patches on the point cloud; p is then calculated using these triangular patches. i Find the nearest neighbor unit edge vector and normal vector, and use these as initial geometric features for input in subsequent steps;

[0008] Step S2: Based on the feature information in different neighborhoods of the point cloud, a shared mapping function g is used. (m) (·) Adaptively generate domain adaptive kernels with different geometries;

[0009] Step S3: Design a spatially aware convolution operator in the point cloud neighborhood, which computes the convolution of each neighborhood node p. ij The corresponding unit direction vector e ij Elevation angle θ ij e Azimuth θ ij a And it participates in the convolution operation as prior geometric information;

[0010] Step S4: Using random sampling and graph attention mechanisms, design graph attention pooling operations on the point cloud to coarsen the point cloud and aggregate neighborhood features;

[0011] Step S5: Stack multiple spatial geometry-aware convolutional layers and graph attention pooling layers to construct the feature extraction branch of the network;

[0012] Step S6: For the point cloud shape classification task, perform global max pooling on the features obtained from the feature extraction branch to obtain the overall shape descriptor of the point cloud, and input it into the fully connected network to achieve target classification;

[0013] Step S7: For the point cloud component segmentation task, design an unpooling operation corresponding to graph attention pooling to gradually decode the extracted features to their original size, and then classify each point through a fully connected network to achieve segmentation of different regions of the target.

[0014] Step S1 specifically includes: querying each point p in the point cloud according to the K-nearest neighbor algorithm.i The two nearest neighbor points p i1 p i2 A series of triangular patches are constructed on the point cloud, and then p is defined on each triangular patch. i Nearest neighbor edge vector and normal vector:

[0015]

[0016] "×" and ||·|| represent the cross product and magnitude of the vectors, respectively. This information is then used as the initial geometric features of the point cloud. As input for subsequent steps;

[0017] Step S2 specifically includes: for any point p i The neighborhood of a point cloud is determined using the K-nearest neighbor algorithm in 3D space. S represents the number of neighboring nodes, thus constructing the point cloud as a graph G(V, E), where V and E represent the vertex set and edge set, respectively. Then, within each neighborhood, the feature F(p) of the neighborhood is used... i ,S)=[f i1 f i2 , ..., f iS}∪{f i} with a shared mapping function g (m) (·) Adaptively generate convolution kernels that match the geometry of the neighborhood:

[0018]

[0019]

[0020] Here Δf ij =f ij -f i For node f ij Edge features, k ij (m) For the m-th convolution kernel K i (m) Core points within;

[0021] Step S3 includes: defining prior geometric information, that is, for the neighborhood Any node p in ij Calculate its unit direction vector e in the local coordinate system. ij Elevation angle θ ij e and azimuth θ ij a :

[0022]

[0023]

[0024]

[0025] Here Δx ij Δy ij Point p ij The relative coordinates on the X and Y axes are then used as input for convolution with the generated domain adaptive kernel:

[0026]

[0027] Step S4 includes: randomly sampling and coarsening the point cloud, and then for any point p... i ∈P down and its corresponding neighborhood The attention coefficients of different nodes are calculated using a graph attention model:

[0028]

[0029] Where, ω i0 Let φ and ψ be two linear mappings, and finally aggregate node features based on the importance reflected by the attention coefficients.

[0030] Step S5 includes: designing a feature extraction branch for point cloud based on the proposed related technologies, mainly including an initial geometric description module and five stacked spatial geometry-aware convolutional layers, and inserting a graph attention pooling layer after the second and fourth convolutional layers, and finally encoding the point cloud into a series of high-dimensional features;

[0031] For the point cloud shape classification task, step S6 includes: aggregating the encoded high-dimensional features into a high-dimensional vector through global max pooling operation and using it as the overall shape descriptor of the point cloud, and inputting it into a fully connected neural network to predict the category of the target;

[0032] For point cloud component segmentation tasks, the method is characterized by the following steps: Step S7 includes: gradually restoring the point cloud to its original size through an unpooling operation corresponding to graph attention pooling. This first involves interpolating the high-dimensional features using nearest-neighbor interpolation, then inputting them together with the encoded features from the previous stage into an attention block, and focusing on the encoded features.

[0033]

[0034]

[0035] Γ1 and Γ2 are two feature maps with ReLU activation function, and Γ3 is a feature map with Sigmoid activation function. The focused features are then concatenated with the interpolated features and input into a multilayer perceptron (MLP) to perform unpooling.

[0036]

[0037] The high-dimensional features obtained from the encoding branch are gradually decoded and the original size of the point cloud is restored through two unpooling operations with attention mechanism, thereby obtaining the feature representation of each point on the point cloud. Finally, each point is classified point by point through a fully connected neural network to achieve component segmentation of the target point cloud. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the initial geometric description module of the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the definition of neighborhood prior geometric information and the domain adaptive kernel of the present invention.

[0040] Figure 3 This is a schematic diagram of the spatial geometry-aware convolution process of the present invention.

[0041] Figure 4 This is a schematic diagram of the attention pooling operation.

[0042] Figure 5 The diagram shows the network structure for point cloud classification and point cloud component segmentation.

[0043] Figure 6 This is a flowchart of attention block and unpooling operations. Detailed Implementation

[0044] The following describes a point cloud shape analysis method based on a spatial geometry-aware graph neural network according to an embodiment of the present invention, with reference to the accompanying drawings. First, the initial feature description module, domain adaptive convolution kernel, spatial geometry-aware convolution, and graph attention pooling provided according to a design example of the present invention will be described with reference to the accompanying drawings. Finally, using these components as basic units, two networks are designed within a unified framework to achieve point cloud shape classification and component segmentation tasks. (Refer to...) Figure 1-6 As shown, the method includes the following steps:

[0045] S1 Initial Geometric Feature Description Module: Input point cloud P = {p1, p2, ..., p...} N}, and with any point p i Search for the distance p centered at the center using the K-nearest neighbor algorithm i The nearest point p i1 p i2 .like Figure 1 As shown, pi p i1 p i2 These three points form a triangular patch, and the initial geometric features of the point cloud are explicitly extracted as follows:

[0046]

[0047] Specifically, Where "×" represents the cross product of vectors, and ||·|| represents the magnitude of the vector. This concise formula allows for the extraction of initial geometric features from the point cloud for use in subsequent processes.

[0048] Construction of S2 domain adaptive convolutional kernel: For a given input point cloud or intermediate point cloud P, its corresponding feature is F (in the input layer, F is the initial geometric feature extracted by the initial geometric feature description module). The K-Nearest Neighbors (KNN) algorithm with self-loops is used to determine the neighborhood of each point, denoted as... Where S reflects the size of the neighborhood. Let F(p) i S)={f i1 f i2 , ..., f iS}∪{f i} represents the feature corresponding to each point in the neighborhood. Then, the feature f of the center point is... i With edge features Δf ij =f ij -f i，j =1, 2, ..., S are input into a shared mapping function g(·), thereby constructing a series of domain-adaptive convolutional kernels. Right now

[0049]

[0050] Where m is the index number of the D convolutional kernels, k ij (m) The weights are the weights of the convolution kernel, and "||" indicates the vector concatenation operation.

[0051] S3 spatial geometry-aware convolution operator: such as Figure 2 As shown, for each neighborhood on the point cloud The unit direction vector e corresponding to each neighboring point ij Elevation angle θ ij e and azimuth θ ij a As geometric prior information:

[0052]

[0053]

[0054]

[0055] Where Δx ij Δy ij Point p ij With point p i The coordinate difference between the X and Y axes. Then, this geometric prior information is used as input and convolved with the domain-adaptive convolution kernel generated in step 2.

[0056]

[0057] Where <·, ·> represent the dot product operation of vectors. Figure 3 The detailed process of spatial geometry-aware convolution operation is explained.

[0058] S4 Graph Attention Pooling Operation: Pooling operations are typically performed after convolutional layers, playing a crucial role in standard CNNs. In the proposed network architecture, this invention also incorporates pooling operations on the point cloud to achieve point cloud coarsening. For example... Figure 4 As shown, a subset of the point cloud is first obtained by randomly sampling from the original point cloud. ρ is the sampling rate. Then, for any point p in the subset of the point cloud... i Its corresponding neighborhood The attention coefficient for each node is calculated using a graph attention mechanism:

[0059]

[0060]

[0061] Where ω i0 φ and ψ are learnable weights, and φ and ψ are two independent linear mapping functions. These learned attention weights can be viewed as a set of soft masks that adaptively select important response features. Finally, fine-to-coarse graph pooling on the point cloud is achieved by aggregating the features of the leaf nodes through attention coefficients, i.e.:

[0062]

[0063] S5-S7 Point Cloud Classification and Segmentation Network: Using the initial geometric feature description module proposed in steps S1-S4, domain-adaptive convolutional kernels, spatial geometry-aware convolutions, and graph attention pooling operations as basic units, a network is constructed for point cloud classification and segmentation tasks, as follows: Figure 5As shown. In the classification task, the point cloud is represented as a high-dimensional vector through a series of convolution and pooling operations, and the point cloud is classified through a fully connected network. The feature extraction stage of the segmentation network adopts the same architecture as the classification network. The difference is that the segmentation task requires classifying each point in the point cloud. Therefore, to obtain the features of each point, this invention designs an additional unpooling operation to achieve coarse-to-fine point cloud refinement and restore the size of the point cloud, specifically as follows... Figure 6 As shown.

[0064] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A point cloud shape analysis method based on a spatial geometric perception map convolutional neural network, characterized in that, The method includes the following steps: Step S1: Extract initial geometric features with translation and scale invariance through the initial geometric feature description module, and find each point p in the point cloud using the K-nearest neighbor algorithm. i The two nearest neighbors p i1 ,p i2 This allows for the construction of a series of triangular patches on the point cloud; p is then calculated using these triangular patches. i Find the nearest neighbor unit edge vector and normal vector, and use these as initial geometric features for input in subsequent steps; Step S2: Based on the feature information in different neighborhoods of the point cloud, a shared mapping function g is used. (m) (·) Adaptively generate domain adaptive kernels with different geometries; Step S3: Design a spatially aware convolution operator in the point cloud neighborhood, which computes the convolution of each neighborhood node p. ij The corresponding unit direction vector e ij Elevation angle θ ij e Azimuth θ ij a And it participates in the convolution operation as prior geometric information; Step S4: Using random sampling and graph attention mechanisms, design graph attention pooling operations on the point cloud to coarsen the point cloud and aggregate neighborhood features; Step S5: Stack multiple spatial geometry-aware convolutional layers and graph attention pooling layers to construct the feature extraction branch of the network; Step S6: For the point cloud shape classification task, perform global max pooling on the features obtained from the feature extraction branch to obtain the overall shape descriptor of the point cloud, and input it into the fully connected network to achieve target classification; Step S7: For the point cloud component segmentation task, design an unpooling operation corresponding to graph attention pooling to gradually decode the extracted features to their original size, and then classify each point through a fully connected network to achieve segmentation of different regions of the target.

2. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S1 includes: querying each point p in the point cloud according to the K-nearest neighbor algorithm. i The two nearest neighbor points p i1 p i2 A series of triangular patches are constructed on the point cloud, and then p is defined on each triangular patch. i Nearest neighbor edge vector and normal vector: "×" and ||·|| represent the cross product and magnitude of the vectors, respectively. This information is then used as the initial geometric features of the point cloud. As input for subsequent steps.

3. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S2 includes: for any point p i The neighborhood of a point cloud is determined using the K-nearest neighbor algorithm in 3D space. S represents the number of neighboring nodes, thus constructing the point cloud as a graph G(V, E), where V and E represent the vertex set and edge set, respectively. Then, within each neighborhood, the feature F(p) of the neighborhood is used... i S)={f i1 f i2 , ..., f iS }∪{f i } with a shared mapping function g (m) (·) Adaptively generate convolution kernels that match the geometry of the neighborhood: Here Δf ij =f ij -f i For node f ij Edge features, k ij (m) For the m-th convolution kernel K i (m) The core point within.

4. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S3 includes: defining prior geometric information, for the neighborhood... Any node p in ij Calculate its unit direction vector e in the local coordinate system. ij Elevation angle θ ij e and azimuth θ ij a : Here Δx ij Δy ij Point p ij The relative coordinates on the X and Y axes are then used as input for convolution with the generated domain adaptive kernel:

5. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S4 includes: randomly sampling and coarsening the point cloud, and then for any point p i ∈P down and its corresponding neighborhood The attention coefficients of different nodes are calculated using a graph attention model: Where, ω i0 Let φ and ψ be two linear mappings, and finally aggregate node features based on the importance reflected by the attention coefficients.

6. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S5 includes: an initial geometry description module, and five stacked spatial geometry-aware convolutional layers, with a graph attention pooling layer inserted after the second and fourth convolutional layers, ultimately encoding the point cloud into a series of high-dimensional features.

7. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S6 includes: aggregating the encoded high-dimensional features into a high-dimensional vector through a global max pooling operation and using it as the overall shape descriptor of the point cloud, and inputting it into a fully connected neural network to predict the category of the target.

8. The point cloud shape analysis method based on a spatial geometric perception map convolutional neural network according to claim 1, characterized in that, Step S7 includes: gradually restoring the point cloud to its original size through an unpooling operation corresponding to graph attention pooling. This first involves interpolating the high-dimensional features using nearest-neighbor interpolation, then inputting them together with the encoded features from the previous stage into an attention block, and focusing on the encoded features. Γ1 and Γ2 are two feature maps with ReLU activation function, and Γ3 is a feature map with Sigmoid activation function. The focused features are then concatenated with the interpolated features and input into a multilayer perceptron (MLP) to perform unpooling. The high-dimensional features obtained from the encoding branch are gradually decoded and the original size of the point cloud is restored through two unpooling operations with attention mechanism, thereby obtaining the feature representation of each point on the point cloud. Finally, each point is classified point by point through a fully connected neural network to achieve component segmentation of the target point cloud.

Images