Compensated attention classification method adapted to lossy point clouds
Patent Information
- Application Number
- CN202311561989.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-21
- Publication Date
- 2026-10-09
- Estimated Expiration
- 2043-11-21
AI Technical Summary
按照点的方法可以分为点卷积神经网络、图神经网络及逐点多层感知机法；点卷积神经网络通常分为连续卷积核与离散卷积核，对硬件的计算能力要求较高；图神经网络方法将原始点云建模为图神经网络，但模型对于有损点云的噪声敏感，无法良好地应用到有损点云目标中；逐点多层感知机方法可以较为良好地平衡精度与计算成本，不需要复杂地映射与处理，但其收敛精度仍需要改进，在有损点云或噪声较大的情况下，模型不够稳定
[0026] 1. Most existing technologies require the division and regularization of the original point cloud, while this invention makes full use of the advantages of the point-by-point multilayer perceptron, which makes the point cloud processing speed faster and the computational cost required for model processing lower.
Smart Images
Figure CN117422933B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a three-dimensional point cloud processing method, specifically to a compensatory attention classification method adapted to lossy point clouds. Background Technology
[0002] Due to the complexity of the scene, sensor aging, and improper data processing, raw point cloud data inevitably suffers from corruption. Current mainstream models often exhibit significant performance fluctuations under lossy point cloud conditions, failing to accurately reflect their performance in real-world environments. General attention methods are also weak in capturing local features of lossy point clouds, making it difficult to meet the requirements for accurate spatial feature description.
[0003] Deep learning methods for point clouds have been maturely applied in fields such as autonomous driving and remote sensing mapping. The implementation of multi-view point cloud processing methods requires jointly optimizing all functions using multiple perspectives of the same target. Furthermore, each perspective's point cloud data has defects, such as occlusion and noise. Effectively fusing and completing this data is a key issue. Voxel-based point cloud processing methods divide the point cloud into volumetric units, thereby regularizing the disordered point cloud and applying convolution to 3D targets. However, related algorithms and methods not only need to efficiently process large-scale data but also require significant model computation costs. Based on point methods, they can be divided into point convolutional neural networks, graph neural networks, and point-wise multilayer perceptron methods. Point convolutional neural networks are usually divided into continuous convolutional kernels and discrete convolutional kernels, which have high requirements for hardware computing power. Graph neural network methods model the original point cloud as a graph neural network, but the model is sensitive to noise in lossy point clouds and cannot be well applied to lossy point cloud targets. Point-wise multilayer perceptron methods can balance accuracy and computational cost relatively well and do not require complex mapping and processing, but their convergence accuracy still needs to be improved, and the model is not stable enough in the case of lossy point clouds or high noise.
[0004] Most of the aforementioned point cloud deep learning methods require projection and partitioning of the original point cloud, i.e., the unordered point cloud needs to be ordered, or they are sensitive to lossy point cloud objects. Therefore, these methods are suitable for scenarios with low real-time requirements and low hardware computing power. Summary of the Invention
[0005] Purpose of the invention: To propose a compensatory attention classification method adapted to lossy point clouds, so as to solve the problems existing in the background technology.
[0006] Technical solution: A compensatory attention classification method adapted to lossy point clouds, comprising the following steps:
[0007] (1) The original point cloud coordinates are projected by a multilayer perceptron to obtain the initial features of the point cloud;
[0008] (2) The original point cloud coordinates and initial point cloud features are sampled and grouped by a grouping layer that combines farthest point sampling and k nearest neighbor sampling to obtain the grouped local coordinates and local features.
[0009] (3) Establish an attention model for coordinates and features, project the grouped local coordinates and local features into a high-dimensional space, and use an attention mechanism to dynamically adjust the local coordinates and local features respectively. Use a learnable matrix to enhance the expression of local coordinates and local features through matrix learning, and adaptively adjust the distribution of high-dimensional features and coordinate information. Use cross attention to associate coordinate information with features and output the neighborhood features enhanced with spatial information.
[0010] (4) Project the neighborhood features into a low-dimensional space and compensate with the neighborhood coordinates additively to obtain the biased amount after coordinate compensation; strengthen the neighborhood coordinates through aggregation operation; project the strengthened neighborhood coordinates into the feature space to obtain the projection bias of the coordinates from low dimension to high dimension; stitch together the enhanced local features through aggregation operation; project the enhanced local features through a multilayer perceptron to obtain the global coordinates.
[0011] (5) Project the global features sequentially to the 512-dimensional, 256-dimensional and classification target number dimensions.
[0012] In one embodiment, batch norm is used to increase the speed of the multilayer perceptron model and the ReLU activation function is used to increase the nonlinearity of the projection.
[0013] In one embodiment, the sampling and grouping process of the original point cloud coordinates in step (2) is as follows:
[0014] For the original point cloud, the farthest point is sampled to obtain the local sampling center of the original point cloud coordinates. Then, for each local sampling center of the original point cloud coordinates, the k-nearest neighbor algorithm is used to find the nearest neighbor point of the center point. The coordinates of the nearest neighbor point are subtracted from the coordinates of the center point to obtain the relative coordinates of the nearest neighbor space. The dimensions are then concatenated with the coordinates of the center point and the coordinates of the nearest neighbor point to obtain the nine-dimensional local coordinates.
[0015] In one embodiment, the sampling and grouping process of the initial point cloud features in step (2) is as follows:
[0016] The sampling grouping layer samples the farthest point from the initial features of the point cloud to obtain the local sampling center of the initial features of the point cloud. Then, the K-nearest neighbor algorithm is used to find the features of the nearest neighbor points of the center for each sampling center. The features of the nearest neighbor points are subtracted from the features of the center point to obtain the relative features of the nearest neighbor space. The relative features of the nearest neighbor points are then concatenated with the features of the center point and the features of the nearest neighbor points to obtain the local coordinates with three times the latitude of the initial features of the point cloud.
[0017] In one embodiment, the attention model in step (3) includes a first branch and a second branch, which are used to process the local coordinates and local features, respectively; specifically
[0018] The first branch projects local features into a high-dimensional space through max pooling and multilayer perceptron, uses an attention mechanism to perform secondary representation of local features, and dynamically adjusts local features by combining a learnable matrix, outputting enhanced local high-dimensional features.
[0019] The second branch projects the local coordinates into a high-dimensional space using max pooling and multilayer perceptron, uses an attention mechanism to perform a secondary representation of the local coordinates, and combines a learnable matrix to dynamically adjust the local coordinates, outputting the enhanced local high-dimensional coordinates.
[0020] Based on a cross-attention mechanism, local features and local coordinates are associated to output spatially enhanced neighborhood features. Preferably, the attention mechanism is a multi-head attention mechanism.
[0021] Furthermore, the cross-attention mechanism in step (3) associates local features and local coordinates to obtain spatially enhanced neighborhood features. The specific process is as follows:
[0022] Project local features into value vectors, and project local coordinates into query vectors and key vectors;
[0023] The initial weight matrix is obtained by multiplying the key vector and the query vector, and the final weight matrix is obtained by adjusting the vector length and normalization function. The weight matrix is then multiplied by the value vector, and enhanced local features, i.e. neighborhood features, are obtained through weight adjustment and additive compensation.
[0024] Preferably, in step (5), the multilayer perceptron uses Leaky ReLU as the activation function.
[0025] Compared with existing methods, the present invention has the following advantages:
[0026] 1. Most existing technologies require the division and regularization of the original point cloud, while this invention makes full use of the advantages of the point-by-point multilayer perceptron, which makes the point cloud processing speed faster and the computational cost required for model processing lower.
[0027] 2. Existing technologies are sensitive to lossy point clouds and have difficulty meeting the accuracy requirements. However, this invention, due to its reasonable local feature description, can better utilize the spatial information of point clouds and greatly improve the feature description capability for lossy point clouds.
[0028] 3. This invention does not require complex feature description and repetitive calculation of point cloud targets. It mainly uses the basic multilayer perceptron method supplemented by the attention mechanism, and can achieve better results with fewer model parameters. Attached Figure Description
[0029] Figure 1 This is a flowchart of a compensatory attention classification method for adapting to lossy point clouds in one embodiment;
[0030] Figure 2 This is an example of the process by which the attention model enhances and adjusts local features and local coordinates. Detailed Implementation
[0031] The technical solution of the present invention will be explained and described in detail below with reference to the accompanying drawings and specific embodiments.
[0032] like Figure 1 As shown, a compensatory attention classification method adapted to lossy point clouds includes the following steps:
[0033] (1) The original point cloud coordinates are projected by a multilayer perceptron to obtain the initial features of the point cloud;
[0034] (2) The original point cloud coordinates and initial point cloud features are sampled and grouped by a grouping layer that combines farthest point sampling and K-nearest neighbor sampling operations to obtain the grouped local coordinates and local features; specifically
[0035] The sampling and grouping process of the original point cloud coordinates is as follows:
[0036] For the original point cloud, the farthest point is sampled to obtain the local sampling center of the original point cloud coordinates. Then, for each local sampling center of the original point cloud coordinates, the K-nearest neighbor algorithm is used to find the nearest neighbor of the center point. The coordinates of the nearest neighbor are subtracted from the coordinates of the center point to obtain the relative coordinates of the nearest neighbor space. The dimensions are then concatenated with the coordinates of the center point and the coordinates of the nearest neighbor to obtain the nine-dimensional local coordinates.
[0037] The sampling and grouping process of the initial features of the point cloud refers to the processing process of the original point cloud coordinates. The sampling and grouping layer samples the farthest point from the initial features of the point cloud to obtain the local sampling center of the initial features of the point cloud. Then, the K-nearest neighbor algorithm is used to find the features of the nearest neighbor points of the center for each sampling center. The features of the nearest neighbor points are subtracted from the features of the center point to obtain the relative features of the nearest neighbor space. The relative features of the nearest neighbor space are then concatenated with the features of the center coordinates and the features of the nearest neighbor points to obtain the local coordinates with three times the latitude of the initial features of the point cloud.
[0038] (3) Considering that the key to processing lossy point clouds lies in the use of spatial information, it is necessary to interact the adjusted point cloud coordinate information with the point cloud feature information in order to compensate for the target features of the lossy point cloud through coordinate information; specifically
[0039] An attention model for coordinates and features is established. The grouped local coordinates and local features are projected into a high-dimensional space using FFN (i.e., multi-layer perceptron). An attention mechanism (multi-head attention mechanism can be used) is used to dynamically adjust the local coordinates and local features respectively. A learnable matrix is used to enhance the representation of local coordinates and local features through matrix learning, and the distribution of high-dimensional features and coordinate information is adaptively adjusted. Cross attention is used to associate coordinate information with features, and the spatial information-enhanced neighborhood features are output.
[0040] In one embodiment, the attention model in step (3) includes a first branch and a second branch, which are used to process the local coordinates and local features, respectively; such as Figure 2 The diagram illustrates the process of enhancing, adjusting, and associating features and coordinates using the attention model.
[0041] The first and second branches project local features and local coordinates into a high-dimensional space using max pooling and multilayer perceptron, respectively. They then use an attention mechanism to perform secondary representation of the local features and local coordinates, and dynamically adjust the local features and local coordinates using a learnable matrix, outputting enhanced local high-dimensional features and local high-dimensional coordinates. Finally, based on a cross-attention mechanism, the local features and local coordinates are associated, outputting spatially enhanced neighborhood features.
[0042] Furthermore, the specific process of associating local features and local coordinates based on the cross-attention mechanism to obtain spatially enhanced neighborhood features is as follows:
[0043] The first and second branches use max pooling and FFN to extract local features f. G Local coordinates p G Projecting onto a higher-dimensional space, we obtain the local features ψ in the higher-dimensional space. f Local coordinates ψ p .
[0044] Combining the learnable matrix H with local features ψ in high-dimensional space f and local coordinates ψ p Dynamic adjustments are made to obtain the enhanced and adjusted local features f. m With local coordinates P m The formula is as follows:
[0045] ψ f =FFN(MaxP(f G ))
[0046] ψ p =FFN(MaxP(p G ))
[0047] fm =MAB2{MAB1(ψ f ,H),ψ f}
[0048] p m =MAB2{MAB1(ψ p ,H),ψ p}
[0049] Among them, MAB1(ψ f H), MAB1(ψ p H) are local features ψ f Local coordinates ψ p The middle representation.
[0050] The enhanced and adjusted local features f m and local coordinates P m Projected onto f v p Q P K space:
[0051] f v =FFN(f m )
[0052] p Q =FFN(p m )
[0053] p K =FFN(p m )
[0054] Where Q, K, and V correspond to query, key, and value, respectively, and local feature f m Projected as value vector f V Local coordinates P m Projected as key vector p K With query vector p Q .
[0055] By querying vector p Q With the key vector p k The product of these terms yields the initial weight matrix, which is then adjusted using the vector length and a normalization function to obtain the final weight matrix p. A ; the weight matrix p A And value vector f V Multiplication, through weight adjustment and additive compensation, yields enhanced local features, namely neighborhood features F. GCA The calculation formula is as follows:
[0056]
[0057]
[0058] in, p represents element-wise multiplication. A dD is the weight matrix for coordinate similarity, and dD is the spatial dimension after FFN projection.
[0059] (4) Since there is less redundant information when high-dimensional features are mapped to low-dimensional space, which is insufficient to resist the information loss caused by nonlinear units, this invention starts from the perspective of feature dimension transformation and, based on a bilateral compensation structure, improves the computational power of the model by avoiding complex local computational operations, in order to resist the information loss caused by nonlinear units when the dimension changes; specifically
[0060] Neighborhood features are projected onto 3D coordinates using a multilayer perceptron. An aggregation operation is then used to enhance the coordinate information, and the enhanced coordinates are projected back onto the feature dimension using another multilayer perceptron, yielding the projection bias from low to high dimensions. The enhanced features are then concatenated using the aggregation operation and projected back onto the global features using another multilayer perceptron. The formula is as follows:
[0061] The neighborhood features are projected into a low-dimensional space and additively compensated using neighborhood coordinates to obtain the biased value after coordinate compensation:
[0062]
[0063]
[0064] Where p knn This represents the neighborhood coordinates obtained by k-nearest neighbor sampling of the coordinates; M represents the number of center points of the sampling; K represents the number of points in the neighborhood obtained by sampling; 3 represents the dimension of the coordinate information;
[0065] Strengthen the coordinates of the points through aggregation operations:
[0066]
[0067] Where p Gc p represents the coordinates of the neighborhood center. Gi p represents neighborhood coordinates. Gc ∈R M×3 .
[0068] From reinforced coordinates Projecting onto the feature space, the resulting neighborhood offset features are:
[0069]
[0070]
[0071] Where f knnThis represents the neighborhood features obtained by sampling the feature from its k nearest neighbors, where d represents the latitude of the neighborhood features.
[0072] The enhanced local features are:
[0073]
[0074]
[0075] Where f Gi f represents any point in the neighborhood. Gc This represents the center point of the neighborhood.
[0076] Finally Global features F are obtained by projecting the layered perceptron. G .
[0077] (5) The decoder uses a multilayer perceptron three times to project the global features into 512-dimensional, 256-dimensional and classification target number dimensions in turn.
[0078] Optionally, the above steps can use batch norm to increase model speed and ReLU activation function to increase projection nonlinearity; in step (5), the multilayer perceptron can use Leaky ReLU as activation function.
[0079] Table 1 shows the classification error rate and number of parameters of different models in an embodiment of the present invention on the lossy dataset ModelNet40C. Double underlines represent the best performance in the experiment, and single underlines represent the second-best performance.
[0080] Table 1. Classification error rate and number of parameters for different models on the lossy ModelNet40C dataset.
[0081]
[0082] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A compensatory attention classification method adapted to lossy point clouds, characterized in that, Includes the following steps: (1) The original point cloud coordinates are projected by a multilayer perceptron to obtain the initial features of the point cloud; (2) The original point cloud coordinates and initial point cloud features are sampled and grouped by a grouping layer that combines farthest point sampling and k nearest neighbor sampling to obtain the grouped local coordinates and local features. (3) Establish an attention model for coordinates and features, project the grouped local coordinates and local features into a high-dimensional space, and use an attention mechanism to dynamically adjust the local coordinates and local features respectively. Use a learnable matrix to enhance the expression of local coordinates and local features through matrix learning, and adaptively adjust the distribution of high-dimensional features and coordinate information. Use cross attention to associate coordinate information with features and output the neighborhood features enhanced with spatial information. (4) Project the neighborhood features into a low-dimensional space and compensate with the neighborhood coordinates additively to obtain the biased amount after coordinate compensation; strengthen the neighborhood coordinates through aggregation operation; project the strengthened neighborhood coordinates into the feature space to obtain the projection bias of the coordinates from low dimension to high dimension; stitch together the enhanced local features through aggregation operation; project the enhanced local features through a multilayer perceptron to obtain the global coordinates. (5) Project the global features sequentially to the 512-dimensional, 256-dimensional and classification target number dimensions.
2. The compensated attention classification method according to claim 1, characterized in that, Using batch norm increases the speed of the multilayer perceptron model, and the ReLU activation function increases the nonlinearity of the projection.
3. The compensated attention classification method according to claim 1, characterized in that, In step (2), the sampling and grouping process of the original point cloud coordinates is as follows: For the original point cloud, the farthest point is sampled to obtain the local sampling center of the original point cloud coordinates. Then, for each local sampling center of the original point cloud coordinates, the k-nearest neighbor algorithm is used to find the nearest neighbor point of the center point. The coordinates of the nearest neighbor point are subtracted from the coordinates of the center point to obtain the relative coordinates of the nearest neighbor space. The dimensions are then concatenated with the coordinates of the center point and the coordinates of the nearest neighbor point to obtain the nine-dimensional local coordinates.
4. The compensated attention classification method according to claim 1, characterized in that, In step (2), the sampling and grouping process of the initial features of the point cloud is as follows: The sampling grouping layer samples the farthest point from the initial features of the point cloud to obtain the local sampling center of the initial features of the point cloud. Then, the K-nearest neighbor algorithm is used to find the features of the nearest neighbor points of the center for each sampling center. The features of the nearest neighbor points are subtracted from the features of the center point to obtain the relative features of the nearest neighbor space. The relative features of the nearest neighbor points are then concatenated with the features of the center point and the features of the nearest neighbor points to obtain the local coordinates with three times the latitude of the initial features of the point cloud.
5. The compensated attention classification method according to claim 1, characterized in that, In step (3), the attention model includes a first branch and a second branch, which are used to process the local coordinates and local features, respectively; specifically The first branch projects local features into a high-dimensional space through max pooling and multilayer perceptron, uses an attention mechanism to perform secondary representation of local features, and dynamically adjusts local features by combining a learnable matrix, outputting enhanced local high-dimensional features. The second branch projects the local coordinates into a high-dimensional space using max pooling and multilayer perceptron, uses an attention mechanism to perform a secondary representation of the local coordinates, and combines a learnable matrix to dynamically adjust the local coordinates, outputting the enhanced local high-dimensional coordinates. Based on the cross-attention mechanism, local features and local coordinates are associated to output spatially enhanced neighborhood features.
6. The compensated attention classification method according to claim 1, characterized in that, In step (3), the attention mechanism is a multi-head attention mechanism.
7. The compensated attention classification method according to claim 5, characterized in that, The cross-attention mechanism associates local features with local coordinates to obtain spatially enhanced neighborhood features. The specific process is as follows: Project local features into value vectors, and project local coordinates into query vectors and key vectors; The initial weight matrix is obtained by multiplying the key vector and the query vector, and the final weight matrix is obtained by adjusting the vector length and normalization function. The weight matrix is then multiplied by the value vector, and enhanced local features, i.e. neighborhood features, are obtained through weight adjustment and additive compensation.
8. The compensated attention classification method according to claim 1, characterized in that, In step (5), the multilayer perceptron uses Leaky ReLU as the activation function.