Large-scale point cloud scene recognition method based on discriminative region feature learning
By constructing a discriminative region-guided Transformer network, the problem of the lack of discriminativeness of local features in point cloud scene recognition is solved, achieving higher recognition accuracy and fewer parameters, thus improving the effect of large-scale point cloud scene recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2024-06-28
- Publication Date
- 2026-07-24
AI Technical Summary
Existing large-scale point cloud scene recognition methods in outdoor point clouds suffer from limited receptive fields, making it difficult to include global context. This results in a lack of discriminative power in local features, and the self-attention layer aggregates noise information, affecting recognition accuracy.
We construct a Transformer network guided by discriminative regions, obtain clues about discriminative regions through a global aggregator, perform long-distance contextual interactions, and use a lightweight global aggregator to generate global descriptors, eliminating the influence of non-discriminative regions and improving the discriminativeness of local features.
It improves the accuracy of point cloud scene recognition, reduces the number of parameters, lowers the risk of overfitting, effectively distinguishes diverse local features, and enhances the recognition effect.
Smart Images
Figure CN118865298B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision technology, specifically, it is a method for large-scale point cloud scene recognition based on discriminative region feature learning. Background Technology
[0002] Large-scale point cloud scene recognition aims to find the point cloud most similar to the currently queried point cloud in a map database, thereby obtaining the approximate location of the current point cloud. This task is of great significance for promoting the practical application and development of mobile robots and autonomous driving. It can assist mobile robots or autonomous vehicles in determining their position in a scene, thus achieving more accurate navigation and path planning.
[0003] Most point cloud scene recognition methods use point cloud feature extraction networks to extract local features, and then use a global aggregator to obtain a global scene descriptor. However, due to the sparsity of outdoor point clouds, ordinary point cloud feature extraction networks have limited receptive fields, making it difficult to include global context to extract discriminative local features. Subsequently, PPTNet, the closest method to this invention, introduced a Transformer to capture global context information, thereby making the extracted local features more discriminative. Specifically, it adds a self-attention layer after the local feature extraction module at each stage of the network to establish long-range dependencies. However, since there are many time-varying objects in the scene, such as cars and pedestrians on the road, directly using the self-attention layer will also aggregate this noise information into the local features, thus impairing the discriminativeness of the local features and leading to incorrect scene recognition results. Summary of the Invention
[0004] The purpose of this invention is to provide a large-scale point cloud scene recognition method based on discriminative region feature learning, in order to solve the problems existing in the existing methods.
[0005] The technical solution for achieving the objective of this invention is: a method for large-scale point cloud scene recognition based on discriminative region feature learning, specifically including the following steps:
[0006] Step 1: Construct a dataset based on continuous frame street data with location coordinates collected by radar on robots or autonomous vehicles;
[0007] Step 2: Construct a discriminative region-guided Transformer network to obtain point-by-point local features of the point cloud;
[0008] Step 3: Based on the training set in Step 1 and the triplet loss for batch hard sample mining, train the discriminative region-guided Transformer network.
[0009] Step 4: Construct a global scene description sub-database using a trained discriminative region-guided Transformer network;
[0010] Step 5: For the query point cloud, use the trained discriminative region-guided Transformer network to obtain the global descriptor of the point cloud, and perform a nearest neighbor query in the scene global descriptor database to obtain the scene recognition result.
[0011] Compared with the prior art, the significant advantages of this invention are:
[0012] (1) This invention constructs a discriminative region-guided attention mechanism. It obtains clues for identifying discriminative regions in a scene through a global aggregator, then performs long-distance contextual interactions within these discriminative regions, and injects this long-distance contextual information into local features through a cross-attention mechanism to improve the discriminativeness of local features. Compared with previous methods, this invention improves the discriminativeness of local features by eliminating the influence of non-discriminative region point clouds such as moving cars and pedestrians on feature extraction, thereby improving the accuracy of the final scene recognition.
[0013] (2) This invention constructs a lightweight global aggregator that can directly aggregate local features of point clouds into a compact global descriptor and allows for more codewords, thereby further improving the effect of global aggregation. Compared with previous methods, this invention significantly reduces the number of parameters, thereby mitigating the impact of overfitting, and allows for more codewords, thus effectively distinguishing diverse local features and improving the accuracy of final scene recognition. Attached Figure Description
[0014] Figure 1 This is an overall framework diagram of the Transformer network guided by the discriminative region in the method of this invention.
[0015] Figure 2 This is a framework diagram of the adaptive local coding module in the method of this invention.
[0016] Figure 3 This is a framework diagram of the discriminative region-guided attention module in the method of this invention.
[0017] Figure 4 This is the overall flowchart of the method of the present invention. Detailed Implementation
[0018] To reduce the impact of non-discriminative point cloud regions such as moving cars and pedestrians on feature extraction, the method of this invention uses a global aggregator to obtain clues for identifying discriminative regions in the scene, and performs global feature interactions only in these discriminative regions. Finally, this global information is propagated to local features to enhance their discriminativeness. Furthermore, to mitigate the overfitting effects of existing global aggregators, this method provides a lightweight global aggregator.
[0019] This invention provides a technical solution: a method for large-scale point cloud scene recognition based on discriminative region feature learning, comprising the following steps:
[0020] Step 1: Construct a dataset based on continuous frame street data with location coordinates collected by radar on robots or autonomous vehicles. The specific process is as follows:
[0021] Step 1.1: Set a keyframe point cloud every α meters, and downsample the keyframe point cloud to N points using voxel downsampling.
[0022] Step 1.2: For a given frame of point cloud, point clouds within a distance of less than α meters are designated as positive samples, and point clouds within a distance of greater than β meters (α < β) meters are designated as negative samples. Record the point cloud indices of the positive and negative samples, and save a set of positive samples and a set of negative samples for each point cloud.
[0023] Step 2: Construct a discriminative region-guided Transformer network to acquire point-by-point local features of the point cloud. This network includes an adaptive local encoding module, a discriminative region-guided attention module, and a feature propagation module. The discriminative region-guided Transformer network comprises three feature downsampling stages, each including an adaptive local encoding module and a discriminative region-guided attention module. These stages yield multi-scale local features. These multi-scale local features will then be input into the feature propagation module and gradually upsampled to the original resolution.
[0024] Step 2.1: Capture local features of the point cloud using an adaptive local coding module. The specific process is as follows:
[0025] First, input the point cloud of this layer. Downsampling to 1 / 4 of the original point cloud size yields the coordinates of the downsampled point cloud. and characteristics Where N l Let d be the number of point clouds input at the current stage, and d be the feature dimension. If no features have been extracted from the input point cloud, then the initial features are set to the coordinates of the point cloud, and a multilayer perceptron is used to increase the dimension to d, i.e., F = MLP(X). The KNN algorithm is used to find the L nearest neighbors of the center point cloud in the input point cloud at this layer. Next, new local features can be obtained by aggregating neighborhood information based on the feature differences between the center point and neighboring points. The specific process is as follows:
[0026] w ij =softmax(MLP([MLP(f j -f i ),MLP(x j -x i )]))
[0027]
[0028] Where MLP stands for Multilayer Perceptron, [·] represents cascade operation, and ⊙ represents element-wise multiplication.
[0029] Step 2.2: Capture discriminative long-range contextual information through an attention module guided by discriminative regions. The specific process is as follows:
[0030] Step 2.2.1: Obtain the center of the discriminative region;
[0031] For the local features updated by the adaptive local coding module Codewords using the global aggregator Calculate the assigned weights This allows us to determine the importance of each local feature:
[0032]
[0033] in Next, for each codeword, the point with the largest assigned weight is selected as the center of the discriminative region. The features of the centers of the K discriminative regions can be obtained according to the following formula. and coordinates
[0034]
[0035] Where I represents the index of the center point of the selected discriminative region, X s [I] indicates that the coordinates of the corresponding center point are selected based on the subscript.
[0036] Step 2.2.2: Obtain the feature description of the discriminative region;
[0037] This step requires feature description of the discriminative region, which can be achieved using an adaptive local coding module for feature aggregation of local features.
[0038]
[0039] Here, the central features of the discriminative region are input to the adaptive local coding module, so the downsampling operation can be omitted in the adaptive local coding module.
[0040] Step 2.2.3: Long-distance interaction in the discriminative region;
[0041] To eliminate the influence of non-discriminative region point clouds such as moving cars and pedestrians on feature extraction, long-distance contextual information interaction is performed only between discriminative region features. The specific interaction process is as follows: discriminative region features are input as query, key, and value features respectively into a multi-head self-attention layer; the resulting features are added to the original discriminative region features; and then passed through a LayerNorm layer to obtain discriminative region features with global contextual information.
[0042]
[0043] Where LN represents LayerNorm.
[0044] Step 2.2.4: Long-distance propagation of contextual information in the discriminative region;
[0045] To enhance local features with long-range contextual information, both local features and discriminative region features are fed into a cross-attention layer. This injects the long-range contextual information of the discriminative region into the local features, resulting in enhanced local features.
[0046]
[0047] Local features Discriminative region features as query features They are used as key and value features input into the cross-attention mechanism.
[0048] Step 2.3: By repeatedly performing the process in Step 2.2, point cloud features at multiple scales can be obtained. These features are then fed into the feature propagation module and continuously upsampled to the original point cloud size. One stage of the feature propagation process can be described as follows:
[0049]
[0050] Here, FP represents the feature propagation layer, which is typically instantiated as a feature propagation layer in PointNet++. Ultimately, this will result in the local features of the point cloud.
[0051] Step 2.4: Generate a global descriptor;
[0052] Next, the local features F are input into a global aggregator to obtain the global aggregated descriptor. Existing global aggregators like NetVLAD typically generate a long global feature set, which is then reduced in dimensionality using a fully connected layer. The large number of parameters in the fully connected layer leads to severe overfitting. Therefore, this method employs a simple and effective global aggregator: LightVLAD. Like NetVLAD, LightVLAD defines a set of learnable codewords. The weights for codewords and local features are obtained using the following formula.
[0053]
[0054] in LightVLAD aggregates local features into global descriptors by assigning weights.
[0055] g = [A1FW1, ..., A K FW K ]
[0056] Where [·] represents a cascading operation. These are learnable parameters.
[0057] Step 3: Train a discriminative region-guided Transformer network;
[0058] Each time, a batch of point clouds is randomly selected and input into a discriminative region-guided Transformer network to obtain a global descriptor for each point cloud. This method uses a triplet loss function for batch hard sample mining to optimize the discriminative region-guided Transformer network.
[0059] L=max(m+d(g,g p )-d(g,g n ),0)
[0060] Where d(·) represents the L2 distance of the feature, m is a predefined interval, and g p G represents the most difficult positive sample in the batch, that is, the positive sample with the largest distance from the current sample. n This represents the most difficult negative sample in the batch, which is the negative sample that is closest to the current sample.
[0061] The loss function is calculated using the above formula. The network parameters in the implicit scene completion model are updated using gradient descent until the model converges, resulting in a trained discriminative region-guided Transformer network.
[0062] Step 4: Construct a global descriptor database for the scene using the trained discriminative region-guided Transformer network; for point cloud data of a new scene, use the discriminative region-guided Transformer network to obtain and save the global descriptor for each point cloud in the dataset.
[0063] Step 5: For the query point cloud, use the trained discriminative region-guided Transformer network to obtain the global descriptor of the point cloud, and perform a nearest neighbor query in the scene global descriptor database to obtain the scene recognition result.
[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments. The described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0065] like Figure 4 As shown, a large-scale point cloud scene recognition method based on discriminative region feature learning includes the following steps:
[0066] Step 1: Construct a dataset based on continuous frame street data with location coordinates collected by radar on robots or autonomous vehicles. The specific process is as follows:
[0067] Step 1.1: Set a keyframe point cloud every η meters, and downsample the keyframe point cloud to N points using voxel downsampling, where η is set to 10 and N is set to 4096;
[0068] Step 1.2: For a given frame of point cloud, point clouds within a distance of less than α meters are designated as positive samples, and point clouds within a distance of more than β meters (α < β) meters are designated as negative samples, where α and β are set to 20 and 50, respectively. Record the point cloud indices of the positive and negative samples, and save a set of positive and negative samples for each point cloud.
[0069] Step 2: See Figure 1 A discriminative region-guided Transformer network is constructed to acquire point-by-point local features of the point cloud. Its network structure is as follows: Figure 2 As shown, this includes an adaptive local encoding module, a discriminative region-guided attention module, and a feature propagation module. See [link to documentation]. Figure 3 The discriminative region-guided Transformer network consists of three feature downsampling stages, each including an adaptive local encoding module and a discriminative region-guided attention module. These stages yield multi-scale local features. These multi-scale local features will then be input into the feature propagation module and gradually upsampled to the original resolution.
[0070] Step 2.1: Capture local features of the point cloud using an adaptive local coding module. The specific process is as follows:
[0071] First, input the point cloud of this layer. Downsampling to 1 / 4 of the original point cloud size yields the coordinates of the downsampled point cloud. and characteristics Where N l Let d be the number of point clouds input at the current stage, and d be the feature dimension, set to 32. If no features have been extracted from the input point cloud, then the initial features are set to the coordinates of the point cloud, and a multilayer perceptron is used to increase the dimension to d, i.e., F = MLP(X). The KNN algorithm is used to find the L nearest neighbors of the center point cloud in the input point cloud at this layer. Here, L is set to 16. Next, new local features can be obtained by aggregating neighborhood information based on the feature differences between the center point and its neighboring points. The specific process is as follows:
[0072] w ij =softmax(MLP([MLP(f j -f i ),MLP(x j -x i )]))
[0073]
[0074] Where MLP stands for Multilayer Perceptron, [·] represents cascade operation, and ⊙ represents element-wise multiplication.
[0075] Step 2.2: Capture discriminative long-range contextual information through an attention module guided by discriminative regions. The specific process is as follows:
[0076] Step 2.2.1: Obtain the center of the discriminative region;
[0077] For the local features updated by the adaptive local coding module Codewords using the global aggregator Calculate the assigned weights This allows us to determine the importance of each local feature:
[0078]
[0079] in K is set to 256. Then, for each codeword, the point with the largest assigned weight is selected as the center of the discriminative region. The features of the centers of the K discriminative regions can be obtained using the following formula. and coordinates
[0080]
[0081] Where I represents the index of the center point of the selected discriminative region, X s [I] indicates that the coordinates of the corresponding center point are selected based on the subscript.
[0082] Step 2.2.2: Obtain the feature description of the discriminative region;
[0083] This step requires feature description of the discriminative region, which can be achieved using an adaptive local coding module for feature aggregation of local features.
[0084]
[0085] Here, the central features of the discriminative region are input to the adaptive local coding module, so the downsampling operation can be omitted in the adaptive local coding module.
[0086] Step 2.2.3: Long-distance interaction in the discriminative region;
[0087] To eliminate the influence of non-discriminative region point clouds such as moving cars and pedestrians on feature extraction, long-distance contextual information interaction is performed only between discriminative region features. The specific interaction process is as follows: discriminative region features are input as query, key, and value features respectively into a multi-head self-attention layer; the resulting features are added to the original discriminative region features; and then passed through a LayerNorm layer to obtain discriminative region features with global contextual information.
[0088]
[0089] Where LN represents LayerNorm.
[0090] Step 2.2.4: Long-distance propagation of contextual information in the discriminative region;
[0091] To enhance local features with long-range contextual information, both local features and discriminative region features are fed into a cross-attention layer. This injects the long-range contextual information of the discriminative region into the local features, resulting in enhanced local features.
[0092]
[0093] Local features Discriminative region features as query features They are used as key and value features input into the cross-attention mechanism.
[0094] Step 2.3: By repeatedly performing the process in Step 2.2, point cloud features at multiple scales can be obtained. These features are then fed into the feature propagation module and continuously upsampled to the original point cloud size. One stage of the feature propagation process can be described as follows:
[0095]
[0096] Here, FP represents the feature propagation layer, which is typically instantiated as a feature propagation layer in PointNet++. Ultimately, this will result in the local features of the point cloud.
[0097] Step 2.4: Generate a global descriptor;
[0098] Next, the local features F are input into a global aggregator to obtain the global aggregated descriptor. Existing global aggregators like NetVLAD typically generate a long global feature set, which is then reduced in dimensionality using a fully connected layer. The large number of parameters in the fully connected layer leads to severe overfitting. Therefore, this method employs a simple and effective global aggregator: LightVLAD. Like NetVLAD, LightVLAD defines a set of learnable codewords. The weights for codewords and local features are obtained using the following formula.
[0099]
[0100] in LightVLAD aggregates local features into global descriptors by assigning weights.
[0101] g = [A1FW1, ..., A K FW K ]
[0102] Where [·] represents a cascading operation. d is a learnable parameter. ′ Set to 1.
[0103] Step 3: Train a discriminative region-guided Transformer network;
[0104] Each time, a batch of point clouds is randomly selected and input into a discriminative region-guided Transformer network to obtain a global descriptor for each point cloud. This method uses a triplet loss function for batch hard sample mining to optimize the discriminative region-guided Transformer network.
[0105]
[0106] Where d(·) represents the L2 distance of the feature, m is a predefined interval set to 0.2, and g p G represents the most difficult positive sample in the batch, that is, the positive sample with the largest distance from the current sample. n This represents the most difficult negative sample in the batch, which is the negative sample that is closest to the current sample.
[0107] The loss function is calculated using the above formula. The network parameters in the implicit scene completion model are updated using gradient descent until the model converges, resulting in a trained discriminative region-guided Transformer network.
[0108] Step 4: Construct a global descriptor database for the scene using the trained discriminative region-guided Transformer network; for point cloud data of a new scene, use the discriminative region-guided Transformer network to obtain and save the global descriptor for each point cloud in the dataset.
[0109] Step 5: For the query point cloud, use the trained discriminative region-guided Transformer network to obtain the global descriptor of the point cloud, and perform a nearest neighbor query in the scene global descriptor database to obtain the scene recognition result.
[0110] The above embodiments should be considered as exemplary and non-limiting, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by those skilled in the art within the scope of the technology disclosed in the invention and within the spirit and principles of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A method for large-scale point cloud scene recognition based on discriminative region feature learning, characterized in that, Includes the following steps: Step 1: Construct a dataset based on continuous frame street data with location coordinates collected by radar on robots or autonomous vehicles; Step 2: Construct a discriminative region-guided Transformer network to obtain local features point by point in the point cloud; the discriminative region-guided Transformer network includes an adaptive local encoding module, a discriminative region-guided attention module, a feature propagation module, and a global aggregator LightVLAD; The specific process of the discriminative region-guided attention module in the discriminative region-guided Transformer network is as follows: Step 2.1.1: Obtain the center of the discriminative region; For the local features updated by the adaptive local coding module Codewords using the global aggregator Calculate the assigned weights This allows us to determine the importance of each local feature: in , All parameters are learnable; then, for each codeword, the point with the largest assigned weight is selected as the center of the discriminative region, and the features of the K discriminative region centers are obtained according to the following formula. and coordinates : Where I represents the index of the center point of the selected discriminative region. This indicates that the coordinates of the corresponding center point are selected based on the subscript. Step 2.1.2: Obtain the feature description of the discriminative region; The discriminative region needs to be characterized using features. The Adaptive Local Encoding (ALE) module is used to aggregate local features. Here, the inputs to the adaptive local coding module are the center coordinates of the discriminative region. and characteristics and point cloud coordinates and local features ;; Step 2.1.3: Long-distance interaction in the discriminative region; The specific interaction process is as follows: Discriminative region features are input as query, key, and value features into a multi-head self-attention (MHSA) layer, the resulting features are added to the original discriminative region features, and then passed through a LayerNorm layer to obtain discriminative region features with global contextual information. : Where LN represents LayerNorm; Step 2.1.4: Long-distance propagation of contextual information in the discriminative region; Local features and discriminative region features are fed into the cross-attention MHCA layer to inject long-range contextual information from the discriminative region into the local features, thereby obtaining enhanced local features. : Local features Discriminative region features as query features These are used as key and value features and input into the cross-attention mechanism; Step 2.2: By repeatedly performing the process in Step 2.1, point cloud features at multiple scales are obtained. These features are then input into the feature propagation module and continuously upsampled to the original point cloud size. One stage of the feature propagation process is described as follows: Where FP represents the feature propagation layer, it is usually instantiated as a feature propagation layer in PointNet++; the final local features of the point cloud are obtained as follows: ; Step 3: Based on the training set in Step 1 and the triplet loss for batch hard sample mining, train the discriminative region-guided Transformer network. Step 4: Construct a global scene description sub-database using a trained discriminative region-guided Transformer network; Step 5: For the query point cloud, use the trained discriminative region-guided Transformer network to obtain the global descriptor of the point cloud, and perform a nearest neighbor query in the scene global descriptor database to obtain the scene recognition result.
2. The large-scale point cloud scene recognition method based on discriminative region feature learning according to claim 1, characterized in that, The specific implementation process of step 1 is as follows: Step 1.1: Every Set a keyframe point cloud, and use voxel downsampling to downsample the keyframe point cloud to N points. Step 1.2: For a given frame of point cloud, the distance to it is less than... Point clouds of meters are used as positive samples, with distances greater than [value missing]. Using the point cloud of meters as a negative sample, Record the point cloud indices of positive and negative samples, and save a set of positive and negative samples for each point cloud.
3. The large-scale point cloud scene recognition method based on discriminative region feature learning according to claim 1, characterized in that: The discriminative region-guided Transformer network in step 2 includes three feature downsampling stages. Each stage includes an adaptive local encoding module and a discriminative region-guided attention module. These stages yield multi-scale local features. Subsequently, these multi-scale local features will be input into the feature propagation module and gradually upsampled to the original resolution.
4. The large-scale point cloud scene recognition method based on discriminative region feature learning according to claim 1, characterized in that: The specific implementation process of the adaptive local coding module of the Transformer network guided by the discriminative region in step 2 is as follows: First, input the point cloud of this layer. Downsampling to 1 / 4 of the original point cloud size yields the coordinates of the downsampled point cloud. and characteristics ,in Here, d represents the number of point clouds input at the current stage, and d is the feature dimension. If no features have been extracted from the input point cloud, the initial features are set to the coordinates of the point cloud, and then a Multilayer Perceptron (MLP) is used to increase the dimension to d. The K-nearest neighbor algorithm is used to find the L nearest neighbors of the center point cloud in the input point cloud of this layer. ; Next, based on the characteristics of the center point and neighborhood point features The feature differences between them are weighted. To aggregate neighborhood information to obtain new local features The specific process is as follows: MLP stands for Multilayer Perceptron. Indicates a cascading operation. This indicates element-wise multiplication.
5. The large-scale point cloud scene recognition method based on discriminative region feature learning as described in claim 1, characterized in that, The specific process of LightVLAD, the global aggregator of the Transformer network guided by the discriminative region in step 2, is as follows: LightVLAD defines a set of learnable codewords. The codewords and local features are obtained using the following formula. Allocation weights : in , , which are learnable parameters; LightVLAD aggregates local features into global descriptors by assigning weights. : in Indicates a cascading operation. For the dimensions of the final global descriptor, For learnable parameters, .
6. The large-scale point cloud scene recognition method based on discriminative region feature learning as described in claim 1, characterized in that, The specific process of training the discriminative region-guided Transformer network in step 3 is as follows: Each time, a batch of point clouds is randomly selected and input into the discriminative region-guided Transformer network to obtain the global descriptor for each point cloud; the triplet loss function of batch hard sample mining is used to optimize the discriminative region-guided Transformer network, and the loss function is defined as: in, The L2 distance represents the feature, where m is a predefined interval. This represents the most difficult positive sample in the batch, which is the positive sample that is furthest from the current sample. This represents the most difficult negative sample in the batch, which is the negative sample that is closest to the current sample. The loss function is calculated using the above formula. The network parameters in the implicit scene completion model are updated using gradient descent until the model converges, resulting in a trained discriminative region-guided Transformer network.