Small sample point cloud target identification method

Abstract

The invention relates to the field of machine learning computer vision and artificial intelligence, in particular to a small sample point cloud target recognition method. The method comprises the following steps that S1, input point cloud data passes through a constructed point cloud feature extraction module, and a final feature vector used for a classification task is obtained; and S2, the final feature vector is processed by a measurement-based point cloud identification module, and the category of the final feature vector is determined by calculating the distance between the feature vector and each prototype. According to the method, multi-scale feature representation with accurate geometric details, rich local structure information, strong global semantics and spatial context understanding capability can be learned, so that high performance and robustness are realized in various point cloud target recognition tasks. Experimental verification is carried out on a public data set and a small sample data set based on the GM-APD laser radar, and experimental results show that compared with other existing algorithms, the algorithm provided by the invention has the advantage that the target recognition accuracy is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the fields of machine learning, computer vision and artificial intelligence, and in particular to a small sample point cloud target recognition method. Background Art

[0002] Currently, 3D point cloud recognition is a hot research topic in the field of computer vision, and has made tremendous progress in recent years. The practical applications of 3D point clouds in autonomous driving, computer graphics, and 3D scene understanding have made the task of 3D point cloud recognition increasingly important.

[0003] Small-shot object recognition networks provide a low-cost, highly adaptable solution for 3D point cloud processing, showing irreplaceable advantages, especially in data-constrained scenarios, and are an important research direction in the field of 3D vision. However, the construction of existing small-shot object recognition networks still has the following limitations. For example, as disclosed in the paper "Agent Prototypes Distillation for Few-Shot Object Detection," due to insufficient accuracy in class-level prototype generation and missing detailed information, they rely too much on global features, which in turn leads to loss of local key information and decreased feature discrimination. It is difficult for the model to fully learn the key features of the target, which ultimately significantly reduces the accuracy of point cloud object recognition. Summary of the Invention

[0004] The present invention proposes a small sample point cloud target recognition method, the purpose of which is to overcome the problem of low accuracy of 3D point cloud target recognition in the prior art.

[0005] In order to achieve the purpose of the present invention, the technical solution proposed by the present invention is: a small sample point cloud target recognition method, comprising the following steps:

[0006] Step S1: The input point cloud data is passed through the constructed point cloud feature extraction module to obtain the final feature vector for the classification task;

[0007] In step S2, the final feature vector is processed by the metric-based point cloud recognition module, and its category is determined by calculating the distance between its feature vector and various prototypes.

[0008] Furthermore, the specific steps of the above step S1 are:

[0009] 1.1 Construct a point cloud feature extraction module, which includes multiple EdgeConv layers and FusionConv layers stacked alternately, an MLP layer and a Max Pool layer;

[0010] 1.2 The input point cloud data is processed by the EdgeConv layer, which dynamically constructs an adjacency graph using the k-nearest neighbor algorithm and extracts local geometric features through point-by-point convolution;

[0011] 1.3 The processed local geometric features are further fused with local geometric structure and context information through the FusionConv layer to construct a local graph, dynamically adjust the neighborhood graph dimension, and calculate the 3D relationship between points.

[0012] 1.4 Repeat steps 1.2 and 1.3 above; by stacking EdgeConv and FusionConv modules level by level, the feature information extracted by the network is effectively fused and spliced ​​to construct a feature vector with strong representation ability.

[0013] 1.5 The feature vector is input into the multi-layer perceptron (MLP) network for processing to obtain the dimension-enhanced features;

[0014] 1.6 The distance features of each point after dimensionality increase are pooled through the Max Pool layer, and the dimensionality reduction is simulated by the human visual system to obtain the local features of the point. The features are then aggregated to obtain the final feature vector for the classification task.

[0015] Furthermore, in the EdgeConv layer in step 1.1 above, after the graph structure is established, the data point x i Its adjacent point x ij The edge features between are quantitatively expressed as:

[0016]

[0017] Furthermore, the FusionConv layer in step 1.1 above

[0018] The local feature map of the feature map is represented as:

[0019] The local feature context is obtained as:

[0020]

[0021] Furthermore, in the above step 1.2, the nonlinear features in the output feature map are:

[0022]

[0023] Furthermore, in the above step 1.2, the local geometric context and local feature context After splicing, the output of the local context fusion module is:

[0024] Furthermore, in the above step S2, the metric-based point cloud recognition module uses a deep neural network model to process the final feature vector of the input, and the final feature vector is gradually converted into a high-dimensional feature vector through the deep neural network model layer, representing its position in the feature space; in the feature space, the prototype is calculated for each class, that is, the mean of the sample features supporting the class.

[0025] Furthermore, in the above step S2, it is assumed that there are N classes, each class has K support samples, and the prototype c of class i i The calculation is as follows:

[0026]

[0027] For each query sample, its category is determined by calculating the distance between its feature vector and each prototype. The commonly used distance metric is Euclidean distance. q The probability of belonging to class i is calculated by the softmax function:

[0028]

[0029] Where d(·,·) represents the squared Euclidean distance d(x,y)=∑(xy) 2 .

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] 1. This invention dynamically adjusts the neighborhood graph dimensions by constructing a local graph and calculating the 3D relationships between points, integrating this information into the feature representation to improve the expressive power of local features. By establishing a correlation model between channels, it can effectively learn the multidimensional differential features of point cloud objects, thereby improving the problem of insufficient feature differentiation between categories.

[0032] 2. This invention uses a metric-based small sample learning method to calculate the Euclidean distance feature mean of the target point cloud to obtain the discriminative features of different categories of targets. The feature mean of the input sample is compared with the sample prototype in the dataset to achieve classification recognition, thereby improving recognition accuracy. A metric-based small sample learning method is used to calculate the Euclidean distance metric similarity between the input sample and the samples in the small sample dataset to achieve point cloud target recognition based on nearest neighbor prototype matching.

[0033] 3. By stacking EdgeConv and FusionConv modules step by step, the feature information extracted by the network is effectively fused and spliced ​​to construct a feature vector with strong representation ability, effectively capturing the local geometric structure and feature context information in the point cloud, thereby improving the expression ability of local features, and effectively improving the problem of small-sample target recognition networks relying too much on global features, resulting in the loss of local key information and decreased feature discrimination. The alternating stacking of the two forms an iterative refinement process. EdgeConv continuously improves feature quality at the local scale, while FusionConv continuously fuses and complements high-quality features of different scales. Ultimately, the network is able to learn multi-scale feature representations that have precise geometric details, rich local structural information, strong global semantics and spatial context understanding capabilities, thereby achieving high performance and robustness in various point cloud target recognition tasks (classification, segmentation, detection). Through experimental verification on public datasets and small-sample datasets based on GM-APD lidar, the experimental results show that the algorithm proposed in this invention has achieved significant improvement in target recognition accuracy compared to other existing algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a flow chart of the method of the present invention;

[0035] Figure 2 A diagram of the network structure for point cloud recognition based on DPCF with few-shot learning;

[0036] Figure 3 This is the feature embedding network structure diagram based on EdgeConv;

[0037] Figure 4 This is the FusionConv module diagram;

[0038] Figure 5 Schematic diagram of the metric-based classification module;

[0039] Figure 6 Target map for data collection in experimental scenario 1;

[0040] Figure 7 A point cloud diagram of the experimental scene. Specific implementation methods

[0041] In order to make the purpose and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings and embodiments.

[0042] The present invention provides a small sample point cloud target recognition method, such as Figure 1 As shown, the following steps are included:

[0043] Step S1: The input point cloud data is used to construct a point cloud feature extraction module to obtain the final feature vector for the classification task. The specific steps are:

[0044] 1.1 Construct a point cloud feature extraction module, including multiple stacked EdgeConv layers and FusionConv layers, MLP layers and Max Pool layers;

[0045] The EdgeConv-based feature embedding layer has the following structure: Figure 3 As shown;

[0046] Given a set of m d-dimensional data points Need to build its k-nearest neighbor graph This graph structure can reflect the data points x in the set V. i The topological relationship between the data point x and its k nearest neighbors, where ε is the adjacency matrix describing the connection status of the nodes in the graph. i Its adjacent point x ij The edge features between are quantitatively expressed as:

[0047]

[0048] Specifically, for each neighboring point in the range j = 1 to k, a nonlinear mapping function f with a trainable parameter is used. γ (.) is processed, and this function is implemented by a network layer composed of 1×1 convolution kernels. The calculation process takes into account x ij With the center point x i Based on this, we can extract point x i and its neighboring point x ij The edge feature between ij . Then the maximum pooling operation is performed on these edge features to obtain the point x i Updated features Its mathematical formula is:

[0049]

[0050] The structure of the FusionConv layer is as follows Figure 4 As shown;

[0051] FusionConv builds a local graph and calculates the 3D relationships between points, integrating this information into the feature representation to improve the expressiveness of local features. The construction of the local graph is based on the geometric relationships and feature context of the input point cloud. This ensures the effective integration of local structural information and enhances the model's perception of complex geometric structures.

[0052] Specifically, FusionConv uses the k-nearest neighbor algorithm to find the specific point p under the 3D Euclidean distance metric. i Neighbor The edge between them defines p i The mathematical formula for the local geometric graph in 3D space is:

[0053]

[0054] Therefore, the local geometric map of all points is usually expressed as:

[0055]

[0056] Similarly, f in the C-dimensional feature map i The local feature map is constructed as follows: where f ik is N(p i ). Therefore, the local feature map of the feature map is expressed as: Get the local feature context:

[0057]

[0058] Among them, M Θ is another shared MLP for encoding local feature maps. Finally, the local geometric context and local feature context Concatenate as the output of our local context fusion module:

[0059]

[0060] In addition to collecting more local details for each point's feature representation, the global bilinear regularization module aims to optimize the feature map by adopting a global perception of the entire point cloud. As in conventional self-attention mechanisms, global perception can capture long-range dependencies (i.e., similarities between points), but this requires relatively high memory consumption. In contrast, in this method, global perception is calculated based on kernel correlations between point-level and channel-level descriptors between feature map elements.

[0061] In the process of constructing the global channel-level descriptor, the weight matrix Fusion output Implement dimensionality reduction. The introduction of the ReLU activation function not only realizes nonlinear transformation but also ensures the non-negative characteristics of the output value. The average pooling operation in the spatial dimension compresses and integrates the spatial features, and finally generates a channel-level descriptor g with global characteristics. c Mathematically, the above operation is as follows:

[0062]

[0063] Among them F L W c It's F L and W c The matrix product of ; “avg” is the average pooling operation, and the reduction factor ξ satisfies ξ ≥ 2. Further, The jth channel represents the global (mean) response in the entire point cloud feature map.

[0064] Introducing the weight matrix Combined with the ReLU activation function and average pooling calculation along the channel dimension, a global point-level feature representation g is constructed. p :

[0065] g p =avg(ReLU(F L W p )),g p ∈R N (8)

[0066] Let g p =[λ1,…,λ N ] Each Represents the global mean response feature of the i-th point in the point cloud feature map. Matrix F L W c The product of g constitutes another feature representation. Different from the simple element-level multiplication operation, the present invention adopts g p and g c The square root of is used to construct low-rank global bilinear features. The specific calculation process can be expressed as the following mathematical expression:

[0067]

[0068] The element η ij Position at row i and column j The mathematical calculation is as follows:

[0069]

[0070] Formula (10) can be interpreted from two aspects. On the feature map, it is an effective way to synthesize the global bilinear response of all elements, combining the corresponding point-level and channel-level descriptors. For each element, λ i and μ j is the arithmetic mean of the i-th point and the j-th channel, η ij is λ i and μ jThe geometric mean of ; it provides a higher-order mean response based on spatial and channel-dependent information.

[0071] By introducing the shared multi-layer perceptron M ψ The low-rank bilinear pooling method is used to design two shortcut connection paths to achieve effective reconstruction of the feature channel dimension. The network structure fuses multi-scale features and ultimately outputs a full-resolution feature map with global context information:

[0072]

[0073] in Next, we hope to find a better way to use the global perception map. The average pooling operation is used to squeeze global information from the spatial and channel spaces respectively, where the pooled mean usually represents the common pattern. For point cloud analysis, we hope that the learned features are more representative. Therefore, in practice, by combining the local context feature F L With the global feature F G , and then the operation result is processed by the nonlinear activation function σ to increase the nonlinear features in the final output feature map:

[0074]

[0075] 1.2 The input point cloud data is processed by the EdgeConv layer, which dynamically constructs an adjacency graph using the k-nearest neighbor algorithm and extracts local geometric features through point-by-point convolution;

[0076] 1.3 The processed local geometric features are further fused with local geometric structure and context information through the FusionConv layer to construct a local graph, dynamically adjust the neighborhood graph dimension, and calculate the 3D relationship between points to enhance the feature expression capability.

[0077] 1.4 Repeat steps 1.2 and 1.3 above; by stacking EdgeConv and FusionConv modules level by level, the feature information extracted by the network is effectively fused and spliced ​​to construct a feature vector with strong representation ability.

[0078] 1.5 The feature vector is input into the multi-layer perceptron (MLP) network for processing to obtain the dimension-enhanced features;

[0079] 1.6 The distance features of each point after dimensionality increase are pooled through the Max Pool layer, and the dimensionality reduction is simulated by the human visual system to obtain the local features of the point. The features are then aggregated to obtain the final feature vector for the classification task, providing feature support for subsequent small sample recognition.

[0080] In step S2, the final feature vector is processed by the metric-based point cloud recognition module, and its category is determined by calculating the distance between its feature vector and each prototype:

[0081] The structure of the metric-based point cloud recognition module is as follows: Figure 5 As shown, the known structure is adopted;

[0082] The metric-based point cloud recognition module uses a deep neural network model to process the final input feature vector. The deep neural network model maps the original 3D coordinate information into a higher-dimensional feature space through multiple layers of nonlinear transformations. The final feature vector is gradually converted through the layers of the deep neural network model into a high-dimensional feature vector representing its position in the feature space. In this feature space, a prototype is calculated for each class, representing the mean of the features of the sample support for that class.

[0083] Assume there are N classes, each class has K support samples, and the prototype c of class i i The calculation is as follows:

[0084]

[0085] For each query sample, its category is determined by calculating the distance between its feature vector and each prototype. The distance metric used in this embodiment is Euclidean distance. q The probability of belonging to class i is calculated by the softmax function:

[0086]

[0087] Where d(·,·) represents the squared Euclidean distance d(x,y)=∑(Xy) 2 .

[0088] To verify the effectiveness of the proposed method, the present invention selects the ModelNet40 dataset and the GM-APD lidar dataset to carry out experimental verification and performance evaluation:

[0089] The present invention uses recognition accuracy as the core indicator for evaluating algorithm performance, and its calculation formula can be expressed as:

[0090]

[0091] 1. Comparative verification on the ModelNet40 dataset.

[0092] All comparison algorithms use randomly initialized parameters and go through 80 complete training rounds. Each round of training includes 400 meta-learning training tasks and 600 meta-learning verification tasks. Specifically, each meta-task sets 5 categories, uses 1 or 5 labeled samples as support sets, and includes 10 unlabeled samples as query sets. After the model training is completed, the system will evaluate 700 meta-test tasks, and the average classification accuracy under 95% confidence interval is used as the evaluation criterion for system performance. The present invention uses cross entropy as the loss function and updates parameters through the Adam optimizer. The specific parameter configuration is as follows: the initial learning rate is 0.0005; the gamma parameter value is 0.5; the learning rate adopts a periodic attenuation strategy, which is reduced by 50% after every 5 training cycles.

[0093] The accuracy of different feature extraction networks on the ModelNet40 dataset is shown in Table 1

[0094] Table 1 Accuracy of different feature extraction networks on the ModelNet40 dataset

[0095]

[0096] Experimental results show that the network architecture proposed in the present invention achieves an accuracy of 71.51% and 86.03% respectively under two different task settings. This feature enables the model to autonomously discover the implicit structured information in the data and achieve a natural clustering effect of feature representation. Experimental results show that compared with the traditional point cloud feature embedding method, the method proposed in the present invention exhibits superior recognition performance. In order to improve the performance of the model, the present invention innovatively introduces the FusionConv module, which effectively enhances the representation ability of the feature vector of the point cloud object. At the same time, by designing the channel self-attention mechanism, the model can automatically learn more discriminative point cloud feature representations. This improvement significantly alleviates the problem of small feature differences between categories, and ultimately achieves a significant improvement in recognition accuracy.

[0097] Second, comparative verification is performed on a small sample dataset of GM-APD lidar:

[0098] The core component of the 3D imaging lidar system used for data collection is the GM-APD array detector, which adopts the staring working mode for information collection. The experimental data collection work scenario 1 is implemented at the height of a fifth floor, with a fixed measurement distance of 320 meters; scenario 2 is implemented at the height of a seventh floor, with a fixed measurement distance of 830 meters. The focus is on the detection and 3D reconstruction of dynamic targets on the ground (including sports utility vehicles, small passenger cars, bicycles and people). The distribution of research objects in the specific scenario is as follows Figure 6As shown in the figure. A laser radar system is used for target detection, and the beam divergence angle of its laser transmitter is 0.6°×0.6°. Considering that a single staring imaging cannot completely cover the entire experimental area, a multi-angle scanning imaging technology is used to achieve comprehensive detection of the scene. To ensure the integrity of the acquired target point cloud data, the data obtained by multi-angle scanning needs to be spliced, as shown in the figure. Figure 7 As shown in the figure, this paper uses a GM-APD lidar system to collect point cloud data and constructs a small sample point cloud dataset for target recognition tasks. This dataset contains three categories: vehicles, bicycles, and pedestrians, and aims to provide basic data support for the classification and recognition of ground moving targets.

[0099] The accuracy on the constructed GM-APD lidar small sample data set is shown in Table 2

[0100] Table 2 Accuracy of the constructed GM-APD lidar small sample data set

[0101]

[0102] Experimental results demonstrate that the proposed method performs well on point cloud data measured by a GM-APD lidar. As shown in Table 2, the proposed method achieves 87.41% recognition accuracy in the 3-way-1-shot task and 95.03% in the 3-way-5-shot task, significantly outperforming existing comparative methods. These data fully demonstrate the effectiveness and significant improvement of the proposed method when processing real-world point cloud data.

Claims

1. A small sample point cloud target recognition method, characterized by: The following steps are included Step S1: The input point cloud data is passed through the constructed point cloud feature extraction module to obtain the final feature vector for the classification task; In step S2, the final feature vector is processed by the metric-based point cloud recognition module, and its category is determined by calculating the distance between its feature vector and various prototypes.

2. The small sample point cloud target recognition method according to claim 1, characterized in that: The specific steps of step S1 are: 1.1 Construct a point cloud feature extraction module, which includes multiple EdgeConv layers and FusionConv layers stacked alternately, an MLP layer and a Max Pool layer; 1.2 The input point cloud data is processed by the EdgeConv layer, which dynamically constructs an adjacency graph using the k-nearest neighbor algorithm and extracts local geometric features through point-by-point convolution; 1.3 The processed local geometric features are further fused with local geometric structure and context information through the FusionConv layer to construct a local graph, dynamically adjust the neighborhood graph dimension, and calculate the 3D relationship between points. 1.4 Repeat steps 1.2 and 1.3 above; by stacking EdgeConv and FusionConv modules level by level, the feature information extracted by the network is effectively fused and spliced ​​to construct a feature vector with strong representation ability. 1.5 The feature vector is input into the multi-layer perceptron (MLP) network for processing to obtain the dimension-enhanced features; 1.6 The distance features of each point after dimensionality increase are pooled through the Max Pool layer, and the dimensionality reduction is simulated by the human visual system to obtain the local features of the point. The features are then aggregated to obtain the final feature vector for the classification task.

3. The small sample point cloud target recognition method according to claim 1, characterized in that: In the EdgeConv layer in step 1.1, after the graph structure is established, the data point x i Its adjacent point x ij The edge features between are quantitatively expressed as: e ij =f γ ([x i ,x ij -x i ]) (1)。 4. The small sample point cloud target recognition method according to claim 1, characterized in that: In the FusionConv layer in step 1.1, the local feature map of the feature map is represented as: The local feature context is obtained as:

5. The small sample point cloud target recognition method according to claim 1, characterized in that: In step 1.2, the nonlinear features in the output feature map are:

6. The small sample point cloud target recognition method according to claim 1, characterized in that: In step 1.2, the local geometric context and local feature context After splicing, the output of the local context fusion module is: 。 7. The small sample point cloud target recognition method according to claim 1, characterized in that: In step S2, the metric-based point cloud recognition module uses a deep neural network model to process the final input feature vector, and the final feature vector is gradually converted into a high-dimensional feature vector through the deep neural network model layer, representing its position in the feature space; in the feature space, the prototype is calculated for each class, that is, the mean of the sample features supporting the class.

8. The small sample point cloud target recognition method according to claim 1, characterized in that: In step S2, it is assumed that there are N classes, each class has K supporting samples, and the prototype c of class i i The calculation is as follows: For each query sample, its category is determined by calculating the distance between its feature vector and each type of prototype; query sample x q The probability of belonging to class i is calculated by the softmax function, and the formula is: Where d(·,·) represents the squared Euclidean distance d(x,y)=∑(xy) 2 .

Images