3D point cloud target identification method based on bidirectional long-short-term memory network

Abstract

The invention discloses a 3D point cloud target identification method based on a bidirectional long-short-term memory network, which comprises the following steps: firstly, preprocessing original point cloud data, carrying out feature embedding on the preprocessed point cloud data, then carrying out data feature extraction based on the bidirectional long-short-term memory network, carrying out local feature aggregation and global feature aggregation on the data, and finally carrying out target identification on the 3D point cloud target. Feature extraction is enhanced, and the data expression ability is improved; and constructing a point cloud data identification model based on a full connection layer, training and evaluating the model, and performing point cloud data identification based on the model. According to the scheme, under the condition that the calculation precision is not reduced, the long-term dependency relationship between the 3D point cloud sequences is captured with few calculation resources and memory resources, aggregation is carried out based on the extracted features, and accurate recognition is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of 3D point cloud recognition, and in particular relates to a 3D point cloud target recognition method based on a bidirectional long short-term memory network. Background Art

[0002] With the rapid development of computer vision and artificial intelligence technologies, 3D point cloud data is increasingly used in many fields, especially in scenarios such as autonomous driving, robot navigation, virtual reality, and 3D reconstruction, becoming an indispensable technical foundation. 3D point cloud data is usually obtained through sensors such as lidar and depth cameras, representing objects and their surroundings in three-dimensional space in a sparse and irregular form. This representation method not only has high-dimensional characteristics, but also has complex structural characteristics, which poses a huge challenge to traditional machine learning and deep learning models.

[0003] Existing 3D point cloud recognition methods mainly focus on direct processing of geometric features. The commonly used convolutional neural network (CNN) performs well when processing regular grid data, but its effectiveness is significantly limited when facing irregular point cloud data, especially in capturing long-term dependencies in the data. In addition, how to effectively extract features from point cloud data remains a key technical problem that needs to be solved urgently.

[0004] Traditional feature extraction methods often rely on manually designed features and cannot fully exploit the inherent complex structure of point cloud data. In recent years, we have introduced the BiLSTM network into the research of point cloud analysis because of its advantages in processing sequence data. The BiLSTM model can simultaneously capture the sequence information between points, thereby improving the accuracy and effectiveness of feature extraction.

[0005] Therefore, in order to improve the performance of 3D point cloud object recognition, a novel and effective method is urgently needed. This method should be able to fully utilize the advantages of BiLSTM and meet the needs of high precision and high robustness in practical applications through effective feature extraction and classification techniques. In this context, the 3D point cloud recognition method based on BiLSTM not only has important research significance, but is also expected to play an important role in the practical application of related technologies and promote the development and progress of this field. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a 3D point cloud target recognition method based on a bidirectional long short-term memory network, which captures the long-term dependencies between 3D point cloud sequences with less computing resources and memory resources without reducing the calculation accuracy, and aggregates the extracted features to achieve recognition.

[0007] The specific technical solution for achieving the purpose of the present invention is:

[0008] A 3D point cloud target recognition method based on a bidirectional long short-term memory network comprises the following steps:

[0009] Step 1: Preprocess the original point cloud data;

[0010] Step 2: embed features of the preprocessed point cloud data;

[0011] Step 3: Extract data features based on bidirectional long short-term memory network;

[0012] Step 4: Perform local feature aggregation and global feature aggregation on the data to enhance feature extraction and improve data expression capabilities;

[0013] Step 5: Build a point cloud data recognition model based on the fully connected layer, and train and evaluate the model;

[0014] Step 6: Use the trained model to perform point cloud data recognition.

[0015] Compared with the prior art, the solution of the present invention takes into account the noise problems such as background interference and occlusion in the acquisition process of 3D point cloud data, and designs a complete algorithm from raw data preprocessing to the final realization of 3D point cloud target recognition. First, the raw data point cloud data is preprocessed, including denoising, data normalization and data downsampling, in order to make the point cloud data used for target recognition cleaner and save more computing resources and memory usage; then, the preprocessed point cloud data is subjected to point feature extraction and position encoding, which helps the network better capture the location information of the point; then, BiLSTM is used to extract features, which contains both forward and backward sequence information, and can better capture the long-term dependency of point cloud sequence data; finally, we perform feature aggregation, including local features and global features, which can make the features learned by the model more comprehensive, so that the model can use the last year's features of the entire point cloud for final recognition. The method of the present invention improves the model's ability to capture the long-term dependency of 3D point cloud sequences, and reduces the computing resources and memory resources occupied to a certain extent, and is a better 3D point cloud recognition method.

[0016] The present invention is further described below in conjunction with specific implementation modes. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the flow of the 3D point cloud target recognition method based on the bidirectional long short-term memory network of the present invention.

[0018] Figure 2 It is a schematic diagram of the network model structure of the present invention. DETAILED DESCRIPTION

[0019] Example

[0020] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an kind" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0022] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps set forth in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the accompanying drawings are not drawn according to the actual proportional relationship. The technology, method and equipment known to those of ordinary skill in the relevant field may not be discussed in detail, but in appropriate cases, the technology, method and equipment should be considered as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so that once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

[0023] Combination Figure 1 , a 3D point cloud target recognition method based on a bidirectional long short-term memory network, comprising the following steps:

[0024] Step 1: Preprocess the original point cloud data:

[0025] The original point cloud data is obtained based on radar acquisition or public data sets. The irregular and sparse characteristics of point cloud data make it impossible to be directly processed by the model, so denoising, normalization and downsampling are performed;

[0026] Original point cloud data P From multiple 3D points The denoising process is to use a statistical filtering method to remove the noise points in the point cloud data P and obtain the denoised point cloud data. :

[0027]

[0028] This method can effectively remove abnormal noise points, making the feature extraction in subsequent steps more accurate;

[0029] In order to eliminate the difference in coordinate scales between different point cloud data, normalization processing is to use linear scaling to scale the coordinate values ​​of all point cloud data to the range of [0,1]. The normalized point cloud data is expressed as :

[0030]

[0031] In order to reduce the amount of calculation and improve processing efficiency, the downsampling process is to use random sampling or feature-based importance sampling method to reduce the number of points in the point cloud data to N. The sampled point cloud data is , , where represents the number of selected sampling points. The purpose of downsampling is to reduce the computational complexity of the model while ensuring the representativeness of the point cloud data.

[0032] Step 2: Feature embedding of preprocessed point cloud data:

[0033] For the sampled point cloud data , the geometric coordinates of each point are nonlinearly mapped through a multi-layer perceptron (MLP), that is, point feature extraction, to generate a high-dimensional feature representation :

[0034]

[0035] This feature extraction process effectively improves the expressiveness of each point;

[0036] Then add the position code generated using sine and cosine functions , combined with geometric features, to obtain a feature representation containing position information :

[0037]

[0038] Step 3: Extract data features based on a bidirectional long short-term memory network. Point cloud data can essentially be regarded as an unstructured sequence data. In order to better capture the temporal relationship between points, we use BiLSTM to process these features. BiLSTM consists of two directional LSTMs, one forward (from front to back) and one reverse (from back to front):

[0039] Based on the feature representation in step 2 , and use the bidirectional long short-term memory network to extract the context feature representation:

[0040] Forward LSTM extracts forward dependencies: the features of each point Input the forward LSTM network, extract the dependency between this point and the previous point, and generate the forward hidden state :

[0041]

[0042] Reverse LSTM extracts backward dependencies: At the same time, the features of each point are Input the reverse LSTM network, extract the dependency between this point and the subsequent points, and generate the backward hidden state :

[0043]

[0044] Bidirectional LSTM feature integration: The output features of the forward LSTM and the reverse LSTM are concatenated to form contextual features containing the temporal information before and after the point cloud. :

[0045]

[0046] in, Represents the forward LSTM network for the i-th point feature The output of represents the hidden state of the forward information of this point when processing the sequence, including the dependency relationship between this point and the previous point. Represents the reverse LSTM network for the i-th point feature The output of represents the hidden state of the backward information of this point when processing the sequence, including the dependency relationship between this point and the subsequent points. The final context feature of the i-th point is represented by the concatenation of the output features of the forward and reverse LSTMs, integrating the previous and next temporal information.

[0047] Step 4: In order to further improve the global expression ability of the point cloud, local feature aggregation and global feature aggregation are performed on the data to enhance feature extraction and improve data expression ability:

[0048] Aggregation of local and global geometric features based on feature extracted data, including:

[0049] Local feature aggregation: Extract local geometric features by pooling the features of points in the neighborhood of each point in the point cloud. The local features of the neighborhood The point features within are pooled by maximum or average pooling to obtain:

[0050]

[0051] Global feature aggregation: In order to capture the global structural information of the entire point cloud, a global pooling operation is used to aggregate the local features of all points to generate a global feature representation. :

[0052]

[0053] in, It is the representation after local feature aggregation, from the neighborhood point N of the current point i Pooling is a pooling operation. It represents the representation after global feature aggregation, obtained from the features of all points. GlobalPooling is a global pooling operation. Data representing the features of the i-th point obtained based on step 3.

[0054] Step 5: Build a point cloud data recognition model based on the fully connected layer, and train and evaluate the model:

[0055] After feature aggregation, the fully connected layer is used to aggregate the global features. Further processing is performed to generate high-dimensional features for classification, that is, feature representation after nonlinear activation , the fully connected layer further maps the global features through linear transformation and nonlinear activation functions (such as ReLU);

[0056] The classification layer uses the Softmax layer to predict the category of the point cloud data. The activation function outputs the probability of each category, and finally generates the probability distribution of the point cloud belonging to different categories. :

[0057]

[0058]

[0059] Among them, h represents the feature representation after processing by the fully connected layer, FC represents the function operation of the fully connected layer, Represents the output category probability distribution of point cloud input x;

[0060] The point cloud data recognition model adopts the cross entropy loss function during the training process. To measure the difference between the model prediction result and the true label, the gradient of the loss relative to the model parameters is calculated through the back-propagation algorithm, and the model parameters are updated using an optimization algorithm (such as Adam or SGD) to reduce the loss function value:

[0061]

[0062] in, Represents the loss function value, which is used to evaluate the model performance. Represents the category indication of the true label, represents the category probability of the model output, represents the parameters of the model, Represents the learning rate, which controls the step size of parameter update.

[0063] In addition, the accuracy rate used in this scheme As an indicator for evaluating the model, the accuracy The correct classification ratio of the model on the validation set is calculated:

[0064]

[0065] in, is the indicator function, when the predicted label With the true label If they are equal, it returns 1, otherwise it returns 0, and N represents the total number of samples;

[0066] The performance of the model is judged by evaluation indicators, and the model parameters are adjusted and optimized accordingly.

[0067] Step 6: Use the trained model to perform point cloud data recognition.

[0068] This solution takes into full account the noise problems such as background interference and occlusion in the acquisition process of 3D point cloud data, and designs a complete algorithm from raw data preprocessing to the final realization of 3D point cloud target recognition. First, the raw data point cloud data is preprocessed, including denoising, data normalization and data downsampling, in order to make the point cloud data used for target recognition cleaner and save computing resources and memory usage; then, the preprocessed point cloud data is subjected to point feature extraction and position encoding, which helps the network better capture the location information of the points; then, BiLSTM is used to extract features, which contains both forward and backward sequence information, and can better capture the long-term dependencies of point cloud sequence data; finally, we perform feature aggregation, including local features and global features, which can make the features learned by the model more comprehensive, so that the model can use the last year's features of the entire point cloud for final recognition.

[0069] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0070] Step 1: Preprocess the original point cloud data;

[0071] Step 2: embed features of the preprocessed point cloud data;

[0072] Step 3: Extract data features based on bidirectional long short-term memory network;

[0073] Step 4: Perform local feature aggregation and global feature aggregation on the data to enhance feature extraction and improve data expression capabilities;

[0074] Step 5: Build a point cloud data recognition model based on the fully connected layer, and train and evaluate the model;

[0075] Step 6: Use the trained model to perform point cloud data recognition.

[0076] The present invention also provides a computer storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0077] Step 1: Preprocess the original point cloud data;

[0078] Step 2: embed features of the preprocessed point cloud data;

[0079] Step 3: Extract data features based on bidirectional long short-term memory network;

[0080] Step 4: Perform local feature aggregation and global feature aggregation on the data to enhance feature extraction and improve data expression capabilities;

[0081] Step 5: Build a point cloud data recognition model based on the fully connected layer, and train and evaluate the model;

[0082] Step 6: Use the trained model to perform point cloud data recognition.

[0083] In order to better illustrate the technical effect of the present invention, the specific data of this embodiment is used for description;

[0084] In this example, the dataset ScanObjectNN is used for experimental verification; ScanObjectNN is a dataset designed for 3D point cloud data object classification, which aims to evaluate the performance of point cloud models in real scenes. The dataset contains 2902 instances divided into 15 categories, including common indoor objects such as beds, bookcases, chairs, sofas, doors, tables, etc. Unlike synthetic datasets such as ModelNet, the data of ScanObjectNN comes from actual scanned objects, with real noise, occlusion, and background interference, making the classification task more challenging.

[0085] Each object is represented as a point cloud, usually downsampled to 2048 points. The main features of ScanObjectNN include realistic noise, partial occlusion, incompleteness, and background interference. In addition, the shape similarity and asymmetry between object categories also increase the difficulty of the task. In order to test the robustness of the model, the dataset provides multiple variants: OBJ-BG (mixed point cloud of objects and background), OBJ-ONLY (only object point cloud), and PB-T50-RS (50% perturbation and rotation added to OBJ-ONLY). The ScanObjectNN dataset is widely used in 3D point cloud classification and segmentation tasks in complex scenes, and supports in-depth research on the generalization ability of the model. We mainly use the variant PB-T50-RS for experimental verification.

[0086] In this embodiment, two classic 3D point cloud processing algorithms are used as comparison algorithms, and their hyperparameter settings are the same as those of the present invention. The network models finally trained by the present invention and other comparison algorithms on the above data sets are saved, and the trained network models are used for testing.

[0087] Table 1 shows the experimental results of 3D object recognition on the same dataset in this embodiment, where OA represents the overall accuracy and mAc represents the average accuracy.

[0088] method mAcc(%) OA(%) PointNet++ 75.4 77.9 PCT 80.5 83.3 Methods 84.3 85.9

[0089] It can be seen that the overall accuracy and average accuracy based on this scheme are higher than the existing conventional algorithms.

[0090] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A 3D point cloud target recognition method based on a bidirectional long short-term memory network, characterized in that: The following steps are involved: Step 1: Preprocess the original point cloud data; Step 2: embed features of the preprocessed point cloud data; Step 3: Extract data features based on bidirectional long short-term memory network; Step 4: Perform local feature aggregation and global feature aggregation on the data to enhance feature extraction and improve data expression capabilities; Step 5: Build a point cloud data recognition model based on the fully connected layer, and train and evaluate the model; Step 6: Use the trained model to perform point cloud data recognition.

2. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 1 is characterized in that: The preprocessing of the point cloud data in step 1 is specifically as follows: Obtain raw point cloud data based on radar acquisition or public data sets, and perform denoising, normalization, and downsampling; The denoising process is to use a statistical filtering method to remove noise points in the point cloud data P to obtain the denoised point cloud data. : ； The normalization process is to use linear scaling to scale the coordinate values ​​of all point cloud data to the range of [0, 1]. The normalized point cloud data is expressed as : ； The downsampling process is to use random sampling or feature-based importance sampling method to reduce the number of points in the point cloud data to N. The sampled point cloud data is , , where represents the number of selected sampling points.

3. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 1 is characterized in that: The feature embedding of the preprocessed point cloud data in step 2 is specifically as follows: For the sampled point cloud data , extract the geometric coordinates of each point through a multi-layer perceptron to generate a high-dimensional feature representation , and add position coding , combined with geometric features, to obtain a feature representation containing location information : ； 。 4. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 3 is characterized in that: The data feature extraction in step 3 is specifically as follows: Based on the feature representation in step 2 , and use the bidirectional long short-term memory network to extract the context feature representation: ； ； ； in, Represents the forward LSTM network for the i-th point feature The output of represents the hidden state of the forward information of this point when processing the sequence, including the dependency relationship between this point and the previous point. Represents the reverse LSTM network for the i-th point feature The output of represents the hidden state of the backward information of this point when processing the sequence, including the dependency relationship between this point and the subsequent points. The final context feature of the i-th point is represented by the concatenation of the output features of the forward and reverse LSTMs, integrating the previous and next temporal information.

5. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 1 is characterized in that: The data feature aggregation in step 4 is specifically as follows: Aggregate local and global geometric features based on feature extracted data: ； ； in, It is the representation after local feature aggregation, from the neighborhood point N of the current point i Pooling is a pooling operation. It represents the representation after global feature aggregation, obtained from the features of all points. GlobalPooling is a global pooling operation. Data representing the features of the i-th point obtained based on step 3.

6. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 5 is characterized in that: The point cloud data recognition model in step 5 is specifically: After completing feature aggregation, the fully connected layer is used to aggregate the global features. Further processing is performed to obtain the feature representation after nonlinear activation , and use the Softmax layer to predict the category of the point cloud data, and output the probability distribution of the point cloud belonging to different categories : ； ； Among them, h represents the feature representation after processing by the fully connected layer, FC represents the function operation of the fully connected layer, Represents the output class probability distribution for point cloud input x.

7. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 6 is characterized in that: The point cloud data recognition model adopts the cross entropy loss function during the training process. To measure the difference between the model prediction result and the true label, the gradient of the loss relative to the model parameters is calculated through the back propagation algorithm, and the model parameters are updated using the optimization algorithm to reduce the loss function value: ； in, Represents the loss function value, which is used to evaluate the model performance. Represents the category indication of the true label, represents the category probability of the model output, represents the parameters of the model, Represents the learning rate, which controls the step size of parameter update.

8. The 3D point cloud target recognition method based on bidirectional long short-term memory network according to claim 6 is characterized in that: Adoption accuracy As an indicator for evaluating the model: ； in, is the indicator function, when the predicted label With the true label If they are equal, it returns 1, otherwise it returns 0. N represents the total number of samples.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer storable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images