An Improved Efficiency Method for Point R-CNN Models

Abstract

This invention discloses an improved PointRCNN model efficiency method, comprising the following steps: inputting a public point cloud dataset into the model; performing preliminary feature extraction on the input data; adding submanifold sparse convolution and feature distillation based on the features initially extracted by the model; selecting a teacher model to extract point cloud features to form new intermediate layer features; comparing these features with those extracted by the student model; and guiding the student model to learn the teacher model's features with fewer parameters. This improved PointRCNN model efficiency method, in the traditional two-stage point cloud object detection model PointRCNN, adds submanifold sparse convolution when the number of points in the first stage is small, ensuring good feature extraction even with sparse points; by performing knowledge distillation on the original model, it optimizes the model structure while significantly improving the speed of point cloud object detection, and ultimately enables accurate point cloud object detection under model training.

Description

Technical Field

[0001] This invention relates to the field of lidar point cloud target detection technology, specifically to an improved PointRCNN model efficiency method. Background Technology

[0002] A lidar is a radar system that uses laser beams to detect the position, velocity, and other characteristics of a target. Its working principle is to first emit a detection laser beam towards the target, then compare the received signal reflected back from the target with the emitted signal, and after appropriate processing, obtain relevant information about the target, such as the target's distance, azimuth, altitude, velocity, attitude, and even shape.

[0003] As autonomous driving technology becomes increasingly popular, improving its effectiveness using LiDAR has become a pressing issue. Point cloud object detection is a technique for object recognition and localization based on 3D point cloud data, widely used in autonomous driving, robotics, virtual reality, and other fields. The core of point cloud object detection is extracting effective features from disordered 3D point cloud data, processing and analyzing these features to accurately detect target objects. Compared to traditional image object detection, point cloud object detection has some unique advantages, such as handling non-planar scenes and providing more accurate object pose information. However, point cloud object detection also has some drawbacks. Point cloud data is typically denser and richer than image data, requiring more computational resources and storage space. The geometric structure of point cloud data is more complex, making feature extraction more difficult than from image data. Because point cloud data is 3D, the occlusion problem between objects is more complex, which can affect the accuracy of object detection. When performing point cloud object detection, the two-stage detection structure is quite complex, and how to improve it to increase recognition accuracy and reduce time has gradually become a key research direction; in view of this, we propose a method to improve the efficiency of the PointRCNN model. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides an improved method for the efficiency of the PointRCNN model, which solves the problem that the occlusion problem between objects is more complex and affects the accuracy of object detection.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an improved PointRCNN model efficiency method, the method comprising the following steps:

[0006] Step S1: Input the point cloud public dataset into the model;

[0007] Step S2: Perform preliminary feature extraction on the input data;

[0008] Step S3: Before performing foreground point segmentation in the first stage, add sub-manifold sparse convolution based on the features initially extracted by the model to ensure that features can be effectively extracted even with sparse points.

[0009] Step S4: Incorporate the feature distillation method. By selecting the teacher model, extract the point cloud features to form new intermediate layer features, and compare them with the features extracted by the student model. This guides the student model to learn the features of the teacher model with fewer parameters, optimizes the model structure, and improves model efficiency.

[0010] Optionally, the principle of sparse convolution of submanifolds in step S3 is as follows:

[0011] After PointRCNN initially extracts features, submanifold sparse convolution is added to further extract point cloud features. Due to the sparsity of point cloud data, submanifold sparse convolution can make full use of the local structure of the point cloud data, reducing the computational cost of the model while improving its efficiency and avoiding feature loss caused by local data sparsity.

[0012] Optionally, in step 4, a knowledge distillation module can be added to the model to optimize the model structure.

[0013] Optionally, step S4 includes the following steps:

[0014] Step S41: The DGCNN dynamic graph convolutional model has good accuracy and can perform point cloud classification and segmentation tasks.

[0015] Step S42: Input the initially extracted features into the teacher model for feature extraction and compare them with the student model;

[0016] Step S43: Based on the characteristics of the teacher model, guide the student model to learn with a smaller amount of data;

[0017] Step S44: Replace the original feature extraction module with the trained student model as the backbone network of the second stage.

[0018] Optionally, the DGCNN can be used as a teacher model.

[0019] Optionally, step S3 includes:

[0020] Step S31: Construct submanifold: Map the point cloud data into a local coordinate system and construct a submanifold for sparse convolution. For each point, define a local coordinate system centered on that point. Construct the local coordinate system by calculating the nearest neighbor of each point in the point cloud data.

[0021] Step S32: Calculate the relative coordinates: Calculate the relative coordinates by subtracting the coordinates of the center of the local coordinate system from the actual coordinates of the point;

[0022] Step S33: Represent the relative coordinates as a set of components and map them to a multidimensional space;

[0023] Step S34: Perform sparse convolution: Use a sparse filter to perform convolution operations on these components. In the convolution operation, only the nearest neighbors of the current point are considered, thus taking advantage of the sparsity of the point cloud data;

[0024] Step S35: Feature Extraction: Combine the convolution results into a feature vector. By using one or more stacked submanifold sparse convolutions, combine the convolution operation results into one or more scale feature vectors, and use them as the feature representation of that point.

[0025] This invention provides a method to improve the efficiency of PointRCNN models. It has the following beneficial effects:

[0026] This improved PointRCNN model efficiency method adds sub-manifold sparse convolution to the traditional two-stage point cloud object detection model PointRCNN when the number of points in the first stage is small, ensuring that feature extraction can be performed well even with sparse points. By performing knowledge distillation on the original model, the speed of point cloud object detection is greatly improved while optimizing the model structure, and the model can accurately detect point cloud objects after training. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the process structure of the present invention;

[0028] Figure 2 This is a diagram of the PointRCNN network model of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1-2 This invention provides a technical solution: an improved PointRCNN model efficiency method, the method comprising the following steps:

[0031] Step S1: Input the point cloud public dataset into the model;

[0032] Step S2: Perform preliminary feature extraction on the input data. Use the original feature extraction module in the model to encode and decode the point cloud data to extract the original point cloud features.

[0033] Step S3: Before foreground point segmentation in the first stage, a sub-manifold sparse convolution is added based on the features initially extracted by the model to ensure effective feature extraction even with sparse points. A bottom-up 3D proposal generation method is used through point cloud segmentation to learn the segmentation of foreground points, capture information from the point cloud context, and make accurate predictions. Before segmenting foreground points, a sub-manifold sparse convolution is added to ensure effective feature extraction even with sparse points. Step S3 includes:

[0034] Step S31: Construct submanifold: Map the point cloud data into a local coordinate system and construct a submanifold for sparse convolution. For each point, define a local coordinate system centered on that point. Construct the local coordinate system by calculating the nearest neighbor of each point in the point cloud data.

[0035] Step S32: Calculate the relative coordinates: Calculate the relative coordinates by subtracting the coordinates of the center of the local coordinate system from the actual coordinates of the point;

[0036] Step S33: Represent the relative coordinates as a set of components and map them to a multidimensional space;

[0037] Step S34: Perform sparse convolution: Use a sparse filter to perform convolution operations on these components. In the convolution operation, only the nearest neighbors of the current point are considered, thus taking advantage of the sparsity of the point cloud data;

[0038] Step S35: Feature Extraction: Combine the convolution results into a feature vector. By using one or more sub-manifold sparse convolutions, combine the convolution operation results into one or more scale feature vectors, and use them as the feature representation of that point.

[0039] Step S4: Incorporate the feature distillation method. By selecting the teacher model, extract the point cloud features to form new intermediate layer features, and compare them with the features extracted by the student model. This guides the student model to learn the features of the teacher model with fewer parameters, optimizes the model structure, and improves model efficiency.

[0040] Step S41: The DGCNN dynamic graph convolutional model has good accuracy and can perform point cloud classification and segmentation tasks, so DGCNN is used as the teacher model.

[0041] Step S42: Input the initially extracted features into the teacher model for feature extraction and compare them with the student model;

[0042] Step S43: Based on the characteristics of the teacher model, guide the student model to learn with a smaller amount of data;

[0043] Step S44: Replace the original feature extraction module with the trained student model as the backbone network of the second stage.

[0044] The principle and method of sparse convolution of submanifolds in step S3 are as follows:

[0045] After PointRCNN initially extracts features, submanifold sparse convolution is added to further extract point cloud features. Due to the sparsity of point cloud data, submanifold sparse convolution can make full use of the local structure of the point cloud data, reducing the computational cost of the model while improving its efficiency and avoiding feature loss caused by local data sparsity.

[0046] In step 4, a knowledge distillation module is added to the model to optimize the model structure.

[0047] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An improved method for Point R-CNN model efficiency, characterized in that: The method includes the following steps: Step S1: Input the point cloud public dataset into the model; Step S2: Perform preliminary feature extraction on the input data; Step S3: Before performing foreground point segmentation in the first stage, add sub-manifold sparse convolution based on the features initially extracted by the model; Step S31: Construct submanifold: Map the point cloud data into a local coordinate system and construct a submanifold for sparse convolution. For each point, define a local coordinate system centered on that point. Construct the local coordinate system by calculating the nearest neighbor of each point in the point cloud data. Step S32: Calculate the relative coordinates: Calculate the relative coordinates by subtracting the coordinates of the center of the local coordinate system from the actual coordinates of the point; Step S33: Represent the relative coordinates as a set of components and map them to a multidimensional space; Step S34: Perform sparse convolution: Use a sparse filter to perform convolution operations on these components. In the convolution operation, only the nearest neighbor of the current point is considered, thus taking advantage of the sparsity of the point cloud data. Step S35: Feature extraction: Combine the convolution results into a feature vector. By using one or more sub-manifold sparse convolutions, combine the convolution operation results into one or more scale feature vectors and use them as the feature representation of the point. Step S4: Incorporate the feature distillation method. By selecting the teacher model, extract the point cloud features to form new intermediate layer features, and compare them with the features extracted by the student model. This guides the student model to learn the features of the teacher model with fewer parameters. Step S41: The DGCNN dynamic graph convolutional model has good accuracy; Step S42: Input the initially extracted features into the teacher model for feature extraction and compare them with the student model; Step S43: Based on the characteristics of the teacher model, guide the student model to learn with a smaller amount of data; Step S44: Replace the original feature extraction module with the trained student model as the backbone network of the second stage.

2. The improved Point R-CNN model efficiency method according to claim 1, characterized in that: The principle and method of submanifold sparse convolution in step S3 are as follows: After PointRCNN initially extracts features, sub-manifold sparse convolution is added to further extract point cloud features.

3. The improved Point R-CNN model efficiency method according to claim 1, characterized in that: In step 4, a knowledge distillation module is added to the model.

4. The improved Point R-CNN model efficiency method according to claim 3, characterized in that: The DGCNN is used as the teacher model.

Images