3D point cloud detection algorithm of embedded computing platform

Abstract

The invention discloses a 3D point cloud detection algorithm of an embedded computing platform, which relates to the technical field of 3D point cloud detection, and comprises the following steps: S1, data processing; s2, performing pretreatment; s3, feature extraction; s4, hardware acceleration; s5, software optimization is carried out; s6, target detection; s7, post-processing; and S8, outputting and displaying. According to the 3D point cloud detection algorithm of the embedded computing platform, the parallel computing capability acceleration algorithm of the embedded GPU is utilized, the programmability of the FPGA is utilized to realize customized hardware acceleration, the signal processing capability of the DSP is utilized to carry out point cloud filtering and feature extraction, hardware acceleration is realized, information processing is ensured to be more timely, and in addition, through optimization of software, the calculation efficiency is improved. According to the technical scheme, the algorithm operation memory is sufficient, the situation that due to the fact that the memory is large, information processing is not smooth is avoided, in addition, through a voxel-based method, a point-based method and a projection-based method, features are extracted from point cloud data through the three methods, and the completeness of 3D information is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of 3D point cloud detection, and in particular to a 3D point cloud detection algorithm for an embedded computing platform. Background Art

[0002] In the field of autonomous driving, perception is an essential key technology for autonomous vehicles. Perception is a process in which autonomous vehicles analyze and observe the surrounding environment data collected by different sensors during driving. The sensor data includes 2D image data based on cameras, 3D laser point cloud data based on lidar and millimeter wave radar, etc. The target detection task based on 2D images is gradually maturing, and there are relatively many industrial applications.

[0003] However, at present, the 3D target detection task based on laser point cloud has not been truly put into application. When the 3D algorithm is applied, the 3D point cloud is projected onto a 2D plane, and then the 2D target detection algorithm is used for detection. This method has a small amount of calculation, but some 3D information will be lost. Alternatively, a voxel-based method is used: the 3D space is divided into voxel grids, and then a 3D convolutional network is used for detection. This method can retain 3D information, but the amount of calculation is large, and it is difficult to fully obtain information, resulting in incomplete data feature extraction.

[0004] Therefore, in view of this, the existing structure and deficiencies are studied and improved, and a 3D point cloud detection algorithm for an embedded computing platform is proposed. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a 3D point cloud detection algorithm for an embedded computing platform, which solves the problems raised in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented by the following technical solutions: a 3D point cloud detection algorithm for an embedded computing platform, the 3D point cloud detection algorithm for the embedded computing platform includes the following operating steps:

[0007] S1. Data processing:

[0008] Acquiring 3D point cloud data through sensors is completed by the driver on the embedded system;

[0009] S2. Preprocessing:

[0010] Preprocess the raw point cloud data to improve the efficiency and robustness of subsequent steps;

[0011] S3. Feature extraction:

[0012] Extract features from the preprocessed point cloud data;

[0013] S4, Hardware Acceleration:

[0014] Use the parallel computing power of embedded GPU to accelerate algorithms, use the programmability of FPGA to achieve customized hardware acceleration, and use the signal processing capabilities of DSP to perform point cloud filtering and feature extraction;

[0015] S5. Software optimization:

[0016] Optimize the code by using C / C++ instead of Python, use multi-core CPU or GPU for parallel computing, and achieve memory optimization by sharing memory and reducing data copying;

[0017] S6. Target Detection:

[0018] Use the pre-trained model to perform target detection on the extracted features;

[0019] S7, post-processing:

[0020] Post-processing the detected structure;

[0021] S8, output display:

[0022] Output the detection results to the system or application for final application of the algorithm.

[0023] Furthermore, in step S1, the sensor uses LiDAR and RGB-D camera to acquire 3D point cloud data.

[0024] Furthermore, in step S2, the preprocessing step includes:

[0025] Downsampling: Reduce the density of point clouds and reduce the amount of calculation. Common methods include voxel downsampling, farthest point sampling (FPS) and random sampling;

[0026] Denoising: remove noise points in the point cloud to improve data quality. Common methods include statistical filtering and radius filtering.

[0027] Coordinate system conversion: convert point cloud data to the agreed coordinate system;

[0028] Data augmentation: Increase the diversity of training data through rotation, translation, and scaling operations.

[0029] Furthermore, in step S3, the feature extraction method includes:

[0030] Voxel-based methods: discretize the cloud space into a voxel grid and then use 3D full convolution to extract features;

[0031] Point-based method: directly process point cloud data and use networks such as PointNet and PointNet++ to extract features;

[0032] Projection-based methods: project the 3D point cloud into a 2D image and then use a 2D convolutional network to extract features.

[0033] Furthermore, in step S6, target detection involves candidate box generation, feature classification and regression, and non-maximum suppression.

[0034] Furthermore, in step S7, the post-processing step includes:

[0035] Object tracking: associating objects in consecutive frames;

[0036] Data fusion: Fusing data from multiple sensors;

[0037] Visualize the results.

[0038] Furthermore, in step S8, the systems for outputting the display are the control system and the navigation system.

[0039] Furthermore, in step S1, the embedded system adopts a lightweight network, and the lightweight network adopts a lightweight network structure such as MobileNet, ShuffleNet, etc. as a backbone.

[0040] The present invention provides a 3D point cloud detection algorithm for an embedded computing platform, which has the following beneficial effects:

[0041] 1. The 3D point cloud detection algorithm of the embedded computing platform uses the parallel computing capability of the embedded GPU to accelerate the algorithm, uses the programmability of the FPGA to achieve customized hardware acceleration, and uses the signal processing capability of the DSP to perform point cloud filtering and feature extraction, thereby accelerating the hardware and ensuring that information processing is more timely. In addition, through software optimization, the algorithm has sufficient memory to run and avoid information processing jams caused by too much memory. In addition, through voxel-based methods, point-based methods, and projection-based methods, the above three methods all extract features from point cloud data to ensure the completeness of 3D information. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the algorithm flow of a 3D point cloud detection algorithm for an embedded computing platform of the present invention. DETAILED DESCRIPTION

[0043] The following embodiments of the present invention are described in further detail in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0044] like Figure 1 As shown, the present invention provides a technical solution: a 3D point cloud detection algorithm for an embedded computing platform, the 3D point cloud detection algorithm for the embedded computing platform includes the following operation steps:

[0045] S1. Data processing:

[0046] The 3D point cloud data is acquired through the sensor, which is completed by the driver on the embedded system. In step S1, the sensor uses LiDAR and RGB-D camera to acquire 3D point cloud data. In step S1, the embedded system uses a lightweight network, and the lightweight network uses a lightweight network structure such as MobileNet and ShuffleNet as the backbone;

[0047] S2. Preprocessing:

[0048] The original point cloud data is preprocessed to improve the efficiency and robustness of subsequent steps. In step S2, the preprocessing steps include:

[0049] Downsampling: Reduce the density of point clouds and reduce the amount of calculation. Common methods include voxel downsampling, farthest point sampling (FPS) and random sampling;

[0050] Denoising: remove noise points in the point cloud to improve data quality. Common methods include statistical filtering and radius filtering.

[0051] Coordinate system conversion: convert point cloud data to the agreed coordinate system;

[0052] Data enhancement: Increase the diversity of training data through rotation, translation, and scaling operations;

[0053] S3. Feature extraction:

[0054] 5. Extract features from the preprocessed point cloud data. In step S3, the feature extraction method includes:

[0055] Voxel-based methods: discretize the cloud space into a voxel grid and then use 3D full convolution to extract features;

[0056] Point-based method: directly process point cloud data and use networks such as PointNet and PointNet++ to extract features;

[0057] Projection-based method: project the 3D point cloud into a 2D image and then use a 2D convolutional network to extract features;

[0058] S4, Hardware Acceleration:

[0059] Use the parallel computing power of embedded GPU to accelerate algorithms, use the programmability of FPGA to achieve customized hardware acceleration, and use the signal processing capabilities of DSP to perform point cloud filtering and feature extraction;

[0060] S5. Software optimization:

[0061] Optimize the code by using C / C++ instead of Python, use multi-core CPU or GPU for parallel computing, and achieve memory optimization by sharing memory and reducing data copying;

[0062] S6. Target Detection:

[0063] Use the pre-trained model to perform target detection on the extracted features. In step S6, target detection involves candidate box generation, feature classification and regression, and non-maximum suppression;

[0064] S7, post-processing:

[0065] The detection structure is post-processed. In step S7, the post-processing step includes:

[0066] Object tracking: associating objects in consecutive frames;

[0067] Data fusion: Fusing data from multiple sensors;

[0068] Visualization of results;

[0069] S8, output display:

[0070] The detection results are output to the system or application for final application of the algorithm. In step S8, the output is displayed to the system as a control system and a navigation system.

[0071] Based on the above description, the parallel computing capability of the embedded GPU is used to accelerate the algorithm, the programmability of the FPGA is used to achieve customized hardware acceleration, and the signal processing capability of the DSP is used to perform point cloud filtering and feature extraction to achieve hardware acceleration and ensure that information processing is more timely. In addition, through software optimization, the algorithm can run with sufficient memory to avoid information processing jams caused by too much memory. In addition, through voxel-based methods, point-based methods, and projection-based methods, the above three methods all extract features from point cloud data to ensure the completeness of 3D information.

[0072] In summary, when the 3D point cloud detection algorithm of the embedded computing platform is used, the 3D point cloud data is first obtained through the sensor, which is completed by the driver on the embedded system. The sensor uses LiDAR and RGB-D camera to obtain 3D point cloud data. The embedded system uses a lightweight network. The lightweight network uses lightweight network structures such as MobileNet and ShuffleNet as the backbone; the original point cloud data is preprocessed to improve the efficiency and robustness of subsequent steps. The preprocessing steps include: downsampling: reduce the point cloud density and reduce the amount of calculation. Common methods include voxel downsampling, farthest point sampling (FPS) and random sampling; denoising: remove noise points in the point cloud to improve data quality. Common methods include statistical filtering and radius filtering; coordinate system conversion: convert the point cloud data to the same coordinate system; data enhancement: increase the diversity of training data through rotation, translation, and scaling operations; extract features from the preprocessed point cloud data. The feature extraction methods include: voxel-based method: discretize the cloud space into a voxel grid, and then use 3D full convolution to extract features; point-based method: directly process the point cloud data According to the data, use PointNet, PointNet++ and other networks to extract features; projection-based method: project the 3D point cloud to a 2D image, and then use a 2D convolutional network to extract features; use the parallel computing power of the embedded GPU to accelerate the algorithm, use the programmability of the FPGA to achieve customized hardware acceleration, and use the signal processing capabilities of the DSP for point cloud filtering and feature extraction; optimize the code by using C / C++ instead of Python, use multi-core CPUs or GPUs for parallel computing, and achieve memory optimization by sharing memory and reducing data copies; use pre-trained models to perform target detection on the extracted features, and target detection involves candidate box generation, feature classification, regression, and non-maximum suppression; post-process the detection structure, and the post-processing steps include: target tracking: associate targets in consecutive frames; data fusion: fuse data from multiple sensors; visualize the results; finally, output the detection results to the system or application for the final application of the algorithm, and the output display to the system is the control system and navigation system. By applying the algorithm to the control system and navigation system in the vehicle, it is convenient for the car to perform autonomous driving.

[0073] The embodiments of the present invention are given for the purpose of illustration and description, and are not intended to be exhaustive or to limit the invention to the disclosed forms. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present invention and to enable those of ordinary skill in the art to understand the present invention and thereby design various embodiments with various modifications suitable for specific uses.

Claims

1. A 3D point cloud detection algorithm for an embedded computing platform, characterized in that: The 3D point cloud detection algorithm of the embedded computing platform includes the following steps: S1. Data processing: Acquiring 3D point cloud data through sensors is completed by the driver on the embedded system; S2. Preprocessing: Preprocess the raw point cloud data to improve the efficiency and robustness of subsequent steps; S3. Feature extraction: Extract features from the preprocessed point cloud data; S4, Hardware Acceleration: Use the parallel computing power of embedded GPU to accelerate algorithms, use the programmability of FPGA to achieve customized hardware acceleration, and use the signal processing capabilities of DSP to perform point cloud filtering and feature extraction; S5. Software optimization: Optimize the code by using C / C++ instead of Python, use multi-core CPU or GPU for parallel computing, and achieve memory optimization by sharing memory and reducing data copying; S6. Target Detection: Use the pre-trained model to perform target detection on the extracted features; S7, post-processing: Post-processing the detected structure; S8, output display: Output the detection results to the system or application for final application of the algorithm.

2. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S1, the sensor uses LiDAR and RGB-D camera to acquire 3D point cloud data.

3. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S2, the pre-processing step includes: Downsampling: Reduce the density of point clouds and reduce the amount of calculation. Common methods include voxel downsampling, farthest point sampling and random sampling; Denoising: remove noise points in the point cloud to improve data quality. Common methods include statistical filtering and radius filtering. Coordinate system conversion: convert point cloud data to the agreed coordinate system; Data augmentation: Increase the diversity of training data through rotation, translation, and scaling operations.

4. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S3, the feature extraction method includes: Voxel-based methods: discretize the cloud space into a voxel grid and then use 3D full convolution to extract features; Point-based method: directly process point cloud data and use PointNet and PointNet++ networks to extract features; Projection-based methods: project the 3D point cloud into a 2D image and then use a 2D convolutional network to extract features.

5. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S6, target detection involves candidate box generation, feature classification and regression, and non-maximum suppression.

6. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S7, the post-processing step includes: Object tracking: associating objects in consecutive frames; Data fusion: Fusing data from multiple sensors; Visualize the results.

7. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S8, the systems that output the display are the control system and the navigation system.

8. The 3D point cloud detection algorithm for an embedded computing platform according to claim 1, characterized in that: In step S1, the embedded system adopts a lightweight network, and the lightweight network adopts a MobileNet or ShuffleNet lightweight network structure as a backbone.

Images