3D object detection system and method based on point cloud and images from different perspectives
A three-dimensional target and detection system technology, applied in the field of target detection, can solve problems such as inability to retrieve it, and achieve the effect of improving accuracy
Active Publication Date: 2022-08-05
ZHEJIANG LAB
View PDF0 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
Back-end fusion, such as the patent "A 3D Target Detection System and Method Based on Point Cloud and Image Data" Easy deployment, high efficiency, etc.; but if the 3D object detection based on the point cloud is missed, the back-end fusion cannot retrieve it
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
Embodiment Construction
[0044] The present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, but not to limit the present invention.
[0045] The embodiment of the present invention adopts self-collected data in the park; wherein, the data set includes 10,000 pairs of laser point clouds (including timestamp information of each point of each frame of point cloud) and image data (including timestamp information of each image), The pose and attitude of the point cloud acquisition equipment lidar, the internal and external parameters of the image acquisition equipment camera, the three-dimensional information label and the two-dimensional information label of the target. Among them, the three-dimensional information label contains the three-dimensional coordinates x, y, z of the target in the lidar coordinate system, length, width and height ...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a three-dimensional target detection system and method based on point clouds and images under different viewing angles. First, the original point cloud data is processed, and the 3D bounding box of the candidate target and the point cloud feature tensor are output; then, according to the image data collected by multi-frame cameras from different perspectives, the collection timestamp of each image, the point cloud collection equipment and different cameras The calibration parameters of the candidate target and the three-dimensional bounding box of the candidate target are obtained to obtain the image feature tensor of the candidate target; then the point cloud feature tensor of the candidate target and the image feature tensor are feature-fused to obtain the fused feature tensor; finally The 3D detection information of the target is obtained by combining the fused feature tensor and the 3D bounding box of the candidate target. The invention solves the problems of time registration generated by multi-sensor fusion and point cloud motion compensation and other errors, and improves the accuracy of three-dimensional target detection.
Description
technical field [0001] The invention belongs to the technical field of target detection, and in particular relates to a three-dimensional target detection system and method based on point clouds and images from different viewing angles. Background technique [0002] Object detection technology, especially 3D object detection technology, is one of the most important tasks in autonomous driving environment perception. Due to the advantages of point cloud in 3D size information, the best 3D object detection technology at this stage is based on point cloud for 3D object detection, including the papers "Sparsely Embedded Convolutional Detection", "3DObject Proposal Generation and Detection from Point Cloud", etc. . However, point cloud features have obvious disadvantages in category discrimination, and can only be discriminated and judged by learning the rough outline of the target and the reflection intensity of some points through the neural network. [0003] Image technology...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06T7/00G06T7/246G06T7/80G06T5/50G06N3/04G06N3/08
CPCG06T7/0002G06T7/251G06T7/80G06T5/50G06N3/08G06T2207/10016G06T2207/20081G06T2207/20084G06T2207/20221G06T2207/30244G06N3/045
Inventor 马也驰华炜韩正勇李金鑫冯权张顺
Owner ZHEJIANG LAB