Three-dimensional object detection and tracking method based on streaming data

Abstract

The invention relates to the field of three-dimensional target detection and tracking, in particular to a three-dimensional object detection and tracking method based on streaming data, which comprises the following steps: inputting a front key frame and a rear key frame comprising point cloud data and image data, performing feature extraction on the data to obtain a feature map, and performing related operation on the two feature maps to obtain a related feature map; inputting the feature map into a candidate box extraction network to obtain a candidate box; and obtaining feature blocks fromthe feature map and the related feature map through the candidate boxes and inputting the feature blocks into the regression network to obtain a three-dimensional box and a three-dimensional box offset of the detected object respectively; solving other frames except the key frames through an interpolation method, and associating targets in all frames to obtain a tracking result. Only the key frames need to be detected, the stream data detection speed is increased, the requirement of an automatic driving environment for real-time performance is met, and meanwhile better stability is achieved; meanwhile, point cloud information and image information are fused, advantages and disadvantages are complementary, and the accuracy of object detection is improved.

Description

technical field [0001] The invention relates to the field of three-dimensional object detection and tracking, and more specifically, to a three-dimensional object detection and tracking method based on flow data. Background technique [0002] At present, autonomous driving and visual perception tasks are mainly divided into image-based, point cloud-based, and image-based and point cloud fusion, specifically: [0003] 1. Image-based methods are mainly represented by Mono3D, 3DOP, etc. Since the image data has no depth information, it is necessary to add additional hand-designed 3D features. However, the single RGB data and specific hand-designed features are not conducive to the neural network to effectively learn 3D spatial information, and also limit the expansion of this scene. In addition, the acquisition of manual features generally takes too long, and such methods currently have limited effects and slow progress. [0004] 2. The point cloud-based scheme can be subdivi...

AI Technical Summary

Problems solved by technology

[0007] In order to overcome the problems of poor detection effect and poor real-time performance in the above-mentioned prior art, the present invention provides a three-dimensional object detection and tracking method based on stream data. Accurate detection and positioning, and improve the detection speed, can realize real-time detection

Images