Systematic target identification and tracking method based on multi-modal data

Abstract

The invention discloses a systematic target identification and tracking method based on multi-modal data. The method comprises the following steps: carrying out online fusion on point cloud data and image data, and generating RGB-D three-dimensional data images; taking the RGB-D three-dimensional data images as input of a deep neural network, and training the deep neural network through a database, wherein the trained deep neural network can output types and distances of various targets in the RGB-D three-dimensional data images and complete identification of the various targets; and for the identified targets, outputting the distances and sizes of the targets and two-dimensional coordinates of target pixels, extracting image features of the targets by using the deep neural network to match identification output of each frame, and predicting, screening and tracking relative positions of the targets by fitting and updating a target motion model so as to track detected specific targets. According to the invention, target identification and tracking of a ground unmanned platform in an unstructured complex environment in which the vehicle driving speed is high, much environmental unknown interference exists, and target appearance positions are random can be realized.

Description

technical field [0001] The invention belongs to the technical field of ground unmanned platform target identification and tracking, and in particular relates to a systematic target identification and tracking method based on multimodal data. Background technique [0002] The ground unmanned platform has the characteristics of high maneuverability and high adaptability. To increase the driving speed of the unmanned platform, it is necessary to improve the detection accuracy and increase the detection frequency. In order to improve its adaptability, multi-modal data is required. Respond to changes in the surrounding environment and the occurrence of extreme situations. When the ground unmanned platform maneuvers autonomously, the multimodal data it collects needs to focus on a wide variety of obstacles or targets. To be able to distinguish different road terrains, it can select one of the detection data for different detection tasks. Different sensors have Corresponding to th...

AI Technical Summary

Problems solved by technology

[0006] 1. The target recognition method based on color images is easily affected by changes in illumination. In the case of blackout, ordinary color images will not be able to provide pixel information for recognition

[0007] 2. The sparseness of the point cloud increases sharply with the increase of the distance, and there is no laser point that can represent the target on the distant target, resulting in the inability to recognize the distant target

[0008] 3. Carry out target recognition and tracking on multiple modal data, and then fuse them at the output end, which can make up for the inherent shortcomings of different data types, but it ignores the correlation between various data types during the recognition process, and at the same time causes the recognition system to Redundancy and increased computational load

[0009] Based on the above shortcomings, the existing unmanned ground platforms cannot work stably for a long time in an unstructured environment

Images