Dynamic object perception system and method based on multi-source sensor information fusion

Abstract

The application discloses a dynamic object perception system and method based on multi-source sensor information fusion, relates to the field of dynamic object perception, and through parallel processing of multi-source heterogeneous data such as RGB-D images, 3D point clouds and millimeter wave radars, and introduction of a cross-modal inconsistency measurement mechanism, differences and conflicts between different sensor features are actively analyzed and quantified to evaluate the perception confidence of each sensing mode under the current environment in real time. Based on this, an adaptive attention mechanism is driven to dynamically generate the fusion weight of each mode, and finally, intelligent weighted fusion of each mode feature vector is realized, and a high-robustness fusion target state is output. In this way, low-quality or conflicting sensor information can be intelligently suppressed, and the contribution of high-quality information can be enhanced, so that more robust and accurate dynamic object perception can be realized in a complex and changeable environment.

Images