Vision and IMU sensor fusion positioning system based on dynamic object semantic segmentation

Abstract

The invention discloses a vision and IMU sensor fusion positioning system based on dynamic object semantic segmentation. The system comprises a front-end algorithm which is divided into a feature extraction module and a tracking module which are used for obtaining the data association of feature points, and tracking the feature points between adjacent frames through a KLT method; an instance segmentation and tracking module performs tracking by using a Deep SORT algorithm and is used for providing data association of semantic information; a dynamic object processing module identifies the dynamic feature points through a dynamic object processing algorithm, and rejects the dynamic feature points in positioning and mapping; and an IMU pre-integration module is used for carrying out integration on an IMU measurement value, IMU pre-integration is adopted as an observation value, and direct integration is carried out after a world coordinate system is converted into a local coordinate system; By utilizing the advantages of the visual sensor and the IMU sensor, the IMU sensor improves the problem of positioning failure caused by image blurring when the monocular camera moves fast; and meanwhile, the visual sensor solves the problem that the IMU accumulative error is relatively large.

Description

technical field [0001] The invention relates to the technical field of visual positioning, in particular to a visual and IMU sensor fusion positioning system based on semantic segmentation of dynamic objects. Background technique [0002] With the gradual improvement of the global satellite navigation system and the rapid development of mobile Internet and wireless communication technology, navigation and location-based services (Location-Based Services, LBS) are of great value to the fields of emergency, national defense, logistics, transportation, advertising and social networking. . According to the "2020 White Paper on the Development of China's Satellite Navigation and Location-Based Service Industry", the annual output value of my country's navigation and location-based service industry reaches hundreds of billions of yuan. With the increasing popularity of smart phones and wearable devices, the demand for navigation and location services is still in the stage of expl...

AI Technical Summary

Problems solved by technology

[0006] In the actual application process, there are often problems with monocular sensor positioning: the image acquired by the camera is easily disturbed by the external environment (occlusion, moving objects, non-textured scenes, lighting changes, etc.); the image will be blurred when the movement is fast, resulting in Positioning failed; the monocular camera cannot obtain the scale information of the real world, so the camera coordinate system cannot be aligned with the real world coordinate system, and there is a scale deviation; the traditional positioning algorithm based on the monocular camera has no ability to handle dynamic scenes, resulting in errors data association, resulting in poor positioning accuracy or even lost positioning

In a dynamic environment, mismatches are prone to occur during data association, which affects positioning accuracy

In addition, ghosting will appear in the constructed point cloud or grid map, which limits application functions such as map positioning, navigation, obstacle avoidance, and interaction

Images