Real-time three-dimensional scene reconstruction system and method based on inertia and depth vision

Abstract

The invention discloses a real-time three-dimensional scene reconstruction system and method based on inertia and depth vision. The relative displacement and relative attitude change obtained by an IMU sensor through integral computation in short time is relatively accurate, and accordingly, the delta PIMU of an IMU can be used as an approximate real value to compare with delta PRGB and delta PD. If the difference between delta PRGB and delta PIMU exceeds a certain threshold value, the relative displacement and relative attitude change calculated by an RGB camera through tracking characteristics on successive frames is determined to be not accurate enough; in a similar way, if the difference between delta PD and delta PIMU exceeds a certain threshold value, the relative displacement and relative attitude change obtained in an ICP algorithm of matching point cloud of successive frames is determined to be not accurate enough; then data is fused based on comparison results, and relative displacement and attitude change of equipment is estimated in real time, so as to improve three-dimensional scene reconstruction precision and equipment fault tolerance.

Description

technical field [0001] The invention relates to the field of computer vision and multi-sensor fusion navigation drawing, in particular to a system and method for real-time three-dimensional scene reconstruction based on inertial sensors and depth vision sensors. Background technique [0002] Around 2010, Israel PrimeSense made a breakthrough in the miniaturization and modularization technology of depth sensors based on structured light, and cooperated with Microsoft to develop the Kienct sensor. A color RGB camera and a depth D camera are integrated on the Kienect sensor, which can quickly obtain point cloud data on the surface of objects within 0.5 meters to 4 meters around. It is a somatosensory device used to capture changes in players' movements. Once the technology was introduced to the market, it attracted the attention of both industry and academia. In the past few years, similar sensors have been launched one after another, such as the Structure Sensor of Occipital ...

AI Technical Summary

Problems solved by technology

If the shape of the surrounding objects changes too much, the algorithm will fail

[0008] At present, the real-time 3D environment reconstruction system based on RGB-D can achieve better 3D reconstruction effect based on visual odometry technology and ICP technology when the surrounding environment feature information is rich. However, when the surrounding environment conditions change, such as Light changes, changes in surrounding environment characteristics, etc., RGB-D real-time 3D environment reconstruction will encounter difficulties

Once the real-time 3D environment reconstruction is interrupted due to changes in the surrounding environmental conditions, it is difficult to continue working, reducing the robustness of the entire system

Images