Monocular vision-based 3D model embedding method in gesture interactive video scene

Abstract

The invention discloses a monocular vision-based 3D model embedding method in a gesture interactive video scene, which belongs to the field of computer graphics and comprises the steps of monocular scene depth reconstruction, fine gesture extraction, 3D model rendering, occlusion judgment and gesture redrawing. According to the monocular vision-based 3D model embedding method in the gesture interactive video scene, the monocular depth recovery technology, the gesture extraction technology, the 3D model rendering technology, the occlusion judgment technology, the gesture redrawing technology and the like are combined, the occlusion relation between the gesture and the model can be effectively processed, and a seamless embedding effect is obtained. Moreover, the shielding problem generated during gesture interaction can be effectively solved, expensive depth detection equipment is not needed, and the actual application requirement is met.

Description

technical field [0001] The invention belongs to the field of computer graphics and relates to a method for embedding a 3D model based on monocular vision in a gesture interactive video scene. Background technique [0002] Augmented reality (Augmented Reality, AR) has the advantages of blending virtual and real, strong expressiveness and good interactivity. By embedding virtual information on images of real scenes, augmented reality technology presents users with an environment with realistic sensory effects. As a natural way of human-computer interaction, gesture interaction is widely used in augmented reality and virtual reality. [0003] Occlusion consistency is the basic principle that the immersive augmented reality environment needs to satisfy, and it requires the correct occlusion relationship between the elements in the augmented reality system. A video image is the projection of a 3D scene on a 2D plane, and restoring the spatial occlusion relationship between the ...

AI Technical Summary

Problems solved by technology

However, most of the augmented reality applications currently ignore the processing of occlusion relations, and the video is only drawn as the background of the virtual model

In gesture-interactive augmented reality applications, gesture interaction is highly dynamic, and the occlusion relationship between gestures and embedded models is not fixed. Drawing the video first and then drawing the embedded model can easily lead to visual confusion.

Although the depth information of the video scene can be obtained by using a depth sensor, or the method of stereo vision by using dual cameras, and the occlusion relationship of the scene can be reconstructed, there are problems in terms of equipment cost or portability

Images