Self-adaptation virtual and actual three-dimensional registration method based on multiple natural characteristics

Abstract

The invention discloses a self-adaptation virtual and actual three-dimensional registration method based on multiple natural characteristics. The self-adaptation virtual and actual three-dimensional registration method comprises steps including that various natural characteristics are generated by means of analyzing video sequences of an actual scene, aggregations of the various natural characteristics are also generated, the states of the various natural characteristics are comprehensively analyzed, priority for virtual and actual three-dimensional registration of the various natural characteristics is obtained, a type of natural characteristics for virtual and actual three-dimensional registration is interactively selected according to the priority or by a user, then the various natural characteristics in the selected type of the natural characteristic aggregation are detected in the video sequences of the actual scene, the natural characteristics with high stability and / or distinctiveness is selected and used as natural characteristics for final registration, and finally, three-dimensional registration of a virtual object in the actual scene is carried out by the aid of the natural characteristics for final registration. The self-adaptation virtual and actual three-dimensional registration method can be widely applied to virtual and actual three-dimensional registration and fusion in virtual reality systems for interactive type digital entertainment, sports researches and training simulation, distance education and training and the like.

Description

technical field [0001] The invention relates to the fields of computer vision, augmented reality and virtual reality, in particular to an adaptive virtual-real three-dimensional registration method based on multiple natural features. Background technique [0002] Augmented reality is a further extension of virtual reality. With the help of necessary equipment, computer-generated virtual objects and objectively existing real environments coexist in the same augmented reality system, presenting virtual objects and real environments to users from the perspective of sensory and experience effects. An integrated augmented reality environment, augmented reality has the basic characteristics of combination of virtual and real, real-time interaction, and three-dimensional registration. [0003] Virtual-real 3D registration technology is an important research content in the field of augmented reality. Virtual-real 3D registration needs to calculate the conversion relationship between...

AI Technical Summary

Problems solved by technology

The virtual-real registration method based on artificial markers is relatively mature, but it has inevitable limitations: users are required to manually set markers, which increases the complexity of user use; destroys the realism of the original scene and makes users feel unreal ; Some scenes cannot set markers, such as live TV, etc.

[0005] In 2003, Davison et al. of the State University of New York at Buffalo proposed a virtual-real 3D registration without markers. This method constructs a sparse 3D model of a scene composed of feature points, but the tracking of feature points is unstable, which limits the use of augmented reality. The scope and quality of the registration cannot make the interaction of augmented reality wider and richer in content

On the other hand, the feature points used for camera pose estimation in this method are all stationary relative to the scene, which limits the user's ability to interact with the augmented reality scene to a certain extent.

In 2006, Julien et al. of Keio University in Japan proposed virtual-real 3D registration based on natural markers. The problem with this method is that when the proportion of natural markers in the image is small, the feature points on the natural markers are in the image. The proportion of the ratio becomes smaller, resulting in a decrease in the performance of the method

[0006] The current virtual-real 3D registration methods usually use a specific type of natural features in the real scene to complete the virtual-real 3D registration. However, a single type of natural feature is difficult to make full use of the rich natural features in the real scene. In some augmented reality scenes Unable to meet the diverse virtual and real 3D registration needs of users

Images