A physical object interaction method and system for real-time fusion of virtual and real objects

Abstract

The present disclosure provides a physical object interaction method and system for real-time fusion of virtual and real objects, which calculates the mapping relationship between the virtual reality equipment coordinate system of the server and the client, including rotation matrix and displacement vector, so as to realize the position of the object on the coordinate representation Unification; obtain the template of the target object, model the template features, track the position of the desktop object, extract the outline of the object, obtain the main direction of the object, obtain two fixed points in the main direction, and calculate through singular value decomposition The rotation matrix of the object relative to the initial state, the rotation angle is calculated by the rotation matrix; the displacement of the real object from the server side relative to the initial position and the rotation relative to the initial orientation are obtained, and the position is adjusted to calculate the current position and orientation of the virtual object , displaying the virtual object on the virtual reality device of the client according to the obtained position and orientation. The disclosure has the advantages of natural interaction, convenient material acquisition, real-time and the like.

Description

technical field [0001] The disclosure belongs to the technical field of mixed reality, and relates to a physical object interaction method and system for real-time fusion of virtual and real objects. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] Mixed reality (MR) is the fusion of the real and virtual worlds to produce new environments and visualizations. Physical and digital objects can co-exist in real time in a mixed reality environment. With the maturity of mixed reality technology, traditional interaction methods are not suitable for interactive scenarios where virtual reality is integrated. Object interaction, as an emerging 3D interaction method, is considered as a natural and easy-to-learn paradigm that people can use. Physical interaction is an interactive way of interacting with digital information through physical entities...

AI Technical Summary

Problems solved by technology

The complexity of the template extraction process limits the types of object recognition; based on the learning method, it is necessary to establish a data set for each object to be recognized. The data set contains image data or point cloud models, etc., and the construction process is complicated.

Images