Real object interaction method and system for real-time fusion of virtual and real objects

Abstract

The invention provides a real object interaction method and system for real-time fusion of virtual and real objects, and the method comprises the steps of carrying out the calculation of a mapping relation between virtual reality equipment coordinate systems of a server and a client, including a rotation matrix and a displacement vector, so as to achieve the unification of the positions of the objects in coordinate representation; acquiring a template of a target object, modeling template features, performing position tracking on a desktop object, extracting an object contour, acquiring a maindirection of the object, acquiring two fixed points in the main direction, calculating a rotation matrix of the object relative to an initial state through singular value decomposition, and calculating a rotation angle through the rotation matrix; obtaining the displacement of the real object from the server side relative to the initial position and the rotation of the real object relative to theinitial orientation, adjusting the position, calculating to obtain the current position and orientation of the virtual object, and displaying the virtual object on the virtual reality equipment of the client side according to the obtained position and orientation. The invention has the advantages of natural interaction, convenient material taking, real-time performance and the like.

Description

technical field [0001] The disclosure belongs to the field of mixed reality technology, and relates to a physical object interaction method and system for real-time fusion of virtual and real objects. Background technique [0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art. [0003] Mixed reality (MR) is the merging of the real world with the virtual world to produce new environments and visualizations. Physical and digital objects can co-exist in real time in a mixed reality environment. With the maturity of mixed reality technology, traditional interaction methods are not suitable for interactive scenarios where virtual reality is integrated. Object interaction, as an emerging 3D interaction method, is considered as a natural and easy-to-learn paradigm that people can use. Object interaction is an interactive way of interacting with digital information through physica...

AI Technical Summary

Problems solved by technology

The complexity of the template extraction process limits the types of object recognition; based on the learning method, it is necessary to establish a data set for each object to be recognized. The data set contains image data or point cloud models, etc., and the construction process is complicated.

Images