Man-machine interaction virtual-real fusion method and device based on mixed reality head-mounted display

Abstract

The invention discloses a human-computer interaction virtual-real fusion method and device based on a mixed reality head-mounted display. The method comprises the following steps: acquiring an internal parameter matrix M < internal > and an external parameter matrix M < external > of a camera; and according to the internal parameter matrix and the external parameter matrix, obtaining tracking coordinate system related information {Pori, Roi} of the MR equipment and a pose relationship between the entity camera and a tracking origin, obtaining an external parameter M'of the camera, and according to the internal parameter and the external parameter of the camera, obtaining a fused image. The method solves the problem that in the prior art, in order to achieve interactivity under the two conditions, a virtual human model of which the clothes, the skin color, the skeleton length and the like are similar to those of a testee is generally constructed in a virtual scene by means of a modeling technology, and the virtual human model and the testee move synchronously by means of data driving. Due to the fact that an existing modeling technology cannot well guarantee the modeling precision, a virtual human model has a certain difference with a tester in the aspects of skin color, clothing, skeleton length and the like, and the authenticity of the interaction effect is reduced.

Description

technical field [0001] The invention proposes a human-computer interaction method for virtual and real dynamic fusion suitable for mixed reality simulation of civil aircraft. The real scene picture is obtained through the camera of the mixed reality head display, the three-dimensional position information of the joint points of the human body is obtained by using a multi-camera system, and then the joint The three-dimensional position information of the point and the image processing technology identify the pixel area of ​​the limb in the picture, and finally extract the pixel area and merge it into the virtual display scene to realize a more realistic and accurate human-computer interaction virtual-real fusion method, so the present invention belongs to Mixed reality, human-computer interaction technology field. Background technique [0002] With the continuous development of intelligent technology, more and more intelligent equipment is used in people's life, work and stud...

AI Technical Summary

Problems solved by technology

As the existing modeling technology cannot guarantee the modeling accuracy well, there are some differences between the virtual human model and the tester in terms of skin color, clothing, bone length, etc., and the authenticity of the interaction effect is reduced.

Images