A virtual-to-real gesture interaction calibration method, system, device and medium

Abstract

The application discloses a virtual and real gesture interaction calibration method, comprising the following steps: acquiring a posture quaternion under a first gesture, a first orientation of a virtual object in a virtual scene, and a first angle difference; determining a first calibration quaternion according to the posture quaternion and the first angle difference; performing first calibration on the posture quaternion according to the first calibration quaternion to obtain a first target quaternion; performing second calibration on a virtual model hand according to the first target quaternion after the first calibration and an initial coordinate of a skeleton root node to obtain a first skeleton root node coordinate; determining a second target quaternion of the virtual model hand and a second skeleton root node coordinate according to a second angle difference; and performing third calibration on the virtual model hand according to the second target quaternion and the second skeleton root node coordinate. The method can improve user experience. The application can be widely applied in the technical field of inertial motion capture.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of inertial motion capture, and particularly to a virtual and real gesture interaction calibration method, system, device and storage medium. BACKGROUND

[0002] Virtual Reality (VR) gesture interaction is generally achieved through a data glove. To achieve correct gesture interaction, a series of calibration work needs to be performed on the interaction glove. For example, an existing inertial motion capture data glove requires a wearer to perform a plurality of different calibration gestures, which takes a long calibration time. After calibration, the virtual model hand can be driven. Moreover, because the interaction function is limited to rotational motion (pitch, roll and yaw), and the root node coordinate is fixed, the first calibration results in the phenomenon of crossed hands, i.e., the virtual right hand is on the left side of the user's field of view, and the virtual left hand is on the right side of the user's field of view. Moreover, to achieve spatial positioning, the traditional technology needs to use peripheral devices such as an HTC Tracker, which requires some tedious calibration work. However, after the above calibration work is completed, how to combine the real world and the virtual world, such as grabbing a virtual world object, is a problem. Because of the limitations of the real environment, the gesture placement orientation and the virtual object placement orientation are quite different, which results in the inability to grab and interact. Therefore, corresponding calibration work is required, which directly affects the user's interactive experience. Therefore, there is an urgent need for a new virtual and real gesture interaction calibration method. SUMMARY

[0003] The present application aims to at least partly solve one of the problems in the prior art.

[0004] To this end, an object of embodiments of the present application is to provide a virtual and real gesture interaction calibration method, system, device and storage medium, which can improve the user experience.

[0005] To achieve the above technical purposes, the technical scheme adopted by the embodiment of the present application comprises: a virtual and real gesture interaction calibration method comprises: acquiring pose quaternions of each inertial motion capture module worn on a real human hand under a first gesture, a first orientation of a virtual object in a virtual scene, an initial gesture of a virtual model hand, and a first angle difference between the first gesture; determining a first calibration quaternion according to the pose quaternions and the first angle difference; performing first calibration on the pose quaternions of the inertial motion capture module according to the first calibration quaternion to obtain a first target quaternion for driving the virtual model hand; acquiring an initial coordinate of a skeleton root node of the virtual model hand; performing second calibration on the virtual model hand according to the first target quaternion after the first calibration and the initial coordinate of the skeleton root node to obtain a first skeleton root node coordinate; calculating a second angle difference between a second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object; determining a second target quaternion of the virtual model hand and a second skeleton root node coordinate according to the second angle difference; and performing third calibration on the virtual model hand according to the second target quaternion and the second skeleton root node coordinate.

[0006] In addition, according to the virtual and real gesture interaction calibration method according to the above embodiment of the present application, the following additional technical features can also be provided:

[0007] Further, in the embodiment of the present application, the step of determining a first calibration quaternion according to the pose quaternions and the first angle difference specifically comprises: performing coordinate system transformation on the pose quaternions to obtain first calibration quaternions of each inertial motion capture module in world coordinates; the first calibration quaternion is a pose quaternion in world coordinates; determining a second calibration quaternion according to the first angle difference; the second calibration quaternion is used to represent the angle difference between the initial gesture of the virtual model hand and the first gesture; and determining the first calibration quaternion according to the first calibration quaternion and the second calibration quaternion.

[0008] Further, in the embodiment of the present application, the step of performing second calibration on the virtual model hand according to the first target quaternion and the initial coordinate of the skeleton root node specifically comprises: extracting azimuth information in the first target quaternion; determining the first skeleton root node coordinate according to the azimuth information, the initial coordinate of the skeleton root node, and the first calibration formula; the first skeleton root node coordinate is a target skeleton root node coordinate after the second calibration; and the first calibration formula is

[0009]

[0010] Wherein, a is azimuth angle information, x1 and y1 are X coordinate and Y coordinate of first target skeleton root node coordinates, × is cross multiplication of matrix, x0 and y0 are X coordinate and Y coordinate of initial coordinates of skeleton root node respectively.

[0011] Further, in the embodiment of the application, the step of determining the second target quaternion of the virtual model hand and the second skeleton root node coordinates according to the second angle difference specifically comprises: determining the second skeleton root node coordinates according to the second angle difference, the first target skeleton root node coordinates and a second calibration formula; wherein the second calibration formula is

[0012]

[0013] Wherein, γ is the second angle difference, x1 and y1 are X coordinate and Y coordinate of first target skeleton root node coordinates, × is cross multiplication of matrix, x2 and y2 are X coordinate and Y coordinate of second skeleton root node coordinates respectively; a rotation quaternion is determined according to the second angle difference; the rotation quaternion represents a rotation angle required by the virtual hand in the third calibration; a second target quaternion of the virtual model hand is determined according to the rotation quaternion and the first target quaternion; the second target quaternion is a quaternion after rotation calibration.

[0014] Further, in the embodiment of the application, the step of determining the first calibration quaternion according to the first calibration quaternion and the second calibration quaternion specifically comprises: determining the first calibration quaternion according to the first calibration quaternion, the second calibration quaternion and a third calibration formula; the third calibration formula is

[0015] Q c (i)=Q sn (i)×Q b

[0016] Wherein, symbol × represents quaternion multiplication, Q c (i) represents the first calibration quaternion, and the first calibration is completed according to the formula, Q sb (i) represents the first calibration quaternion, and the first calibration is completed according to the formula, Q b represents the second calibration quaternion.

[0017] Further, in the embodiment of the application, the step of performing coordinate system transformation on the posture quaternion to obtain the first calibration quaternion of each inertial motion capture module in the world coordinate system specifically comprises: the coordinate system transformation satisfies the following formula:

[0018] Q sn (i)=[Q s (i)] -1

[0019] wherein -1 is the inverse operation of a quaternion, Q sn (i) is a first calibration quaternion, Q s (i) is a pose quaternion.

[0020] Further, in the embodiments of the present application, the step of calibrating the pose quaternion of the inertial motion capture module according to the first calibration quaternion to obtain a first target quaternion for driving the virtual model hand, specifically comprises: obtaining the pose quaternion of each inertial motion capture module under the first gesture; determining the first target quaternion according to the pose quaternion and the first calibration quaternion; the pose quaternion, the first calibration quaternion and the first target quaternion satisfy the following relationship:

[0021] Q m (i) = Q s (i) x Q c (i)

[0022] wherein Q c (i) is a first calibration quaternion, Q s (i) is the pose quaternion of each inertial motion capture module under the first gesture; Q m (i) is a first target quaternion.

[0023] In another aspect, the embodiments of the present application also provide a virtual and real gesture interaction calibration system, comprising:

[0024] A first obtaining module is configured to obtain the pose quaternion of each inertial motion capture module worn on a real human hand under a first gesture, a first orientation of a virtual object in a virtual scene, and a first angle difference between an initial gesture of a virtual model hand and the first gesture. A first processing module is configured to determine a first calibration quaternion according to the pose quaternion and the first angle difference. A first calibration module is configured to calibrate the pose quaternion of the inertial motion capture module according to the first calibration quaternion to obtain a first target quaternion for driving the virtual model hand. A second obtaining module is configured to obtain an initial coordinate of a skeleton root node of the virtual model hand. A second calibration module is configured to calibrate the virtual model hand according to the first target quaternion and the initial coordinate of the skeleton root node to obtain a first coordinate of the skeleton root node. A second processing module is configured to calculate a second angle difference between a second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object. A third processing module is configured to determine a second target quaternion and a second coordinate of the skeleton root node of the virtual model hand according to the second angle difference. A third calibration module is configured to calibrate the virtual model hand according to the second target quaternion and the second coordinate of the skeleton root node.

[0025] In another aspect, the present application also provides a virtual and real gesture interaction calibration device, comprising:

[0026] at least one processor;

[0027] at least one memory for storing at least one program;

[0028] When the at least one program is executed by the at least one processor, the at least one processor implements the virtual and real gesture interaction calibration method as any one of the above.

[0029] In addition, the present application also provides a storage medium, wherein the storage medium stores processor executable instructions, and the processor executable instructions are used for executing the virtual and real gesture interaction calibration method as any one of the above when executed by a processor.

[0030] The advantages and beneficial effects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be known by the practice of the present application:

[0031] The calibration method of the present application can obtain the attitude quaternion of each inertial motion capture module worn on a real human hand under a first gesture, the first orientation of a virtual object in a virtual scene, the first angle difference between the initial gesture of a virtual model hand and the first gesture; determine the first calibration quaternion according to the attitude quaternion and the first angle difference; perform the first calibration on the attitude quaternion of the inertial motion capture module according to the first calibration quaternion, to obtain the first target quaternion for driving the virtual model hand; obtain the initial coordinates of the skeleton root node of the virtual model hand; perform the second calibration on the virtual model hand according to the first target quaternion and the initial coordinates of the skeleton root node, to obtain the first skeleton root node coordinates; calculate the second angle difference between the second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object; determine the second target quaternion of the virtual model hand and the second skeleton root node coordinates according to the second angle difference; perform the third calibration on the virtual model hand according to the second target quaternion and the second skeleton root node coordinates. The present application can successively complete three calibrations under the same gesture action, which can improve the accuracy of virtual and real gesture interaction, simplify the calibration operation, and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 The steps of a virtual and real gesture interaction calibration method in a specific embodiment of the present application are shown in the schematic diagram;

[0033] Figure 2 The structure of a virtual and real gesture interaction calibration system in a specific embodiment of the present application is shown in the schematic diagram;

[0034] Figure 3 Figure 1 is a schematic diagram of a virtual and real gesture interaction calibration device according to an embodiment of the present application;

[0035] Figure 4 Figure 2 is a schematic diagram of a first gesture in a virtual and real gesture interaction calibration method according to an embodiment of the present application;

[0036] Figure 5 Figure 3 is a schematic diagram of a position of a virtual article placement orientation and a virtual model hand orientation during secondary calibration according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The principles and processes of the virtual and real gesture interaction calibration method, system, device and storage medium according to the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] Reference Figure 1 A virtual and real gesture interaction calibration method according to the present application comprises the following steps:

[0039] S1, obtaining attitude quaternions of each inertial motion capture module worn on a real human hand in a first gesture, a first orientation of a virtual article in a virtual scene, and a first angle difference between an initial gesture of a virtual model hand and the first gesture;

[0040] In this step, the inertial motion capture module can be an integrated wearable device, and the integrated wearable device can have multiple inertial motion capture sub-modules, which can be worn on various positions of the human hand. The first gesture can be a specific calibration action, which can be holding both hands forward (generally facing the display terminal device), with the palm facing down and the fingers straight, and the thumb at a 45-degree angle to the other four fingers. Alternatively, the action can be placing both hands on the table according to the above action. When obtaining the attitude quaternions, the real human hand can be placed on the display terminal device according to the first gesture, and the processor of the display terminal device can obtain the attitude quaternions of each inertial motion capture module. The first orientation of the virtual article in the virtual scene refers to the initial position of the virtual article, which can be set artificially according to specific requirements. After setting the orientation, the processor of the display terminal device can also obtain the first orientation of the virtual article. The first angle difference can be the angle difference between the initial gesture of the virtual model hand and the first gesture. The acquisition method can be wireless or wired data transmission between the display terminal device and the inertial motion capture module.

[0041] S2, determining a first calibration quaternion according to the attitude quaternions and the first angle difference;

[0042] In this step, the first angle difference needs to be converted into a quaternion of angle through parameter conversion; through mathematical operation of the attitude quaternion and the quaternion of angle, a first calibration quaternion can be obtained, which can be used for first calibration of the attitude quaternion of each inertial motion capture module under the first gesture.

[0043] S3, calibrating the attitude quaternion of the inertial motion capture module according to the first calibration quaternion to obtain a first target quaternion for driving the virtual model hand;

[0044] In this step, after the first calibration of the attitude quaternion of each inertial motion capture module under the first gesture through the first calibration quaternion, a calibrated first target quaternion can be obtained, which can be used as a quaternion for subsequent driving of the virtual model hand to move.

[0045] S4, obtaining an initial coordinate of a skeleton root node of the virtual model hand;

[0046] In this step, the initial coordinate of the skeleton root node of the virtual model hand can be obtained through the internal processor of the display terminal device. The initial coordinate of the skeleton root node can be the root node coordinate of the virtual model hand after the first calibration. In this application, the root node coordinate before and after the first calibration does not change. The initial coordinate of the skeleton root node can be obtained through the processor at the beginning of calibration.

[0047] S5, according to the first target quaternion after the first calibration and the initial coordinate of the skeleton root node, performing second calibration on the virtual model hand to obtain a first skeleton root node coordinate;

[0048] In this step, the first target quaternion after the first calibration can be solved into the rotation motion information (pitch, roll and yaw) of the model hand. The yaw information is extracted, and the second calibration target coordinate, that is, the first skeleton root node coordinate, can be obtained according to the yaw information and the initial coordinate of the skeleton root node.

[0049] S6, calculating a second angle difference between the second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object;

[0050] In this step, the second orientation is the orientation of the model hand after the second calibration, and the orientation of the virtual model hand after the second calibration will change. At this time, the angle difference between the second orientation and the first orientation needs to be calculated. The orientation can be determined according to the orientation coordinates in the commonly used world coordinates. Since the orientation of the model hand after the second calibration is consistent with that of the real hand, the orientation of the model hand can also be determined through the orientation of the real hand. The processor can obtain the second orientation through wired or wireless means, and the orientation of the virtual object can be determined according to the orientation of the object placed at the beginning.

[0051] S7, determining a second target quaternion of the virtual model hand and a second skeleton root node coordinate according to the second angle difference;

[0052] In this step, the second target quaternion of the virtual hand can be determined according to the angle difference between the orientations of the virtual hand and the virtual object in the world coordinates. The second target quaternion is the quaternion required for the third calibration. Through the quaternion, the calibrated angle and other parameters can be calculated. The first skeleton root node coordinate obtained through the second calibration and the orientation angle difference can be used to obtain the second skeleton root node coordinate. The second skeleton root node coordinate is the root node coordinate after calibration

[0053] S8, performing a third calibration on the virtual model hand according to the second target quaternion and the second skeleton root node coordinate.

[0054] In this step, the virtual model hand can be calibrated for the third time according to the second target quaternion required for calibration and the second skeleton root node coordinate obtained after calibration, to obtain the final driving virtual model hand quaternion and virtual model hand root node.

[0055] Further, the step of determining the first calibration quaternion according to the attitude quaternion and the first angle difference can specifically include:

[0056] S11, performing coordinate system transformation on the attitude quaternion to obtain a first calibration quaternion of each inertial motion capture module in the world coordinates; the first calibration quaternion is the attitude quaternion in the world coordinates;

[0057] S12, determining a second calibration quaternion according to the first angle difference; the second calibration quaternion is used to represent the angle difference between the initial gesture of the virtual model hand and the first gesture;

[0058] S13, determining the first calibration quaternion according to the first calibration quaternion and the second calibration quaternion;

[0059] In some embodiments of the present application, the pose quaternion of the inertial motion capture module can be converted into a quaternion in world coordinates first, and then a second calibration quaternion for the first calibration can be determined by the initial gesture of the virtual model hand and the angle difference between the real hand and the first gesture.

[0060] Further, the step of performing the second calibration on the virtual model hand according to the first target quaternion and the initial coordinates of the skeleton root node can specifically include:

[0061] S21, extract azimuth information in the first target quaternion;

[0062] S22, determine first skeleton root node coordinates according to the azimuth information, the initial coordinates of the skeleton root node, and the first calibration formula; the first skeleton root node coordinates are target skeleton root node coordinates for the second calibration; wherein the first calibration formula is

[0063]

[0064] wherein a is the azimuth information, x1 and y1 are the X and Y coordinates of the first target skeleton root node coordinates, x is the cross product of the matrix, and x0 and y0 are the X and Y coordinates of the initial coordinates of the skeleton root node, respectively;

[0065] In some embodiments of the present application, the azimuth information in the first target quaternion after the first calibration can be extracted, and the above two parameters can be input into the first calibration formula according to the azimuth information and the initial coordinates of the skeleton root node to determine the skeleton root node coordinates. The first calibration formula is

[0066]

[0067] wherein a is the azimuth information, x1 and y1 are the X and Y coordinates of the first target skeleton root node coordinates, x is the cross product of the matrix, and x0 and y0 are the X and Y coordinates of the initial coordinates of the skeleton root node, respectively. It should be noted that the root node coordinates include three coordinates XYZ, and since the second calibration is performed on the same plane, the Z coordinate does not change, so the first calibration formula does not operate on the Z coordinate.

[0068] Further, the step of determining the second target quaternion and the second skeleton root node coordinates of the virtual model hand according to the second angle difference can specifically include:

[0069] S31, determining a second skeleton root node coordinate according to the second angle difference, the first target skeleton root node coordinate, and a second calibration formula; wherein the second calibration formula is

[0070]

[0071] wherein γ is the second angle difference, x1 and y1 are X and Y coordinates of the first target skeleton root node coordinate, × is cross multiplication of a matrix, x2 and y2 are X and Y coordinates of the second skeleton root node coordinate respectively; and for determination of the second target quaternion, a rotation quaternion can be determined according to the second angle difference, and then the second target quaternion can be determined according to the rotation quaternion and the first target quaternion obtained after the first calibration.

[0072] S32, determining a rotation quaternion according to the second angle difference; the rotation quaternion represents a rotation angle required by the virtual hand in the third calibration;

[0073] S33, determining a second target quaternion of the virtual model hand according to the rotation quaternion and the first target quaternion; the second target quaternion is a quaternion after rotation calibration.

[0074] In some embodiments of the present application, a second angle difference between a second position of the virtual model hand after the second calibration and a first position of the virtual object, and a preset second calibration formula can be used to determine a second skeleton root node coordinate of the virtual model hand

[0075] wherein the second calibration formula can be

[0076]

[0077] wherein γ is the second angle difference, x1 and y1 are X and Y coordinates of the first target skeleton root node coordinate, × is cross multiplication of a matrix, x2 and y2 are X and Y coordinates of the second skeleton root node coordinate respectively; and for determination of the second target quaternion, a rotation quaternion can be determined according to the second angle difference, and then the second target quaternion can be determined according to the rotation quaternion and the first target quaternion obtained after the first calibration.

[0078] Further, the step of determining the first calibration quaternion according to the first calibration quaternion and the second calibration quaternion can specifically include:

[0079] determining the first calibration quaternion according to the first calibration quaternion, the second calibration quaternion, and a third calibration formula; wherein the third calibration formula is

[0080] Q c (i)=Q sn (i)×Q b

[0081] wherein the symbol × represents quaternion multiplication, Q c(i) represents the first calibration quaternion, and the first calibration is completed according to the formula, Q sn (i) represents the i-th inertial motion capture module first calibration quaternion, Q b represents the second calibration quaternion.

[0082] Specifically, the relationship among the first calibration quaternion, the second calibration quaternion and the first calibration quaternion satisfies the calculation formula

[0083] Q c (i) = Q sn (i) x Q b

[0084] Wherein, the symbol x represents quaternion multiplication, Q c (i) represents the first calibration quaternion, and the first calibration is completed according to the formula, Q sn (i) represents the i-th inertial motion capture module first calibration quaternion, Q b represents the second calibration quaternion.

[0085] Further, the step of performing coordinate system transformation on the attitude quaternion to obtain the first calibration quaternion of each inertial motion capture module in the world coordinate system can specifically include:

[0086] The coordinate system transformation satisfies the following formula:

[0087] Q sn (i) = [Q s (i)] -1

[0088] Wherein [] -1 is the inverse operation of quaternion, Q sn (i) is the first calibration quaternion, Q s (i) is the attitude quaternion.

[0089] Specifically, the first calibration quaternion and the attitude quaternion satisfy the relationship

[0090] Q sn (i) = [Q s (i)] -1

[0091] Wherein [] -1 is the inverse operation of quaternion, Q sn (i) is the first calibration quaternion, Q s (i) is the attitude quaternion.

[0092] Further, the step of performing first calibration on the attitude quaternion of the inertial motion capture module according to the first calibration quaternion to obtain the first target quaternion for driving the virtual model hand can specifically include:

[0093] Obtain the attitude quaternions of each inertial motion capture module under the first gesture.

[0094] Based on the attitude quaternion and the first calibration quaternion, a first target quaternion is determined; the attitude quaternion, the first calibration quaternion, and the first target quaternion satisfy the following relationship:

[0095] Q m (i)=Q s (i)×Q c (i)

[0096] in

[0097] Q c (i) is the first calibration quaternion, Q s (i) represents the attitude quaternion of each inertial motion capture module under the first gesture; Q m (i) is the first objective quaternion;

[0098] Specifically, the attitude quaternions, the first calibration quaternion, and the first target quaternion of each inertial motion capture module under the first gesture satisfy the following relationship:

[0099] Q m (i)=Q s (i)×Q c (i)

[0100] in

[0101] Q c (i) is the first calibration quaternion, Q s (i) represents the attitude quaternion of each inertial motion capture module under the first gesture; Q m (i) is the first objective quaternion.

[0102] The principles of this application will be explained below with reference to specific embodiments:

[0103] The data processing procedure of this application can be illustrated using an inertial motion capture module as an example. In this embodiment, the attitude quadruple is Q. s The first calibration quaternion is Q. c (i) The initial coordinates of the skeleton root node are (x0, y0, z0), and the first target quaternion is Q. m (i)

[0104] The first step is for the wearer to put on the data gloves (an integrated wearable device that is easy and quick to put on), such as... Figure 2 As shown, there is an inertial motion capture module in each of the eight key parts of the hand, and the thumb and index finger each have two inertial motion capture modules, in order to capture more precise and complex hand gestures.

[0105] The second step, as Figure 4 As shown, have the wearer raise both hands forward (generally facing the display device), palms down and fingers straight, with the thumb at a 45-degree angle to the other four fingers. Alternatively, you can place both hands flat on a table to perform the calibration, which is simpler and easier to do. This is called the calibration action.

[0106] The third step is to obtain the quaternion Q of each inertial motion capture module under the calibration action state. s This serves as the attitude quaternion for each inertial motion capture module under the first gesture, because the calibration action is relatively static, Q s We can take the mean value and rotate it to the world coordinate system to obtain Q. sn It is named the first calibrated quaternion, and its formula is as follows:

[0107] Q sn (i)=[Q s (i)] -1

[0108] Where i represents the i-th inertial motion capture module, Q san (i) represents the first calibration quaternion output by the i-th inertial motion capture module in the world coordinate system, Q. s (i) represents the attitude quaternion output by the i-th inertial motion capture module;

[0109] Fourth step: Generally, the virtual model hand is in a Tpose state, with the right palm facing down and fingers extended horizontally pointing in the positive x-axis direction, and the left palm also facing down and fingers extended horizontally pointing in the negative x-axis direction. Both thumbs form a 45-degree angle with the other four fingers. Due to calibration, the left hand of the virtual model hand needs to rotate 90 degrees clockwise horizontally, and the right hand needs to rotate 90 degrees counterclockwise horizontally. Therefore, the initial hand gesture and the first angle difference between the initial hand gesture and the first hand gesture are 90°. This 90° is then converted into a quaternion Q using Euler angle transformation. b Q b Named the second calibration quaternion;

[0110] Fifth, calibrate the attitude data of the inertial motion capture module based on the first and second calibration quaternions obtained in the third and fourth steps above. The calibration formula is as follows:

[0111] Q c (i)=Q sn (i)×Q b

[0112] Where the symbol × represents quaternion multiplication, Q c(i) represents the first calibration quaternion of the i-th inertial motion capture module to the corresponding virtual model skeleton, and the first calibration is completed according to the formula;

[0113] After obtaining the first calibration quaternion Q c , the quaternion output by each inertial motion capture module can be calibrated, and the calibrated quaternion is Q m , then:

[0114] Q m (i) = Q s (i) * Q c (i)

[0115] Wherein, Q m (i) represents the first target quaternion of the i-th inertial motion capture module after the first calibration;

[0116] In the sixth step, Q m , the virtual model hand can be driven, and at this time, the rotation motion information (pitch, roll and yaw) can be extracted from the virtual model hand, because the root node coordinate information is fixed, which is the initial model skeleton information coordinate Pos0(x0, y0, z0), wherein y0 is equal to 0 and z0 is equal to 0, which will cause the right hand of the virtual model to be on the left side and the left hand of the virtual model hand to be on the right side, forming a crossed hand phenomenon, which is poor for the experience of the user, and the root node coordinate needs to be calibrated for the second time for such problems, because the calibration action is horizontal, only the horizontal coordinate rotation information needs to be considered, and the calibration formula is as follows:

[0117]

[0118] Through the above formula, the root node coordinate after the second calibration, that is, the first skeleton root node coordinate Pos1(x1, y1, z1), can be obtained, wherein z1 remains unchanged and is equal to 0, and a is the yaw information of the hand at the current calibration action, which is the second calibration, and after completion, the virtual model hand and the real hand can be consistent in space and orientation information, and if more accurate spatial position information is wanted, real-time fusion such as HTC Tracker can be used.

[0119] Seventh step, after the second calibration, the virtual model hand and the real hand in the real world keep consistency in space and orientation information, but due to the limitation of the scene used by the user in the real world, it is not necessarily consistent with the scene in the virtual space in 1:1, so that the user cannot interact well in the virtual scene and the experience is poor, for example, we want to grab a virtual object in the virtual scene, and the object is placed in the south, while the display end and the calibration action orientation are in the north, and the hand in the front cannot successfully grab the object. In order to solve such problems, either rotate the object in the north, or the user turns to grab, which does not meet the user's experience, so the third calibration is needed, that is, the orientation rotation of the virtual space and the real space; as shown in Figure 5 The third calibration can be completed according to the horizontal orientation angle between the orientation of the virtual object and the orientation of the virtual model hand in the second calibration, that is, the second angle difference γ (converted into quaternion Q r ) between the second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object.

[0120] The transformation of the root node coordinates of the virtual model hand is as follows:

[0121]

[0122] That is, the transformed root node coordinates, that is, the second skeleton root node coordinates Pos2 (x2, y2, z2) can be obtained, wherein z2 remains unchanged and is equal to z1, and the quaternion for driving the model hand is transformed, and the formula is as follows:

[0123] Q out (i) = Q m (i) x Q r (i)

[0124] Q out (i) represents the final driving quaternion of the virtual model hand, that is, the second target quaternion, and the virtual model hand root node Pos2 and the real-time driving quaternion Q out (i) can be obtained after the third calibration; through the first calibration, the second calibration and the third calibration, the accurate gesture interaction between the virtual and the real can be realized.

[0125] In addition, referring to Figure 2 , and Figure 1 , the embodiments of the present application also provide a virtual and real gesture interaction calibration system, which can include:

[0126] The first obtaining module 101 is configured to obtain a pose quaternion of each inertial motion capture module worn on a real hand in a first gesture, a first orientation of a virtual object in a virtual scene, and a first angle difference between an initial gesture of a virtual model hand and the first gesture.

[0127] The first processing module 102 is configured to determine a first calibration quaternion according to the pose quaternion and the first angle difference.

[0128] The first calibration module 103 is configured to perform first calibration on the pose quaternion of the inertial motion capture module according to the first calibration quaternion, to obtain a first target quaternion for driving the virtual model hand.

[0129] The second obtaining module 104 is configured to obtain an initial coordinate of a skeleton root node of the virtual model hand.

[0130] The second calibration module 105 is configured to perform second calibration on the virtual model hand according to the first target quaternion and the initial coordinate of the skeleton root node, to obtain a first skeleton root node coordinate.

[0131] The second processing module 106 is configured to calculate a second angle difference between a second orientation of the virtual model hand after the second calibration and the first orientation of the virtual object.

[0132] The third processing module 107 is configured to determine a second target quaternion and a second skeleton root node coordinate of the virtual model hand according to the second angle difference.

[0133] The third calibration module 108 is configured to perform third calibration on the virtual model hand according to the second target quaternion and the second skeleton root node coordinate.

[0134] Corresponding to the method, the embodiments of the present application also provide a virtual and real gesture interaction calibration device, and the specific structure can refer to the method Figure 1 , and the device comprises: Figure 3

[0135] at least one processor 1001;

[0136] at least one memory 1002 configured to store at least one program;

[0137] When the at least one program is executed by the at least one processor, the at least one processor implements the virtual and real gesture interaction calibration method.

[0138] The contents in the above method embodiments are applicable to the device embodiments, the device embodiments specifically implement the functions same as the above method embodiments, and achieve the same beneficial effects as the above method embodiments.​

[0139] corresponding to the method of Figure 1 The embodiments of the present application also provide a storage medium having stored therein processor-executable instructions for performing the virtual and real gesture interaction calibration method when executed by a processor.

[0140] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operational illustrations. For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality / operations involved. Also, the embodiments presented and described in the flowcharts are only examples of implementations and are not limiting. The processes can be implemented in alternative embodiments without departing from the scope of the application. Alternative embodiments are possible where the order of the blocks has been modified, where various blocks from different embodiments have been combined into single blocks, and where the blocks have been implemented in hardware or software without departing from the scope of the application.

[0141] Furthermore, although the present application is described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also to be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation is within the routine of an engineer's knowledge given the property, functionality and internal relationships of the various functional modules disclosed herein. Accordingly, the present application is not limited to purely hardware or software implementations, but rather encompasses hybrid implementations within the scope of the appended claims and their equivalents.

[0142] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts of the prior art that make contributions or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes several programs for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0143] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of ordered steps for implementing logical functions, which can be embodied in any computer readable medium for use by a program execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can take programs from a program execution system, device or apparatus and execute them) or in conjunction with these program execution systems, devices or apparatus. For the purpose of this specification, "computer readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by program execution systems, devices or apparatus or in conjunction with these program execution systems, devices or apparatus.

[0144] More specific examples (non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be stored in a computer memory.

[0145] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable

[0146] In the above description of the present specification, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in one or more embodiments or examples.

[0147] Although the embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, alternatives and variations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

[0148] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and these equivalent modifications or substitutions are included in the scope defined by the claims of the present application.

Claims

1. A method for calibrating virtual and real-world gesture interaction, characterized in that, Includes the following steps: The attitude quaternions of each inertial motion capture module worn on a real person's hand under the first gesture, the first position of the virtual object in the virtual scene, and the first angle difference between the initial gesture of the virtual model's hand and the first gesture are obtained. The attitude quaternions are transformed to obtain the first calibration quaternions of each inertial motion capture module in world coordinates; the first calibration quaternions are attitude quaternions in world coordinates; the second calibration quaternion is determined based on the first angle difference; the second calibration quaternion is used to characterize the angle difference between the initial gesture and the first gesture of the virtual model hand; the first calibration quaternion is determined based on the first calibration quaternion and the second calibration quaternion; The attitude quaternion of the inertial motion capture module is calibrated for the first time based on the first calibration quaternion to obtain the first target quaternion used to drive the virtual model hand; Obtain the initial coordinates of the root node of the skeleton of the virtual model hand; Based on the first target quaternion after the first calibration and the initial coordinates of the skeleton root node, the virtual model hand is calibrated a second time to obtain the coordinates of the first skeleton root node. Calculate the second angular difference between the second orientation of the virtual model hand and the first orientation of the virtual object after the second calibration; Based on the second angle difference, determine the second target quaternion of the virtual model hand and the coordinates of the second skeleton root node; The virtual model hand is calibrated for the third time based on the second target quaternion and the coordinates of the second skeleton root node.

2. The virtual and real gesture interaction calibration method according to claim 1, characterized in that, The step of performing a second calibration of the virtual model hand based on the first target quaternion and the initial coordinates of the skeleton root node specifically includes: Extract the azimuth information from the first target quaternion; Based on the azimuth information, the initial coordinates of the skeleton root node, and the first calibration formula, the coordinates of the first skeleton root node are determined; the coordinates of the first skeleton root node are the target skeleton root node coordinates for the second calibration; wherein the first calibration formula is... in The azimuth information is provided, where x1 and y1 are the X and Y coordinates of the root node of the first target skeleton. Let x0 and y0 be the cross product of the matrices, and let x0 and y0 be the X and Y coordinates of the initial coordinates of the skeleton root node, respectively.

3. The virtual and real gesture interaction calibration method according to claim 1, characterized in that, The step of determining the second target quaternion and the second skeleton root node coordinates of the virtual model hand based on the second angle difference specifically includes: The coordinates of the second skeleton root node are determined based on the second angle difference, the coordinates of the first target skeleton root node, and the second calibration formula; wherein the second calibration formula is: in The second angle difference, where x1 and y1 are the X and Y coordinates of the root node of the first target skeleton. Let x2 be the cross product of the matrices, and let x2 and y2 be the X and Y coordinates of the root node of the second skeleton, respectively. The rotation quaternion is determined based on the second angle difference; the rotation quaternion represents the rotation angle required for the virtual hand in the third calibration; Based on the rotation quaternion and the first target quaternion, a second target quaternion for the virtual model hand is determined; the second target quaternion is the quaternion after rotation calibration.

4. The virtual and real gesture interaction calibration method according to claim 1, characterized in that, The step of determining the first calibration quaternion based on the first calibration quaternion and the second calibration quaternion specifically includes: The first calibration quaternion is determined based on the first calibration quaternion, the second calibration quaternion, and the third calibration formula; the third calibration formula is... Among them, symbols To represent quaternion multiplication, This represents the first calibration quaternion, and the first calibration is completed according to this formula. Indicates the first The first quaternion of the inertial motion capture module is calibrated. This represents the second calibrated quaternion.

5. The virtual and real gesture interaction calibration method according to claim 1, characterized in that, The step of transforming the attitude quaternions to obtain the first calibration quaternions of each inertial motion capture module in world coordinates specifically includes: The coordinate system transformation satisfies the following formula: in[] -1 This is the inverse operation for quaternions. For the first quaternion, It is a quaternion of attitude.

6. The virtual and real gesture interaction calibration method according to claim 1, characterized in that, The step of performing the first calibration of the attitude quaternion of the inertial motion capture module based on the first calibration quaternion to obtain the first target quaternion used to drive the virtual model hand specifically includes: Obtain the attitude quaternions of each inertial motion capture module under the first gesture; Based on the attitude quaternion and the first calibration quaternion, a first target quaternion is determined; the attitude quaternion, the first calibration quaternion, and the first target quaternion satisfy the following relationship: in For the first calibration quaternion, The attitude quaternion for each inertial motion capture module under the first gesture; It is the first objective quaternion.

7. A virtual and real-world gesture interaction calibration system, characterized in that, include: The first acquisition module is used to acquire the attitude quaternion of each inertial motion capture module worn on the real person's hand under the first gesture, the first position of the virtual object in the virtual scene, and the first angle difference between the initial gesture of the virtual model's hand and the first gesture. The first processing module is used to transform the attitude quaternions into coordinate systems to obtain the first calibration quaternions of each inertial motion capture module in world coordinates; the first calibration quaternion is the attitude quaternion in world coordinates; the second calibration quaternion is determined based on the first angle difference; the second calibration quaternion is used to characterize the angle difference between the initial gesture and the first gesture of the virtual model hand; the first calibration quaternion is determined based on the first calibration quaternion and the second calibration quaternion. The first calibration module is used to perform a first calibration of the attitude quaternion of the inertial motion capture module based on the first calibration quaternion, so as to obtain the first target quaternion used to drive the virtual model hand; The second acquisition module is used to acquire the initial coordinates of the root node of the skeleton of the virtual model hand; The second calibration module is used to perform a second calibration on the virtual model hand based on the first target quaternion and the initial coordinates of the skeleton root node, so as to obtain the coordinates of the first skeleton root node. The second processing module is used to calculate the second angle difference between the second orientation of the virtual model hand and the first orientation of the virtual object after the second calibration. The third processing module is used to determine the second target quaternion and the coordinates of the second skeleton root node of the virtual model hand based on the second angle difference. The third calibration module performs a third calibration on the virtual model hand based on the second target quaternion and the coordinates of the second skeleton root node.

8. A virtual and real gesture interaction calibration device, characterized in that... include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a virtual and real gesture interaction calibration method as described in any one of claims 1-6.

9. A storage medium storing processor-executable instructions, characterized in that, The processor-executable instructions, when executed by the processor, are used to perform a virtual and real gesture interaction calibration method as described in any one of claims 1-6.

Images