Pose acquisition method and device, display system, storage medium and computer program

Abstract

A pose acquisition method and apparatus, a display system, a storage medium and a computer program, the method comprising: acquiring first pose data at a current frame moment, the first pose data being used for representing a pose of a first marker fixed on a mobile device under a reference coordinate system; based on the first pose data and the first relative pose data, obtaining second pose data of the mobile device under the reference coordinate system; acquiring third pose data at the current frame moment, wherein the third pose data is used for representing the pose of a second marker fixed on the display equipment under the reference coordinate system; on the basis of the third pose data and the second relative pose data, fourth pose data of the display equipment under the reference coordinate system is obtained; and based on the second pose data and the fourth pose data, obtaining the pose of the mobile device in the coordinate system of the display device. The accuracy of obtaining the pose of the mobile device under the coordinate system of the display device is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of posture tracking technology, and in particular to a posture acquisition method and device, a display system, a storage medium, and a computer program. Background Art

[0002] The field of object tracking and positioning plays an increasingly important role in modern science and technology. With the continuous advancement of computer technology and the rapid development of artificial intelligence, object tracking and positioning technology has been widely used in virtual reality, intelligent transportation, industrial production and other aspects, and has achieved remarkable results.

[0003] However, the robustness and user experience of pose tracking still need to be improved. Summary of the Invention

[0004] The problem solved by the embodiments of the present invention is to provide a posture acquisition method and device, a display system, a storage medium and a computer program to improve the robustness of posture tracking and user experience.

[0005] To solve the above problems, an embodiment of the present invention provides a posture acquisition method, which is suitable for acquiring the posture of a movable device in the coordinate system of a display device, and the posture acquisition method includes: acquiring first posture data at a current frame moment, the first posture data being used to represent the posture of a first marker fixed on the movable device in a reference coordinate system; acquiring second posture data of the movable device in the reference coordinate system based on the first posture data and the first relative posture data, the first relative posture data being used to represent the relative posture relationship between the movable device and the first marker; acquiring third posture data at the current frame moment, the third posture data being used to represent the posture of a second marker fixed on the display device in the reference coordinate system; acquiring fourth posture data of the display device in the reference coordinate system based on the third posture data and the second relative posture data, the second relative posture data being used to represent the relative posture relationship between the display device and the second marker; and acquiring the posture of the movable device in the coordinate system of the display device based on the second posture data and the fourth posture data.

[0006] Correspondingly, an embodiment of the present invention also provides a posture acquisition device, including: a first posture acquisition module, used to acquire first posture data at a current frame moment, wherein the first posture data is used to represent the posture of a first marker fixed on a movable device in a reference coordinate system; a second posture acquisition module, used to acquire second posture data of the movable device in the reference coordinate system based on the first posture data and the first relative posture data, wherein the first relative posture data is used to represent the relative posture relationship between the movable device and the first marker; a third posture acquisition module, used to acquire third posture data at the current frame moment, wherein the third posture data is used to represent the posture of a second marker fixed on a display device in the reference coordinate system; a fourth posture acquisition module, used to acquire fourth posture data of the display device in the reference coordinate system based on the third posture data and the second relative posture data, wherein the second relative posture data is used to represent the relative posture relationship between the display device and the second marker; a fifth posture acquisition module, used to acquire the posture of the movable device in the coordinate system of the display device based on the second posture data and the fourth posture data.

[0007] Correspondingly, an embodiment of the present invention also provides a display system, including: a movable device, on which a first marker is fixed; a display device, on which a second marker is fixed; and a processor, which is used to execute the posture acquisition method provided by an embodiment of the present invention.

[0008] Correspondingly, an embodiment of the present invention further provides a storage medium, which stores one or more computer instructions, and the one or more computer instructions are used to implement the posture acquisition method provided by an embodiment of the present invention.

[0009] Correspondingly, an embodiment of the present invention further provides a computer program product, comprising computer instructions, which, when executed by a processor, are used to implement the posture acquisition method provided by an embodiment of the present invention.

[0010] Compared with the prior art, the technical solution of the embodiment of the present invention has the following advantages:

[0011] In the posture acquisition method provided by an embodiment of the present invention, first posture data of the current frame moment is obtained, and the first posture data is used to represent the posture of the first marker fixed on the movable device in the reference coordinate system. Based on the first posture data and the first relative posture data, second posture data of the movable device in the reference coordinate system is obtained, and the first relative posture data is used to represent the relative posture relationship between the movable device and the first marker. Third posture data of the current frame moment is obtained, and the third posture data is used to represent the posture of the second marker fixed on the display device in the reference coordinate system. Based on the third posture data and the second relative posture data, fourth posture data of the display device in the reference coordinate system is obtained, and the second relative posture data is used to represent the relative posture relationship between the display device and the second marker. Based on the second posture data and the fourth posture data, the posture of the movable device in the coordinate system of the display device is obtained. Compared to a solution that directly obtains the position and posture of a movable device through a display device, an embodiment of the present invention obtains the position and posture of a first marker on the movable device and a second marker on the display device in a reference coordinate system outside the display device, and obtains second position and posture data of the movable device in the reference coordinate system by means of a fixed relative position between the movable device and the first marker, and obtains fourth position and posture data of the display device in the reference coordinate system by means of a fixed relative position between the display device and the second marker. Since the movable device and the display device share the same reference coordinate system, the position and posture of the movable device in the coordinate system of the display device can be obtained based on the second and fourth position and posture data, thereby reducing dependence on the positioning of the movable device and improving the problem of inaccurate positioning data collection of the display device due to the influence of the environment on the characteristic information of the movable device itself. At the same time, the first marker is fixed on the movable device and the second marker is fixed on the display device. The geometric structure between the first marker and the movable device is stable, and the geometric structure between the second marker and the display device is also stable, thereby improving the accuracy of obtaining the position and posture of the movable device in the coordinate system of the display device, thereby improving the robustness of tracking the position and posture of the movable device and the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of an embodiment of a posture acquisition method of the present invention;

[0013] Figure 2 It is a structural diagram of the display system of the present invention;

[0014] Figure 3 (a) is an enlarged view of the display device and the second fixture. Figure 3 (b) is an enlarged view of the movable device and the first fixture;

[0015] Figure 4 It is a functional block diagram of an embodiment of the posture acquisition system of the present invention. DETAILED DESCRIPTION

[0016] As can be seen from the background technology, the robustness and user experience of current posture tracking still need to be improved.

[0017] Research has found that, in the current process of tracking the posture of a mobile device, the posture of the mobile device is mainly obtained by tracking the key feature points of the mobile device itself. However, when the key feature points of the mobile device itself are not obvious enough, or when the mobile device is subject to complex background environments or interference from multiple objects, it is easy to lead to a reduction in the effectiveness of tracking the posture of the mobile device, thereby reducing the accuracy of tracking the posture of the mobile device, and further affecting the robustness of tracking the posture of the mobile device and user experience.

[0018] In order to solve the technical problem, the embodiment of the present invention provides a posture acquisition method. Figure 1 , Figure 1 It is a flow chart of an embodiment of a posture acquisition method of the present invention.

[0019] In an embodiment of the present invention, a posture acquisition method is suitable for acquiring the posture of a movable device in a coordinate system of a display device. The posture acquisition method includes the following basic steps:

[0020] Step S1: Acquire first pose data at a current frame moment, where the first pose data is used to represent a pose of a first marker fixed on the movable device in a reference coordinate system;

[0021] Step S2: acquiring second pose data of the movable device in the reference coordinate system based on the first pose data and the first relative pose data, wherein the first relative pose data is used to represent the relative pose relationship between the movable device and the first marker;

[0022] Step S3: Acquire third posture data at the current frame moment, where the third posture data is used to represent the posture of the second marker fixed on the display device in the reference coordinate system;

[0023] Step S4: acquiring fourth pose data of the display device in the reference coordinate system based on the third pose data and the second relative pose data, wherein the second relative pose data is used to represent a relative pose relationship between the display device and the second marker;

[0024] Step S5: Based on the second posture data and the fourth posture data, obtain the posture of the movable device in the coordinate system of the display device.

[0025] The posture acquisition method provided by the embodiment of the present invention obtains the postures of a first marker on a movable device and a second marker on a display device in a reference coordinate system outside the display device, and obtains second posture data of the movable device in the reference coordinate system by means of a fixed relative posture between the movable device and the first marker, and obtains fourth posture data of the display device in the reference coordinate system by means of a fixed relative posture between the display device and the second marker. Since the movable device and the display device share the same reference coordinate system, the posture of the movable device in the coordinate system of the display device can be obtained based on the second posture data and the fourth posture data, thereby reducing the dependence on the positioning of the movable device, which is beneficial to improving the problem of inaccurate positioning data collection of the display device due to the characteristic information of the movable device itself being affected by the environment. At the same time, the first marker is fixed on the movable device and the second marker is fixed on the display device. The geometric structure between the first marker and the movable device is stable, and the geometric structure between the second marker and the display device is also stable, thereby improving the accuracy of obtaining the posture of the movable device in the coordinate system of the display device, thereby improving the robustness of the posture tracking of the movable device and the user experience.

[0026] In order to make the above-mentioned objects, features and advantages of the embodiments of the present invention more obvious and easy to understand, the following Figures 2 to 3 The specific embodiments of the present invention are described in detail.

[0027] In this embodiment, the posture acquisition method is used to acquire the posture of the movable device 110 in the coordinate system of the display device 160 .

[0028] In this embodiment, the mobile device 110 includes a handheld controller. For example, the mobile device 110 is a handheld controller used in VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality).

[0029] VR, AR, and MR technologies can provide a simulated three-dimensional digital experience, allowing us to immerse ourselves in any virtual or real world, regardless of the constraints of our physical location. Controllers are a crucial interaction method in VR, AR, and MR. To meet the demands of high-precision, high-robustness, and low-latency applications, robust controller tracking is crucial.

[0030] It is understandable that the movable device 110 is not limited to a handle. In other embodiments, the movable device 110 may also be other movable devices 110 with posture tracking requirements.

[0031] In this embodiment, the display device 160 includes a wearable display device.

[0032] The wearable display device is used in conjunction with the movable device 110. Specifically, the wearable display device 160 can be VR smart glasses, AR smart glasses or MR smart glasses.

[0033] It is understandable that the display device 160 is not limited to a wearable display device. In other embodiments, the display device can also be other devices that need to track and display the position of the movable device.

[0034] It should be noted that the display device 160 has an image acquisition device 196 , and therefore the coordinate system of the display device 160 is the camera coordinate system corresponding to the image acquisition device 196 .

[0035] Combined with reference Figures 2 to 3 , execute step S1 to obtain the first pose data of the current frame moment, where the first pose data is used to represent the pose of the first marker 103 fixed on the movable device 110 in the reference coordinate system.

[0036] It should be noted that by obtaining the posture of the first marker 103 in the reference coordinate system, in the subsequent process of obtaining the posture of the movable device 110 in the reference coordinate system, the posture of the movable device 110 in the reference coordinate system can be obtained based on the first posture data and the fixed relative posture between the movable device 110 and the first marker 103. At the same time, since the first marker 103 is fixed on the movable device 110, a rigid geometric structure is formed between the second marker 103 and the display device 110. Therefore, the geometric structure between the first marker 103 and the movable device 110 is stable, which also improves the accuracy of obtaining the posture of the movable device 110 in the reference coordinate system.

[0037] It should also be noted that the reference coordinate system and the coordinate system of the display device are two different coordinate systems. Therefore, there is no need to directly obtain the position of the movable device 110 in the coordinate system of the display device 160 through the display device 160. Instead, the position of the movable device 110 in the coordinate system of the display device is indirectly obtained with the help of the reference coordinate system, thereby reducing the dependence of the display device 160 on the positioning of the movable device 110.

[0038] In this embodiment, the reference coordinate system is a three-dimensional coordinate system.

[0039] By adopting a three-dimensional coordinate system, the first pose data and the subsequently obtained third pose data are both six-degree-of-freedom (DoF) data, thereby obtaining the pose of the movable device 110 without the need for inertial measurement data. In other words, the pose acquisition method can achieve pose tracking of a movable device that is not equipped with an inertial measurement unit (IMU).

[0040] In a specific embodiment, the reference coordinate system includes the coordinate system of the motion capture device 180 .

[0041] Specifically, the motion capture device 180 can accurately obtain the position and posture of the first marker 103 in the coordinate system of the motion capture device 180 (i.e., the first pose data), thereby obtaining the position and posture of the movable device 110 in the coordinate system of the motion capture device 180 based on the first pose data and the fixed relative pose between the movable device 110 and the first marker 103.

[0042] As an example, the motion capture device 180 includes an Optitrack device or a Mocap (Motion capture) device.

[0043] As an example, the number of the first markers 103 is at least four, and any four of the first markers 103 are non-coplanar.

[0044] It should be noted that the number of first markers 103 is at least four, and any four of the first markers 103 are non-coplanar, so that the structure composed of multiple first markers 103 is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the first posture data, the posture information of the first marker 103 in different dimensional directions can be obtained.

[0045] As an example, Figure 3 As shown, the number of the first markers 103 is 5, and any four first markers 103 are non-coplanar.

[0046] In this embodiment, the first marker 103 includes a light-emitting element.

[0047] Specifically, the signal light emitted by the light-emitting element is more conducive to the motion capture device 180 capturing the first marker.

[0048] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0049] Therefore, as an example, the step of obtaining the first position data at the current frame moment includes: obtaining multiple first 3dof data, the number of the first 3dof data is at least four, and is used to respectively represent the position of each first marker 103 in the reference coordinate system; based on the multiple first 3dof data, obtaining the first position data at the current frame moment.

[0050] Specifically, the first bit pose data is 6dof data.

[0051] Combined with reference Figures 2 to 3 , execute step S2, and obtain second posture data of the movable device 110 in the reference coordinate system based on the first posture data and the first relative posture data, wherein the first relative posture data is used to represent the relative posture relationship between the movable device 110 and the first marker 103.

[0052] Specifically, the movable device 110 and the display device 160 share a reference coordinate system. Therefore, the posture of the movable device 110 in the coordinate system of the display device 160 can be obtained through the second posture data and the subsequently obtained fourth posture data, thereby reducing the dependence on the positioning of the movable device 110, which is conducive to improving the problem of inaccurate positioning data collection of the display device 160 due to the characteristic information of the movable device 110 itself being affected by the environment.

[0053] It should be noted that since the first marker 103 is fixed on the movable device 110, a rigid geometric structure is formed between the first marker 103 and the movable device 110. Therefore, the movable device 110 and the first marker 103 have a fixed relative posture relationship. After the posture of the first marker 103 in the reference coordinate system (that is, the first posture number) is obtained, the second posture data of the movable device 110 in the reference coordinate system can be converted based on the first posture data and the first relative posture data.

[0054] As an example, the first relative posture data includes six degrees of freedom (DoF) data.

[0055] Specifically, the first relative posture data includes a rotation offset between the movable device 110 and the first marker 103 , and a translation offset between the movable device 110 and the first marker 103 .

[0056] It should be noted that by making the first relative posture data between the movable device 110 and the first marker 103 include six-degree-of-freedom data, in the process of obtaining the second posture data of the movable device 110 in the reference coordinate system, the first relative posture data and the first posture data can be multiplied by each other to obtain the second posture data of the movable device 110 in the reference coordinate system.

[0057] As an example, by formula P oc =P oc' ·P c'c Obtain the second position data of the movable device 110 in the reference coordinate system, wherein P oc Refers to the second pose data, P oc' Refers to the first pose data, P c'c Refers to the first relative pose data.

[0058] Combined with reference Figures 2 to 3 , execute step S3 to obtain the third posture data of the current frame moment, where the third posture data is used to represent the posture of the second marker 100 fixed on the display device 160 in the reference coordinate system.

[0059] It should be noted that by obtaining the pose of the second marker 100 in the reference coordinate system, in the subsequent process of obtaining the pose of the display device 160 in the reference coordinate system, the pose of the display device 160 in the reference coordinate system can be obtained based on the third pose data and the fixed relative pose between the display device and the second marker 100. At the same time, because the second marker 100 is fixed to the display device 160, a rigid geometric structure is formed between the second marker 100 and the display device 160. Therefore, the geometric structure between the second marker 100 and the display device 160 is stable, thereby also improving the accuracy of obtaining the pose of the display device 160 in the reference coordinate system.

[0060] As an example, the number of the second markers 100 is at least four, and any four of the second markers 100 are non-coplanar.

[0061] It should be noted that the number of second markers 100 is at least four, and any four of the second markers 100 are non-coplanar, so that the structure composed of multiple second markers 100 is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the third posture data, the posture information of the second marker 100 in different dimensional directions can be obtained.

[0062] As an example, Figure 3 As shown, the number of the second markers 100 is 5, and any four second markers 100 are non-coplanar.

[0063] In this embodiment, the second marker 100 includes a light-emitting element.

[0064] Specifically, the signal light emitted by the light-emitting element is more conducive to the motion capture device 180 capturing the second marker.

[0065] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0066] As an example, the step of obtaining the third posture data at the current frame moment includes: obtaining multiple second 3dof data, the number of the second 3dof data is at least four, and is used to respectively represent the posture of each second marker 100 in the reference coordinate system; based on the multiple second 3dof data, obtaining the third posture data at the current frame moment.

[0067] Specifically, the third posture data is 6dof data.

[0068] Combined with reference Figures 2 to 3 , executing step S31, before the subsequent step of obtaining the fourth posture data of the display device 160 in the reference coordinate system based on the third posture data and the second relative posture data, it also includes obtaining the second relative posture data between the display device 160 and the second marker 100.

[0069] It should be noted that by obtaining the second relative posture data, the second relative posture data and the third posture data can be used to obtain the fourth posture data of the display device 160 in the reference coordinate system, which is conducive to the subsequent use of the fourth posture data and the second posture data to obtain the posture of the movable device 110 in the coordinate system of the display device 160, thereby reducing the dependence on the positioning of the movable device 110, and is conducive to improving the problem of inaccurate positioning data collection of the display device 160 due to the characteristic information of the movable device 110 itself being affected by the environment.

[0070] It should also be noted that the second marker 100 is fixed on the display device 160, and a rigid geometric structure is formed between the second marker 100 and the display device 160. The second relative posture data between the display device 160 and the second marker 100 is fixed. Therefore, by first obtaining the second relative posture data, in the subsequent process of obtaining the fourth posture data of the display device 160 in the reference coordinate system at the current frame moment, the posture of the display device 160 in the reference coordinate system can be directly calculated based on the posture of the second marker 100 in the reference coordinate system.

[0071] It can be understood that when the second marker 100 does not change, the second relative posture data between the display device 160 and the second marker 100 is fixed. Therefore, it is only necessary to obtain the second relative posture data between the display device 160 and the second marker 100 before obtaining the fourth posture data for the first time. Once the second relative posture data is obtained, there is no need to repeatedly calculate the second relative posture data between the display device 160 and the second marker 100 when obtaining the posture of the movable device 110 at subsequent frame times.

[0072] As an example, the step of obtaining the second relative posture data includes: obtaining the first static posture data of the display device 160 in the reference coordinate system, and the second static posture data of the second marker 100 in the reference coordinate system, the first static posture data being used to represent the posture of the display device 160 when it is in a static state, and the second static posture data being used to represent the posture of the second marker 100 when it is in a static state; based on the first static posture data and the second static posture data, obtaining the second relative posture data between the display device 160 and the second marker 100.

[0073] The first static posture data and the second static posture data are respectively the posture data of the display device 160 and the second marker 100 in the same reference coordinate system, and the display device 160 and the second marker 100 have a fixed relative posture. Therefore, after obtaining the first static posture data and the second static posture data, it is easy to obtain the second relative posture data between the display device 160 and the second marker 100.

[0074] It should be noted that, in the process of obtaining the second relative posture data, keeping the display device 160 and the second marker 100 in a stationary state can improve the accuracy of obtaining the second relative posture data between the display device 160 and the second marker 100. Accordingly, when subsequently obtaining the posture of the display device 160 in the reference coordinate system, the accuracy of obtaining the posture data of the display device 160 in the reference coordinate system is also improved.

[0075] As an example, the step of obtaining the first static posture data includes: obtaining the first position information of the first marker 103 in the coordinate system of the display device 160, and the second position information of the first marker 103 in the reference coordinate system; based on the first position information, the second position information, and the internal parameters of the image acquisition device in the display device 160, obtaining the first static posture data of the display device 160 in the reference coordinate system.

[0076] Compared with directly obtaining the first static posture data of the display device 160 in the reference coordinate system through the motion capture device 180, this embodiment obtains the second position information of the first marker 103 in the reference coordinate system through the motion capture device 180 to infer the first static posture data. The motion capture device 180 is more likely to receive the specific marking signal (such as an infrared light signal) generated by the first marker 103, which can further improve the accuracy of obtaining the first static posture data of the display device 160 in the reference coordinate system.

[0077] Specifically, through the first position information and the second position information, the corresponding relationship between the position information of the first marker 103 in the coordinate system of the display device 160 and the position information of the first marker 103 in the reference coordinate system can be obtained, and the internal parameters of the image acquisition device in the display device 160 are used to facilitate the subsequent calculation of the first static posture data of the display device 160 in the reference coordinate system through the PnP algorithm.

[0078] As an example, the step of acquiring first position information of the first marker 103 in the coordinate system of the display device 160 includes: acquiring a two-dimensional pixel position of the first marker 103 in the coordinate system of the display device 160 .

[0079] It should be noted that by obtaining the two-dimensional pixel position of the first marker 103 in the coordinate system of the display device 160, the first static posture data of the display device 160 in the reference coordinate system can be converted through the PnP algorithm.

[0080] In this embodiment, the step of obtaining the two-dimensional pixel position of the first marker 103 in the coordinate system of the display device 160 includes: obtaining a picture of the first marker 103 at the current frame moment through the image acquisition device of the display device 160; obtaining the two-dimensional pixel position of the first marker 103 in the picture in the coordinate system of the display device 160.

[0081] As an example, the step of acquiring the second position information of the first marker 103 in the reference coordinate system includes: acquiring the three-dimensional coordinate position of the first marker 103 in the reference coordinate system.

[0082] It should be noted that, by obtaining the three-dimensional coordinate position of the first marker 103 in the reference coordinate system, the first static posture data of the display device 160 in the reference coordinate system can be converted through the PnP algorithm.

[0083] In this embodiment, the three-dimensional coordinate position of the first marker 103 in the coordinate system of the motion capture device 180 is obtained by the motion capture device 180. Specifically, 3D of data of a plurality of first markers 103 is obtained by the motion capture device 180, wherein the number of the 3D of data is at least four and each of the 3D of data is used to represent the position of each first marker 103 in the reference coordinate system; based on the plurality of 3D of data, the three-dimensional coordinate position of the first marker 103 in the coordinate system of the motion capture device 180 is obtained.

[0084] As an example, the first static pose data is calculated by a PnP (Perspective-n-Point) algorithm.

[0085] Specifically, through the formula P oh =PnP(pixel, 3d points, K) to obtain the first static posture data of the display device 160 in the reference coordinate system, where P oh It refers to the first static posture data in the reference coordinate system when the display device 160 is in a static state, pixel refers to the two-dimensional pixel position of the first marker 103 in the coordinate system of the display device 160, 3d points refers to the three-dimensional coordinate position of the first marker 103 in the reference coordinate system, and K refers to the internal parameter of the image acquisition device.

[0086] Specifically, after obtaining the first static posture data of the display device 160 in the reference coordinate system, it can be calculated by the formula P h'h =P oh' -1 ·P oh Acquire the second relative posture data between the display device 160 and the second marker 100, wherein P h'h Refers to the second relative pose data, P oh' Refers to the second static pose data, P oh Refers to the first static pose data.

[0087] By obtaining the second relative posture data between the display device 160 and the second marker 100, in the subsequent process of obtaining the fourth posture data of the display device 160 in the reference coordinate system at the current frame moment, the posture of the display device 160 in the reference coordinate system can be directly calculated based on the posture of the second marker 100 in the reference coordinate system.

[0088] It should be noted that since the second marker 100 is fixed on the display device 160, a rigid geometric structure is formed between the second marker 100 and the display device 160. Therefore, there is a fixed relative posture relationship between the display device 160 and the second marker 100. After the posture of the second marker 100 in the reference coordinate system (that is, the third posture number) is obtained, the fourth posture data of the display device 160 in the reference coordinate system can be converted according to the third posture data and the second relative posture data.

[0089] As an example, the second relative posture data includes six degrees of freedom (DoF) data.

[0090] Specifically, the second relative posture data includes a rotation offset between the display device 160 and the second marker 100 , and a translation offset between the movable device 110 and the first marker 103 .

[0091] It should be noted that by making the second relative posture data between the display device 160 and the second marker 100 include six-degree-of-freedom data, in the process of obtaining the fourth posture data of the display device 160 in the reference coordinate system, the second relative posture data and the third posture data can be multiplied by each other to obtain the fourth posture data of the display device 160 in the reference coordinate system.

[0092] Combined with reference Figures 2 to 3 , execute step S4, and obtain fourth posture data of the display device 160 in the reference coordinate system based on the third posture data and the second relative posture data, wherein the second relative posture data is used to represent the relative posture relationship between the display device 160 and the second marker 100.

[0093] Specifically, the movable device 110 and the display device 160 share a reference coordinate system. Therefore, the second posture data and the fourth posture data can be used to obtain the posture of the movable device 110 in the coordinate system of the display device 160, thereby reducing the dependence on the positioning of the movable device 110, which is conducive to improving the problem of inaccurate positioning data collection of the display device 160 due to the influence of the environment on the characteristic information of the movable device 110 itself.

[0094] As an example, by formula P oh =P oh' ·P h'h Acquire the fourth posture data of the display device 160 in the reference coordinate system, wherein P oh Refers to the fourth pose data, P oh' Refers to the third pose data, P h'hRefers to the second relative pose data.

[0095] Combined with reference Figures 2 to 3 , execute step S5, based on the second posture data and the fourth posture data, obtain the posture of the movable device 110 in the coordinate system of the display device 160.

[0096] Compared with the solution of directly obtaining the posture of the movable device 110 through the display device 160, the posture of the movable device 110 in the coordinate system of the display device 160 is obtained based on the second posture data and the fourth posture data, thereby reducing the dependence on the positioning of the movable device 110, which is beneficial to improving the problem of inaccurate positioning data collection of the display device 160 due to the influence of the environment on the characteristic information of the movable device 110 itself.

[0097] At the same time, the first marker is fixed on the movable device, and the second marker is fixed on the display device. The geometric structure between the first marker and the movable device is stable, and the geometric structure between the second marker and the display device is also stable, thereby improving the accuracy of obtaining the position and posture of the movable device in the coordinate system of the display device, thereby improving the robustness of tracking the position and posture of the movable device and user experience.

[0098] In addition, the second posture data and the fourth posture data are both six-degree-of-freedom data, which is conducive to obtaining the posture of the movable device 110 in the coordinate system of the display device 160 through a multiplication operation.

[0099] As an example, by formula P hc =P oh -1 ·P oc Obtain the position of the movable device 110 in the coordinate system of the display device 160, where P hc Refers to the position of the mobile device 110 in the coordinate system of the display device 160, P oh Refers to the fourth pose data, P oc Refers to the second pose data.

[0100] Correspondingly, an embodiment of the present invention also provides a posture acquisition device. Figure 4 It is a functional block diagram of an embodiment of a posture acquisition device of the present invention.

[0101] In this embodiment, the posture acquisition device 300 is suitable for acquiring the posture of the movable device in the coordinate system of the display device.

[0102] In this embodiment, the posture acquisition device 300 includes: a first posture acquisition module 301, used to acquire the first posture data of the current frame moment, the first posture data is used to represent the posture of the first marker fixed on the movable device in the reference coordinate system; a second posture acquisition module 302, used to acquire the second posture data of the movable device in the reference coordinate system based on the first posture data and the first relative posture data, the first relative posture data is used to represent the relative posture relationship between the movable device and the first marker; a third posture acquisition module 303, used to acquire the third posture data of the current frame moment, the third posture data is used to represent the posture of the second marker fixed on the display device in the reference coordinate system; a fourth posture acquisition module 304, used to acquire the fourth posture data of the display device in the reference coordinate system based on the third posture data and the second relative posture data, the second relative posture data is used to represent the relative posture relationship between the display device and the second marker; a fifth posture acquisition module 305, used to acquire the posture of the movable device in the coordinate system of the display device based on the second posture data and the fourth posture data.

[0103] Specifically, by obtaining the posture of the first marker on the movable device and the second marker on the display device in a reference coordinate system outside the display device, and using the fixed relative posture between the movable device and the first marker to obtain the second posture data of the movable device in the reference coordinate system, and using the fixed relative posture between the display device and the second marker to obtain the fourth posture data of the display device in the reference coordinate system, since the movable device and the display device share the same reference coordinate system, the posture of the movable device in the coordinate system of the display device can be obtained based on the second posture data and the fourth posture data, thereby reducing the dependence on the positioning of the movable device, which is beneficial to improving the problem of inaccurate positioning data collection of the display device due to the influence of the environment on the characteristic information of the movable device itself. At the same time, the first marker is fixed on the movable device and the second marker is fixed on the display device. The geometric structure between the first marker and the movable device is stable, and the geometric structure between the second marker and the display device is also stable, thereby improving the accuracy of obtaining the posture of the movable device in the coordinate system of the display device, thereby improving the robustness of tracking the posture of the movable device and the user experience.

[0104] In this embodiment, the posture acquisition device 300 is used to acquire the posture of the movable device in the coordinate system of the display device.

[0105] In this embodiment, the movable device includes a handheld controller. For example, the movable device is a handheld controller used in VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality).

[0106] VR, AR, and MR technologies can provide a simulated three-dimensional digital experience, allowing us to immerse ourselves in any virtual or real world, regardless of the constraints of our physical location. Hand controllers are a crucial interaction method in VR, AR, and MR. To meet the requirements of high-precision, high-robustness, and low-latency applications, the robustness of controller position tracking needs to be improved.

[0107] It is understandable that the movable device is not limited to the handle. In other embodiments, the movable device can also be other movable devices with posture tracking requirements.

[0108] In this embodiment, the display device includes a wearable display device.

[0109] The wearable display device is used in conjunction with a movable device. Specifically, the wearable display device can be VR smart glasses, AR smart glasses or MR smart glasses.

[0110] It is understandable that the display device is not limited to a wearable display device. In other embodiments, the display device may also be other devices that need to track and display the position of a movable device.

[0111] It should be noted that the display device is provided with an image acquisition device, and therefore the coordinate system of the display device is the camera coordinate system corresponding to the image acquisition device.

[0112] It should be noted that the first pose acquisition module 301 obtains the pose of the first marker in the reference coordinate system. In the subsequent process of obtaining the pose of the movable device in the reference coordinate system, the pose of the movable device in the reference coordinate system can be obtained based on the first pose data and the fixed relative pose between the movable device and the first marker. At the same time, since the first marker is fixed on the movable device, a rigid geometric structure is formed between the second marker and the display device. Therefore, the geometric structure between the first marker and the movable device is stable, which also improves the accuracy of obtaining the pose of the movable device in the reference coordinate system.

[0113] It should also be noted that the reference coordinate system and the coordinate system of the display device are two different coordinate systems. Therefore, there is no need to directly obtain the position and posture of the movable device in the coordinate system of the display device through the display device. Instead, the position and posture of the movable device in the coordinate system of the display device is indirectly obtained with the help of the reference coordinate system, thereby reducing the display device's dependence on the positioning of the movable device.

[0114] In this embodiment, the reference coordinate system is a three-dimensional coordinate system.

[0115] By adopting a three-dimensional coordinate system, the first pose data and the subsequently obtained third pose data are both six-degree-of-freedom (DoF) data, thereby obtaining the pose of the movable device without the need for inertial measurement data. In other words, the pose acquisition method can achieve pose tracking of movable devices that are not equipped with an inertial measurement unit (IMU).

[0116] In a specific embodiment, the reference coordinate system includes a coordinate system of a motion capture device.

[0117] Specifically, the motion capture device can accurately obtain the position and posture of the first marker in the coordinate system of the motion capture device (i.e., the first pose data), thereby obtaining the position and posture of the movable device in the coordinate system of the motion capture device based on the first pose data and the fixed relative pose between the movable device and the first marker.

[0118] As an example, the motion capture device includes an Optitrack device or a Mocap (Motion capture) device.

[0119] As an example, the number of the first markers is at least four, and any four of the first markers are non-coplanar.

[0120] It should be noted that the number of first markers is at least four, and any four of the first markers are non-coplanar, so that the structure composed of multiple first markers is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the first posture data, the posture information of the first marker in different dimensional directions can be obtained.

[0121] As an example, Figure 3 As shown, the number of first markers is 5, and any four first markers are non-coplanar.

[0122] In this embodiment, the first marker includes a light-emitting element.

[0123] Specifically, the signal light emitted by the light-emitting element is more conducive to the motion capture device capturing the first marker.

[0124] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0125] Therefore, as an example, the step of obtaining the first position data at the current frame moment includes: obtaining multiple first 3dof data, the number of the first 3dof data is at least four, and is used to respectively represent the position of each first marker in the reference coordinate system; based on the multiple first 3dof data, obtaining the first position data at the current frame moment.

[0126] Specifically, the first bit pose data is 6dof data.

[0127] Specifically, the movable device and the display device share a reference coordinate system. Therefore, the posture of the movable device in the coordinate system of the display device can be obtained through the second posture data and the subsequently obtained fourth posture data, thereby reducing the dependence on the positioning of the movable device, which is beneficial to improving the problem of inaccurate positioning data collection of the display device due to the characteristic information of the movable device itself being affected by the environment.

[0128] It should be noted that since the first marker is fixed on the movable device, a rigid geometric structure is formed between the first marker and the movable device. Therefore, the movable device and the first marker have a fixed relative posture relationship. After the posture of the first marker in the reference coordinate system (that is, the first posture number) is obtained, the second posture data of the movable device in the reference coordinate system can be converted based on the first posture data and the first relative posture data.

[0129] As an example, the first relative posture data includes six degrees of freedom (DoF) data.

[0130] Specifically, the first relative posture data includes a rotation offset between the movable device and the first marker, and a translation offset between the movable device and the first marker.

[0131] It should be noted that by obtaining the first relative posture data between the movable device and the first marker including six-degree-of-freedom data, in the process of obtaining the second posture data of the movable device in the reference coordinate system, the first relative posture data and the first posture data can be multiplied by each other to obtain the second posture data of the movable device in the reference coordinate system.

[0132] As an example, by formula P oc =P oc' ·P c'cObtain the second pose data of the movable device in the reference coordinate system, wherein P oc Refers to the second pose data, P oc' Refers to the first pose data, P c'c Refers to the first relative posture data.

[0133] It should be noted that the third pose acquisition module 303 is used to acquire the pose of the second marker in the reference coordinate system. In the subsequent process of acquiring the pose of the display device in the reference coordinate system, the pose of the display device in the reference coordinate system can be acquired based on the third pose data and the fixed relative pose between the display device and the second marker. Furthermore, since the second marker is fixed to the display device, a rigid geometric structure is formed between the second marker and the display device. Therefore, the geometric structure between the second marker and the display device is stable, thereby improving the accuracy of acquiring the pose of the display device in the reference coordinate system.

[0134] As an example, the number of the second markers is at least four, and any four of the second markers are non-coplanar.

[0135] It should be noted that the number of second markers is at least four, and any four of the second markers are non-coplanar, so that the structure composed of multiple second markers is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the third posture data, the posture information of the second markers in different dimensional directions can be obtained.

[0136] As an example, Figure 3 As shown, the number of second markers is 5, and any four second markers are non-coplanar.

[0137] In this embodiment, the second marker includes a light-emitting element.

[0138] Specifically, the signal light emitted by the light-emitting element is more conducive to the dynamic capture device capturing the second marker.

[0139] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0140] As an example, the step of obtaining the third posture data at the current frame moment includes: obtaining multiple second 3dof data, the number of the second 3dof data is at least four, and is used to respectively represent the posture of each second marker in the reference coordinate system; based on the multiple second 3dof data, obtaining the third posture data at the current frame moment.

[0141] Specifically, the third posture data is 6dof data.

[0142] As an example, the posture acquisition apparatus 300 further includes: a sixth posture acquisition module 306, configured to acquire second relative posture data between the display device and the second marker.

[0143] It should be noted that the sixth posture acquisition module 306 can use the second relative posture data and the third posture data to obtain the fourth posture data of the display device in the reference coordinate system by obtaining the second relative posture data, which is conducive to the subsequent use of the fourth posture data and the second posture data to obtain the posture of the movable device in the coordinate system of the display device, thereby reducing the dependence on the positioning of the movable device, and is conducive to improving the problem of inaccurate positioning data collection of the display device due to the influence of the environment on the characteristic information of the movable device itself.

[0144] It should also be noted that the second marker is fixed on the display device, and a rigid geometric structure is formed between the second marker and the display device. The second relative posture data between the display device and the second marker is fixed. Therefore, by first obtaining the second relative posture data, in the subsequent process of obtaining the fourth posture data of the display device in the reference coordinate system at the current frame moment, the posture of the display device in the reference coordinate system can be directly calculated based on the posture of the second marker in the reference coordinate system.

[0145] It can be understood that when the second marker does not change, the second relative posture data between the display device and the second marker is fixed. Therefore, it is only necessary to obtain the second relative posture data between the display device and the second marker before obtaining the fourth posture data for the first time. Once the second relative posture data is obtained, there is no need to repeatedly calculate the second relative posture data between the display device and the second marker when obtaining the posture of the movable device at subsequent frame times.

[0146] As an example, the sixth posture acquisition module 306 includes: a first acquisition unit 3051, used to obtain first static posture data of the display device in the reference coordinate system, and second static posture data of the second marker in the reference coordinate system, the first static posture data is used to represent the posture of the display device when it is in a static state, and the second static posture data is used to represent the posture of the second marker when it is in a static state; a second acquisition unit 3052, used to obtain second relative posture data between the display device and the second marker based on the first static posture data and the second static posture data.

[0147] The first static posture data and the second static posture data are respectively the posture data of the display device and the second marker in the same reference coordinate system, and the display device and the second marker have a fixed relative posture. Therefore, after obtaining the first static posture data and the second static posture data, it is easy to obtain the second relative posture data between the display device and the second marker.

[0148] It should be noted that in the process of obtaining the second relative posture data, keeping the display device and the second marker in a stationary state can improve the accuracy of obtaining the second relative posture data between the display device and the second marker. Accordingly, when subsequently obtaining the posture of the display device in the reference coordinate system, the accuracy of obtaining the posture data of the display device in the reference coordinate system is also improved.

[0149] As an example, the step of obtaining the first static posture data includes: obtaining the first position information of the first marker in the coordinate system of the display device, and the second position information of the first marker in the reference coordinate system; based on the first position information, the second position information, and the internal parameters of the image acquisition device in the display device, obtaining the first static posture data of the display device in the reference coordinate system.

[0150] Compared with directly obtaining the first static posture data of the display device in the reference coordinate system through a motion capture device, this embodiment obtains the second position information of the first marker in the reference coordinate system through a motion capture device to infer the first static posture data. The motion capture device is more likely to receive the specific marker signal (such as an infrared light signal) generated by the first marker, which can further improve the accuracy of obtaining the first static posture data of the display device in the reference coordinate system.

[0151] Specifically, through the first position information and the second position information, the corresponding relationship between the position information of the first marker in the coordinate system of the display device and the position information of the first marker in the reference coordinate system can be obtained, and the internal parameters of the image acquisition device in the display device are used to facilitate the subsequent calculation of the first static posture data of the display device in the reference coordinate system through the PnP algorithm.

[0152] As an example, the step of obtaining first position information of the first marker in the coordinate system of the display device includes: obtaining a two-dimensional pixel position of the first marker in the coordinate system of the display device.

[0153] It should be noted that, by obtaining the two-dimensional pixel position of the first marker in the coordinate system of the display device, the first static posture data of the display device in the reference coordinate system can be converted through the PnP algorithm.

[0154] In this embodiment, the step of obtaining the two-dimensional pixel position of the first marker in the coordinate system of the display device includes: obtaining a picture of the first marker at the current frame moment through the image acquisition device of the display device; and obtaining the two-dimensional pixel position of the first marker in the picture in the coordinate system of the display device.

[0155] As an example, the step of acquiring the second position information of the first marker in the reference coordinate system includes: acquiring the three-dimensional coordinate position of the first marker in the reference coordinate system.

[0156] It should be noted that, by obtaining the three-dimensional coordinate position of the first marker in the reference coordinate system, the first static posture data of the display device in the reference coordinate system can be converted through the PnP algorithm.

[0157] In this embodiment, a three-dimensional coordinate position of a first marker in a coordinate system of the motion capture device is obtained by a motion capture device. Specifically, 3D-of data of a plurality of first markers is obtained by the motion capture device, wherein the number of 3D-of data is at least four and each of the 3D-of data is used to represent the position of each first marker in a reference coordinate system; and based on the plurality of 3D-of data, the three-dimensional coordinate position of the first marker in the coordinate system of the motion capture device is obtained.

[0158] As an example, the first static pose data is calculated by a PnP (Perspective-n-Point) algorithm.

[0159] Specifically, through the formula P oh =PnP(pixel, 3d points, K) obtains the first static posture data of the display device in the reference coordinate system, where P oh It refers to the first static posture data in the reference coordinate system when the display device is in a stationary state, pixel refers to the two-dimensional pixel position of the first marker in the coordinate system of the display device, 3d points refers to the three-dimensional coordinate position of the first marker in the reference coordinate system, and K refers to the internal parameter of the image acquisition device.

[0160] Specifically, after obtaining the first static posture data of the display device in the reference coordinate system, it can be calculated by formula P h'h =P oh' -1 ·P oh Acquire the second relative posture data between the display device and the second marker, wherein P h'h Refers to the second relative pose data, P oh' Refers to the second static pose data, P ohRefers to the first static pose data.

[0161] By obtaining the second relative posture data between the display device and the second marker, in the subsequent process of obtaining the fourth posture data of the display device in the reference coordinate system at the current frame moment, the posture of the display device in the reference coordinate system can be directly calculated based on the posture of the second marker in the reference coordinate system.

[0162] It should be noted that since the second marker is fixed on the display device, a rigid geometric structure is formed between the second marker and the display device. Therefore, there is a fixed relative posture relationship between the display device and the second marker. After the posture of the second marker in the reference coordinate system (that is, the third posture number) is obtained, the fourth posture data of the display device in the reference coordinate system can be converted based on the third posture data and the second relative posture data.

[0163] As an example, the second relative posture data includes six degrees of freedom (DoF) data.

[0164] Specifically, the second relative posture data includes a rotation offset between the display device and the second marker, and a translation offset between the movable device and the first marker.

[0165] It should be noted that by making the second relative posture data between the display device and the second marker include six-degree-of-freedom data, in the process of obtaining the fourth posture data of the display device in the reference coordinate system, the second relative posture data and the third posture data can be multiplied by each other to obtain the fourth posture data of the display device in the reference coordinate system.

[0166] Specifically, the fourth posture acquisition module 304 is used to obtain the fourth posture data of the display device in the reference coordinate system based on the third posture data and the second relative posture data, and the movable device and the display device share a reference coordinate system, so that the posture of the movable device in the coordinate system of the display device can be obtained according to the second posture data and the fourth posture data, reducing the dependence on the positioning of the movable device, which is conducive to improving the problem of inaccurate positioning data collection of the display device due to the influence of the environment on the characteristic information of the movable device itself.

[0167] As an example, by formula P oh =P oh' ·P h'h Acquire the fourth posture data of the display device in the reference coordinate system, wherein P oh Refers to the fourth pose data, P oh' Refers to the third pose data, P h'hRefers to the second relative pose data.

[0168] Compared with the solution of directly obtaining the posture of the movable device through the display device, the fifth posture acquisition module 305 obtains the posture of the movable device in the coordinate system of the display device based on the second posture data and the fourth posture data, thereby reducing the dependence on the positioning of the movable device, which is beneficial to improving the problem of inaccurate positioning data collection of the display device due to the influence of the environment on the characteristic information of the movable device itself.

[0169] At the same time, the first marker is fixed on the movable device, and the second marker is fixed on the display device. The geometric structure between the first marker and the movable device is stable, and the geometric structure between the second marker and the display device is also stable, thereby improving the accuracy of obtaining the position and posture of the movable device in the coordinate system of the display device, thereby improving the robustness of tracking the position and posture of the movable device and user experience.

[0170] At the same time, the second posture data and the fourth posture data are both six-degree-of-freedom data, which is conducive to obtaining the posture of the movable device in the coordinate system of the display device through multiplication operation.

[0171] As an example, by formula P hc =P oh -1 ·P oc Obtain the position of the movable device in the coordinate system of the display device, where P hc Refers to the position of the mobile device in the coordinate system of the display device, P oh Refers to the fourth pose data, P oc Refers to the second pose data.

[0172] Correspondingly, an embodiment of the present invention further provides a display system. Figures 2 to 3 : is a system diagram of a display system according to an embodiment of the present invention. Figure 2 It is a schematic diagram showing the structure of the system. Figure 3 (a) is an enlarged view of the display device and the second fixture. Figure 3 (b) is an enlarged view of the movable device and the first fixture.

[0173] In this embodiment, the display system includes: a movable device 110, on which a first marker 103 is fixed; a display device 160, on which a second marker 100 is fixed; and a processor, which is used to execute the posture acquisition method described in any of the aforementioned embodiments.

[0174] In this embodiment, the display system is used to obtain the position and posture of the movable device 110 in the coordinate system of the display device 160 .

[0175] In this embodiment, the mobile device 110 includes a handheld controller. For example, the mobile device 110 is a handheld controller used in VR (Virtual Reality), AR (Augmented Reality), or MR (Mixed Reality).

[0176] VR, AR, and MR technologies can provide a simulated three-dimensional digital experience, allowing us to immerse ourselves in any virtual or real world, regardless of the constraints of our physical location. Controllers are a crucial interaction method in VR, AR, and MR. To meet the demands of high-precision, high-robustness, and low-latency applications, robust controller tracking is crucial.

[0177] It is understandable that the movable device 110 is not limited to a handle. In other embodiments, the movable device 110 may also be other movable devices 110 with posture tracking requirements.

[0178] In this embodiment, a first marker 103 is fixed on the movable device 110 .

[0179] It should be noted that, since the first marker 103 is fixed on the movable device 110 , a rigid geometric structure is formed between the first marker 103 and the movable device 110 . Therefore, the movable device 110 and the first marker 103 have a fixed relative posture relationship.

[0180] It should also be noted that by fixing the first marker 103 on the movable device 110, the posture of the movable device 110 can be obtained by obtaining the posture of the first marker 103 and the relative posture relationship between the movable device 110 and the first marker 103, thereby reducing the dependence on the positioning of the movable device 110.

[0181] In this embodiment, the number of the first markers 103 is at least four, and any four of the first markers 103 are not coplanar.

[0182] It should be noted that the number of first markers 103 is at least four, and any four of the first markers 103 are non-coplanar, so that the structure composed of multiple first markers 103 is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the posture data of the first marker 103, the posture information of the first marker 103 in different dimensional directions can be obtained.

[0183] As an example, Figure 3As shown, the number of the first markers 103 is 5, and any four first markers 103 are non-coplanar.

[0184] In this embodiment, the first marker 103 includes a light-emitting element.

[0185] Specifically, the signal light emitted by the light-emitting element is more conducive to the motion capture device capturing the first marker 103.

[0186] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0187] In this embodiment, the display system also includes: a first fixture 105, fixed on the movable device 110, the first fixture 105 includes a first rigid bracket 104 and the first marker 103, one end of the first rigid bracket 104 is fixed on the movable device 110, and the other end is fixed to the first marker 103.

[0188] Specifically, the first rigid bracket 104 serves to fix and support the first marker 103 , so that the first marker 103 can be firmly fixed on the movable device 110 , reducing the probability of shaking of the first marker 103 , thereby improving the accuracy of obtaining the posture data of the first marker 103 .

[0189] In this embodiment, the display device 160 includes a wearable display device 160 .

[0190] The wearable display device 160 is used in conjunction with the movable device 110. Specifically, the wearable display device 160 can be VR smart glasses, AR smart glasses or MR smart glasses.

[0191] In this embodiment, a second marker 100 is fixed on the display device 160 .

[0192] It should be noted that, since the second marker 100 is fixed on the display device 160 , a rigid geometric structure is formed between the second marker 100 and the display device 160 . Therefore, there is a fixed relative posture relationship between the display device 160 and the second marker 100 .

[0193] It should also be noted that by fixing the second marker 100 on the display device 160 , the posture of the display device 160 can be obtained by obtaining the posture of the second marker 100 and the relative posture relationship between the display device 160 and the second marker 100 .

[0194] In this embodiment, the number of the second markers 100 is at least four, and any four of the second markers 100 are not coplanar.

[0195] It should be noted that the number of second markers 100 is at least four, and any four of the second markers 100 are non-coplanar, so that the structure composed of multiple second markers 100 is a three-dimensional structure rather than a plane or a line, so that in the process of obtaining the posture data of the second marker 100, the posture information of the second marker 100 in different dimensional directions can be obtained.

[0196] As an example, Figure 3 As shown, the number of the second markers 100 is 5, and any four second markers 100 are non-coplanar.

[0197] In this embodiment, the second marker 100 includes a light-emitting element.

[0198] Specifically, the signal light emitted by the light-emitting element is more conducive to the motion capture device capturing the second marker 100.

[0199] As an example, the light-emitting element is a self-luminous light source, such as an LED lamp.

[0200] In this embodiment, the display system also includes: a second fixture 102, fixed on the display device 160, the second fixture 102 includes a second rigid bracket 101 and a second marker 100, one end of the second rigid bracket 101 is fixed on the display device 160, and the other end is fixed to the second marker 100.

[0201] Specifically, the second rigid bracket 101 serves to fix and support the second marker 100 , so that the second marker 100 can be firmly fixed on the display device 160 , reducing the probability of the second marker 100 shaking, thereby improving the accuracy of obtaining the posture data of the second marker 100 .

[0202] In this embodiment, the display system further includes: a motion capture device 180, which is used to provide a reference coordinate system and to obtain the first posture data and the third posture data.

[0203] Specifically, the motion capture device 180 can accurately obtain the posture of the first marker 103 and the second marker 100 in the coordinate system of the motion capture device 180 (i.e., the first posture data and the third posture data), thereby obtaining the posture of the movable device 110 in the coordinate system of the motion capture device 180 based on the first posture data and the fixed relative posture between the movable device 110 and the first marker 103, and obtaining the posture of the display device 160 in the coordinate system of the motion capture device 180 based on the third posture data and the fixed relative posture between the display device 160 and the second marker 100.

[0204] As an example, the motion capture device 180 includes an Optitrack device or a Mocap (Motion capture) device.

[0205] In this embodiment, the timestamp of the motion capture device 180 is aligned with the timestamp of the display device 160 .

[0206] Specifically, the timestamp of the motion capture device 180 is aligned with the timestamp of the display device 160, which can ensure the temporal correspondence between the posture of the first marker 103 and the second marker 100 acquired by the motion capture device 180 and the images of the first marker 103 and the second marker 100 acquired by the image acquisition device in the display device 160, thereby providing an accurate time reference for subsequent data processing.

[0207] In this embodiment, the processor is used to execute relevant computer instructions to implement the posture acquisition method provided by this embodiment.

[0208] As an example, the processor can be a central processing unit (CPU), a field programmable gate array (FPGA), a programmable logic controller (PLC), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the posture acquisition method described in an embodiment of the present invention.

[0209] An embodiment of the present invention further provides a storage medium, wherein the storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the posture acquisition method provided in the aforementioned embodiment.

[0210] An embodiment of the present invention also provides a computer program product, including computer instructions, which, when executed by a processor, is used to implement the posture acquisition method provided in the aforementioned embodiment.

[0211] The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise mentioned, the elements or features may be considered as optional. Each element or feature may be put into practice without being combined with other elements or features. In addition, the embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some configurations of any one embodiment may be included in another embodiment and may be replaced by the corresponding configuration of another embodiment. It is obvious to those skilled in the art that claims that do not have a clear reference relationship to each other in the appended claims may be combined into embodiments of the present invention, or may be included as new claims in amendments after submitting this application.

[0212] The embodiments of the present invention may be implemented by various means such as hardware, firmware, software, or a combination thereof. In a hardware configuration, the method according to the exemplary embodiment of the present invention may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0213] In a firmware or software configuration, the embodiments of the present invention may be implemented in the form of modules, procedures, functions, and the like. Software codes may be stored in a memory unit and executed by a processor. The memory unit may be located inside or outside the processor and may send and receive data to and from the processor via various known means.

[0214] Although the embodiments of the present invention are disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

Claims

1. A posture acquisition method, characterized in that: The posture acquisition method is suitable for acquiring the posture of a movable device in a coordinate system of a display device, and the posture acquisition method includes: Acquire first pose data at a current frame moment, where the first pose data is used to represent a pose of a first marker fixed on the movable device in a reference coordinate system; Acquire second pose data of the movable device in the reference coordinate system based on the first pose data and first relative pose data, wherein the first relative pose data is used to represent a relative pose relationship between the movable device and the first marker; Acquire third posture data at the current frame moment, where the third posture data is used to represent the posture of a second marker fixed on the display device in a reference coordinate system; Acquire fourth pose data of the display device in the reference coordinate system based on the third pose data and the second relative pose data, wherein the second relative pose data is used to represent a relative pose relationship between the display device and the second marker; Based on the second posture data and the fourth posture data, the posture of the movable device in the coordinate system of the display device is obtained.

2. The posture acquisition method according to claim 1, wherein: The reference coordinate system is a three-dimensional coordinate system; the first relative posture data and the second relative posture data both include six-degree-of-freedom data.

3. The posture acquisition method according to claim 1, wherein: Before obtaining fourth posture data of the display device in the reference coordinate system based on the third posture data and the second relative posture data, the method further includes: obtaining second relative posture data between the display device and the second marker; The step of obtaining the second relative posture data includes: obtaining the first static posture data of the display device in the reference coordinate system, and the second static posture data of the second marker in the reference coordinate system, the first static posture data being used to indicate the posture of the display device when it is in a static state, and the second static posture data being used to indicate the posture of the second marker when it is in a static state; based on the first static posture data and the second static posture data, obtaining the second relative posture data between the display device and the second marker.

4. The posture acquisition method according to claim 3, wherein: The step of acquiring the first static posture data includes: acquiring first position information of the first marker in the coordinate system of the display device and second position information of the first marker in the reference coordinate system; Based on the first position information, the second position information, and the internal parameters of the image acquisition device in the display device, first static posture data of the display device in the reference coordinate system is acquired.

5. The posture acquisition method according to claim 4, wherein: The step of obtaining first position information of the first marker in the coordinate system of the display device includes: obtaining a two-dimensional pixel position of the first marker in the coordinate system of the display device; The step of acquiring second position information of the first marker in the reference coordinate system includes: acquiring the three-dimensional coordinate position of the first marker in the reference coordinate system; By formula P oh =PnP(pixel, 3d points, K) obtains the first static posture data of the display device in the reference coordinate system, where P oh It refers to the first static posture data in the reference coordinate system when the display device is in a stationary state, pixel refers to the two-dimensional pixel position of the first marker in the coordinate system of the display device, 3d points refers to the three-dimensional coordinate position of the first marker in the reference coordinate system, and K refers to the internal parameter of the image acquisition device.

6. The posture acquisition method according to claim 3, wherein: By formula P h'h =P oh' -1 ·P oh Acquire the second relative posture data between the display device and the second marker, wherein P h'h Refers to the second relative pose data, P oh' Refers to the second static pose data, P oh Refers to the first static pose data.

7. The posture acquisition method according to any one of claims 1 to 6, characterized in that: By formula P oh =P oh' ·P h'h Acquire the fourth posture data of the display device in the reference coordinate system, wherein P oh Refers to the fourth pose data, P oh' Refers to the third pose data, P h'h Refers to the second relative pose data.

8. The posture acquisition method according to any one of claims 1 to 6, characterized in that: The step of obtaining the first pose data at the current frame moment includes: obtaining a plurality of first 3dof data, the number of the first 3dof data being at least four and being used to respectively represent the poses of each first marker in the reference coordinate system; obtaining the first pose data at the current frame moment based on the plurality of the first 3dof data; The step of obtaining the third posture data at the current frame moment includes: obtaining multiple second 3dof data, the number of the second 3dof data is at least four, and is used to respectively represent the posture of each second marker in the reference coordinate system; based on the multiple second 3dof data, obtaining the third posture data at the current frame moment.

9. The posture acquisition method according to any one of claims 1 to 6, characterized in that: By formula P oc =P oc' ·P c'c Obtain second position data of the movable device in the reference coordinate system, wherein P oc Refers to the second pose data, P oc' Refers to the first pose data, P c'c Refers to the first relative pose data.

10. The posture acquisition method according to any one of claims 1 to 6, characterized in that: By formula P hc =P oh -1 ·P oc Obtain the position of the movable device in the coordinate system of the display device, where P hc Refers to the position of the mobile device in the coordinate system of the display device, P oh Refers to the fourth pose data, P oc Refers to the second pose data.

11. The posture acquisition method according to any one of claims 1 to 6, characterized in that: The reference coordinate system includes the coordinate system of the motion capture device.

12. A posture acquisition device, adapted to acquire the posture of a movable device in a coordinate system of a display device, characterized in that: include: a first pose acquisition module, configured to acquire first pose data at a current frame moment, wherein the first pose data is used to represent a pose of a first marker fixed on the movable device in a reference coordinate system; a second posture acquisition module, configured to acquire second posture data of the movable device in the reference coordinate system based on the first posture data and first relative posture data, wherein the first relative posture data is used to represent a relative posture relationship between the movable device and the first marker; A third posture acquisition module is used to obtain third posture data at the current frame moment, wherein the third posture data is used to represent the posture of the second marker fixed on the display device in the reference coordinate system; a fourth posture acquisition module, configured to acquire fourth posture data of the display device in the reference coordinate system based on the third posture data and second relative posture data, wherein the second relative posture data is used to represent a relative posture relationship between the display device and the second marker; The fifth posture acquisition module is used to obtain the posture of the movable device in the coordinate system of the display device based on the second posture data and the fourth posture data.

13. A display system, characterized in that: include: A movable device, wherein a first marker is fixed on the movable device; a display device, wherein a second marker is fixed to the display device; A processor, wherein the processor is used to execute the posture acquisition method according to any one of claims 1 to 11.

14. The display system according to claim 13, wherein: The display system further includes a motion capture device for providing a reference coordinate system and for acquiring the first posture data and the third posture data.

15. The display system according to claim 14, wherein: The timestamp of the motion capture device is aligned with the timestamp of the display device.

16. The display system according to claim 13, wherein: The number of the first markers is at least four, and any four of the first markers are non-coplanar; The number of the second markers is at least four, and any four of the second markers are non-coplanar.

17. The display system according to claim 13, wherein: The first marker and the second marker both include a light-emitting element.

18. The display system according to claim 13, wherein: The display system further includes: a first fixture fixed on the movable device, the first fixture including a first rigid bracket and the first marker, one end of the first rigid bracket being fixed on the movable device, and the other end being fixed to the first marker; The second fixture is fixed on the display device. The second fixture includes a second rigid bracket and the second marker. One end of the second rigid bracket is fixed on the display device, and the other end is fixed on the second marker.

19. The display system according to any one of claims 13 to 18, wherein: The movable device includes a handle.

20. The display system according to any one of claims 13 to 18, wherein: The display device includes a wearable display device.

21. A storage medium, characterized in that The storage medium stores one or more computer instructions, and the one or more computer instructions are used to implement the posture acquisition method according to any one of claims 1 to 11.

22. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, they are used to implement the posture acquisition method according to any one of claims 1 to 11.

Images