Head-mounted display device, tracking device, and data calibration method

Abstract

A head-mounted display device is disclosed. The head-mounted display device is communicatively coupled to a tracking device. The head-mounted display device includes a host memory and a host processor. The host memory is configured to store a host map. The host processor is configured to perform the following steps: obtain client capability information from the tracking device; generate a local map from the host map based on the client capability information; and transmit the local map to the tracking device. By reducing the amount of transmission data transmitted from the head-mounted display device to the tracking device in an initial stage, a user can obtain a better user experience.

Description

Technical Field

[0001] This disclosure relates to a head-mounted display device, a tracking device, and a data calibration method, and more particularly to a head-mounted display device, a tracking device, and a data calibration method for a virtual reality system. Background Technology

[0002] In the initial stages of a virtual reality system, to align the client coordinate system of the tracking device with the host coordinate system of the head-mounted display, the tracking device needs to download the host map stored on the head-mounted display. However, the longer the virtual reality system is used, the larger the host map becomes. The longer it takes to download the host map from the head-mounted display to the tracking device, the worse the user experience will be. Summary of the Invention

[0003] One embodiment of this disclosure discloses a head-mounted display device. The head-mounted display device is communicatively connected to a tracking device. The head-mounted display device includes a host-side memory and a host-side processor. The host-side memory stores a host-side map. The host-side processor performs the following steps: obtaining client capability data from the tracking device; generating a local map from the host-side map based on the client capability data; and transmitting the local map to the tracking device.

[0004] In some embodiments, the feature is that the client capability data includes a client baseline value of the tracking device.

[0005] In some embodiments, the local map is characterized by being generated based on a ratio between a host baseline value of the head-mounted display device and a client baseline value.

[0006] In some embodiments, the feature is that the client capability data further includes a preset location of the tracking device.

[0007] In some embodiments, the local map is characterized in that it is generated based on the client baseline value and the preset location.

[0008] In some embodiments, the client capability data includes a client effective depth distance of the tracking device.

[0009] In some embodiments, the local map is characterized in that it is generated based on the client's effective depth distance.

[0010] Another embodiment of this disclosure discloses a tracking device. The tracking device is communicatively connected to a head-mounted display device. The tracking device includes a client processor and a client memory. The client processor is used to perform the following steps: transmitting client capability data to the head-mounted display device; and receiving a local map generated based on the client capability data. The client memory is used to store the local map.

[0011] In some embodiments, the feature is that the client capability data includes a client baseline value of the tracking device.

[0012] Another embodiment of this disclosure discloses a data calibration method. The data calibration method is applicable to a virtual reality system including a head-mounted display device and a tracking device. The data calibration method includes the following steps: transmitting client capability data from the tracking device to the head-mounted display device; generating a local map from a host-side map stored in the head-mounted display device based on the client capability data; and transmitting the local map from the head-mounted display device to the tracking device. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a virtual reality (VR) system according to some embodiments of the present invention;

[0014] Figure 2 As shown in some embodiments of the present invention Figure 1 A schematic diagram of the virtual reality system in the image;

[0015] Figure 3 This is a flowchart illustrating a data calibration method according to some embodiments of the present invention;

[0016] Figure 4 A schematic diagram illustrating the calculation of effective depth distance according to some embodiments of the present invention; and

[0017] Figure 5 This is a schematic diagram of an embodiment of a local map according to some embodiments of the present invention.

[0018] Symbol explanation:

[0019] E: Environment

[0020] 100: Virtual Reality System

[0021] 110: Head-mounted display device

[0022] 130A, 130B: Tracking devices

[0023] 130: Tracking device

[0024] 112: Host-side processor

[0025] 114: Host-side storage

[0026] 116: Host-side input / output circuit

[0027] 118A, 118B: Main unit camera

[0028] 132: Client Processor

[0029] 134: Client Storage

[0030] 136: Client Input / Output Circuit

[0031] 138A, 138B: Client-side camera

[0032] 300: Data Calibration Method

[0033] S310, S330, S350: Steps

[0034] P: Point

[0035] Z: Effective depth distance

[0036] OL: Aperture Center

[0037] OR: Center of the aperture

[0038] U: Imaging plane

[0039] UL: Coordinates

[0040] UR: Coordinates

[0041] PL: Position

[0042] PR: Location

[0043] BL: Client Baseline

[0044] F: Focal length

[0045] X: axis

[0046] MH: Host-side Map

[0047] DH:radius

[0048] MC1, MC2: Local Maps

[0049] DC1, DC2: Effective depth distance of the client

[0050] PC1, PC2: Preset positions Detailed Implementation

[0051] The spirit of this disclosure will be clearly illustrated below with reference to the accompanying drawings and detailed description. Anyone skilled in the art who understands the embodiments of this disclosure may make changes and modifications based on the technology taught in this disclosure without departing from the spirit and scope of this disclosure.

[0052] Please see Figure 1 . Figure 1 This is a schematic diagram of a virtual reality (VR) system 100 according to some embodiments of the present invention.

[0053] like Figure 1 As shown, the virtual reality system 100 includes a head-mounted display (HMD) 110 and tracking devices 130A and 130B. The HMD 110 can be mounted on a VR headset and worn by a user. When the user wears the VR headset, the HMD 110 covers the user's field of vision and provides virtual reality images to the user. In some embodiments, the tracking devices 130A and 130B are self-tracking devices, held and controlled by the user.

[0054] In some embodiments, the head-mounted display device 110 serves as a host device, while the tracking devices 130A and 130B serve as client devices. In some embodiments, the head-mounted display device 110 is communicatively connected to the tracking devices 130A and 130B.

[0055] like Figure 1 As shown, a user operates the virtual reality system 100 in environment E. In some embodiments, the virtual reality system 100 generates and updates a host-side map of environment E based on the characteristics of environment E. In some embodiments, the virtual reality system 100 is a simultaneous localization and mapping (SLAM) system.

[0056] Please see Figure 2 . Figure 2 As shown in some embodiments of the present invention Figure 1 A schematic diagram of the virtual reality system 100. (See diagram below.) Figure 2 As shown, the head-mounted display device 110 includes a host processor 112, a host memory 114, a host input / output circuit 116, a host camera 118A, and a host camera 118B. The host memory 114 is coupled to the host processor 112, the host input / output circuit 116 is coupled to the host processor 112, the host camera 118A is coupled to the host processor 112, and the host camera 118B is coupled to the host processor 112.

[0057] like Figure 2 The tracking device 130 shown can be used to represent, for example, Figure 1The tracking devices 130A and 130B are shown.

[0058] like Figure 2 As shown, the tracking device 130 includes a client processor 132, a client memory 134, a client input / output circuit 136, a client camera 138A, and a client camera 138B. The client memory 134 is coupled to the client processor 132, the client input / output circuit 136 is coupled to the client processor 132, the client camera 138A is coupled to the client processor 132, and the client camera 138B is coupled to the client processor 132.

[0059] In some embodiments, the head-mounted display device 110 transmits or receives information / data or signals via the host input / output circuit 116, while the tracking devices 130A and 130B transmit or receive information / data or signals via the client input / output circuit 136.

[0060] like Figure 1 and Figure 2 The operation method of the virtual reality system 100 shown will be referred to in the following paragraphs. Figure 3 Please provide an explanation.

[0061] Please see Figure 3 . Figure 3 This is a flowchart illustrating a data calibration method 300 according to some embodiments of the present invention. The data calibration method 300 can be performed by... Figure 1 and Figure 2 The virtual reality system 100 in the middle is executed. The data calibration method 300 includes steps S310 to S350.

[0062] In some embodiments, the data calibration method 300 is performed in the initial stage when the user begins to use the virtual reality system 100.

[0063] In step S310, the tracking device transmits the client capability data to the head-mounted display device. In some embodiments, step S310 is performed by... Figure 2 The host processor 112 of the head-mounted display device 110 shown executes the commands.

[0064] Please also refer to Figure 1 and Figure 2 In some embodiments, during the initial stage of the virtual reality system 100, the host-side memory 114 of the head-mounted display device 110 stores the host-side map of the environment E.

[0065] In some embodiments, the client processor 132 of the tracking device 130A transmits client capability data of the tracking device 130A to the head-mounted display device 110, while the tracking device 130B transmits client capability data of the tracking device 130B to the head-mounted display device 110.

[0066] In step S330, the head-mounted display device generates a local map from the host-side map stored in the head-mounted display device based on the client capability data. In some embodiments, step S330 is performed by... Figure 2 The host processor 112 of the head-mounted display device 110 in the middle executes.

[0067] Please also refer to Figure 1 as well as Figure 2 In some embodiments, the client capability data is the client baseline value of the tracking device 130, while the local map is generated based on the ratio between the host baseline value of the head-mounted display device 110 and the client baseline value.

[0068] In some embodiments, the client baseline value refers to the distance between client camera 138A and client camera 138B. The host baseline value refers to the distance between host camera 118A and host camera 118B. In some embodiments, the host baseline value is 15 cm to 25 cm, while the client baseline value is 7.5 m to 12.5 m. The client baseline values ​​and host baseline values ​​described above are merely illustrative examples, and the implementation of this invention is not limited to the above.

[0069] In some embodiments, the ratio between the area of ​​the local map and the area of ​​the host map is proportional to the ratio between the cube of the client baseline value and the cube of the host baseline value.

[0070] For example, if the client baseline value is half the host baseline value, then the ratio between the cube of the client baseline value and the cube of the host baseline value is 1 / 8, and the ratio between the area of ​​the local map and the area of ​​the host map is also 1 / 8. In other words, the area of ​​the local map is 1 / 8 of the area of ​​the host map.

[0071] In some embodiments, the host processor 112 calculates the effective depth distance of the tracking device 130 based on the client baseline value. In some embodiments, the host processor 112 also calculates the effective depth distance of the head-mounted display device 110 based on the host baseline value.

[0072] Please see Figure 4 . Figure 4 This is a schematic diagram illustrating the calculation of effective depth distance according to some embodiments of the present invention. Figure 4 As shown, assuming Figure 2The client cameras 138A and 138B shown are horizontally positioned on the X-axis. OL represents the aperture center of client camera 138A on the X-axis, and OR represents the aperture center of client camera 138B on the X-axis. The distance between client cameras 138A and 138B is the client baseline BL. F represents the focal length. UL represents the coordinates on the imaging plane U corresponding to client camera 138A, while -UR represents the coordinates on the imaging plane U corresponding to client camera 138B. Figure 1 In the environment E shown, point P forms the left image at PL, and its position PR on the imaging plane U forms the right image. Z represents the effective depth distance of the client.

[0073] Based on the similarity between triangles P-PL-PR and P-OL-OR, we obtain formula (1):

[0074]

[0075] Based on formula (1), the effective depth distance of the client is obtained, as shown in formula (2):

[0076]

[0077] The method for calculating the effective depth distance of the host based on the host baseline value is similar to the method for calculating the effective depth distance of the client based on the client baseline value, therefore the method for calculating the effective depth distance of the host will not be described in detail here.

[0078] In some embodiments, the client capability data includes the client's effective depth distance, which is transmitted from the tracking device 130 to the head-mounted display device 110.

[0079] In some embodiments, the effective depth distance is proportional to the baseline value. That is, the larger the baseline value, the larger the effective depth distance. The smaller the baseline value, the smaller the effective depth distance. The relationship between the effective depth distance and the baseline value can also be seen from the formula (2) above. That is, the larger the baseline value, the larger the effective depth distance.

[0080] For example, if the client baseline value is half of the host baseline value, the client effective depth distance is half of the host effective depth distance.

[0081] In some embodiments, the effective depth distance for the client is 5 to 6 meters, while the effective depth distance for the host is 10 to 12 meters. The effective depth distances for the client and host described above are illustrative examples, and the implementation of this invention is not limited to these descriptions.

[0082] In some embodiments, the host processor 112 generates a local map from the host map based on the effective depth distance of the client.

[0083] In some embodiments, the client capability data further includes the preset location of the tracking device 130. The preset location of the tracking device 130 is transmitted by the tracking device 130 to the head-mounted display device 110, and the local map is generated by the host-side map based on the client baseline value and the preset location.

[0084] In some embodiments, when the tracking device 130 is worn on the user's hand, the preset position of the tracking device 130 is 10 centimeters in front of the user's chest. In some embodiments, when the tracking device 130 is worn on the user's foot, the preset position of the tracking device 130 is 160 centimeters below the head-mounted display device 110.

[0085] Please refer to the following: Figure 5 . Figure 5 This is a schematic diagram of an embodiment of a local map according to some embodiments of the present invention. Figure 5 As shown, the user operates the virtual reality system 100 (including a head-mounted display device 110 and tracking devices 130A and 130B) in environment E.

[0086] like Figure 5 As shown, the preset position of tracking device 130A is preset position PC1, while the preset position of tracking device 130B is preset position PC2. The effective depth distance of the client side of tracking device 130A is effective depth distance DC1, while the effective depth distance of the client side of tracking device 130B is effective depth distance DC2. The effective depth distance of the host side of head-mounted display device 110 is effective depth distance DH.

[0087] like Figure 2 The host-side processor 112 generates a local map MC1 from the host-side map MH based on the preset position PC1 and the effective depth distance DC1 of the client, while the host-side processor 112 generates a local map MC2 from the host-side map HM based on the preset position PC2 and the effective depth distance DC2 of the client.

[0088] Specifically, within the area of ​​the host-side map MH, a local map MC1 is generated with a preset location PC1 as the center and the client's effective depth distance DC1 as the radius. Furthermore, within the area of ​​the host-side map MH, a local map MC2 is generated with a preset location PC2 as the center and the client's effective depth distance DC2 as the radius. The areas of local map MC1, local map MC2, and the area of ​​the host-side map MH constitute a 3D area (3D map).

[0089] Please refer to it again. Figure 3 In step S350, the head-mounted display device transmits the local map to the tracking device. In some embodiments, step S350 is performed as follows: Figure 2 The host processor 112 of the head-mounted display device 110 shown executes this. That is to say, Figure 2 The host processor 112 of the head-mounted display device 110 transmits a local map to the tracking device 130.

[0090] Please refer to the following: Figure 5 In some embodiments, the host processor 112 of the head-mounted display device 110 generates and transmits a local map MC1 to the tracking device 130A, while the host processor 112 of the head-mounted display device 110 generates and transmits a local map MC2 to the tracking device 130B.

[0091] In summary, the embodiments of this disclosure provide a head-mounted display device, a tracking device, and a data calibration method. A local map is generated based on client capability data from the tracking device, and the head-mounted display device transmits this local map, rather than a host-side map, to the tracking device. Thus, in the initial stage, the amount of data transmitted from the head-mounted display device to the tracking device is reduced, resulting in a better user experience.

[0092] Please refer to it again. Figure 2 In some embodiments, the host processor 112 and the client processor 132 may be, but are not limited to, a single processor or a collection of multiple microprocessors, such as a CPU or a GPU. The microprocessors are electrically coupled to memory to access at least one instruction. Based on the at least one instruction, the data calibration method described above can be executed. In some embodiments, the memory (including host memory 114 or client memory 134) may be flash memory, HDD, SSD (Solid State Drive), DRAM (Dynamic Random Access Memory), or SRAM (Static Random Access Memory). In some embodiments, host memory 114 and client memory 134 may be non-transitory computer storage media storing at least one instruction containing the data calibration method. The at least one instruction can be accessed and executed by the host processor 112 or the client processor 132.

[0093] Additionally, it should be noted that, unless otherwise specified, there is no particular order requirement for the steps in the above-described data calibration method 300. Furthermore, these steps can be performed simultaneously, or their execution times can at least partially overlap.

[0094] Unless otherwise specified, the terms used herein generally have their ordinary meaning in the context of this art, the content of this disclosure, and the specific content thereof. Certain terms used to describe this disclosure will be discussed elsewhere in this specification to provide additional guidance to those skilled in the art in describing this disclosure.

[0095] While specific embodiments of this disclosure have been disclosed with respect to the above embodiments, these embodiments are not intended to limit this disclosure. Various alternatives and modifications can be made by those skilled in the art without departing from the principles and spirit of this disclosure. Therefore, the scope of protection of this disclosure is determined by the appended claims.

Claims

1. A head-mounted display device, characterized in that, The communication connection is to a tracking device, wherein the head-mounted display includes: A host-side memory for storing a host-side map; and A host-side processor, used for: The tracking device obtains client capability information; Based on the client's capability data, a partial map is generated in the area of ​​the host-side map stored in the head-mounted display device; and The local map is transmitted to the tracking device.

2. The head-mounted display device as described in claim 1, characterized in that, The client capability data includes a client baseline value for the tracking device.

3. The head-mounted display device as described in claim 2, characterized in that, The local map is generated based on a ratio between a host baseline value and a client baseline value of the head-mounted display device.

4. The head-mounted display device as described in claim 2, characterized in that, The client's capability data also includes a preset location of the tracking device.

5. The head-mounted display device as described in claim 4, characterized in that, The local map is generated based on the client's baseline value and the preset location.

6. The head-mounted display device as claimed in claim 1, characterized in that, The client capability data includes the effective depth distance of the tracking device for one client.

7. The head-mounted display device as claimed in claim 6, characterized in that, The local map is generated based on the client's effective depth and distance.

8. A tracking device, characterized in that, The communication connection is to a head-mounted display device, wherein the tracking device includes: A client processor, used to: Transmitting client capability data to the head-mounted display device; and Receive a partial map, wherein the partial map is generated based on the client's capability data within an area of ​​a host-side map stored in the head-mounted display device; and A client-side storage device is used to store the local map.

9. The tracking device as claimed in claim 8, characterized in that, The client capability data includes a client baseline value for the tracking device.

10. A data calibration method, characterized in that, Applicable to a virtual reality system comprising a head-mounted display and a tracking device, wherein the data calibration method includes: The tracking device transmits client capability data to the head-mounted display device; The head-mounted display device generates a local map in an area of ​​a host-side map stored on the head-mounted display device based on the client's capability data; as well as The local map is transmitted from the head-mounted display to the tracking device.

Images