A virtual reality system and a method implemented therein

Abstract

In some aspects, a method implemented in a virtual reality system comprising at least one tracked device and at least one tracking device is provided. The at least one tracked device has a sensor unit configured to measure parameters relating to a position and / or pose. The method include: transmitting the parameters obtained by the sensor unit, to the at least one tracking device; tracking the at least one tracked device to obtain information relating to the position and / or pose; computing the position and / or pose based on the information and the parameters obtained by the sensor unit; transmitting the position and / or pose to a host device. The method further includes: relaying or transmitting, by the at least one tracked device, a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracking device and the at least one tracked device.

Description

A VIRTUAL REALITY SYSTEM AND A METHOD IMPLEMENTED THEREINTECHNICAL FIELD

[0001] The present disclosure generally relates to a virtual reality system, in particular, a virtual reality system comprising a tracking device employed therein and a method implemented in the virtual reality system.BACKGROUND

[0002] Virtual reality is a simulated 3D environment that enables users to explore and interact with a virtual surrounding in a way that approximates reality, as it is perceived through the users' senses. The environment is created with computer hardware and software, and users often need to wear headset devices such as helmets or goggles to interact with the environment. Tracking solutions built into these headsets with inside-out tracking system have limitations in tracking the devices around it because of physical limitations caused by occlusion between the tracking sensor(s) (e.g., cameras) built into the headset or by the distance between the tracking sensor(s) and the tracked devices.

[0003] Therefore, there exists a need for virtual reality systems to have a solution that can complement and improve the capabilities of existing inside out tracking system in capturing movement of a user, thereby augmenting user experience.SUMMARY

[0004] According to a first aspect of the present disclosure, a method implemented in a virtual reality system is provided. The virtual reality system may include at least one tracked device and at least one tracking device. The at least one tracked device may have a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device. The method may include: transmitting, by the at least one tracked device, the parameters obtained by the sensor unit relating to a position and / or pose of the at least one tracked device, to the at least one tracking device; tracking the at least one tracked device, by the at least one tracking device, to obtain information relating to the position and / or pose of the at least one tracked device; computing, by the at least one tracking device, the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit of the at least one tracked device; transmitting, by the at least one tracking device, the position and / orpose of the at least one tracked device to a host device. The method may further include: transmitting a timing synchronization signal to the at least one tracked device and relaying, by the at least one tracked device, the timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracking device and the at least one tracked device; and / or transmitting, by the at least one tracked device, a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

[0005] According to a second aspect of the present disclosure, a virtual reality system is provided. The virtual reality system may include: at least one tracked device having a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device; and at least one tracking device, wherein the at least one tracked device is configured to transmit the parameters obtained by the sensor unit relating to the position and / or pose of the at least one tracked device to the at least one tracking device; wherein the at least one tracking device is configured to track the at least one tracked device to obtain information relating to the position and / or pose of the at least one tracked device and compute the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit, and transmit the position and / or pose of the at least one tracked device to a host device; wherein the at least one tracked device is configured to transmit and / or relay a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

[0006] According to a third aspect of the present disclosure, a non-transitory computer- readable medium comprising computer executable instructions stored therein, which when executed by one or more processors, cause the virtual reality system as described herein to perform the method as described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a block diagram showing an example virtual reality system according to an embodiment of the present disclosure.

[0008] FIG. 2 is a flow chart showing a method implemented in the example virtual reality system of FIG. 1.

[0009] FIG. 3 is a diagram showing a perspective view of the example virtual reality system shown in FIG. 1.

[0010] FIG. 4 is a diagram showing an example virtual reality system according to an embodiment of the present disclosure.

[0011] FIG. 5 is a diagram showing signal transmission occurred in the example virtual reality system of FIG. 4.

[0012] FIG. 6 is a diagram showing an example virtual reality system according to an embodiment of the present disclosure.

[0013] FIG. 7 is a block diagram showing an example electronic device, according to an embodiment of the present disclosure.DETAILED DESCRIPTION

[0014] Embodiments described below in the context of a device, apparatus, or system are analogously valid for the respective methods, and vice versa. Furthermore, it will be understood that the embodiments described below may be combined, for example, a part of one embodiment may be combined with a part of another implementation, and a part of one embodiment may be combined with a part of another embodiment.

[0015] It should be understood that the terms "on", "over", "top", "bottom", "down", "side", "back", "left", "right", "front", “back”, "lateral", "side", "up", "down", “vertical”, “horizontal” etc., when used in the following description are used for convenience and to aid understanding of relative positions or directions, and not intended to limit the orientation of any device, or structure or any part of any device or structure. In addition, the singular terms "a", "an", and "the" include plural references unless context clearly indicates otherwise. Similarly, the word "or" is intended to include "and" unless the context clearly indicates otherwise.

[0016] It will be further understood that the terms “comprise” (and any form of comprise, such as “comprises” and “comprising”), “have” (and any form of have, such as “has” and “having”), “include” (and any form of include, such as “includes” and “including”), and “contain” (and any form of contain, such as “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises,” “has,” “includes” or “contains” one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more steps or elements. Likewise, a step of a method or an element of a device that “comprises,” “has,” “includes” or “contains” one or more features possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that isconfigured in a certain way is configured in at least that way, but may also be configured in ways that are not listed.

[0017] Approximating language, as used herein throughout the specification and claims, may be applied to modify any quantitative representation that could permissibly vary without resulting in a change in the basic function to which it is related. Accordingly, a value modified by a term or terms, such as “about,” “substantially”, is not limited to the precise value specified but within tolerances that are acceptable for operation of the embodiment for an application for which it is intended. In some instances, the approximating language may correspond to the precision of an instrument for measuring the value. In the context of various embodiments, and as used herein, terms like ‘first’ and ‘second’ are used to distinguish elements of the disclosure and do not necessarily indicate that the ‘first’ comes before the ‘second’. Explanations for the first tracked device may be applicable to the second or more tracked device.

[0018] In an inside-out concept, body motion may be captured by tracking sensors (e.g., cameras) in-built in head-mounted displays (HMDs), thereby capturing views of the controllers. It may use computer vision SLAM (Simultaneous Localization & Mapping) technology to compute its position. The arrangement of the in-built sensors in the HMDs may improve occlusion by providing computer vision from the HMDs.

[0019] In another concept where tracking sensors are mounted on the HMDs for tracking the VR controllers, the tracking sensors may need to be positioned at the edge of HMDs to have the best vantage point. The tracking sensors together with a mounting fixture or frame may not block the HMD’s in-built sensors that are used for HMD tracking.

[0020] Various aspects of what is described here seek to provide a virtual reality (VR) system, particularly, the VR system may include: a head-mounted display (HMD); at least one tracked device (e.g. two VR controllers for using by each hand of a user) having a sensor unit configured to measure a position and / or pose of the at least one tracked device (e.g. an inertial measurement unit (IMU), a gyroscope sensor, a six degrees of freedom (6DOF) electromagnetic motion tracking sensor, etc.); and a tracking device (e.g. sensors, tracking cameras). The at least one tracked device may be configured to transmit parameters obtained by the sensor unit relating to a position and / or pose of the at least one tracked device (i.e. IMU data) to the tracking device. The tracking device may be configured to compute the position and / or pose of the at least one tracked device based on the parameters obtained bythe sensor unit and information obtained by its own tracking sensor that tracks transducer design into the tracked devices, and transmit the position and / or pose of the at least one tracked device to the HMD (e.g. in a hostless manner where the tracking device communicates with the HMD and no host device is necessary) or to a host device that is in communication with the HMD (e.g. in a hosted manner where the HMD and the track device both communicate with the host device). In the hosted manner, the VR system may communicate with the host device and the position and / or pose of the at least one tracked device transmitted to the host device by the tracking device may supplement (e.g. complement) information transmitted to the host device by the HMD, such that a more accurate position and / or pose of the at least one tracked device may be obtained. Similarly, in the hostless manner, the position and / or pose of the at least one tracked device transmitted to the HMD by the tracking device may supplement (e.g. complement) information obtained (e.g. captured images or calculated based on the captured images) by the HMD, such that a more accurate position and / or pose of the at least one tracked device may be obtained. In the hostless manner, the HMD may perform the functions of the host device as described herein.

[0021] According to various aspects, the at least one tracked device may include a first tracked device (e.g. a right controller) and a second tracked device (e.g. a left controller). In the hosted manner, the first and second tracked devices may be configured to transmit to the tracking device over a first radio frequency channel using a plurality of time slots and transmit to the HMD over a second radio frequency channel using the plurality of time slots (i.e. the same plurality of time slots used by the first radio frequency channel). The first radio frequency channel may be different from the second radio frequency channel. The wireless communication link should not be limited to the radio frequency radiation link but include other communication link, e.g. an infra-red radiation link. Each time slot of the plurality of timeslots may be divided into multiple sub-slots. Each sub-slot may have equal length in duration.

[0022] The first tracked device may be configured to transmit parameters obtained by a sensor unit (e.g. an IMU, a gyroscope sensor, a six degrees of freedom (6DOF) electromagnetic motion tracking sensor, etc) thereof to the tracking device at the first subslot of each time slot over the first radio frequency channel; and the second tracked device may be configured to transmit a user input and / or parameters obtained by a sensor unit (e.g. an IMU, a gyroscope sensor, a six degrees of freedom (6DOF) electromagnetic motiontracking sensor, etc) thereof to the HMD at the first sub- slot of each time slot over the second radio frequency channel. The first tracked device may be further configured to transmit a user input and / or the parameters obtained by the sensor unit thereof to the HMD at the second sub-slot of each time slot over the second radio frequency channel; and the second tracked device may be further configured to transmit the parameters obtained by the sensor unit thereof to the tracking device at the second sub- slot of each time slot over the first radio frequency channel. In other words, the first tracked device may be further configured to transmit, the same parameters obtained by the sensor unit thereof that has been transmitted to the tracking device, to the HMD at the second sub- slot of each time slot over the second radio frequency channel; and the second tracked device may be further configured to transmit, the same parameters obtained by the sensor unit thereof that has been transmitted to the HMD, to the tracking device at the second sub-slot of each time slot over the second radio frequency channel. Furthermore, in addition to the parameters obtained by the sensor unit of the first tracked device / the second tracked device, the first tracked device / the second tracked device may also transmit the respective user input to the HMD, respectively. The HMD may function in response to the user input or further transmit the user input to the host device and function in response to the consequent command from the host device (e.g. in the hosted manner).

[0023] The transmission from the second tracked device to the HMD over the second radio frequency channel may concurrently occur with the transmission from the first tracked device to the tracking device over the first radio frequency channel, and the transmission by the first tracked device to the HMD over the second radio frequency channel may concurrently occur with the transmission from the second tracked device to the tracking device over the first radio frequency channel. Either of the first and second radio frequency channels may support one transmission at any sub-slot. The first / second tracked devices may alternatively use the first / second radio frequency channels.

[0024] According to some aspects, the at least one tracked device may be further configured to transmit a user input to the tracking device, and the tracking device may be configured to transmit a command and / or a timing synchronization signal to the at least tracked device so as to achieve synchronization between tracking sensor operation in the tracking device and the transducers operation in at least one tracked device. The at least tracked device may be configured not to communicate with the HMD and exclusively communicate with the tracking device. The tracking device may be configured to transmitthe position and / or pose of the at least one tracked device to the HMD in the hostless manner, or to the host device in the hosted manner. The tracked device may be configured to transmit a command and / or a timing synchronization signal to tracking device and / or HMD so as to achieve synchronization between tracking sensor operation in the tracking device and / or HMD and the transducers operation in at least one tracked device.

[0025] In some aspects of what is described here, the tracking device may include a plurality of tracking sensors, and the plurality of tracking sensors may include one master tracking sensor and multiple slave tracking sensors, the multiple slave tracking sensors communicating with the master tracking sensor. The plurality of tracking sensors may be so disposed that each of the plurality of tracking sensors is configured to capture a signal (e.g. an image) of the at least one tracked device that is different from a signal (e.g. an image) of the at least one tracked device that is captured by another tracking sensor. The tracking device may detect signals from the plurality of transducers of the tracked device(s). The plurality of transducers of the tracked device(s) may include piezo transducer, magnetic transducers, optical transducers (e.g. LED), or any type of transducers that transmit signals (e.g. frequency, light, sound waves) to be detected by the tracking devices (e.g. sensors).

[0026] For example, the plurality of sensors may surround the HMD (e.g. a place where the user is supposedly standing), (equally or non-equally) spaced apart from each other. From a top view, the plurality of sensors may be spaced apart by a certain degrees and orientated to face toward the HMD (e.g. a place where the user is supposedly standing). From a front view (e.g. from the HMD’s view), the plurality of sensors may be disposed at different heights, e.g. with one sensor higher or lower than its adjacent sensor(s). Therefore, the signals (e.g. images) captured by the plurality of sensors may complement each other and provide a more accurate detection (estimation of position) of the at least one tracked device. The multiple slave sensors may collectively communicate with the master sensor and the master sensor may consolidate the captured signals (e.g. images) by the multiple slave sensors and the master sensor. The tracking device may compute the pose / position of the at least one tracked device based on the consolidated captured signals (e.g. images).

[0027] In some instances, aspects of the systems and techniques described here provide technical improvements and advantages over existing approaches. For example, the proposed VR system may provide an improved tracking of the at least one VR controller and consequently improved tracking of the motion of the user. This may in turn provide animproved mapping of the motion of the user into the virtual reality displayed in the HMD. In an example, the proposed VR system may include a tracking device (e.g. a tracking sensor) that captures the movement of the VR controller(s) (e.g. tracked devices) external to the HMD (e.g. the user). This may provide more accurate images as the images may be taken from a farther distance than the HMD and consequently supplement or complement information obtained (e.g. captured images or calculated based on the captured images) by the HMD for determining the pose / position of the controller(s) (e.g. orientation or position). Specifically, in some embodiments, the communication between the controllers and the tracking device may be alternatively configured with the communication between the controllers and the HMD over the first and second radio frequency channels, thereby providing a smooth, fast and efficient communication, an improved mapping of the user’s motion into the virtual reality in the HMD and consequently better user experience.

[0028] The following examples pertain to various aspects of the present disclosure.

[0029] Example 1 is a method implemented in a virtual reality system including at least one tracked device, and at least one tracking device, wherein the at least one tracked device has a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device. The method may include: transmitting, by the at least one tracked device, the parameters obtained by the sensor unit relating to a position and / or pose of the at least one tracked device, to the at least one tracking device; tracking the at least one tracked device, by the at least one tracking device, to obtain information relating to the position and / or pose of the at least one tracked device; computing, by the at least one tracking device, the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit of the at least one tracked device; transmitting, by the at least one tracking device, the position and / or pose of the at least one tracked device to a host device. The method may further include: transmitting a timing synchronization signal to the at least one tracked device and relaying, by the at least one tracked device, the timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracking device and the at least one tracked device; and / or transmitting, by the at least one tracked device, a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

[0030] In Example 2, the subject matter of Example 1 may optionally include that the transmitting by the at least one tracked device to the at least one tracking device occurs over a first radio frequency channel using a plurality of time slots, wherein each time slot of the plurality of time slots is divided into at least one section in a manner that each of the at least one tracked device communicates with the at least one tracking device in a different one section of the at least one section.

[0031] In Example 3, the subject matter of Example 2 may optionally include that the at least one tracked device comprises a first tracked device and a second tracked device, and each time slot of the plurality of time slots is divided into multiple sub- slots, wherein the first tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a first sub-slot of each time slot over the first radio frequency channel and the second tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a second sub-slot of each time slot over the first radio frequency channel.

[0032] In Example 4, the subject matter of Example 3 may optionally include that the second tracked device is configured to transmit the parameters obtained by the sensor unit thereof to a head-mounted display (HMD) at the first sub- slot of each time slot over a second radio frequency channel; and the first tracked device is configured to transmit the parameters obtained by the sensor unit thereof to the HMD at the second sub- slot of each time slot over the second radio frequency channel.

[0033] In Example 5, the subject matter of Example 4 may optionally include that the first radio frequency channel is different from the second radio frequency channel.

[0034] In Example 6, the subject matter of Example 4 may optionally include that the transmitting by the second tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmitting by the first tracked device to the tracking device over the first radio frequency channel, wherein the transmitting by the first tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmitting by the second tracked device to the tracking device over the first radio frequency channel.

[0035] In Example 7, the subject matter of Example 1 may optionally include that the tracking device is configured to communicate with a head-mounted display (HMD) and transmit the position and / or pose of the at least one tracked device to the HMD in a hostless manner.

[0036] In Example 8, the subject matter of Example 1 may optionally include that the tracking device is configured to communicate with the host device in a hosted manner, and transmit the position and / or pose of the at least one tracked device to the host device.

[0037] In Example 9, the subject matter of Example 1 may optionally include that the at least one tracking device comprises a plurality of tracking devices, wherein each of the at least one tracked device communicates with each of the at least one tracking device.

[0038] In Example 10, the subject matter of Example 1 may optionally include that the at least one tracked device comprises inbuilt tracking transducers, each of the inbuilt tracking transducers being fixedly or detachably positioned on the at least one tracked device in a manner to form a transducer constellation, in order to send the information relating to the position and / or pose of the at least one tracked device to the at least one tracking device.

[0039] Example 11 is a virtual reality system including: at least one tracked device having a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device; and at least one tracking device, wherein the at least one tracked device is configured to transmit the parameters obtained by the sensor unit relating to the position and / or pose of the at least one tracked device to the at least one tracking device; wherein the at least one tracking device is configured to track the at least one tracked device to obtain information relating to the position and / or pose of the at least one tracked device and compute the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit, and transmit the position and / or pose of the at least one tracked device to a host device; wherein the at least one tracked device is configured to transmit and / or relay a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

[0040] In Example 12, the subject matter of Example 11 may optionally include that the at least one tracked device is configured to transmit to the at least one tracking device over a first radio frequency channel using a plurality of time slots, wherein each time slot of the plurality of time slots is divided into at least one section in a manner that each of the at least one tracked device communicates with the at least one tracking device in a different one section of the at least one section.

[0041] In Example 13, the subject matter of Example 12 may optionally include that the at least one tracked device comprises a first tracked device and a second tracked device, and each time slot of the plurality of time slots is divided into multiple sub- slots, wherein thefirst tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a first sub-slot of each time slot over the first radio frequency channel and the second tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a second sub-slot of each time slot over the first radio frequency channel.

[0042] In Example 14, the subject matter of Example 13 may optionally include that the second tracked device is configured to transmit the parameters obtained by the sensor unit thereof to a head-mounted display (HMD) at the first sub- slot of each time slot over a second radio frequency channel; and the first tracked device is configured to transmit the parameters obtained by the sensor unit thereof to the HMD at the second sub- slot of each time slot over the second radio frequency channel.

[0043] In Example 15, the subject matter of Example 14 may optionally include that the transmission from the second tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmission from the first tracked device to the tracking device over the first radio frequency channel, wherein the transmission by the first tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmission from the second tracked device to the tracking device over the first radio frequency channel.

[0044] In Example 16, the subject matter of Example 11 may optionally include that the tracking device is configured to communicate with a head-mounted display (HMD), and transmit the position and / or pose of the at least one tracked device to the HMD in a hostless manner.

[0045] In Example 17, the subject matter of Example 11 may optionally include that the tracking device is configured to communicate with the host device, and transmit the position and / or pose of the at least one tracked device to the host device in a hosted manner.

[0046] In Example 18, the subject matter of Example 11 may optionally include that the tracking device comprises a plurality of tracking sensors, wherein each of the at least one tracked device communicates with each of the at least one tracking device.

[0047] In Example 19, the subject matter of Example 11 may optionally include that the at least one tracked device comprises inbuilt tracking transducers, each of the inbuilt tracking transducers being fixedly or detachably positioned on the at least one tracked devicein a manner to form a transducer constellation, in order to send the information relating to the position and / or pose of the at least one tracked device to the at least one tracking device.

[0048] Example 20 is a non-transitory computer-readable medium comprising computer executable instructions stored therein, which when executed by one or more processors, cause the virtual reality system of Example 11 to perform the method of Example 1.

[0049] FIG. 1 is a block diagram showing an example virtual reality (VR) system 100 according to an embodiment of the present disclosure. FIG. 2 is a flow chart showing a method 200 implemented in the VR system 100 of FIG. 1. FIG. 3 is a diagram showing a perspective view of the example VR system 100. Now with reference to FIGS. 1 to 3, the VR system 100 is described.

[0050] According to various non-limiting embodiments, referring to FIG. 1, the VR system 100 may include a head-mounted display (HMD) 120, a tracking device 130 and at least one tracked device 140. The tracked device 140 may be a VR controller and / or body trackers that are mounted to body parts like wrists or ankles to track the body movements, or any device that can be tracked by the tracking device 130 and communicate with the tracking device as described below.

[0051] According to various embodiments, the VR system 100 may include the headmounted display (HMD) 120, the tracking device 130 and at least one tracked device 140. The head-mounted display (HMD) 120, the tracking device 130 and the at least one tracked device 140, and / or the host device 101 may be electrically and / or mechanically coupled with one another to facilitate VR activities. The HMD 120 and the tracking device 130 may be in wireless or wired communication with the VR host device 101 (shown as two-directional arrows 104, 102, respectively). The wireless communication may be performed in accordance with a wireless communication protocol, e.g. Wi-Fi, Bluetooth, radio frequency (RF) protocol, etc., to communicate via a wireless link located within the HMD 120 and / or the tracking device 130. The HMD 120 and the tracking device 130 may have circuitry (e.g., transmitter(s), receiver(s), transceiver(s)) that allow for wireless communication protocols to be employed between them. The wired communication may be performed via, e.g. a cable connection. A tracking device may be a device that have tracking sensors that can sense the six degrees of freedom (6DOF) position of tracked devices and it may need to use sensor unit (e.g. IMU) data in the tracked device, to supplement its sensor information for reliable 6DOF position computation

[0052] While FIG. 1 shows an example VR system 100 including one HMD 120, one tracked device 140, and one tracking device 130, in other embodiments any number of these components may be included in the VR system 100. For example, there may be multiple tracking device 130 that track from different positions of the tracked device 140 and / or the HMD 120. The tracking device 130 may track 6DOF of movement of the user’s body. In some embodiments, different and / or additional components may be included in the VR system 100. In some embodiments, the VR system 100 may optionally include a VR host device 101 (e.g., a personal computer, a laptop) as shown in FIG.l (i.e. a hosted manner). In some embodiments, the VR system 100 may include two tracked devices, such as a left VR controller and a right VR controller operated by the left hand and right hand of a user, respectively. Some features of the VR system 100 that are shown in FIG. 1 are not described in details for purpose of brevity and the VR system 100 may include further features not shown in FIG. 1.

[0053] The HMD 120 is a head-mounted display that presents media to a user. Examples of media presented by the HMD 120 include one or more images, video, audio, or some combination thereof. In some embodiments, audio is presented via an external device (e.g., speakers and / or headphones) that receives audio information from the HMD 120, the VR host 101, or both, and presents audio data based on the audio information. In some embodiments, the HMD 120 may include an inertial measurement unit (IMU) sensor 121, tracking cameras 123, a wireless IC 125 and a tracking CPU 127. The HMD may also include an electronic display (not shown). The electronic display may display images to the user in accordance with data received from the VR host 101 (e.g. in a hosted manner). In various embodiments, the electronic display may comprise a single electronic display or multiple electronic displays (e.g., a display for each eye of a user).

[0054] In various embodiments, images captured by the camera 123 of the HMD 120 may be communicated to the VR host 101 via the tracking CPU 127. The images may include at least a partial view of the tracked device 140. Additionally, the HMD 120 may receive one or more calibration parameters from the VR host 101 to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.) for capturing images.

[0055] The IMU 121 is an electronic device that generates IMU data indicating a position of the HMD 120 based on measurement signals received from one or more sensors.A sensor may generate one or more measurement signals in response to motion of the HMD 120. Examples of sensors may include: one or more (3-axis) accelerometers, one or more (3-axis) gyroscopes, one or more (3-axis) magnetometers, another suitable type of sensor that detects motion, a type of sensor used for error correction, or some combination thereof. The sensors may be located external to the IMU 121, internal to the IMU 121, or some combination thereof. For example, the sensors may include multiple accelerometers to measure translational motion (forward / back, up / down, left / right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, roll). In some embodiments, the IMU 121 may rapidly sample the measurement signals from various sensors and calculate the inertial movement of the HMD 120 from the sampled data. For example, the IMU 121 may integrate the measurement signals received from one or more accelerometers over time to estimate a velocity vector and integrate the velocity vector over time to determine a movement of the HMD 120. In some embodiments, the IMU 121 may generate the IMU data itself (e.g. by the tracking CPU 127) or provide the IMU data to the host device 101 to determine the IMU data. The IMU data may describe the position of the HMD 120 with respect to a reference point. While the reference point may generally be defined as a point in space; however, in practice the reference point may be defined as a point within the HMD 120 (e.g., a center of the IMU 121).

[0056] The IMU 121 may receive one or more calibration parameters from the VR host 101, which may be used to maintain tracking of the HMD 120. Based on a received calibration parameter, the IMU 121 may adjust one or more IMU parameters (e.g., sample rate). In some embodiments, certain calibration parameters may cause the IMU 121 to update the position of the reference point so it corresponds to a next calibrated position of the reference point. Updating the position of the reference point as the next calibrated position of the reference point may help reduce accumulated error associated with the determined position. The accumulated error, also referred to as drift error, may cause the position of the reference point to "drift" away from the actual position of the reference point over time. The HMD 120 may include one or more IMU sensor or any other sensor capable of detecting a pose / position of the tracked device 140. As used herein, the "pose" of the tracked device 140 may refer to its location and rotational orientation. “Position” of the tracked device 140 and the "pose" of the tracked device 140 may be used interchangeably herein.

[0057] According to various non-limiting embodiments, the tracking device 130 may include a tracking CPU 131, at least one tracking sensor 133 and a wireless IC 135. The tracking device 130 may include a plurality of sensors beside the sensor 133 (e.g. cameras) that are spaced apart from each other (as shown in FIG. 6). The tracking device 130 (e.g. each sensor of the tracking device 130) may be stationary or move as the user moves with the tracked device 140 in a manner that the distance from the tracking device 130 to the tracked device 140 remains substantially constant. Each sensor of the tracking device 130 may provide a respective view of the tracked device 140 from a (changing or constant) distance away from the user's location (the tracked device 140). The tracking device 130 may include one or more sensors or any other device capable of capturing signals (e.g. images) of the tracked device 140 and the observable areas (e.g. 133a as shown in FIG. 3 in which the VR system does not have a HMD). The observable area of a sensor may be defined by its field of view (FOV). The sensors of the tracking device 130 may be configured in a variety of ways, such as mounted on a stand, which may include one or more stationary or movable arms. The sensors of the tracking device 130 may capture one or more signals (e.g. images) of the observable areas of the tracked device 140. The sensors of the tracking device 130 may be typically placed in a direction facing the front of the tracked device 140. The sensors of the tracking device 130 may be placed at a generally known (e.g., 1-2 meters) distance from the tracked device 140.

[0058] In various embodiments, signals (e.g. images) captured by the at least sensor 133 of the tracking device 130 may be communicated to the VR host 101 via the tracking CPU 131 (e.g. in the hosted manner). The images may include at least a partial view of the tracked device 140. Additionally, the tracking device 130 may receive one or more calibration parameters from the VR host 101 to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.) for capturing images within an observable area 133a (as shown in FIG. 3). The tracking device 130 (e.g. the tracking CPU 131) may calibrate the sensor(s) 133 using one or more calibration parameters and may adjust one or more calibration parameters to reduce error in determination of the position of the tracked device 140. For example, the tracking device 130 adjusts the focus of the sensor(s) 133 to obtain a more accurate position for observed images. Moreover, if tracking of the tracked device 140 is lost (e.g., the sensor(s) 133 loses line of sight of the tracked device 140), the tracking device 130 may re-calibrate.

[0059] According to various non-limiting embodiments, the tracked device 140 may include a sensor unit 144 configured to measure a position and / or pose of the at least one tracked device 140 (e.g. an IMU, a gyroscope sensor, a six degrees of freedom (6DOF) electromagnetic motion tracking sensor, etc.), a plurality of transducers 143 (e.g. one or more) and a wireless IC 145. The tracked device 140 may include various sensors to provide control inputs to the VR host 101. The control inputs may include various buttons, control sticks, touch screen, position sensors, and so forth for control of the VR host 101 or an application of the VR host 101. The VR host 101 may perform an action within an application executing thereon in response to the input received from the tracked device 140 and provide feedback to the user that the action was performed. The provided feedback may be visual or audible feedback or haptic feedback via the tracked device 140. The tracked device 140 may include in-built sensors.

[0060] According to various embodiments, the sensor unit 144 may include an IMU 141. The sensor unit 144 will be described with respect to the IMU 141; however, it shall be appreciated the sensor unit 144 may include any sensor device that can measure a pose / position of the tracked device 140.

[0061] The IMU 141 of the tracked device 140 may capture IMU data, e.g. parameters including an acceleration and / or velocity data of the tracked device 140), and provide the IMU data to the tracking device 130 (e.g. shown as two-directional arrow 106) and optionally to the HMD 120 (e.g. shown as dashed two-directional arrow 108). The IMU data for IMU 141 of the tracked device 140 may be captured similarly to the IMU data described above with respect to IMU 121 of the HMD 120. The IMU 141 may receive one or more calibration parameters from the VR host 101. The one or more calibration parameters may be used to maintain tracking of the tracked device 140.

[0062] According to various embodiments, the plurality of transducers 143 may include piezo transducer, magnetic transducers, optical transducers (e.g. LED), or any type of transducers that transmit signals (e.g. frequency, light, sound waves) to be detected by the tracking devices (e.g. sensors 133). In some embodiments, the plurality of transducers 143 may include a set of light-emitting diodes (LEDs) disposed on the surface of the tracked device 140 that may be easily detected by the tracking device 130 and distinguished from other features captured by the tracking device 130. Each transducer may be respectively located in a specific position on the tracked device 140 relative to one or more components,such as another transducer, of the tracked device 140, in a manner to form a transducer constellation. In one embodiment, the plurality of transducers 143 may be a set of LEDs that may emit light in a portion of the spectrum that is not normally present in a room, such as infrared (IR) or ultraviolet (UV) light. In another embodiment, the plurality of transducers 143 may be a set of LEDs that may be a contrasting color of a portion of the external surface of the tracked device 140 in any suitable pattern, order, or array. The plurality of transducers 143 may be disposed on the tracked device 140 in various patterns, such that each side or orientation of the tracked device 140 may present a different orientation of the plurality of transducers 143. In this way, different views of the plurality of transducers 143 may be captured by the tracking device 130 (e.g. the sensors 133) and the cameras 123 of the HMD 120. The plurality of transducers may be fixedly or detachably positioned on, and thus coupled to, the tracked device 140 by any appropriate method. In some embodiments, the plurality of transducers 143 may be replaced by or supplemented by any identifiers that are configured to be detected by the tracking device 130 and the HMD 120.

[0063] As the user uses the tracked device 140, the user may move the tracked device 140 through space in front of the user's body, and / or rotate the tracked device 140. The movement of the tracked device 140 as the user performs these motions may be detected by the tracking cameras 123 of the HMD 120 and the at least one tracking sensor 133 of the tracking device 130, for an example, in a manner of capturing signals (e.g. images) of the tracked device 140. By virtue of the transducers being positioned on the tracked device 140 detectable to the sensors 133, cameras 123, signals (e.g. of light, of magnetic forces, of sound waves) produced by the plurality of transducers 143 that are detected by the sensor 133, cameras 123 may be used as an indication of the positions and motion of the tracked device 140.

[0064] Further, the tracking cameras 123 of the HMD 120 may capture a view of the tracked device 140 from the HMD’s view, while the at least one tracking sensor 133 of the tracking device 130 may capture a view of the tracked device 140 from the tracking device’s view. In this way, views of the tracked device 140 may be captured from multiple different perspectives, and as the tracking device 130 is typically apart from the user, the views provided of the tracked device 140 by the HMD 120 and the tracking device 130 may typically substantially differ in the angle of view to the tracked device 140, providing different perspectives on the tracked device 140's location and rotation, such as by providingopposite or orthogonal views of the tracked device 140. Motion of the tracked device 140 tracked by the sensors 133, cameras 123 may allow for corresponding virtual-reality hand motions to be shown on the HMD 120. For example, when the user makes a punching motion while playing a boxing game, movement of the plurality of transducers in a manner corresponding to a punch may be detected and used to model the user's motion for the image subject displayed on the HMD 120 in the VR system 100.

[0065] Now referring to FIG. 2, according to various non-limiting embodiments, the method 200 may include transmitting, by the at least one tracked device, parameters obtained by the sensor unit relating to a position and / or pose of the at least one tracked device, to the at least one tracking device (step 201); tracking the at least one tracked device, by the at least one tracking device, to obtain information relating to a position and / or pose of the at least one tracked device (step 203); computing, by the at least one tracking device, the position and / or pose of the at least one tracked device based on the parameters obtained by the sensor unit and information obtains from its tracking sensor by the at least one tracking device (step 205); and transmitting, by the at least one tracking device, the position and / or pose of the at least one tracked device to the HMD or a host device that is in communication with the HMD (step 207). The method 200 may further include transmitting, by the HMD, a timing synchronization signal to the at least one tracked device (step 209) and relaying, by the at least one tracked device, the timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracking device and the at least one tracked device (step 211). Alternatively to steps 209 and 211, the method 200 may include transmitting, by the tracked device, a timing synchronization signal to the HMD and the at least one tracking device so as to achieve synchronization between the at least one tracked device, the HMD and the at least one tracking device.

[0066] According to various non-limiting embodiments, the IMU 141 included in the tracked device 140 may generate parameters based at least in part on movements of the tracked device 140 and that parameters may be conveyed to the tracking device 130 and optionally to the HMD 120 in some embodiments as described herein by, for example, the wireless IC of the tracked device 140. The tracking device 130 may process (e.g., by the tracking CPU 131) the parameters it receives from the tracked device 140 and compute (e.g., with a computing module of the tracking CPU 131, e.g., circuitry, electronics) the position and / or pose of the at least tracked device 140. The tracking device 130 may use theparameters to supplement its sensor information for reliable 6D0F computation. In some embodiments, the tracking device 130 may also compute the position and / or pose of the at least tracked device 140 based on the captured signals (e.g. images) from the sensor(s) 133. The tracking device may subsequently communicate to the HMD 120 or the host device 101 that is in communication with the HMD 120, the position and / or pose of the at least tracked device 140 for mapping the corresponding positions in the VR system 100.

[0067] To compute the position and / or pose of the at least tracked device 140 based on the captured signals (e.g. images) from the sensor (s) 133, the tracking device 130 may analyze the signals (e.g. images) captured by the sensor(s) 133 to identify features of the signals (e.g. images) that correspond to the transducer constellation of the tracked device 140. For example, the tracking device 130 may identify bright points of light that correspond to the transducer constellation (e.g. LED constellation) of the tracked device 140. For example, the tracking device 130 may identify strong magnetic interaction that corresponds to the transducer constellation (e.g. magnetic transducers). The tracking device 130 may then analyze the detected features relative to known positions of the tracked device 140. The tracking device 130 may further compute the position and / or pose of the at least tracked device 140 based on the parameters it receives from the tracked device 140. The parameters may be used to determine the position and rotation of the tracked device 140 and the tracking device 130 may compute the position and / or pose of the at least tracked device 140 by referring to the position and rotation of the tracked device 140 included in the parameters and / or with the transducer constellation of the tracked device 140.

[0068] In some embodiments, if the captured image is lost or no longer includes a view of the tracked device 140, the tracking device 130 may compute the position and / or pose of the at least tracked device 140 merely based on the parameters. For example, the user may turn away from the tracking device 130, thus occluding the tracked device 140 from the tracking device 130.

[0069] In some embodiments, the VR host device 101 may receive the captured signals (e.g. images) from the HMD 120 and the position and / or pose of the at least one tracked device from the tracking device 130, and further modify the position and / or pose of the at least one tracked device 140 (e.g. in the hosted manner). In some embodiments, the VR host device 101 may receive the position and / or pose of the at least one tracked device 140 from the HMD 120 and the position and / or pose of the at least one tracked device 140 from thetracking device 130, and further modify the position and / or pose of the at least one tracked device 140 (e.g. in the hosted manner). In some embodiments, the HMD 120 may further modify the position and / or pose of the at least one tracked device 140 based on the captured signals (e.g. images) from the HMD 120 and the position and / or pose of the at least one tracked device from the tracking device 130 (e.g. in the hostless manner).

[0070] FIG. 4 is a diagram showing an example virtual reality (VR) system 400 according to an embodiment of the present disclosure. FIG. 5 is a diagram showing signal transmission occurred in the example VR system 400 of FIG. 4. According to various nonlimiting embodiments, the VR system 400 may include features of the VR system 100 as described above in connection with FIGS. 1 to 3. Specifically, the method 200 implemented in the VR system 100 may be similarly implemented in the VR system 400.

[0071] According to various non-limiting embodiments, the VR system 400 may include a head-mounted display (HMD) 420, one or more tracking device (s) 430 and one or more of tracked device(s) 440. The one or more tracked device(s) 440 may comprise a first tracked device 441 and a second tracked device 442, and nLhtracked device 443. The HMD 420 and the one or more tracking device(s) 430 may be optionally communicate in wireless or wired communication with a VR host device 401 as denoted as arrows 407 and 403 in a hosted manner, as mentioned above. Alternatively, the one or more tracking device(s) 430 may communicate in wireless or wired communication with the HMD 420 in a hostless manner. The HMD 420, the one or more tracking device(s) 430 and the one or more tracked device(s) 440 may function similarly to the HMD 120, the one or more tracking device(s) 430 and the tracked device 140 of the VR system 100 as described above. The one or more tracking device(s) 430 may have an observable area 430a and the HMD 420 may have an observable area 420a, whereby the tracked devices 440 may be covered by both the observable areas 430a, 420a such that both the one or more tracking device(s) 430 and the HMD 420 may track the movement of the tracked devices 440.

[0072] Now referring to FIG. 4, the method implemented in the VR system 400 is described. The one or more tracked device(s) 440 may initiate a connection with the HMD 420 by a connection user input (e.g. pressing any button or moving any joystick of the one or more tracked device(s) 440, or rotating the one or more tracked device(s) 440). The one or more tracked device(s) 440 may then connect to the one or more tracking device(s) 430.

[0073] The one or more tracked device(s) 440 may transmit a user input to the HMD, denoted as arrow 408 in FIG. 4. The one or more tracked device(s) 440 may receive a command and / or a timing synchronization signal from the HMD 420, denoted as arrow 406 in FIG. 4, so as to achieve synchronization between the HMD 420 and the one or more tracked device(s) 440"Command" may refer to various control command (e.g., turn on haptic, perform calibration) and get information commands (e.g., firmware version, status, etc.). “Timing synchronization" may refer to radio communication synchronization and activation time of the transducers (e.g., LEDs) in the tracked devices to have synchronization with the sensors activation in the tracking devices. The timing synchronization may be sent at every capture interval (@ Sensor Activation Frequency (Hz)), or it may be sent every few seconds to inform the one or more tracked device(s) 440 to turn on transducer for a specific duration (e.g. 500 us) at a specific interval (Sensor Activation Frequency (Hz)) which is aligned to the capture time of the sensor (e.g. the one or more tracking device(s) 430). In response to the command and / or the timing synchronization signal received from HMD 420, the one or more tracked device(s) 440 may subsequently relay the command and / or the timing synchronization signal to the one or more tracking device(s) 430, denoted as arrow 404 in FIG. 4, so as to achieve synchronization between the one or more tracking device(s) 430 and the one or more tracked device(s) 440. The operations of the transducers (e.g. 143) may need to be synchronized with the tracking sensors (e.g. 133) in the tracking devices (e.g. 430, 130). Alternatively, the tracked device(s) may send command and / or a timing synchronization signal to the HMD and the tracking device(s), denoted as arrow 408, 402 respectively, as to achieve synchronization between the tracked device(s), HMD and tracking devices. “Timing synchronization" may refer to radio communication synchronization and activation time of the transducers (e.g., LEDs) in the tracked devices to have synchronization with the sensor activation in the tracking devices. The timing synchronization may be sent at every capture interval (@ Sensor Activation Frequency (Hz)), or it may be sent every few seconds to inform the one or more tracking device(s) 430 to turn on sensor for a specific duration (e.g. 500 us) at a specific interval (Sensor Activation Frequency (Hz)) which is aligned to the LED activation time of the tracked device 440.

[0074] The one or more tracked device(s) 440 may transmit the user input and parameters obtained by the IMU relating to a position and / or pose of the one or more tracked device(s) 440 to the HMD 420, denoted as arrow 408. The one or more tracked device(s)440 may also transmit the same parameters obtained by the IMU relating to the position and / or pose of the one or more tracked device(s) 440 to the one or more tracking device(s) 430, denoted as arrow 402. The one or more tracking device(s) 430 may compute the position and / or pose of the one or more tracked device(s) 440 based on the parameters obtained by the IMU of the one or more tracked device(s) 440 and the transducer (e.g., LED constellation) of the one or more tracked device(s) 440. The one or more tracking device(s) 430 may provide the one or more tracked device(s) 440’ s pose / position parameters to the VR host 401 in the hosted manner and to the HMD 420 in the hostless manner (denoted as arrow 405), which consequently complements the tracking function of the HMD 420 in both the hostless and hosted manners.

[0075] FIG. 5 shows an example slot configuration showing interactions between the tracked devices 441, 442, 443 and the one or more tracking device(s) 430, and interactions between the tracked devices 441, 442, 443 and the HMD 420 according to various embodiments. With reference to FIG. 5, the transmitting by the tracked devices 440 to the one or more tracking device(s) 430 and the transmitting by the tracked devices 440 to the HMD 420 are described. In a timeslot (for example, Timeslot #1), the first tracked device441 may transmit parameters obtained by the IMU relating to the position and / or pose of the one or more tracked device(s) 440 to the one or more tracking device(s) 430 and transmit a user input and the same parameters obtained by the IMU relating to the position and / or pose of the one or more tracked device(s) 440 to the HMD 420; and the second tracked device 442 may transmit a user input and parameters obtained by the IMU relating to the position and / or pose of the one or more tracked device(s) 440 to the HMD 420 and transmit the same parameters obtained by the IMU relating to the position and / or pose of the one or more tracked device(s) 440 to the one or more tracking device(s) 430. The tracked devices 440 may communicate with the HMD 420 and / or the one or more tracking device(s) 430 wirelessly. Wireless communications between the HMD 420 and the tracked devices 440 may be timeslot based. The length of the timeslot (in second) may be inversely proportional to a polling rate of the VR system 400.

[0076] The transmitting by the tracked devices 440 (e.g. 441, 442) to the one or more tracking device(s) 430 may occur over a first radio frequency channel 504 using a plurality of time slots (e.g. #1, #2, # 3, etc.), and each time slot of the plurality of time slots may be divided into multiple sub-slots 501, 502, 503. The first sub-slot 501 may have a same lengthas the second sub-slot 502. The transmitting by the one or more tracked device(s) 440 to the HMD 420 may occur over a second radio frequency channel 505 using the same plurality of time slots (e.g. #1, #2, # 3, etc.).

[0077] The first tracked device 441 may be configured to transmit the parameters obtained by an IMU thereof to the one or more tracking device(s) 430 at the 1st sub-slot 501 of each time slot over the first radio frequency channel 504 and the second tracked device 442 may be configured to transmit the parameters obtained by an IMU thereof to the one or more tracking device(s) 430 at the 2ndsub-slot 502 of each time slot over the first radio frequency channel 504.

[0078] The second tracked device 442 may be configured to transmit the user input and / or the parameters obtained by the IMU thereof to the HMD 420 at the 1st sub-slot 501 of each time slot over the second radio frequency channel 505. The subsequent tracked device 443 may be configured to transmit the user input and / or the parameters obtained by the IMU thereof to the HMD 420 at the 2ndsub-slot 502 of each time slot over the second radio frequency channel 505. The first radio frequency channel 504 may be different from the second radio frequency channel 505.

[0079] The transmitting by the second tracked device 442 to the HMD 420 over the second radio frequency channel 505 may concurrently occur with the transmitting by the first tracked device 441 to the one or more tracking device(s) 430 over the first radio frequency channel 504. That is, the transmitting by the second tracked device 442 to the HMD 420 over the second radio frequency channel 505 may occur with the transmitting by the first tracked device 441 to the one or more tracking device(s) 430 over the first radio frequency channel 504 in a synchronous manner. The transmitting by the first tracked device 441 to the HMD 420 over the second radio frequency channel 505 may concurrently occur with the transmitting by the nLhtracked device 443 to the one or more tracking device(s) 430 over the first radio frequency channel 504. That is, the transmitting by the first tracked device 441 to the HMD 420 over the second radio frequency channel 505 may occur with the transmitting by the nLhtracked device 443 to the one or more tracking device(s) 430 over the first radio frequency channel 504 in a synchronous manner.

[0080] While FIG. 5 shows the first and second tracked devices 441, 442 communicate with the one or more tracking device(s) 430, it should be appreciated that two more tracked device(s) 440 may similarly communicate with the one or more tracking device(s) 430concurrently. That may mean the transmitting by the at least one tracked device to the at least one tracking device occurs over a first radio frequency channel using a plurality of time slots, wherein each time slot of the plurality of time slots is divided into at least one section in a manner that each of the at least one tracked device communicates with the at least one tracking device in a different one section of the at least one section. That may also mean the transmitting by the at least one tracked device to the at least one tracking device occurs over at least one radio frequency channel, wherein the at least one tracked device communicates with each of the at least one tracking device in a different one radio frequency channel of the at least one radio frequency channel. For example, the first and second tracked devices 441, 442 may communicate with a first tracking device of the one or more tracking device(s) 430 over a first radio frequency channel, the first and second tracked devices 441, 442 may communicate with the HMD 420 over a second radio frequency channel, and the first and second tracked devices 441, 442 may communicate with a second tracking device of the one or more tracking device(s) 430 over a third radio frequency channel.

[0081] Furthermore, each of the one or more tracking device(s) 430 may communicate with each of the one or more tracked device(s) 440 and each of the one or more tracking device(s) 430 may communicate with the host 401 / HMD 420 as described herein.

[0082] FIG. 6 is a diagram showing an example virtual reality system 700 according to an embodiment of the present disclosure. According to various non-limiting embodiments, the VR system 700 may include features of the VR system 100 as described above in connection with FIGS. 1 to 3. Specifically, the method 200 implemented in the VR system 100 may be similarly implemented in the VR system 700.

[0083] According to various non-limiting embodiments, the VR system 700 may include a head-mounted display (HMD) 720, a tracking device 730 and a set of tracked devices 740. The tracking device 730 may comprise a plurality of tracking sensors (e.g. tracking cameras or devices with computing function) forming a tracking network, and the plurality of sensors may comprise one master sensor 731 and multiple slave sensors e.g. 733 to 737, wherein the multiple slave sensors 733-737 may communicate with the master sensor 731 in a wireless manner or a wired manner. The HMD 720 and the tracking device 730 (i.e. the master sensor 731) may be optionally communicate in wireless or wired communication with a VR host device 701 as denoted as arrows 707 and 703 in a hosted manner, similarly to the VR system 400. Alternatively, the tracking device 730 (i.e. the master sensor 731)may communicate in wireless or wired communication with the HMD 720 in a hostless manner. The HMD 720, the tracking device 730 and the tracked devices 740 may function similarly to the HMD 120, the tracking device 130 and the tracked device 140 of the VR system 100 as mentioned above. The sensors 731-737 of the tracking device 730 may each have an observable area 731a-737a, and the HMD 720 may have an observable area (not shown in FIG. 6), whereby the tracked devices 740 may be covered by all the observable areas such that both the tracking device 730 and the HMD 720 may track the movement of the tracked devices 740.

[0084] Referring to FIG. 6, the method implemented in the VR system 700 is described. The tracked device 740 may transmit a user input to the tracking device in addition to the parameters obtained by the IMU relating to a position and / or pose of the at least one tracked device, denoted as arrow 702. The tracking device 730 may transmit a command and / or a timing synchronization signal to the tracked devices so as to achieve synchronization between the tracking device and the tracked device 740, denoted as arrow 704. The tracked devices may not transmit the user input and the parameters obtained by the IMU relating to the position and / or pose of the at least one tracked device to the HMD 720 in the present embodiment. Alternatively, the tracked device 740 may transmit a command and / or a timing synchronization signal to the tracking devices so as to achieve synchronization between the tracked device and the tracking devices 730, denoted as arrow 702.

[0085] The tracking device 730 (e.g. the master sensor 731) may compute the position and / or pose of the tracked device 740 based on the parameters obtained by the IMU of the one or more tracked device(s) 440 and the captured signals (e.g. images) by the sensors 731- 737. In some embodiments, the slave sensors 733-737 may capture the signals (e.g. images) of the tracked device 740 and send the signals (e.g. images) to the master tracking sensor 731, and the master sensor 731 may compute the position / pose of each tracked device 740. In some embodiments, the slave sensors 733-737 may perform independent tracked device tracking by tracking the transducer constellation of each tracked device 740. Each tracked device may have a unique transducer constellation for the tracking device(s) to identify its identity. Each of the slave sensors 733-737 may provide the tracked device(s) position to the master sensors 731. The master sensor 731 may consolidate the position from all the slave sensors 733-737 and refine the position of each tracked device 740. The tracking device 730 may provide the tracked device pose / po sition to the VR host 701 in the hostedmanner and to the HMD 720 in the hostless manner, which consequently complements the tracking function of the HMD 720 in both the hostless and hosted manners. The master tracking sensor 731 may consolidate all the user inputs and tracked devices’ pose / position and send to the HMD 720. The tracking device 730 may be able to track multiple tracked devices or accessories. Tracked devices / accessories may connect to the tracking device (e.g. the sensor(s)) directly. The tracking device 730 may control transducer timing synchronization of the tracked devices / accessories. Alternatively, tracked device 740 may operate the transducer locally and control sensor timing synchronization of the tracking device 730.

[0086] FIG. 7 is a block diagram showing an example electronic device 900, according to an embodiment of the present disclosure. The electronic device 900 may be a laptop computer, a desktop computer, a tablet computer, an automobile computer, a gaming device, a smart phone, a personal digital assistant, a server, or other electronic devices capable of running computer applications. The VR systems 100, 400 and 700 may have computing modules (e.g., circuitry, electronics), discussed below in connection with FIG. 9, for implementing the operations disclosed herein. In some embodiments, the electronic device 900 includes a processor 902, an input / output (VO) module 904, memory 906, a power unit 908, and one or more network interfaces 910. The electronic device 900 can include additional components. In some embodiments, the processor 902, input / output (VO) module 904, memory 906, power unit 908, and the network interface(s) 910 are housed together in a common housing or other assembly.

[0087] The example processor 902 can execute instructions, for example, to generate output data based on data inputs. The instructions can include programs, codes, scripts, modules, or other types of data stored in memory (e.g., memory 906). Additionally or alternatively, the instructions can be encoded as pre-programmed or re-programmable logic circuits, logic gates, or other types of hardware or firmware components or modules. The processor 902 may be, or may include, a multicore processor having a plurality of cores, and each such core may have an independent power domain and can be configured to enter and exit different operating or performance states based on workload. Additionally or alternatively, the processor 902 may be, or may include, a general-purpose microprocessor, as a specialized co-processor or another type of data processing apparatus. In some cases, the processor 902 performs high-level operation of the electronic device 900. For example,the processor 902 may be configured to execute or interpret software, scripts, programs, functions, executables, or other instructions stored in the memory 906.

[0088] The example I / O module 904 may include a mouse, keypad, touch screen, scanner, optical reader, and / or stylus (or other input device(s)) through which a user of the electronic device 900 may provide input to the electronic device 900, and may also include one or more of a speaker for providing audio output and a video display device for providing textual, audiovisual, and / or graphical output.

[0089] The example memory 906 may include computer-readable storage media, for example, a volatile memory device, a non-volatile memory device, or both. The memory 906 may include one or more read-only memory devices, random-access memory devices, buffer memory devices, or a combination of these and other types of memory devices. In some instances, one or more components of the memory can be integrated or otherwise associated with another component of the electronic device 900. The memory 906 may store instructions that are executable by the processor 902. In some examples, the memory 906 may store instructions for an operating system 912 and for application programs 914. The memory 906 may also store a database 916.

[0090] The example power unit 908 provides power to the other components of the electronic device 900. For example, the other components may operate based on electrical power provided by the power unit 908 through a voltage bus or other connection. In some embodiments, the power unit 908 includes a battery or a battery system, for example, a rechargeable battery. In some embodiments, the power unit 908 includes an adapter (e.g., an AC adapter) that receives an external power signal (from an external source) and coverts the external power signal to an internal power signal conditioned for a component of the electronic device 900. The power unit 908 may include other components or operate in another manner.

[0091] The electronic device 900 may be configured to operate in a wireless, wired, or cloud network environment (or a combination thereof). In some embodiments, the electronic device 900 can access the network using the network interface(s) 910. The network interface(s) 910 can include one or more adapters, modems, connectors, sockets, terminals, ports, slots, and the like. The wireless network that the electronic device 900 accesses may operate, for example, according to a wireless network standard or another type of wireless communication protocol. For example, the wireless network may be configured to operate as a Wireless Local Area Network (WLAN), a Personal Area Network (PAN), ametropolitan area network (MAN), or another type of wireless network. Examples of WLANs include networks configured to operate according to one or more of the 802.11 family of standards developed by IEEE (e.g., Wi-Fi networks), and others. Examples of PANs include networks that operate according to short-range communication standards (e.g., BLUETOOTH®, Near Field Communication (NFC), ZigBee), millimeter wave communications, and others. The wired network that the electronic device 900 accesses may, for example, include Ethernet, SONET, circuit- switched networks (e.g., using components such as SS7, cable, and the like), and others.

[0092] Various aspects of what is described here have provided a VR system with improved capabilities of capturing movement of a user, thereby augmenting user experience.

[0093] Some of the subject matter and operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Some of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by, or to control the operation of, data-processing apparatus. A computer storage medium can be, or can be included in, a computer-readable storage device, a computer- readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. Moreover, while a computer storage medium is not a propagated signal, a computer storage medium can be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0094] Some of the operations described in this specification can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0095] The term “data processing apparatus” encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). The apparatuscan also include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them.

[0096] A computer program (also known as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0097] Some of the processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0098] While this specification contains many details, these should not be understood as limitations on the scope of what may be claimed, but rather as descriptions of features specific to particular examples. Certain features that are described in this specification or shown in the drawings in the context of separate embodiments can also be combined. Conversely, various features that are described or shown in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

[0099] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may beadvantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single product or packaged into multiple products.

[0100] While embodiments of the invention have been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced. It will be appreciated that common numerals, used in the relevant drawings, refer to components that serve a similar or the same purpose.

Claims

CLAIMSWhat is claimed is:

1. A method implemented in a virtual reality system comprising at least one tracked device, and at least one tracking device, wherein the at least one tracked device has a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device, the method comprising: transmitting, by the at least one tracked device, the parameters obtained by the sensor unit relating to a position and / or pose of the at least one tracked device, to the at least one tracking device; tracking the at least one tracked device, by the at least one tracking device, to obtain information relating to the position and / or pose of the at least one tracked device; computing, by the at least one tracking device, the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit of the at least one tracked device; transmitting, by the at least one tracking device, the position and / or pose of the at least one tracked device to a host device; the method further comprising: transmitting a timing synchronization signal to the at least one tracked device and relaying, by the at least one tracked device, the timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracking device and the at least one tracked device; and / or transmitting, by the at least one tracked device, a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

2. The method of claim 1, wherein the transmitting by the at least one tracked device to the at least one tracking device occurs over a first radio frequency channel using a plurality of time slots, wherein each time slot of the plurality of time slots is divided into at least one section in a manner that each of the at least one tracked device communicates with the at least one tracking device in a different one section of the at least one section.

3. The method of claim 2, wherein the at least one tracked device comprises a first tracked device and a second tracked device, and each time slot of the plurality of time slots is divided into multiple sub- slots, wherein the first tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a first sub- slot of each time slot over the first radio frequency channel and the second tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a second sub-slot of each time slot over the first radio frequency channel.

4. The method of claim 3, wherein the second tracked device is configured to transmit the parameters obtained by the sensor unit thereof to a head-mounted display (HMD) at the first sub-slot of each time slot over a second radio frequency channel; and the first tracked device is configured to transmit the parameters obtained by the sensor unit thereof to the HMD at the second sub-slot of each time slot over the second radio frequency channel.

5. The method of claim 4, wherein the first radio frequency channel is different from the second radio frequency channel.

6. The method of claim 4, wherein the transmitting by the second tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmitting by the first tracked device to the tracking device over the first radio frequency channel, wherein the transmitting by the first tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmitting by the second tracked device to the tracking device over the first radio frequency channel.

7. The method of claim 1, wherein the tracking device is configured to communicate with a head-mounted display (HMD) and transmit the position and / or pose of the at least one tracked device to the HMD in a hostless manner.

8. The method of claim 1, wherein the tracking device is configured to communicate with the host device in a hosted manner, and transmit the position and / or pose of the at least one tracked device to the host device.

9. The method of claim 1, wherein the at least one tracking device comprises a plurality of tracking devices, wherein each of the at least one tracked device communicates with each of the at least one tracking device.

10. The method of claim 1, wherein the at least one tracked device comprises inbuilt tracking transducers, each of the inbuilt tracking transducers being fixedly or detachably positioned on the at least one tracked device in a manner to form a transducer constellation, in order to send the information relating to the position and / or pose of the at least one tracked device to the at least one tracking device.

11. A virtual reality system comprising: at least one tracked device having a sensor unit configured to measure parameters relating to a position and / or pose of the at least one tracked device; and at least one tracking device, wherein the at least one tracked device is configured to transmit the parameters obtained by the sensor unit relating to the position and / or pose of the at least one tracked device to the at least one tracking device; wherein the at least one tracking device is configured to track the at least one tracked device to obtain information relating to the position and / or pose of the at least one tracked device and compute the position and / or pose of the at least one tracked device based on the information obtained by the at least one tracking device and the parameters obtained by the sensor unit, and transmit the position and / or pose of the at least one tracked device to a host device; wherein the at least one tracked device is configured to transmit and / or relay a timing synchronization signal to the at least one tracking device so as to achieve synchronization between the at least one tracked device and the at least one tracking device.

12. The virtual reality system of claim 11, wherein the at least one tracked device is configured to transmit to the at least one tracking device over a first radio frequency channel using a plurality of time slots, wherein each time slot of the plurality of time slots is divided into at least one section in a manner that each of the at least one tracked devicecommunicates with the at least one tracking device in a different one section of the at least one section.

13. The virtual reality system of claim 12, wherein the at least one tracked device comprises a first tracked device and a second tracked device, and each time slot of the plurality of time slots is divided into multiple sub-slots, wherein the first tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a first sub-slot of each time slot over the first radio frequency channel and the second tracked device is configured to transmit parameters obtained by a sensor unit thereof to the at least one tracking device at a second sub-slot of each time slot over the first radio frequency channel.

14. The virtual reality system of claim 13, wherein the second tracked device is configured to transmit the parameters obtained by the sensor unit thereof to a headmounted display (HMD) at the first sub- slot of each time slot over a second radio frequency channel; and the first tracked device is configured to transmit the parameters obtained by the sensor unit thereof to the HMD at the second sub-slot of each time slot over the second radio frequency channel.

15. The virtual reality system of claim 14, wherein the transmission from the second tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmission from the first tracked device to the tracking device over the first radio frequency channel, wherein the transmission by the first tracked device to the HMD over the second radio frequency channel concurrently occurs with the transmission from the second tracked device to the tracking device over the first radio frequency channel.

16. The virtual reality system of claim 11, wherein the tracking device is configured to communicate with a head-mounted display (HMD), and transmit the position and / or pose of the at least one tracked device to the HMD in a hostless manner.

17. The virtual reality system of claim 11, wherein the tracking device is configured to communicate with the host device, and transmit the position and / or pose of the at least one tracked device to the host device in a hosted manner.

18. The virtual reality system of claim 11, wherein the tracking device comprises a plurality of tracking sensors, wherein each of the at least one tracked device communicates with each of the at least one tracking device.

19. The virtual reality system of claim 11, wherein the at least one tracked device comprises inbuilt tracking transducers, each of the inbuilt tracking transducers being fixedly or detachably positioned on the at least one tracked device in a manner to form a transducer constellation, in order to send the information relating to the position and / or pose of the at least one tracked device to the at least one tracking device.

20. A non-transitory computer-readable medium comprising computer executable instructions stored therein, which when executed by one or more processors, cause the virtual reality system of claim 11 to perform the method of claim 1.

Images