Method and system for resolving hemispherical ambiguity in six-degree-of-freedom attitude measurement

Abstract

Techniques for resolving hemispheric ambiguity are disclosed. One or more magnetic fields are emitted at a handheld controller. The one or more magnetic fields are detected by one or more sensors positioned relative to a headset. Motion data corresponding to the handheld controller or the headset is detected. During a first time interval, a first position and a first orientation of the handheld controller within a first hemisphere are determined based on the detected one or more magnetic fields, and a first difference is calculated based on the first position, the first orientation, and the motion data. During a second time interval, a second position and a second orientation of the handheld controller within a second hemisphere are determined based on the detected one or more magnetic fields, and a second difference is calculated based on the second position, the second orientation, and the motion data.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 62 / 797,776, filed on January 28, 2019, entitled “METHOD AND SYSTEM FOR RESOLVING HEMISPHERE AMBIGUITY IN SIX DEGREE OF FREEDOM POSE MEASUREMENTS”. Background Art

[0003] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images, or portions thereof, are presented to a user in a manner that looks or can be perceived as real. Virtual reality or "VR" scenarios generally involve presenting digital or virtual image information opaque to other actual real-world visual input; augmented reality or "AR" scenarios generally involve presenting digital or virtual image information as an enhancement to the visualization of the actual world around the user.

[0004] Despite advances in these display technologies, there is a need in the art for improved methods, systems, and apparatus related to augmented reality systems, and particularly display systems. Summary of the invention

[0005] The present disclosure generally relates to technologies for improving the performance and user experience of optical systems. More particularly, embodiments of the present disclosure provide methods for operating an augmented reality (AR) or virtual reality (VR) device in which a handheld controller is used to assist in the operation of the device. The overview of the present disclosure is described with reference to the examples given below. As used below, any reference to a series of examples will be understood as a reference to each of those examples separately (e.g., "Examples 1-4" will be understood as "Examples 1, 2, 3, or 4").

[0006] Example 1 is a method of resolving hemispheric ambiguity at a system including one or more sensors, the method comprising: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields by one or more sensors positioned within an earphone or a belt pack of the system; running a first processing stack during a first time interval, wherein running the first processing stack comprises: determining a first position and a first orientation of the handheld controller within a first hemisphere with respect to the earphone based on the one or more magnetic fields; and calculating a first difference based on the first position, the first orientation, and motion data corresponding to the handheld controller or the earphone; running a second processing stack during a second time interval, wherein running the second processing stack comprises: determining a second position and a second orientation of the handheld controller within a second hemisphere with respect to the earphone based on the one or more magnetic fields, wherein the second hemisphere is diametrically opposite to the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the motion data; and terminating the first processing stack or the second processing stack based on one or both of the first difference and the second difference.

[0007] Example 2 is a method according to Example 1, further comprising: detecting the motion data by an inertial measurement unit (IMU) positioned within the handheld controller.

[0008] Example 3 is a method according to Example 1, further comprising: detecting the motion data by an inertial measurement unit (IMU) positioned within the handheld headset.

[0009] Example 4 is a method according to Example 2 or 3, wherein the motion data is detected during one or both of the first time interval and the second time interval.

[0010] Example 5 is a method according to Example 2 or 3, wherein the motion data is detected before both the first time interval and the second time interval.

[0011] Example 6 is a method according to Example 1, wherein the first time interval is concurrent with the second time interval.

[0012] Example 7 is a method according to Example 1, wherein the first time interval is simultaneous with the second time interval.

[0013] Example 8 is a method according to Example 1, wherein the first time interval has a first start time and the second time interval has a second start time, and wherein the first start time and the second start time are simultaneous or separated by less than a threshold.

[0014] Example 9 is a method according to Example 1, further comprising: comparing the first difference with a threshold; determining that the first difference exceeds the threshold; and in response to determining that the first difference exceeds the threshold: terminating the first processing stack; and allowing the second processing stack to continue running.

[0015] Example 10 is a method according to Example 1, further comprising: comparing the second difference with a threshold; determining that the second difference exceeds the threshold; and in response to determining that the second difference exceeds the threshold: terminating the second processing stack; and allowing the first processing stack to continue running.

[0016] Example 11 is a method according to Example 1, further comprising: comparing the first difference with the second difference; determining that the first difference exceeds the second difference; and in response to determining that the first difference exceeds the second difference: terminating the first processing stack; and allowing the second processing stack to continue running.

[0017] Example 12 is a method according to Example 1, further comprising: comparing the first difference with the second difference; determining that the second difference exceeds the first difference; and in response to determining that the second difference exceeds the first difference: terminating the second processing stack; and allowing the first processing stack to continue running.

[0018] Example 13 is a method according to Example 1, wherein the first hemisphere is about a front hemisphere of the headset, and the second hemisphere is about a back hemisphere of the headset.

[0019] Example 14 is a method according to Example 1, further comprising: delivering virtual content to the user based on one of: the first position and the first orientation; or the second position and the second orientation.

[0020] Example 15 is a method according to Example 1, wherein the system is an optical device.

[0021] Example 16 is a system comprising: a handheld controller comprising a magnetic field transmitter configured to emit one or more magnetic fields; a headset or a belt pack comprising one or more magnetic field sensors configured to detect the one or more magnetic fields; a processor configured to perform operations comprising: running a first processing stack during a first time interval, wherein running the first processing stack comprises: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the one or more magnetic fields; and calculating a first difference based on the first position, the first orientation, and motion data corresponding to the handheld controller or the headset; running a second processing stack during a second time interval, wherein running the second processing stack comprises: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the one or more magnetic fields, wherein the second hemisphere is diametrically opposite to the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the motion data; and terminating the first processing stack or the second processing stack based on one or both of the first difference and the second difference.

[0022] Example 17 is a system according to Example 16, further comprising: detecting the motion data by an inertial measurement unit (IMU) positioned within the handheld controller.

[0023] Example 18 is a system according to Example 16, wherein the operation also includes: detecting the motion data by an inertial measurement unit (IMU) positioned within the headset.

[0024] Example 19 is a system according to Example 17 or 18, wherein the motion data is detected during one or both of the first time interval and the second time interval.

[0025] Example 20 is a system according to Example 17 or 18, wherein the motion data is detected before both the first time interval and the second time interval.

[0026] Example 21 is a system according to Example 16, wherein the first time interval is concurrent with the second time interval.

[0027] Example 22 is a system according to Example 16, wherein the first time interval is simultaneous with the second time interval.

[0028] Example 23 is a system according to Example 16, wherein the first time interval has a first start time and the second time interval has a second start time, and wherein the first start time and the second start time are simultaneous or separated by less than a threshold.

[0029] Example 24 is a system according to Example 16, wherein the operation also includes: comparing the first difference with a threshold; determining that the first difference exceeds the threshold; and in response to determining that the first difference exceeds the threshold: terminating the first processing stack; and allowing the second processing stack to continue running.

[0030] Example 25 is a system according to Example 16, wherein the operation also includes: comparing the second difference with a threshold; determining that the second difference exceeds the threshold; and in response to determining that the second difference exceeds the threshold: terminating the second processing stack; and allowing the first processing stack to continue running.

[0031] Example 26 is a system according to Example 16, wherein the operation also includes: comparing the first difference with the second difference; determining that the first difference exceeds the second difference; and in response to determining that the first difference exceeds the second difference: terminating the first processing stack; and allowing the second processing stack to continue running.

[0032] Example 27 is a system according to Example 16, wherein the operation also includes: comparing the first difference with the second difference; determining that the second difference exceeds the first difference; and in response to determining that the second difference exceeds the first difference: terminating the second processing stack; and allowing the first processing stack to continue running.

[0033] Example 28 is a system according to Example 16, wherein the first hemisphere is about a front hemisphere of the headset and the second hemisphere is about a back hemisphere of the headset.

[0034] Example 29 is a system according to Example 16, wherein the operation further includes: delivering virtual content to the user based on one of the following: the first position and the first orientation; or the second position and the second orientation.

[0035] Example 30 is a system according to Example 16, wherein the system is an optical device.

[0036] Example 31 is a method for resolving hemispheric ambiguity, the method comprising: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields by one or more sensors positioned relative to a headset; detecting motion data corresponding to the handheld controller or the headset; during a first time interval: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; and calculating a first difference based on the first position, the first orientation, and the motion data; during a second time interval: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposite to the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the motion data.

[0037] Example 32 is a system comprising: a handheld controller comprising a magnetic field transmitter configured to emit one or more magnetic fields; an earphone comprising one or more magnetic field sensors configured to detect the one or more magnetic fields; a motion sensor configured to detect motion data corresponding to the handheld controller or the earphone; and one or more processors configured to perform operations comprising: during a first time interval: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the earphone based on the detected one or more magnetic fields; and calculating a first difference based on the first position, the first orientation, and the motion data; during a second time interval: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the earphone based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposite to the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the motion data.

[0038] Example 33 is a non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields by one or more sensors positioned relative to the headset; detecting motion data corresponding to the handheld controller or the headset; during a first time interval: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; and calculating a first difference based on the first position, the first orientation, and the motion data; during a second time interval: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposite to the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the motion data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated into and form a part of this specification, are included to provide a further understanding of the present disclosure, illustrate embodiments of the present disclosure, and together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show the structural details of the present disclosure in more detail than is necessary for a basic understanding of the present disclosure or in various ways in which it can be practiced.

[0040] Figure 1 An augmented reality (AR) scene as viewed through a wearable AR device is shown in accordance with some embodiments.

[0041] Figure 2 Various possible components of an AR system are shown.

[0042] Figure 3 An example system diagram of an electromagnetic tracking system is shown.

[0043] Figure 4 An example is shown of how an electromagnetic tracking system may be merged with an AR system.

[0044] Figure 5 Illustrated is the hemispherical ambiguity problem that can exist in an electromagnetic tracking system.

[0045] Figure 6 A method for resolving hemispherical ambiguity at a system or device including one or more sensors is shown.

[0046] Figure 7 A method for resolving hemispherical ambiguity at a system or device including one or more sensors is shown.

[0047] Figure 8Shown are results of simulating a desired totem pose when the head moves and the totem is stationary.

[0048] Fig. 9 Experimental data is shown when the head is moving and the totem is stationary, where the totem is initialized in the correct hemisphere.

[0049] Fig.10 Experimental totem motion data is shown when the totem is stationary, where the totem is initialized in the correct hemisphere.

[0050] Fig.11 Experimental data is shown when the head is moving and the totem is stationary, where the totem is initialized in the wrong hemisphere.

[0051] Fig.12 Shows no Fig.11 The displayed posture data corresponds to the experimental totem motion data of the totem motion.

[0052] Fig.13 A computer system according to some embodiments described herein is shown. DETAILED DESCRIPTION

[0053] A typical head-mounted augmented reality (AR) display is at least loosely coupled to the user's head and therefore moves when the user's head moves. If the user's head movement is detected by the display system, the displayed data can be updated to take into account the change in head posture. As an example, if a user wearing a head-mounted display views a virtual representation of a three-dimensional (3D) object on the display and walks around the area where the 3D object appears, the 3D object can be re-rendered for each viewpoint, providing the user with the perception that they are walking around objects occupying real space. If the head-mounted display is used to present multiple objects within a virtual space (e.g., a rich virtual world), measurements of head posture (e.g., the position and orientation of the user's head) can be used to re-render the scene to match the user's dynamically changing head position and orientation and provide increased immersion in the virtual space.

[0054] Thus, detection or calculation of head pose can facilitate the display system rendering virtual objects so that they appear to occupy space in the real world in a manner that makes sense to the user. In addition, detection of the position and / or orientation of real objects, such as handheld devices or controllers (which may also be referred to as "totems"), haptic devices, or other real physical objects, relative to the user's head or AR system can also facilitate the display system presenting display information to the user so that the user can effectively interact with certain aspects of the AR system. At least for AR applications, the placement of virtual objects in spatial relationship to physical objects (e.g., presented to appear to spatially approximate two-dimensional or three-dimensional physical objects) can be an important issue.

[0055] For example, head motion can significantly complicate the placement of virtual objects in the view of the surrounding environment. This is true whether the view is captured as an image of the surrounding environment and then projected or displayed to the end user, or whether the end user directly perceives the view of the surrounding environment. For example, head motion will likely cause the end user's field of view to change, which will likely require updating the positions where various virtual objects are displayed in the end user's field of view.

[0056] In addition, head motion can occur within a variety of ranges and speeds. The head motion speed can change not only between different head motions, but also within the range of a single head motion or across the range of a single head motion. For example, the head motion speed can initially increase (e.g., linearly or nonlinearly) from a starting point, and can decrease when an end point is reached, with a maximum speed being obtained somewhere between the starting point and the end point of the head motion. Rapid head motion can even exceed the ability of a specific display or projection technology to render an image that appears uniform and / or as smooth motion to an end user.

[0057] Head tracking accuracy and latency (i.e., the time that elapses between when a user moves his or her head and when the image is updated and displayed to the user) has been a challenge for virtual reality (VR) and AR systems. Particularly for display systems that fill a significant portion of a user's field of view with virtual elements, it may be important that the accuracy of head tracking is high and that the overall system latency from the first detection of head motion to the update of light delivered by the display to the user's visual system is very low. If the latency is high, the system may produce a mismatch between the user's vestibular and visual sensory systems and produce a user-perceived scene that may result in motion sickness or simulator sickness. If the system latency is high, the apparent location of virtual objects will appear unstable during rapid head movements.

[0058] In addition to head mounted display systems, other display systems can benefit from accurate and low latency head pose detection. These include head tracking display systems where the display is not worn on the user's body but is mounted, for example, on a wall or other surface. The head tracking display acts as a window onto the scene, and as the user moves his head relative to the "window", the scene is re-rendered to match the user's changing viewpoint. Other systems include head mounted projection systems, where the head mounted display projects light onto the real world.

[0059] In addition, in order to provide a realistic augmented reality experience, the AR system can be designed to interact with the user. For example, multiple users can play ball games with virtual balls and / or other virtual objects. A user can "catch" a virtual ball and throw it back to another user. In another embodiment, a first user can be provided with a totem (e.g., a real racket communicatively coupled to the AR system) to hit the virtual ball. In other embodiments, a virtual user interface can be presented to the AR user to allow the user to select one of many options. The user can use a totem, a tactile device, a wearable component, or simply contact a virtual screen to interact with the system.

[0060] Detecting the user's head pose and orientation and detecting the physical position in the real object space enables the AR system to display virtual content in an effective and pleasing manner. However, although these capabilities are critical to AR systems, they are difficult to implement. For example, an AR system may need to identify the physical position of a real object (e.g., a user's head, a totem, a haptic device, a wearable component, a user's hand, etc.) and correlate the physical coordinates of the real object with the virtual coordinates corresponding to one or more virtual objects displayed to the user. This may require highly accurate sensors and sensor recognition systems that track the position and orientation of one or more objects at a fast rate.

[0061] Current methods do not perform positioning at a satisfactory speed and precision standard. Therefore, a better positioning system is needed in the context of AR and VR devices. In the following description, various embodiments will be described. For the purpose of explanation, specific configurations and details are set forth to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments can be practiced without the specific details. In addition, well-known features may be omitted or simplified so as not to make the described embodiments unclear.

[0062] Figure 1 An AR scene as viewed through a wearable AR device is shown according to some embodiments described herein. An AR scene 100 is depicted in which a user of the AR technology sees a real-world park-like setting 106 featuring people, trees, buildings in the background, and a concrete platform 120. In addition to these items, the user of the AR technology also perceives that he "sees" a robotic statue 110 standing on the real-world platform 120, and a cartoon-like avatar character 102 flying by, which appears to be an avatar of a bumblebee, even though these elements (character 102 and statue 110) do not exist in the real world. Due to the extreme complexity of human visual perception and the nervous system, it is challenging to produce VR or AR technology that facilitates comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0063] Figure 2Various possible components of an AR system are shown. In the illustrated embodiment, an AR system user 260 is depicted wearing a head-mounted component 258 characterized by a frame 264 structure coupled to a display system 262 positioned in front of the user's eyes. A speaker 266 is coupled to the frame 264 in the depicted configuration and is positioned near the user's ear canal (in one embodiment, another speaker not shown is positioned near the user's other ear canal to provide stereo / shapeable sound control). The display 262 is operably coupled (as indicated by 268) to a local processing and data module 270, such as by a wired conductor or wireless connection, which can be mounted in various configurations, such as fixedly attached to the frame 264, fixedly attached to a helmet or hat, removably attached to the torso of the user 260 in a backpack-type configuration, or removably attached to the hip of the user 260 in a belt-coupled configuration.

[0064] The local processing and data module 270 may include an energy-efficient processor or controller, and digital memory, such as flash memory, both of which may be used to assist in the processing, caching, and storage of data, including 1) data captured from sensors operably coupled to the frame 264, such as an image capture device (such as a camera), a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a radio, and / or a gyroscope; and / or (2) data obtained and / or processed using a remote processing module 272 and / or a remote data repository 274, possibly for transmission to the display 262 after such processing or retrieval.

[0065] The local processing and data module 270 may be operably coupled (as indicated by 276, 278) to a remote processing module 272 and a remote data repository 274, such as via one or more wired or wireless communication links, so that these remote modules 272, 274 are operably coupled to each other and can be used as resources for the local processing and data module 270. In one embodiment, the remote processing module 272 may include one or more relatively powerful processors or controllers configured to analyze and process data and / or image information. In one embodiment, the remote data repository 274 may include a relatively large-scale digital data storage facility that may be obtained through the Internet or other network configurations in a "cloud" resource configuration. In one embodiment, all data is stored and all calculations are performed in the local processing and data module, which allows for fully autonomous use from any remote module.

[0066] One method of achieving high-precision positioning can involve the use of electromagnetic fields coupled with electromagnetic sensors strategically placed on a user's AR headset, belt pack, and / or other auxiliary devices (e.g., totems, haptic devices, gaming devices, etc.). An electromagnetic tracking system typically includes at least one electromagnetic field transmitter and at least one electromagnetic field sensor. The sensor can measure an electromagnetic field with a known distribution. Based on these measurements, the position and orientation of the field sensor relative to the transmitter is determined.

[0067] Figure 3 An example system diagram of an electromagnetic tracking system is shown, which may have components similar to those developed by organizations such as Polhemus (RTM) of Colchester, Vermont, manufactured by Sixense (RTM) Entertainment, Inc. of Los Gatos, California, the Biosense (RTM) division of Johnson & Johnson, and other tracking companies. In one or more embodiments, the electromagnetic tracking system includes an electromagnetic field transmitter 302 configured to emit a known magnetic field. Figure 3 As shown, the electromagnetic field transmitter 302 may be coupled to a power source 310 (eg, an electric current, a battery, etc.) to provide power to the electromagnetic field transmitter 302 .

[0068] In one or more embodiments, the electromagnetic field transmitter 302 includes multiple coils that generate magnetic fields (e.g., at least three coils positioned perpendicular to each other to generate fields in the X, Y, and Z directions). These electromagnetic fields are used to establish a coordinate space that allows the system to map the position of the sensor relative to a known magnetic field and helps determine the position and / or orientation of the sensor. In one or more embodiments, the electromagnetic sensors 304A, 304B, etc. can be attached to one or more real objects. The electromagnetic sensor 304 can include a smaller coil that can induce an electric current through the emitted electromagnetic field.

[0069] Typically, components of the electromagnetic field sensor 304 may include small coils or loops, such as a collection of three differently oriented (i.e., such as orthogonally oriented relative to each other) coils coupled together within a small structure, such as a cube or other container, that are positioned / oriented to capture input magnetic flux from the magnetic field emitted by the electromagnetic field transmitter 302, and by comparing the currents induced through these coils, and knowing the relative positioning and orientation of the coils relative to each other, the relative position and orientation of the sensor relative to the transmitter can be calculated.

[0070] If you will refer to Figure 4As further described, one or more motion sensors such as an inertial measurement unit (IMU) may be operably coupled to each of the electromagnetic field transmitter 302 and the electromagnetic field sensor 304 to detect the position and orientation of each component relative to each other and / or relative to the coordinate system. In one or more embodiments, multiple sensors (possibly including an IMU) may be used to detect the position and orientation of each component with respect to the electromagnetic field transmitter 302 and the electromagnetic field sensor 304. In some instances, the electromagnetic tracking system may provide a position in three directions (i.e., X, Y, and Z directions) and further in two or three orientation angles. In some embodiments, the measurements of one or more IMUs may be compared with the measurements of the coils to determine the position and orientation of the sensor. In one or more embodiments, both electromagnetic (EM) data and motion data, along with various other data sources such as cameras, depth sensors, and other sensors, may be combined to determine the position and orientation. This information may be sent (e.g., wireless communication, Bluetooth, etc.) to the processing unit 306. In some embodiments, the posture (or position and orientation) may be reported at a relatively high refresh rate in a conventional system.

[0071] Typically, the electromagnetic transmitter is coupled to a relatively stable and large object, such as a table, counter, wall or ceiling, and one or more sensors are coupled to a smaller object, such as a medical device, a handheld gaming component, etc. Alternatively, as described below with reference to Figure 4 As described, the various features of the electromagnetic tracking system can be used to produce a configuration that can track changes or differences in position and / or orientation between two objects moving in space relative to a more stable global coordinate system; in other words, Figure 4 A configuration is shown in which variations of an electromagnetic tracking system can be used to track position and orientation differences between a head-mounted component and a handheld component, while the head pose relative to a global coordinate system (say, local to the user's room environment) is otherwise determined, such as by using a simultaneous localization and mapping (SLAM) technique using an outward-facing capture camera that can be coupled to the head-mounted component of the system.

[0072] The processing unit 306 can control the electromagnetic field transmitter 420 and can also capture data from the various electromagnetic field sensors 304. It should be understood that the various components of the system can be coupled to each other by any electromechanical or wireless / Bluetooth means. The processing unit 306 can also include data about known magnetic fields, as well as a coordinate space relative to the magnetic field. This information is then used to detect the position and orientation of the sensor relative to the coordinate space corresponding to the known electromagnetic field. The processing unit 306 can also be coupled to a threshold module 312 (as shown in the illustrated embodiment), or alternatively or additionally, include the threshold module 312 as a subcomponent. In some embodiments, the threshold module 312 can be configured to dynamically adjust a threshold value, and the difference value calculated by the processing unit 306 can be compared with the threshold value for determining the posture of the handheld controller, as described in more detail below.

[0073] One advantage of an electromagnetic tracking system is that it produces highly accurate tracking results with minimal latency and high resolution. In addition, the electromagnetic tracking system does not have to rely on optical trackers, and sensors / objects that are not within the user's line of sight can be easily tracked. It should be understood that the intensity v of the electromagnetic field decreases as a cubic function of the distance r from the coil transmitter (e.g., electromagnetic field transmitter 302). Therefore, the processing unit 306 can be configured to perform certain functions, such as an algorithm that predicts the distance based on the measured intensity, to determine the position and orientation of the sensor / object at different distances away from the electromagnetic field transmitter 302.

[0074] Given the rapid drop in the strength of the electromagnetic field as one moves farther from the electromagnetic transmitter, the best results in terms of accuracy, efficiency, and low latency can be achieved at closer distances. In a typical electromagnetic tracking system, the electromagnetic field transmitter is powered by an electric current (e.g., plugged into a power source) and has sensors located within a 20 ft radius from the electromagnetic field transmitter. A shorter radius between the sensor and the field transmitter may be more desirable in many applications, including AR applications.

[0075] Figure 4An example of how an electromagnetic tracking system may be incorporated into an AR system is shown, wherein an electromagnetic field transmitter 402 is included as part of a handheld controller 406. In one or more embodiments, the handheld controller may be a totem to be used in a gaming scene. In other embodiments, the handheld controller may be a haptic device. However, in yet other embodiments, the electromagnetic field transmitter may simply be included as part of a belt pack 470. The handheld controller 406 may include a battery 410 or other power source to power the electromagnetic field transmitter 402. It should be understood that the electromagnetic field transmitter 402 may also include or be coupled to an IMU 450 or other motion sensor that is configured to assist in determining the location and / or orientation of the electromagnetic field transmitter 402 relative to other components. This may be particularly important in situations where both the field transmitter 402 and the sensor 404 are moving. As Figure 4 As shown in the embodiment of FIG. 4 , placing the electromagnetic field transmitter 402 in the handheld controller rather than in the belt pack ensures that the electromagnetic field transmitter does not compete for resources at the belt pack, but instead uses its own battery source at the handheld controller 406 .

[0076] In some embodiments, electromagnetic sensors 404 can be positioned relative to AR headset 458, such as disposed at one or more locations on AR headset 458 and / or at one or more locations on belt pack 470. Sensors disposed on AR headset 458 can be disposed along with other sensing devices, such as one or more IMUs or additional magnetic flux capture coils 408. For example, Figure 4 As shown, sensors 404, 408 can be placed on either side of the headset 458. Since these sensors are designed to be quite small (and therefore, in some cases, can be less sensitive), having multiple sensors can improve efficiency and accuracy. In one or more embodiments, one or more sensors can also be arranged on the belt bag 470 or any other part of the user's body. Sensors 404, 408 can be wirelessly or via Bluetooth with a computing device that determines the posture and orientation of the sensor (and the AR headset to which it is attached). In one or more embodiments, the computing device can reside at the belt bag 470. In other embodiments, the computing device can reside at the headset itself or even at the handheld controller 406. In one or more embodiments, the computing device can further include a mapping database 430 (e.g., a traversable world model, a coordinate space, etc.) to detect postures, determine the coordinates of real objects and virtual objects, and can even connect to cloud resources and traversable world models.

[0077] In many instances, conventional electromagnetic transmitters may be too bulky for AR devices. Therefore, using a smaller coil than a conventional system, the electromagnetic field transmitter can be designed to be compact. However, given that the strength of the electromagnetic field decreases as a cubic function of the distance away from the field transmitter, a shorter radius (e.g., about 3-3.5ft) between the electromagnetic sensor 404 and the electromagnetic field transmitter 402 can reduce power consumption when compared to conventional systems. In one or more embodiments, this aspect can be used to extend the life of a battery 410 that can power a handheld controller 406 and an electromagnetic field transmitter 402. Alternatively, in other embodiments, this aspect can be used to reduce the size of the coil that generates the magnetic field at the electromagnetic field transmitter 402. However, in order to obtain a magnetic field of the same strength, increased power may be required. This allows a compact electromagnetic field transmitter unit 402 that can be compactly adapted to the handheld controller 406.

[0078] A number of other changes may be made when using an electromagnetic tracking system for an AR device, which may require a more efficient attitude reporting rate than other applications. For example, motion-based or IMU-based attitude tracking may be used. In many cases, the increased stability of the IMU may result in increased efficiency of the attitude detection process. The IMU may be designed so that it remains stable for up to 50-100 milliseconds. It should be understood that some embodiments may utilize an external attitude estimator module that may enable attitude updates to be reported at a rate of 10-20 Hz (i.e., the IMU may drift over time). By keeping the IMU stable at a reasonable rate, the rate of attitude updates can be reduced to 10-20 Hz (compared to higher frequencies in conventional systems).

[0079] If the electromagnetic tracking system can run at a 10% duty cycle (e.g., only checking for ground truth every 100 milliseconds), this would be an additional way to save power at the AR system. This would mean that the electromagnetic tracking system wakes up every 10 milliseconds of every 100 milliseconds to generate a pose estimate. This translates directly into power savings, which can in turn affect the size, battery life, and cost of the AR device. In one or more embodiments, this reduction in duty cycle can be strategically utilized by providing two handheld controllers (not shown) instead of just one handheld controller. For example, a user can play a game that requires two totems, etc. Alternatively, in a multi-user game, two users can have their own totem / handheld controller to play the game. When two controllers (e.g., symmetrical controllers for each hand) are used instead of one controller, the controller can operate at an offset duty cycle. For example, the same concept can also apply to controllers utilized by two different users playing a multiplayer game.

[0080] Figure 5The hemisphere ambiguity problem that can exist in an electromagnetic tracking system such as the one described herein is illustrated. For six degrees of freedom (DOF) tracking, a handheld controller 502 (labeled "TX"), which may also be referred to as a handheld device or totem, may generate EM signals modulated at three separate frequencies, one frequency for each axis X, Y, and Z. A wearable device 504, which may be implemented as an AR headset, has an EM receiving component that receives the EM signals at the X, Y, and Z frequencies. The position and orientation (i.e., pose) of the handheld controller may be derived based on the characteristics of the received EM signals. However, due to the symmetric nature of the EM signals, it may be impossible to determine in which hemisphere (e.g., the front hemisphere 506A or the rear hemisphere 506B) the handheld controller is without an additional reference frame. That is, the same EM value may be obtained at the wearable device for two diametrically opposed totem poses (one in each hemisphere), where the selected plane passes through the center of the sphere dividing the two hemispheres. This is in Figure 5 It is shown in Figure 5 A single screenshot of the EM signal received by the wearable device 504 is shown, either pose may be valid. However, as the handheld controller 502 moves, the tracking algorithm will typically encounter errors if the wrong hemisphere is selected due to inconsistencies in the data from the various sensors. Hemisphere ambiguity arises in part due to the fact that at the beginning of a six-DOF tracking session, the initial EM totem data does not have a clear absolute position. Instead, it provides a relative distance that can be interpreted as one of two positions in a 3D volume that is divided into two equal spheres using a wearable device centered between the two hemispheres (e.g., an AR headset mounted on the user's head). Therefore, embodiments of the present disclosure provide methods and systems that resolve hemisphere ambiguity so that the actual position of a handheld controller can be successfully tracked.

[0081] According to some embodiments of the present disclosure, a method for solving the hemisphere ambiguity problem includes starting two processing stacks at the beginning of the tracking pipeline: one processing stack associated with the reference posture in the front hemisphere and another processing stack associated with the reference posture in the back hemisphere. When the two processing stacks are running, their outputs are checked - the processing stack with the wrong assumption of the hemisphere will quickly have problems tracking the actual movement of the handheld controller. At that time, the error processing stack can be stopped (e.g., destroyed or terminated), and the remaining instances are allowed to continue processing. In some embodiments, the check is periodically performed to calculate the divergence between the estimated posture (such as the fusion of position data and electromagnetic data determined by a motion sensor such as an IMU) and each reference posture (e.g., the reference posture in hemisphere 506A and the reference posture in hemisphere 506B). If any of the calculated divergences exceeds a threshold, the corresponding instance of the tracking module for the specific estimated posture stops. Additional descriptions related to these methods are provided more fully with respect to the description provided below.

[0082] Figure 6 A method 600 for resolving hemispherical ambiguity at a system or device including one or more sensors is shown. Among other possibilities, the system may be an electromagnetic tracking system or an optical device, such as an AR device, or any system that supports the transmission and reception of electromagnetic or magnetic signals. One or more steps of the method 600 may be omitted during the execution of the method 600, and the steps need not be performed in the order shown. Among other possibilities, one or more steps of the method 600 may be performed by the processing unit 306, the local processing and data module 270, and / or the remote processing module 272.

[0083] At step 602, one or more magnetic fields may be emitted by a magnetic field transmitter positioned at the handheld controller. The magnetic field transmitter may generate a magnetic field by generating a magnetic field in one direction (e.g., X, Y, or Z) by each coil. The magnetic field may be generated using any wavelength. In one or more embodiments, each of the axes may oscillate at a slightly different frequency. Although magnetic fields are discussed in some embodiments, this discussion of magnetic fields is not intended to limit embodiments of the present disclosure, and other fields including electric fields and electromagnetic fields are included within the scope of the present disclosure.

[0084] At step 604, one or more magnetic fields may be detected by one or more sensors positioned within the headset or belt pack. In some embodiments, a coordinate space corresponding to the magnetic or electromagnetic field may be determined. For example, a coordinate space around a transmitter (e.g., a handheld controller) may be determined based on the detected magnetic field. In some embodiments, the behavior of a coil at a sensor (which may be attached to a known object) may be detected. For example, the current induced at the coil may be calculated. In other embodiments, the rotation of the coil or any other quantifiable behavior may be tracked and measured.

[0085] At step 606A, the first processing stack is initiated so that the first processing stack runs during the first time interval. Running the first processing stack may include one or more of execution steps 608A, 612A, and 614A. The first processing stack may run on the local processing and data module 270, the remote processing module 272, or on a server remote from the device. At step 606B, the second processing stack is initiated so that the second processing stack runs during the second time interval. Running the second processing stack may include one or more of execution steps 608B, 612B, and 614B. The second processing stack may run on the local processing and data module 270, the remote processing module 272, or on a server remote from the device. The first time interval may be simultaneous, concurrent, or non-concurrent (i.e., non-overlapping) with the second time interval. In some embodiments, the first processing stack and the second processing stack are initialized simultaneously. In some embodiments, the first processing stack and the second processing stack are initialized sequentially.

[0086] At step 608A, the position and orientation of the handheld controller in the first hemisphere is determined based on the detected magnetic field. At step 608B, the position and orientation of the handheld controller in the second hemisphere is determined based on the detected magnetic field. The first hemisphere and the second hemisphere can be relative along the diameter, and in some embodiments, can correspond to the front hemisphere and the rear hemisphere about the headset. However, the first hemisphere and the second hemisphere can be defined in any of various configurations, including front / back, up / down, left / right. In some embodiments, the interface between the spheres is defined by a plane with a normal pointing downward to 10, 20, 30, 40, 50, 60, 70, or 80 degrees from the forward direction of the headset. In one example, the controller 406 can call a mapping table that makes the behavior of the coil at the sensor related to various positions or orientations. Based on these calculations, the position in the coordinate space can be determined together with the orientation of the sensor.

[0087] At step 610, motion data is detected by a motion sensor, such as an IMU sensor. The motion data may correspond to (e.g., may indicate) the motion of a handheld controller and / or headset. In addition to other possibilities, the motion data may include linear acceleration, angular rate, and / or orientation data. In some embodiments, the motion data may be referred to as IMU data when detected by an IMU sensor. In some instances, an IMU sensor may be positioned within a handheld controller or headset. For example, an IMU sensor may be mounted within a handheld controller or headset. In some embodiments, the motion data is detected by two different IMUs, one in a handheld controller and the other in a headset. In such an embodiment, the motion data may be used to determine the relative motion of the handheld controller with respect to the headset.

[0088] At steps 612A, 612B, one or more performance statistics associated with the first processing stack and the second processing stack are analyzed, respectively. In the illustrated embodiment, a difference is calculated between the determined position and orientation from steps 608A, 608B and the relative motion of the handheld controller with respect to the headset as determined using the motion data. In some examples, for each processing stack, multiple postures may be determined and compared with the motion of the handheld controller as indicated by the motion data. If, for example, the motion data indicates that the handheld controller is not moving (with respect to the headset), and the determined one or more postures (determined using the detected magnetic data) indicate that the handheld controller is moving, a difference with a high value may be calculated.

[0089] At steps 614A, 614B, the differences calculated in steps 612A, 612B are compared to a predetermined threshold. If any one of the differences exceeds the threshold, the corresponding processing stack is suspended or terminated while the other processing stack is allowed to continue processing (e.g., continue to determine the posture of the handheld controller in the corresponding hemisphere based on the additional detected magnetic field). In some embodiments, the threshold acts as an acceptable accuracy threshold so that if any one of the differences is less than the threshold, the corresponding processing stack is allowed to continue processing while the other processing stack is suspended. In some alternative embodiments, if the difference is greater than the threshold but less than a second threshold, indicating a relatively small difference, which can be referred to as an intermediate difference, parts of the process can be repeated. For example, if the intermediate difference is calculated at 612A, parts 608A, 610, 612A, and 614A can be repeated until the difference exceeds the second threshold, which will result in termination of the first hemisphere processing. In some embodiments, in contrast to a fixed value, the threshold is dynamic and can allow larger or smaller differences over time depending on the estimated posture. Non-limiting examples of dynamic thresholds include determining a large change in estimated pose within a short period of time, and then the controller 306 (intermediate decision block) can temporarily permit a threshold with a larger difference, or the number of samples that must remain below the threshold in a given time interval can be adjusted.

[0090] Figure 7 A method 700 for resolving hemispherical ambiguity at a system or device including one or more sensors is shown. Among other possibilities, the system may be an electromagnetic tracking system or an optical device, such as an AR device, or any system that supports the transmission and reception of magnetic signals. During execution of method 700, one or more steps of method 700 may be omitted, and the steps need not be performed in the order shown. Among other possibilities, one or more steps of method 700 may be performed by controller 306, local processing and data module 270, and / or remote processing module 272. One or more steps of method 700 may correspond to one or more steps of method 600.

[0091] At step 702, one or more magnetic fields may be emitted by a magnetic field transmitter positioned at a handheld controller. At step 704, one or more magnetic fields may be detected by one or more sensors positioned within the headset or belt pack. At step 706A, a first processing stack is initiated such that the first processing stack runs during a first time interval. At step 706B, a second processing stack is initiated such that the second processing stack runs during a second time interval. At step 708A, a position and orientation of the handheld controller within a first hemisphere is determined based on the detected magnetic field. At step 708B, a position and orientation of the handheld controller within a second hemisphere is determined based on the detected magnetic field. The first hemisphere and the second hemisphere may be diametrically opposed, and in some embodiments, may correspond to a front hemisphere and a rear hemisphere with respect to the headset.

[0092] At step 710, motion data is detected by a motion sensor, similar to the motion sensor described with reference to step 610. At steps 712A, 712B, one or more performance statistics of the first processing stack and the second processing stack, respectively, are analyzed. In the illustrated embodiment, a difference is calculated between the determined position and orientation from steps 708A, 708B and the relative motion of the handheld controller with respect to the headset as determined using the motion data.

[0093] At step 714, the differences calculated in steps 712A, 712B are compared to each other. If the difference calculated in step 712A (i.e., the "first difference") exceeds the difference calculated in step 712B (i.e., the "second difference"), then the first processing stack is aborted and the second processing stack is allowed to continue processing (i.e., continue to determine the pose of the handheld controller in the corresponding hemisphere based on the additional detected magnetic field). On the other hand, if the first difference is less than the second difference, then the second processing stack is aborted and the first processing stack is allowed to continue processing.

[0094] In some embodiments described herein, the term "EM tracking" may refer to the use of EM waves to determine the position of an object. In some embodiments described herein, the term "pose" may refer to the (x, y, z) definition of the object's position with respect to a world pose. In some embodiments described herein, the term "world pose" may refer to the position of an object with respect to a (0, 0, 0) reference point. In some embodiments described herein, the term "head pose" may refer to the position of the head in the world. In some embodiments described herein, the term "totem pose" may refer to the position of a totem / handheld controller in the world. In some embodiments, the totem pose may be calculated by adding a fused pose to the head pose. In some embodiments described herein, the term "IMU pose" corresponds to the motion of the totem as measured by the IMU. In some embodiments described herein, the term "EM pose" corresponds to the motion of the totem as measured by EM tracking. In some embodiments described herein, the term "fused pose" is calculated by decomposing the IMU pose into the EM pose along with some additional filtering. In some embodiments, the pose is run through the tracking algorithm at the same time to obtain two versions of all totem poses: EM, IMU, and fused poses. Based on the method used, the appropriate attitude velocity (position change) is compared with the expected change assumed depending on the observed conditions.

[0095] In some embodiments, a totem pose hemisphere is established when no totem motion is detected concurrently with a changed head pose. The method relies on the absence of totem motion detected by the totem IMU and a changed head pose at the same time point. The premise is that if the totem pose will be initialized in the wrong hemisphere, then a change in head pose will result in a change in totem pose, all within the world coordinate system or inversely thereto, i.e., a totem pose initialized in the correct hemisphere should not see a change in its pose even if the head pose changes.

[0096] In some embodiments, the totem pose is calculated by combining the EM sensor measured distances between the EM transmitter and the receiver and adding it to the head pose after passing the data through the necessary transformations to align the reference frame. If the totem has not moved, and the head moves away from the totem, the EM distance should change by the same absolute value to ensure that the calculated totem pose remains unchanged. In some embodiments, the method provides an algebraic solution that the inequality of the calculated totem pose is an indication of incorrect hemisphere initialization. Multiple samples can be collected using a stationary totem, and if all samples result in a moving totem pose within the world coordinate system when the head moves, the selected hemisphere can be considered to be an incorrect hemisphere.

[0097] Figure 8 The results of simulating the expected totem pose when the head moves and the totem is stationary are shown. The simulation data is obtained from reference Figure 9-12 The experimental data shown supports and validates the embodiments of the present disclosure that can determine that the actual totem pose will not change within the world reference frame when the head moves and the totem is stationary. In this example, a fused totem pose is shown because the EM totem pose tends to have significant noise and the fused pose includes processing and necessary filtering to account for the motion data.

[0098] refer to Figure 8 , the totem gesture is constant in Examples 0 to 4, shown as position 8 in the world coordinate system. In Example 0, the head gesture is static at position 6 in the world coordinate system, and the ghost totem gesture exists at position 4 in the world coordinate system. Referring to Example 1, the head gesture is displaced from position 6 to approximately position 3 in the world coordinate system. In response to this change in head gesture, although the actual totem position is static at position 8, the ghost totem gesture is displaced from position 4 to position -2 in the world coordinate system. Referring to Example 2, the head gesture is displaced from position 6 to approximately position 4, and further the actual totem position is static, while the ghost totem gesture is displaced from position 4 to approximately position 0. Referring to Example 3, the head gesture is displaced from position 6 to position 8, and the ghost totem gesture is also moved to position 8. Referring to Example 4, the head gesture is displaced from position 6 to approximately position 9, and the ghost totem gesture is displaced from position 4 to approximately position 10. As can be seen in Examples 0 to 4, in some embodiments, the relative distance between the head pose and each of the totem poses (actual and ghosted) within the world coordinate system can be the same over a wide range of the location of the head pose.

[0099] Fig. 9 Experimental data is shown when the head is moving and the totem is stationary, where the totem is initialized in the correct hemisphere. Fig. 9 As shown, when the totem is initialized in the correct hemisphere, the totem fusion posture remains almost unchanged regardless of head movement, as shown by Figure 8 This is inferred from the actual totem pose demonstrated by the simulated data in , which shows no motion of the actual totem pose across examples 0 to 4.

[0100] Fig.10 Experimental totem motion data is shown when the totem is stationary, where the totem is initialized in the correct hemisphere. Fig.10 The data shown may correspond to Fig. 9 The posture data shown. Fig.10 The above graph shows the rotation (in degrees) measurements obtained by the IMU gyroscope as a function of time, and Fig.10 The following figure shows the acceleration (in m / s) obtained by the IMU accelerometer 2 The measurement results are shown as a function of time. Fig.10As shown, the IMU gyroscope data has some noise around the zero degree rotation value. The IMU acceleration data shows that no translation changes are seen in the X, Y, and Z axes. Since the change in acceleration is measured with respect to the acceleration of gravity, constant offsets in the Y and Z axes are expected even without totem motion.

[0101] Fig.11 Experimental data is shown when the head moves and the totem is stationary, where the totem is initialized in the wrong hemisphere. The experimental data shows that the totem pose changes with head movement even when the totem is stationary. It is clear that when initialized in the wrong hemisphere, the totem fusion pose is an amplified version of the head pose, which is consistent with the pose generated by Figure 8 , which shows significant motion of the ghosting totem pose across examples 0 to 4.

[0102] Fig.12 Shows no Fig.11 The displayed posture data corresponds to the experimental totem motion data of the totem motion. Fig.12 The above graph shows the rotation (in degrees) measurements obtained by the IMU gyroscope as a function of time, and Fig.12 The following figure shows the acceleration (in m / s) obtained by the IMU accelerometer 2 The IMU gyroscope data shows very little noise around the zero degree rotation value, and the IMU acceleration data shows that no translation changes are seen in the X, Y, and Z axes.

[0103] Fig.13 1 shows a simplified computer system 1300 according to the embodiments described herein. Fig.13 The computer system 1300 shown may be incorporated into the devices described herein. Fig.13 A schematic diagram of one embodiment of a computer system 1300 is provided that can perform some or all of the steps of the methods provided by various embodiments. It should be noted that Fig.13 It is intended only to provide a generalized illustration of any or all of the various components that may be used as appropriate. Fig.13 It is broadly shown how individual system elements may be implemented in a relatively discrete or relatively more integrated manner.

[0104] A computer system 1300 is shown including hardware elements that may be electrically coupled via a bus 1305 or that may otherwise communicate as appropriate. The hardware elements may include one or more processors 1310, including but not limited to one or more general purpose processors and / or one or more special purpose processors such as digital signal processing chips, graphics acceleration processors, and / or the like; one or more input devices 1315, which may include but are not limited to a mouse, keyboard, camera, and / or the like; and one or more output devices 1320, which may include but are not limited to a display device, a printer, and / or the like.

[0105] The computer system 1300 may also include and / or communicate with one or more non-transitory storage devices 1325, which may include, but are not limited to, local and / or network access storage, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices, such as random access memory ("RAM") and / or read-only memory ("ROM"), which may be programmable, flash-updated, and / or the like. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, and / or the like.

[0106] The computer system 1300 may also include a communication subsystem 1319, which may include but is not limited to a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device and / or a chipset such as Bluetooth. TM Devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication facilities, etc. and / or the like. The communication subsystem 1319 may include one or more input and / or output communication interfaces that permit data to be exchanged with a network (such as the network described below, to name just one example), other computer systems, televisions, and / or any other devices described herein. Depending on the desired functionality and / or other implementation issues, a portable electronic device or similar device may transmit images and / or other information via the communication subsystem 1319. In other embodiments, a portable electronic device, such as a first electronic device, may be incorporated into the computer system 1300, for example, as an electronic device of the input device 1315. In some embodiments, the computer system 1300 will also include a working memory 1335, which may include a RAM or ROM device, as described above.

[0107] The computer system 1300 may also include software elements shown as currently located within the working memory 1335, including an operating system 1340, device drivers, executable libraries, and / or other code, such as one or more application programs 1345, which may include computer programs provided by various embodiments, and / or may be designed to implement methods and / or configure systems provided by other embodiments, as described above. By way of example only, one or more programs described with respect to the methods discussed above may be implemented as code and / or instructions executable by a computer and / or a processor within a computer; in aspects, such code and / or instructions may then be used to configure and / or adapt a general purpose computer or other device to perform one or more operations according to the described methods.

[0108] The set of these instructions and / or codes may be stored on a non-transitory computer-readable storage medium, such as one or more storage devices 1325 described above. In some cases, the storage medium may be contained within a computer system, such as computer system 1300. In other embodiments, the storage medium may be separate from the computer system, for example, a removable medium, such as an optical disk, and / or provided in an installation package so that the storage medium can be used to program, configure, and / or adapt a general-purpose computer using the instructions / codes stored thereon. These instructions may take the form of executable code that can be executed by computer system 1300 and / or may take the form of source and / or installable code, which is based on compilation and / or installation on computer system 1300, for example, using any of the commonly available compilers, installers, compression / decompression utilities, etc., and then takes the form of executable code.

[0109] It will be apparent to those skilled in the art that a large number of variations can be made according to specific requirements. For example, custom hardware can also be used, and / or specific elements can be implemented in hardware, software, including portable software such as applets, etc., or both. Furthermore, connections to other computing devices, such as network input / output devices, can be employed.

[0110] As mentioned above, in one aspect, some embodiments may use a computer system, such as computer system 1300, to perform methods according to various embodiments of the technology. According to one set of embodiments, some or all of the processes of such methods are performed by computer system 1300 in response to processor 1310 executing one or more sequences of one or more instructions, which may be included in operating system 1340 and / or other code, such as application 1345 contained in working memory 1335. Such instructions may be read into working memory 1335 from another computer-readable medium, such as one or more of storage devices 925. By way of example only, execution of the sequences of instructions contained in working memory 1335 may cause one or more processors 1310 to perform one or more processes of the methods described herein. Additionally or alternatively, portions of the methods described herein may be performed by dedicated hardware.

[0111] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any medium that participates in providing data that causes a machine to operate in a particular manner. In an embodiment implemented using computer system 1300, various computer-readable media may be involved in providing instructions / code to one or more processors 1310 for execution and / or may be used to store and / or carry such instructions / code. In many implementations, computer-readable media are physical and / or tangible storage media. Such media may take the form of non-volatile media or volatile media. Non-volatile media include, for example, optical disks and / or magnetic disks, such as one or more storage devices 1325. Volatile media include, but are not limited to, dynamic memory, such as working memory 1335.

[0112] Common forms of physical and / or tangible computer readable media include, for example, floppy disks, diskettes, hard disks, magnetic tape or any other magnetic medium, CD-ROMs, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cassette or any other medium from which a computer can read instructions and / or code.

[0113] Various forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to one or more processors 1310 for execution. By way of example only, the instructions may initially be carried on a magnetic disk and / or optical disk of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and / or executed by the computer system 1300.

[0114] The communication subsystem 1319 and / or its components will typically receive the signal, and the bus 1305 may then carry the signal and / or the data, instructions, etc. carried by the signal to the working memory 1335, from which the one or more processors 1310 retrieve and execute the instructions. The instructions received by the working memory 1335 may optionally be stored on the non-transitory storage device 1325 before or after execution by the one or more processors 1310.

[0115] The methods, systems and devices discussed above are examples. Various configurations may omit, replace or add various processes or components as appropriate. For example, in alternative configurations, the method may be performed in an order different from the order described, and / or the stages may be added, omitted and / or combined. Moreover, the features described with respect to certain configurations may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Moreover, technology evolves and therefore many of the elements are examples and do not limit the scope of the present disclosure or claims.

[0116] Specific details are given in the description that provides a thorough understanding of exemplary configurations, including implementations. However, the configuration can be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary details to avoid making the configuration difficult to understand. This description only provides example configurations and does not limit the scope, applicability, or configuration of the claims. On the contrary, the foregoing description of the configuration will provide an enabling description for implementing the described technology to those skilled in the art. Various changes may be made to the functions and arrangements of the elements without departing from the spirit or scope of the present disclosure.

[0117] Moreover, the configuration may be described as a process depicted as a schematic flow chart or block diagram. Although each operation may be described as a sequential process, many operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. The process may have additional steps not included in the accompanying drawings. Moreover, the example of the method may be implemented by hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segment performing the necessary tasks may be stored in a non-transitory computer-readable medium, such as a storage medium. The processor may perform the described tasks.

[0118] Several example configurations have been described, and various modifications, alternative configurations, and equivalents may be used without departing from the spirit of the present disclosure. For example, the above elements may be components of a larger system, wherein other rules may be superior to or otherwise modify the application of the technology. Moreover, several steps may be performed before, during, or after the above elements are considered. Therefore, the above description does not restrict the scope of the claims.

[0119] As used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Thus, for example, reference to "a user" includes one or more such users and reference to "a processor" includes reference to one or more processors and equivalents thereof known to those skilled in the art, and so forth.

[0120] Furthermore, when used in this specification and the following claims, the words "comprises", "contains" and "includes" are intended to specify the presence of stated features, integers, components or steps, but do not exclude the presence or addition of one or more other features, integers, components, steps, actions or combinations.

[0121] It should also be understood that the examples and embodiments described herein are for illustrative purposes only and that various slight modifications or changes thereof will be suggested to those skilled in the art and are to be included within the spirit and purview of the present application and the scope of the appended claims.

Claims

1. A method for solving hemispherical blur, the method comprising: emitting one or more magnetic fields at the handheld controller; detecting the one or more magnetic fields by one or more sensors positioned relative to the earphone; detecting motion data corresponding to the handheld controller or the headset; Running a first processing stack during a first time interval, wherein running the first processing stack comprises: determining a first position and a first orientation of the handheld controller within a first hemisphere about the headset based on the detected one or more magnetic fields; and calculating a first difference between the first position, the first orientation, and the motion data; and executing a second processing stack during a second time interval, wherein executing the second processing stack comprises: determining a second position and a second orientation of the handheld controller within a second hemisphere about the headset based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposed to the first hemisphere; and calculating a second difference between the second position, the second orientation, and the motion data; and The first processing stack or the second processing stack is aborted based on one or both of the first difference and the second difference.

2. The method according to claim 1, further comprising: comparing the first difference to a threshold; determining that the first difference exceeds the threshold; as well as In response to determining that the first difference exceeds the threshold: terminating the first processing stack; and The second processing stack is allowed to continue.

3. The method according to claim 1, further comprising: comparing the first difference to the second difference; determining that the first difference exceeds the second difference; as well as In response to determining that the first difference exceeds the second difference: terminating the first processing stack; and The second processing stack is allowed to continue.

4. The method according to claim 1, wherein: The motion data corresponds to the handheld controller, and wherein the motion data is detected by an inertial measurement unit (IMU) positioned within the handheld controller.

5. The method according to claim 1, wherein: The motion data corresponds to the earphone, and wherein the motion data is detected by an inertial measurement unit (IMU) positioned within the earphone.

6. The method according to claim 1, wherein: The motion data is detected during one or both of the first time interval and the second time interval.

7. The method according to claim 1, wherein: The first time interval and the second time interval are concurrent or simultaneous.

8. The method according to claim 1, wherein: The first hemisphere is about the front hemisphere of the earphone, and the second hemisphere is about the back hemisphere of the earphone.

9. A system for resolving hemispherical blur, comprising: a handheld controller including a magnetic field transmitter configured to emit one or more magnetic fields; a headset comprising one or more magnetic field sensors configured to detect the one or more magnetic fields; a motion sensor configured to detect motion data corresponding to the handheld controller or the headset; as well as One or more processors configured to perform operations comprising: Running a first processing stack during a first time interval, wherein running the first processing stack comprises: determining a first position and a first orientation of the handheld controller within a first hemisphere about the headset based on the detected one or more magnetic fields; and calculating a first difference between the first position, the first orientation, and the motion data; and executing a second processing stack during a second time interval, wherein executing the second processing stack comprises: determining a second position and a second orientation of the handheld controller within a second hemisphere about the headset based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposed to the first hemisphere; and calculating a second difference between the second position, the second orientation, and the motion data; and The first processing stack or the second processing stack is aborted based on one or both of the first difference and the second difference.

10. The system according to claim 9, wherein: The motion data corresponds to the handheld controller, and wherein the motion data is detected by an inertial measurement unit (IMU) positioned within the handheld controller.

11. The system according to claim 9, wherein: The first time interval and the second time interval are concurrent or simultaneous.

12. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: emitting one or more magnetic fields at the handheld controller; detecting the one or more magnetic fields by one or more sensors positioned relative to the earphone; detecting motion data corresponding to the handheld controller or the headset; Running a first processing stack during a first time interval, wherein running the first processing stack comprises: determining a first position and a first orientation of the handheld controller within a first hemisphere about the headset based on the detected one or more magnetic fields; and calculating a first difference between the first position, the first orientation, and the motion data; and and executing a second processing stack during a second time interval, wherein executing the second processing stack comprises: determining a second position and a second orientation of the handheld controller within a second hemisphere about the headset based on the detected one or more magnetic fields, wherein the second hemisphere is diametrically opposed to the first hemisphere; calculating a second difference between the second position, the second orientation, and the motion data; and The first processing stack or the second processing stack is aborted based on one or both of the first difference and the second difference.

13. The non-transitory computer readable medium of claim 12, wherein: The operations also include: comparing the first difference to a threshold; determining that the first difference exceeds the threshold; and In response to determining that the first difference exceeds the threshold: terminating the first processing stack; and The second processing stack is allowed to continue.

14. The non-transitory computer readable medium of claim 12, wherein: The operations also include: comparing the first difference to the second difference; determining that the first difference exceeds the second difference; and In response to determining that the first difference exceeds the second difference: terminating the first processing stack; and The second processing stack is allowed to continue.

Images