Method and system for resolving hemispheric ambiguity in six degree of freedom attitude measurements - Patents.com

Abstract

A technique for resolving hemisphere ambiguity is disclosed. One or more magnetic fields are emitted at a handheld controller. The one or more magnetic fields are detected by one or more sensors positioned relative to a headset. Movement data corresponding to the handheld controller or the headset is detected. During a first time interval, a first position and a first orientation of the handheld controller within a first hemisphere are determined based on the detected one or more magnetic fields, and a first difference is calculated based on the first position, the first orientation, and the movement data. During a second time interval, a second position and a second orientation of the handheld controller within a second hemisphere are determined based on the detected one or more magnetic fields, and a second difference is calculated based on the second position, the second orientation, and the movement data.

Description

[Background technology]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 797,776, filed Jan. 28, 2019, and entitled “METHOD AND SYSTEM FOR RESOLVING HEMISPHERE AMBIGUITY IN SIX DEGREE OF FREEDOM POSE MEASUREMENTS,” the contents of which are incorporated herein in their entirety.

[0002] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or may be perceived as real. Virtual reality, or "VR", scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs, while augmented reality, or "AR", scenarios typically involve the presentation of digital or virtual image information as an augmentation to the visualization of the real world around the user.

[0003] Despite the advances made in these display technologies, there remains a need in the art for improved methods, systems, and devices relating to augmented reality systems, and particularly display systems. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure generally relates to techniques for improving the performance and user experience of optical systems. More specifically, embodiments of the present disclosure provide a method for operating an augmented reality (AR) or virtual reality (VR) device in which a handheld controller is employed to assist in the operation of the device. The inventive summary of the present disclosure is described with reference to the examples given below. As used below, any reference to a series of examples is to be understood as a disjunctive reference to each of those examples (e.g., "Example 1-4" is to be understood as "Example 1, 2, 3, or 4").

[0005] Example 1 is a method of resolving hemisphere ambiguity in a system with one or more sensors, comprising the steps of emitting one or more magnetic fields at a handheld controller, detecting the one or more magnetic fields with one or more sensors positioned in a headset or beltpack of the system, and performing a first processing stack during a first time interval, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the one or more magnetic fields, and generating a first hemisphere-based ambiguity map of the first position, first orientation, and second hemisphere-based ambiguity map corresponding to either the handheld controller or the headset. performing a second processing stack during a second time interval, the second processing stack including: calculating a first difference based on the movement data; determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on one or more magnetic fields, the second hemisphere being diametrically opposite the first hemisphere; calculating a second difference based on the second position, second orientation, and movement data; and aborting either the first processing stack or the second processing stack based on one or both of the first difference and the second difference.

[0006] Example 2 is the method of example 1, further comprising detecting the movement data by an inertial measurement unit (IMU) located in the handheld controller.

[0007] Example 3 is the method of example 1, further comprising detecting the movement data by an inertial measurement unit (IMU) located in the headset.

[0008] Example 4 is the method of example 2 or 3, wherein the movement data is detected during one or both of a first time interval and a second time interval.

[0009] Example 5 is the method of example 2 or 3, wherein the movement data is detected prior to both the first time interval and the second time interval.

[0010] Example 6 is the method of example 1, wherein the first time interval occurs in parallel with the second time interval.

[0011] Example 7 is the method of example 1, wherein the first time interval occurs simultaneously with the second time interval.

[0012] Example 8 is the method of example 1, wherein the first time interval has a first start time and the second time interval has a second start time, and the first start time and the second start time occur simultaneously or are separated by less than a threshold value.

[0013] Example 9 is the method of Example 1, further including the steps of comparing the first difference with a threshold, determining that the first difference exceeds the threshold, and in response to determining that the first difference exceeds the threshold, aborting the first processing stack and allowing the second processing stack to continue.

[0014] Example 10 is the method of Example 1, further including the steps of comparing the second difference with a threshold, determining that the second difference exceeds the threshold, and in response to determining that the second difference exceeds the threshold, aborting the second processing stack and allowing the first processing stack to continue.

[0015] Example 11 is the method of Example 1, further including the steps of comparing the first difference with the second difference, determining that the first difference exceeds the second difference, and in response to the step of determining that the first difference exceeds the second difference, aborting the first processing stack and allowing the second processing stack to continue.

[0016] Example 12 is the method of Example 1, further including the steps of comparing the first difference to the second difference, determining that the second difference exceeds the first difference, and in response to the step of determining that the second difference exceeds the first difference, aborting the second processing stack and allowing the first processing stack to continue.

[0017] Example 13 is the method of example 1, wherein the first hemisphere is a front hemisphere relative to the headset, and the second hemisphere is a back hemisphere relative to the headset.

[0018] Example 14 is the method of example 1, further comprising delivering virtual content to the user based on either a first location and a first orientation or a second location and a second orientation.

[0019] Example 15 is the method of example 1, wherein the system is an optical device.

[0020] Example 16 is a system including a handheld controller including a magnetic field emitter configured to emit one or more magnetic fields, and a headset or beltpack including one or more magnetic field sensors configured to detect one or more magnetic fields, and performing a first processing stack during a first time interval, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the one or more magnetic fields, and calculating a first difference based on the first position, first orientation, and movement data corresponding to either the handheld controller or the headset. a processor configured to perform operations including: performing a second processing stack for a second time interval, the second processing stack including determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on one or more magnetic fields, the second hemisphere being diametrically opposite the first hemisphere; calculating a second difference based on the second position, second orientation, and movement data; and aborting either the first processing stack or the second processing stack based on one or both of the first difference and the second difference.

[0021] Example 17 is the system of example 16, further comprising detecting the movement data by an inertial measurement unit (IMU) located in the handheld controller.

[0022] Example 18 is the system of example 16, wherein the operations further include detecting the movement data by an inertial measurement unit (IMU) positioned within the headset.

[0023] Example 19 is the system of example 17 or 18, wherein the movement data is detected during one or both of the first time interval and the second time interval.

[0024] Example 20 is the system of example 17 or 18, wherein the movement data is detected prior to both the first time interval and the second time interval.

[0025] Example 21 is the system of example 16, wherein the first time interval occurs in parallel with the second time interval.

[0026] Example 22 is the system of example 16, wherein the first time interval occurs simultaneously with the second time interval.

[0027] Example 23 is the system of Example 16, wherein the first time interval has a first start time, the second time interval has a second start time, and the first start time and the second start time occur simultaneously or are separated by less than a threshold value.

[0028] Example 24 is the system described in Example 16, wherein the operations further include the steps of comparing the first difference with a threshold, determining that the first difference exceeds the threshold, and in response to the step of determining that the first difference exceeds the threshold, aborting the first processing stack and allowing the second processing stack to continue.

[0029] Example 25 is the system described in Example 16, wherein the operations further include the steps of comparing the second difference with a threshold, determining that the second difference exceeds the threshold, and in response to the step of determining that the second difference exceeds the threshold, aborting the second processing stack and allowing the first processing stack to continue.

[0030] Example 26 is the system described in Example 16, wherein the operations further include the steps of comparing the first difference and the second difference, determining that the first difference exceeds the second difference, and in response to the step of determining that the first difference exceeds the second difference, aborting the first processing stack and allowing the second processing stack to continue.

[0031] Example 27 is the system described in Example 16, wherein the operations further include the steps of comparing the first difference to the second difference, determining that the second difference exceeds the first difference, and in response to the step of determining that the second difference exceeds the first difference, aborting the second processing stack and allowing the first processing stack to continue.

[0032] Example 28 is the system of example 16, wherein the first hemisphere is a front hemisphere relative to the headset and the second hemisphere is a back hemisphere relative to the headset.

[0033] Example 29 is the system described in Example 16, wherein the operations further include a step of delivering virtual content to the user based on either a first position and a first orientation or a second position and a second orientation.

[0034] Example 30 is the system of example 16, wherein the system is an optical device.

[0035] Example 31 is a method of resolving hemisphere ambiguity, the method including the steps of emitting one or more magnetic fields at a handheld controller, detecting the one or more magnetic fields by one or more sensors positioned relative to a headset, detecting movement data corresponding to the handheld controller or the headset, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields during a first time interval, calculating a first difference based on the first position, the first orientation, and the movement data, determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields during a second time interval, the second hemisphere being diametrically opposite the first hemisphere, and calculating a second difference based on the second position, the second orientation, and the movement data.

[0036] Example 32 is a system comprising: a handheld controller comprising a magnetic field emitter configured to emit one or more magnetic fields; a headset comprising one or more magnetic field sensors configured to detect one or more magnetic fields; a movement sensor configured to detect movement data corresponding to the handheld controller or the headset; and one or more processors configured to perform operations including: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the one or more magnetic fields detected during a first time interval; calculating a first difference based on the first position, the first orientation, and the movement data; determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the one or more magnetic fields detected during a second time interval, the second hemisphere being diametrically opposite the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the movement data.

[0037] Example 33 is a non-transitory computer readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform operations including emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields with one or more sensors positioned relative to a headset; detecting movement data corresponding to the handheld controller or the headset; determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields during a first time interval; calculating a first difference based on the first position, the first orientation, and the movement data; determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields during a second time interval, the second hemisphere being diametrically opposite the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the movement data. The present invention further provides the following: (Item 1) 1. A method for resolving hemisphere ambiguity, the method comprising: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields with one or more sensors positioned relative to the headset; Detecting movement data corresponding to the handheld controller or the headset; During the first time interval, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference based on the first position, the first orientation, and the movement data; During the second time interval, determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the movement data; and A method comprising: (Item 2) performing a first process stack during the first time interval, the performing the first process stack including determining the first position and the first orientation and calculating the first difference; performing a second process stack during the second time interval, the performing the second process stack including determining the second position and the second orientation and calculating the second difference; The method of claim 1, further comprising: (Item 3) 3. The method of claim 2, further comprising aborting either the first processing stack or the second processing stack based on one or both of the first difference and the second difference. (Item 4) comparing the first difference to a threshold; and determining that the first difference exceeds the threshold; and in response to determining that the first difference exceeds the threshold, Aborting the first process stack; and allowing the second processing stack to continue; and The method of claim 3, further comprising: (Item 5) comparing the first difference with the second difference; and determining that the first difference exceeds the second difference; in response to determining that the first discrepancy exceeds the second discrepancy, Aborting the first process stack; and allowing the second processing stack to continue; and The method of claim 3, further comprising: (Item 6) The movement data corresponds to the handheld controller, and the movement data is measured by an inertial measurement unit (IMU) located within the handheld controller. The method according to item 1, wherein the (Item 7) 2. The method of claim 1, wherein the movement data corresponds to the headset, and the movement data is detected by an inertial measurement unit (IMU) positioned within the headset. (Item 8) 2. The method of claim 1, wherein the movement data is detected during one or both of the first time interval and the second time interval. (Item 9) 2. The method of claim 1, wherein the first time interval occurs in parallel or simultaneously with the second time interval. (Item 10) 2. The method according to claim 1, wherein the first hemisphere is a front hemisphere relative to the headset, and the second hemisphere is a back hemisphere relative to the headset. (Item 11) 1. A system comprising: a handheld controller comprising a magnetic field emitter configured to emit one or more magnetic fields; a headset comprising one or more magnetic field sensors configured to detect the one or more magnetic fields; a movement sensor configured to detect movement data corresponding to the handheld controller or the headset; one or more processors, the one or more processors comprising: During the first time interval, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference based on the first position, the first orientation, and the movement data; During the second time interval, determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the movement data; and one or more processors configured to perform operations including: A system comprising: (Item 12) The operation further comprises: performing a first process stack during the first time interval, the performing the first process stack including determining the first position and the first orientation and calculating the first difference; performing a second process stack during the second time interval, the performing the second process stack including determining the second position and the second orientation and calculating the second difference; Item 12. The system according to item 11, comprising: (Item 13) The operation further comprises: Item 13. The system of item 12, further comprising aborting either the first processing stack or the second processing stack based on one or both of the first difference and the second difference. (Item 14) Item 12. The system of item 11, wherein the movement data corresponds to the handheld controller, and the movement data is detected by an inertial measurement unit (IMU) positioned within the handheld controller. (Item 15) Item 12. The system of item 11, wherein the first time interval occurs in parallel or simultaneously with the second time interval. (Item 16) A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields with one or more sensors positioned relative to the headset; Detecting movement data corresponding to the handheld controller or the headset; During the first time interval, determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference based on the first position, the first orientation, and the movement data; During the second time interval, determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference based on the second position, the second orientation, and the movement data; and A non-transitory computer readable medium for performing operations including: (Item 17) The operation further comprises: performing a first process stack during the first time interval, the performing the first process stack including determining the first position and the first orientation and calculating the first difference; performing a second process stack during the second time interval, the performing the second process stack including determining the second position and the second orientation and calculating the second difference; 17. The non-transitory computer-readable medium of item 16, comprising: (Item 18) The operation further comprises: 20. The non-transitory computer-readable medium of claim 17, further comprising: aborting either the first processing stack or the second processing stack based on one or both of the first difference and the second difference. (Item 19) The operation further comprises: comparing the first difference to a threshold; and determining that the first difference exceeds the threshold; and in response to determining that the first difference exceeds the threshold, Aborting the first process stack; and allowing the second processing stack to continue; and 20. The non-transitory computer-readable medium of item 18, comprising: (Item 20) The operation further comprises: comparing the first difference with the second difference; and determining that the first difference exceeds the second difference; in response to determining that the first discrepancy exceeds the second discrepancy, Aborting the first process stack; and allowing the second processing stack to continue; and 20. The non-transitory computer-readable medium of item 18, comprising: [Brief description of the drawings]

[0038] The accompanying drawings, which are included to provide a further understanding of the present disclosure, are incorporated in and form a part of this specification, illustrate embodiments of the present disclosure, and together with the detailed description serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than may be necessary for a fundamental understanding of the disclosure and various ways in which it may be practiced.

[0039] [Figure 1] FIG. 1 illustrates an AR scene as viewed through a wearable augmented reality (AR) device, according to some embodiments.

[0040] [Diagram 2] FIG. 2 illustrates various possible components of an AR system.

[0041] [Diagram 3] FIG. 3 illustrates an exemplary system diagram of an electromagnetic tracking system.

[0042] [Figure 4] FIG. 4 illustrates one example of how an electromagnetic tracking system can be incorporated with an AR system.

[0043] [Diagram 5]FIG. 5 illustrates the hemispheric ambiguity problem that can exist in electromagnetic tracking systems.

[0044] [Figure 6] FIG. 6 illustrates a method for resolving hemisphere ambiguity in a system or device that includes one or more sensors.

[0045] [Figure 7] FIG. 7 illustrates a method for resolving hemisphere ambiguity in a system or device that includes one or more sensors.

[0046] [Figure 8] FIG. 8 illustrates the results of a simulation showing the expected totem pose when the head is moving and the totem is stationary.

[0047] [Figure 9] FIG. 9 illustrates experimental data when the head is moving and the totem is stationary, with the totem initialized in the correct hemisphere.

[0048] [Figure 10] FIG. 10 illustrates experimental totem movement data when the totem is stationary with the totem initialized in the correct hemisphere.

[0049] [Figure 11] FIG. 11 illustrates experimental data when the head is moving and the totem is stationary, with the totem initialized in the incorrect hemisphere.

[0050] [Figure 12] FIG. 12 illustrates experimental totem movement data without the totem motion corresponding to the posture data displayed in FIG.

[0051] [Figure 13]FIG. 13 illustrates a computer system according to some embodiments described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0052] A typical head-mounted augmented reality (AR) display is at least loosely coupled to the user's head and therefore moves as the user's head moves. When the user's head movement is detected by the display system, the displayed data can be updated to account for the change in head pose. As an example, when a user wearing a head-mounted display views a virtual representation of a three-dimensional (3D) object on the display and walks around the area in which the 3D object appears, the 3D object can be re-rendered for each viewpoint, giving the user the perception that they are walking around an object that occupies real space. When a head-mounted display is used to present multiple objects in a virtual space (e.g., a rich virtual world), measurements of head pose (e.g., the user's head location and orientation) can be used to re-render the scene to match the user's dynamically changing head location and orientation and provide an improved sense of immersion in the virtual space.

[0053] Thus, head pose detection or calculation can facilitate a display system to render virtual objects so that they appear to occupy space in the real world in a manner that makes sense to the user. In addition, detection of the position and / or orientation of real objects, such as a handheld device or controller (which may also be referred to as a "totem"), a haptic device, or other real physical object, relative to the user's head or the AR system may also facilitate the display system in presenting display information to the user and enabling the user to effectively interact with certain aspects of the AR system. At least with respect to AR applications, placing virtual objects in a spatial relationship with physical objects (e.g., presented to appear spatially proximate to the physical objects in two or three dimensions) can be a non-trivial problem.

[0054] For example, head movement can significantly complicate the placement of virtual objects within a view of the surrounding environment. This is true whether the view is captured as an image of the surrounding environment and then projected or displayed to the end user, or whether the end user directly perceives the view of the surrounding environment. For example, head movement is likely to change the end user's field of view, which will likely require updates to where various virtual objects are displayed within the end user's field of view.

[0055] In addition, head movements may occur within a wide variety of ranges and speeds. Head movement speed may vary not only between different head movements, but also within or across the range of a single head movement. For example, head movement speed may initially increase (e.g., linearly or non-linearly) from a starting point, decrease as an end point is reached, and obtain a maximum speed somewhere between the beginning and end points of the head movement. Rapid head movements may further exceed the capabilities of a particular display or projection technology to render an image that appears to an end user as a uniform and / or smooth motion.

[0056] Head tracking accuracy and latency (i.e., the time elapsed between the time a user moves their head and the time the image is updated and displayed to the user) are challenges for virtual reality (VR) and AR systems. In particular, for display systems that fill a substantial portion of the user's field of view with virtual elements, it can be important that head tracking accuracy is high and that the overall system latency from the initial detection of head movement to the update of the light delivered by the display to the user's visual system is very low. If the latency is high, the system can create a mismatch between the user's vestibular and visual sensory systems, generating user perception scenarios that can lead to motion or simulator sickness. If the system latency is high, the apparent location of virtual objects will appear unstable during rapid head movements.

[0057] In addition to head-mounted display systems, other display systems can also benefit from accurate and low-latency head pose detection. These include eye-tracking display systems, where the display is not worn on the user's body, but is mounted, for example, on a wall or other surface. Eye-tracking displays act like a window on a scene, and as the user moves his or her head relative to the "window," the scene is re-rendered to match the user's changing viewpoint. Other systems include head-mounted projection systems, where a head-mounted display projects light onto the real world.

[0058] In addition, the AR system may be designed to interact with the user to provide a realistic augmented reality experience. For example, multiple users may play a ball game with a virtual ball and / or other virtual objects. One user may "catch" the virtual ball and throw it back to another user. In another embodiment, a first user may be provided with a totem (e.g., a real bat communicatively coupled to the AR system) to hit the virtual ball. In other embodiments, a virtual user interface may be presented to the AR user, allowing the user to select one of many options. The user may use a totem, a haptic device, a wearable component, or simply touch a virtual screen to interact with the system.

[0059] Detecting the pose and orientation of a user's head and the physical location of real objects in space enable an AR system to display virtual content in an effective and enjoyable manner. However, while these capabilities are essential for an AR system, they are difficult to achieve. For example, an AR system may need to recognize the physical location of real objects (e.g., the user's head, a totem, a haptic device, a wearable component, the user's hand, etc.) and correlate the physical coordinates of the real objects with the virtual coordinates corresponding to one or more virtual objects being displayed to the user. This may require highly accurate sensors and sensor recognition systems to track the position and orientation of one or more objects at a rapid rate.

[0060] Current approaches do not perform localization at satisfactory speed or accuracy standards. Thus, there is a need for a better localization system in the context of AR and VR devices. In the following description, various embodiments will be described. For purposes of explanation, specific configurations and details are set forth to provide a thorough understanding of the embodiments. However, it will also be apparent to those skilled in the art that the embodiments may be practiced without the specific details. Furthermore, well-known features may be omitted or simplified so as not to obscure the described embodiments.

[0061] FIG. 1 illustrates an AR scene as viewed through a wearable AR device according to some embodiments described herein. An AR scene 100 is depicted in which a user of the AR technology sees a real-world park-like setting 106 featuring people, trees, buildings in the background, and a concrete platform 120. In addition to these items, the user of the AR technology also perceives that they "see" a robotic figure 110 standing on the real-world platform 120 and a cartoon-like avatar character 102, which appears to be an anthropomorphic bumblebee, flying nearby, although these elements (the character 102 and the figure 110) do not exist in the real world. Due to the extreme complexity of the human visual perception and nervous system, it is challenging to produce VR or AR technology that promotes a comfortable, natural-feeling, rich presentation of virtual image elements, among other virtual or real-world image elements.

[0062] 2 illustrates various possible components of an AR system. In the illustrated embodiment, an AR system user 260 is depicted wearing a head-mounted component 258 featuring a frame 264 structure coupled to a display system 262 positioned in front of the user's eyes. A speaker 266 is coupled to the frame 264 in the depicted configuration and positioned adjacent to the user's ear canal (in one embodiment, another speaker, not shown, is positioned adjacent to the user's other ear canal to provide stereo / adjustable sound control). The display 262 is operably coupled (as shown by 268), such as by wired leads or wireless connectivity, to a local processing and data module 270, which may be mounted in various configurations, such as fixedly attached to the frame 264, fixedly attached to a helmet or hat, removably attached to the torso of the user 260 in a backpack-style configuration, or removably attached to the waist of the user 260 in a belt-style configuration.

[0063] The local processing and data module 270 may include a power-efficient processor or controller and digital memory, such as flash memory, both of which may be utilized to assist in processing, caching, and storing: (1) data captured from sensors, such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, and / or gyroscopes, which may be operatively coupled to the frame 264; and / or (2) data that is potentially obtained and / or processed using the remote processing module 272 and / or remote data repository 274 for passing to the display 262 after processing or retrieval.

[0064] The local processing and data module 270 may be operatively coupled to a remote processing module 272 and a remote data repository 274 (as indicated by 276, 278), such as via one or more wired or wireless communication links, such that these remote modules 272, 274 are operatively coupled to one another and available as resources to the local processing and data module 270. In one embodiment, the remote processing module 272 may include one or more relatively powerful processors or controllers configured to analyze and process the data and / or image information. In one embodiment, the remote data repository 274 may include a relatively large capacity digital data storage facility, which may be available through the Internet or other networking configuration in a "cloud" resource configuration. In one embodiment, all data is stored within the local processing and data module, and all calculations are performed there, allowing for fully autonomous use from any remote module.

[0065] One approach to achieving high accuracy localization may involve the use of electromagnetic fields combined with electromagnetic sensors strategically placed on the user's AR headset, beltpack, and / or other auxiliary devices (e.g., totems, haptic devices, gaming equipment, etc.). Electromagnetic tracking systems typically include at least an electromagnetic field emitter and at least one electromagnetic field sensor. The sensors may measure an electromagnetic field with known dispersion. Based on these measurements, the position and orientation of the field sensor relative to the emitter is determined.

[0066] 3 illustrates an example system diagram of an electromagnetic tracking system developed by organizations such as The Biosense (RTM) division of Johnson & Johnson Corporation, Polhemus (RTM), Inc. (Colchester, Vermont), and may have components similar to those manufactured by companies such as Sixense (RTM) Entertainment, Inc. (Los Gatos, California), and other follower companies. In one or more embodiments, the electromagnetic tracking system includes an electromagnetic field emitter 302 configured to emit a known magnetic field. As shown in FIG. 3, the electromagnetic field emitter 302 may be coupled to a power source 310 (e.g., electrical current, a battery, etc.) to provide power to the electromagnetic field emitter 302.

[0067] In one or more embodiments, the electromagnetic field emitter 302 includes several coils (e.g., at least three coils positioned perpendicular to each other and producing fields in the X, Y, and Z directions) that generate magnetic fields. These magnetic fields are used to establish a coordinate space that allows the system to map the location of the sensor relative to known magnetic fields and help determine the sensor's location and / or orientation. In one or more embodiments, the electromagnetic sensors 304A, 304B, etc. may be attached to one or more real objects. The electromagnetic sensors 304 may include smaller coils in which a current can be induced through the emitted electromagnetic field.

[0068] In general, the components of the electromagnetic field sensor 304 may include small coils or loops, such as a set of three differently oriented (i.e., oriented orthogonally with respect to each other) coils bound together in a small structure such as a cube or other container, positioned / oriented to capture incoming magnetic flux from the magnetic field emitted by the electromagnetic field emitter 302, and by comparing the currents induced through these coils and knowing the relative positioning and orientation of the coils with respect to each other, the relative position and orientation of the sensor with respect to the emitter may be calculated.

[0069] As will be further described with reference to FIG. 4, one or more motion sensors, such as an inertial measurement unit (IMU), may be operatively coupled to each of the electromagnetic field emitters 302 and the electromagnetic field sensors 304 to detect the position and orientation of each component relative to one another and / or to a coordinate system. In one or more embodiments, multiple sensors (possibly including an IMU) may be used in conjunction with the electromagnetic field emitters 302 and the electromagnetic field sensors 304 to detect the position and orientation of each component. In some cases, an electromagnetic tracking system may provide positions in three directions (i.e., X, Y, and Z directions) and also at two or three orientation angles. In some embodiments, IMU measurements may be compared to coil measurements to determine the position and orientation of the sensor. In one or more embodiments, both electromagnetic (EM) data and motion data may be combined to determine the position and orientation, in addition to various other sources of data such as cameras, depth sensors, and other sensors. This information may be transmitted (e.g., via wireless communication, Bluetooth, etc.) to the processing unit 306. In some embodiments, pose (or position and orientation) may be reported at a relatively high refresh rate in conventional systems.

[0070] Traditionally, an electromagnetic emitter is coupled to a relatively stable, large object, such as a table, operating table, wall, or ceiling, and one or more sensors are coupled to a smaller object, such as a medical device, handheld gaming component, or the like. Alternatively, as described below with reference to Figure 4, various features of an electromagnetic tracking system may be employed to produce a configuration in which changes or differences in position and / or orientation between two objects moving in space relative to a more stable global coordinate system may be tracked; in other words, a variation of the electromagnetic tracking system may be utilized to track position and orientation differences between a head-mounted component and a handheld component, while the head pose relative to a global coordinate system (e.g., a room environment local to the user) is otherwise determined, such as by simultaneous localization and mapping (SLAM) techniques using an outward-facing capture camera that may be coupled to the head-mounted component of the system.

[0071] The processing unit 306 may control the electromagnetic field emitter 302 and may also capture data from the various electromagnetic field sensors 304. It should be understood that the various components of the system may be coupled to one another through any electromechanical or wireless / Bluetooth means. The processing unit 306 may also include data regarding the known magnetic field and the coordinate space associated with that magnetic field. This information is then used to detect the position and orientation of the sensor relative to the coordinate space corresponding to the known electromagnetic field. The processing unit 306 may further be coupled to a threshold module 312 (as shown in the illustrated embodiment) or may alternatively or additionally include the threshold module 312 as a subcomponent. In some implementations, the threshold module 312 may be configured to dynamically adjust a threshold against which a difference value calculated by the processing unit 306 may be compared to determine the attitude of the handheld controller, as will be described in more detail below.

[0072] One advantage of electromagnetic tracking systems is that they produce very accurate tracking results with minimal latency and high resolution. In addition, electromagnetic tracking systems do not necessarily rely on optical trackers, and sensors / objects that are not within the user's line of sight can be easily tracked. It should be appreciated that the strength of the electromagnetic field v drops off as a cubic function of the distance r from the coil transmitter (e.g., the electromagnetic field emitter 302). Thus, the processing unit 306 may be configured to perform certain functions, such as an algorithm that predicts distance based on the measured strength, to determine the position and orientation of the sensor / object at variable distances away from the electromagnetic field emitter 302.

[0073] Given the rapid drop in strength of the electromagnetic field as one moves further away from the electromagnetic emitter, best results in terms of accuracy, efficiency, and low latency may be achieved at closer distances. In a typical electromagnetic tracking system, the electromagnetic field emitter is powered by an electric current (e.g., a plug-in power supply) and has a sensor located within a 20-foot radius away from the electromagnetic field emitter. A shorter radius between the sensor and the field emitter may be more desirable in many applications, including AR applications.

[0074] FIG. 4 illustrates an example of how an electromagnetic tracking system may be incorporated with an AR system and an electromagnetic field emitter 402 may be incorporated as part of a handheld controller 406. In one or more embodiments, the handheld controller may be a totem to be used in a gaming scenario. In other embodiments, the handheld controller may be a haptic device. In still other embodiments, the electromagnetic field emitter may simply be incorporated as part of a belt pack 470. The handheld controller 406 may include a battery 410 or other power source that powers the electromagnetic field emitter 402. It should be appreciated that the electromagnetic field emitter 402 may also include and / or be coupled to an IMU 450 or other movement sensor configured to assist in determining the positioning and / or orientation of the electromagnetic field emitter 402 relative to other components. This may be important, especially when the field emitter 402 and the sensor 404 are both mobile. Locating the electromagnetic field emitter 402 in the handheld controller rather than the beltpack, as shown in the embodiment of FIG. 4, ensures that the electromagnetic field emitter is not competing for resources in the beltpack, but rather uses its own battery source in the handheld controller 406.

[0075] In some embodiments, the electromagnetic sensor 404 may be positioned relative to the AR headset 458, such as being placed on one or more locations on the AR headset 458 and / or on one or more locations on the belt pack 470. The sensors placed on the AR headset 458 may be placed in addition to other sensing devices, such as one or more IMUs or additional flux capture coils 408. For example, as shown in FIG. 4, the sensors 404, 408 may be placed on both sides of the headset 458. Because these sensors are engineered to be fairly small (and therefore may not be very sensitive in some cases), having multiple sensors may improve efficiency and accuracy. In one or more embodiments, one or more sensors may also be placed on the belt pack 470 or any other part of the user's body. The sensors 404, 408 may communicate wirelessly or through Bluetooth to a computing device that determines the attitude and orientation of the sensor (and the AR headset to which it is attached). In one or more embodiments, the computing device may reside on the belt pack 470. In other embodiments, the computing device may reside in the headset itself, or even in the handheld controller 406. The computing device may in turn include a mapping database 430 (e.g., a passable world model, coordinate space, etc.) for detecting poses and determining coordinates of real and virtual objects, and may further connect to cloud resources and passable world models in one or more embodiments.

[0076] In many instances, conventional electromagnetic emitters may be too bulky for an AR device. Thus, the electromagnetic field emitter may be engineered to be compact using smaller coils compared to conventional systems. However, a shorter radius (e.g., about 3-3.5 feet) between the electromagnetic sensor 404 and the electromagnetic field emitter 402 may reduce power consumption compared to conventional systems, provided that the strength of the electromagnetic field decreases as a cubic function of distance away from the field emitter. This aspect may also be utilized to extend the life of the battery 410, which may power the handheld controller 406 and the electromagnetic field emitter 402 in one or more embodiments. Alternatively, in other embodiments, this aspect may be utilized to reduce the size of the coils that generate the magnetic field in the electromagnetic field emitter 402. However, to obtain the same strength of the magnetic field, the power may need to be increased. This allows for a smaller electromagnetic field emitter unit 402 that may fit compactly in the handheld controller 406.

[0077] Some other modifications may be made when using an electromagnetic tracking system for an AR device, which may require a more efficient attitude reporting rate than other applications. For example, movement-based or IMU-based attitude tracking may be employed. In many cases, the improved stability of the IMU may lead to improved efficiency of the attitude detection process. IMUs may be engineered such that they remain stable for up to 50-100 milliseconds. It should be understood that some embodiments may utilize an external attitude estimator module, which may allow attitude updates to be reported at a rate of 10-20 Hz (i.e., the IMU may drift over time). By keeping the IMU stable at a reasonable rate, the rate of attitude updates may be reduced to 10-20 Hz (compared to higher frequencies in conventional systems).

[0078] If the electromagnetic tracking system could be activated at a 10% duty cycle (e.g., only pinging for ground truth every 100 ms), this would be an additional way to save power in the AR system. This would mean that the electromagnetic tracking system would wake up and generate a pose estimate every 10 ms every 100 ms. This would translate directly into power consumption savings, which in turn could impact the size, battery life, and cost of the AR device. In one or more embodiments, this reduction in duty cycle can be strategically exploited by providing two handheld controllers (not shown) instead of only one. For example, a user may play a game that requires two totems, etc. Or, in a multi-user game, two users may have their own totem / handheld controller to play the game. When two controllers (e.g., symmetrical controllers per hand) are used instead of one, the controllers may operate at an offset duty cycle. The same concept can also be applied to controllers utilized by, for example, two different users playing a multiplayer game.

[0079] FIG. 5 illustrates the hemisphere ambiguity problem that may be present in an electromagnetic tracking system such as that described herein. For six degrees of freedom (DOF) tracking, a handheld controller 502 (labeled "TX"), which may also be referred to as a handheld device or totem, may generate EM signals modulated on three separate frequencies, one for each of the axes X, Y, and Z. A wearable 504 (labeled "RX"), which may be implemented as an AR headset, has an EM receiving component receiving the EM signals on the X, Y, and Z frequencies. The position and orientation (i.e., pose) of the handheld controller can be derived based on the characteristics of the received EM signal. However, due to the symmetrical nature of the EM signal, it may not be possible to determine the hemisphere in which the handheld controller resides (e.g., front hemisphere 506A or back hemisphere 506B) without an additional frame of reference. That is, the same EM value can be obtained at the wearable for two diametrically opposed totem poses (one for each hemisphere) with a chosen plane passing through the center of the sphere that divides the two hemispheres. This is illustrated in FIG. 5, which shows that either pose can be valid for a single snapshot of the EM signal received by the wearable 504. However, when the handheld controller 502 is moved, the tracking algorithm will typically encounter errors if an inappropriate hemisphere is chosen due to inconsistencies in the data from the various sensors. The hemisphere ambiguity arises, in part, due to the fact that when a 6DOF tracking session begins, the initial EM totem data does not have a clear absolute position. Instead, it provides a relative distance that can be interpreted as one of two positions in a 3D volume that is divided into two equal spheres with the wearable (e.g., an AR headset mounted on the user's head) centered between the two hemispheres. Thus, embodiments of the present disclosure provide methods and systems that resolve hemisphere ambiguity to enable successful tracking of the actual position of a handheld controller.

[0080] According to some embodiments of the present disclosure, a method that can be used to solve the hemisphere ambiguity problem includes starting two processing stacks when the tracking pipeline is started, one associated with the reference pose of the front hemisphere and the other associated with the reference pose of the back hemisphere. While both processing stacks are running, their outputs are checked, and the processing stack with an incorrect assumption of the hemisphere will soon have a problem tracking the actual movement of the handheld controller. At that point, the erroneous processing stack can be stopped (e.g., destroyed or terminated) and the remaining instances are allowed to continue processing. In some embodiments, a check is performed periodically to calculate the divergence between the estimated pose (a fusion of electromagnetic data and position data as determined by a movement sensor such as an IMU) and each reference pose (e.g., the reference pose of hemisphere 506A and the reference pose of hemisphere 506B). If any calculated divergence exceeds a threshold, the corresponding instance of the tracking module for the particular estimated pose is stopped. Additional descriptions related to these methods are provided more fully with respect to the descriptions provided below.

[0081] 6 illustrates a method 600 for resolving hemispheric ambiguity in a system or device including one or more sensors. The system may be, among other possibilities, an electromagnetic tracking system, such as an AR device, or an optical device, or any system that supports the emission and reception of electromagnetic or magnetic signals. One or more steps of the method 600 may be omitted during the implementation of the method 600, and the steps need not be performed in the order shown. One or more steps of the method 600 may be performed by the processing unit 306, the local processing and data module 270, and / or the remote processing module 272, among other possibilities.

[0082] In step 602, one or more magnetic fields may be emitted by a magnetic field emitter located on the handheld controller. The magnetic field emitter may generate a magnetic field with each coil generating a magnetic field in one direction (e.g., X, Y, or Z). The magnetic fields may be generated with an arbitrary waveform. In one or more embodiments, each axis may oscillate at a slightly different frequency. Although magnetic fields are discussed in some embodiments, this discussion of magnetic fields is not intended to limit the embodiments of the present disclosure, and other fields, including electric and electromagnetic fields, are included within the scope of the present disclosure.

[0083] In step 604, one or more magnetic fields may be detected by one or more sensors positioned in a headset or beltpack. In some embodiments, a coordinate space corresponding to the magnetic or electromagnetic field may be determined. For example, a coordinate space around an emitter (e.g., a handheld controller) may be determined based on the detected magnetic fields. In some embodiments, the behavior of a coil in the sensor (which may be attached to a known object) may be detected. For example, a current induced in the coil may be calculated. In other embodiments, the rotation of the coil, or any other quantifiable behavior, may be tracked and measured.

[0084] In step 606A, a first process stack is initiated, causing the first process stack to run for a first time interval. Running the first process stack may include performing one or more of steps 608A, 612A, and 614A. The first process stack may run on the local processing and data module 270, the remote processing module 272, or on a server remote to the device. In step 606B, a second process stack is initiated, causing the second process stack to run for a second time interval. Running the second process stack may include performing one or more of steps 608B, 612B, and 614B. The second process stack may run on the local processing and data module 270, the remote processing module 272, or on a server remote to the device. The first time interval may occur simultaneously, in parallel, or non-concurrently (i.e., not overlapping) with the second time interval. In some embodiments, the first process stack and the second process stack are initialized at the same time. In some embodiments, the first process stack and the second process stack are initialized sequentially.

[0085] In step 608A, the position and orientation of the handheld controller in the first hemisphere is determined based on the detected magnetic field. In step 608B, the position and orientation of the handheld controller in the second hemisphere is determined based on the detected magnetic field. The first and second hemispheres may be diametrically opposed and, in some embodiments, may correspond to the front and back hemispheres relative to the headset. However, the first and second hemispheres may be defined in any of a variety of configurations, including front / back, up / down, left / right. In some embodiments, the interface between the hemispheres is defined by a plane with a normal pointing 10, 20, 30, 40, 50, 60, 70, or 80 degrees downward from the forward direction of the headset. In one example, the controller 406 may reference a mapping table that correlates the behavior of the coils in the sensor to various positions or orientations. Based on these calculations, the position in the coordinate space may be determined in addition to the orientation of the sensor.

[0086] At step 610, movement data is detected by a movement sensor, such as an IMU sensor. The movement data may correspond to (e.g., may be indicative of) movement of the handheld controller and / or headset. The movement data may include linear acceleration, angular velocity, and / or orientation data, among other possibilities. In some embodiments, the movement data may be referred to as IMU data when detected by an IMU sensor. In some cases, the IMU sensor may be located in the handheld controller or the headset. For example, the IMU sensor may be mounted in the handheld controller or the headset. In some embodiments, the movement data is detected by two different IMUs, one in the handheld controller and another in the headset. In such embodiments, the movement data may be used to determine the relative movement of the handheld controller with respect to the headset.

[0087] In steps 612A, 612B, one or more performance statistics associated with the first and second processing stacks, respectively, are analyzed. In the illustrated embodiment, a discrepancy is calculated between the position and orientation determined from steps 608A, 608B and the relative movement of the handheld controller relative to the headset as determined using the movement data. In some examples, multiple poses may be determined for each processing stack and compared to the movement of the handheld controller as indicated by the movement data. For example, if the movement data indicates that the handheld controller is not moving (relative to the headset) while the determined pose (determined using the detected magnetic data) indicates that the handheld controller is moving, a discrepancy with a high value may be calculated.

[0088] In steps 614A, 614B, the differences calculated in steps 612A, 612B are compared to a predefined threshold. If any difference exceeds the threshold, the corresponding processing stack is stopped or terminated and the other processing stack is allowed to continue processing (e.g., continue determining the orientation of the handheld controller in the respective hemisphere based on additional detected magnetic fields). In some embodiments, the threshold acts as an acceptable accuracy threshold such that if any difference is below the threshold, the corresponding processing stack is allowed to continue processing and the other processing stack is stopped. In some alternative embodiments, if the difference exceeds the threshold but falls below a second threshold, indicating a relatively small value of difference, which may be referred to as an intervening difference, portions of the process may be repeated. For example, if the intervening difference is calculated in 612A, portions 608A, 610, 612A, and 614A may be repeated until the difference exceeds the second threshold, which would result in the termination of the first hemisphere processing. In some embodiments, the threshold may be dynamic, as opposed to a fixed value, allowing for greater or lesser variance depending on the estimated posture over time. A non-limiting example of a dynamic threshold includes determining a large change in estimated posture over a short period of time, and then the controller 306 (or an intervening decision block) may temporarily allow a threshold with a greater variance, or a number of samples that must remain below the threshold within a given time interval may be adjusted.

[0089] FIG. 7 illustrates a method 700 for resolving hemispheric ambiguity in a system or device including one or more sensors. The system may be, among other possibilities, an electromagnetic tracking system or optical device, such as an AR device, or any system that supports the emission and reception of magnetic signals. One or more steps of the method 700 may be omitted during the implementation of the method 700, and the steps need not be performed in the order shown. One or more steps of the method 700 may be performed by the controller 306, the local processing and data module 270, and / or the remote processing module 272, among other possibilities. One or more steps of the method 700 may correspond to one or more steps of the method 600.

[0090] In step 702, one or more magnetic fields may be emitted by a magnetic field emitter located in the handheld controller. In step 704, one or more magnetic fields may be detected by one or more sensors located in the headset or beltpack. In step 706A, a first processing stack is initiated, causing the first processing stack to run for a first time interval. In step 706B, a second processing stack is initiated, causing the second processing stack to run for a second time interval. In step 708A, a position and orientation of the handheld controller in a first hemisphere is determined based on the detected magnetic field. In step 708B, a position and orientation of the handheld controller in a second hemisphere is determined based on the detected magnetic field. The first and second hemispheres may be diametrically opposed and, in some embodiments, may correspond to the front and back hemispheres relative to the headset.

[0091] In step 710, movement data is detected by a movement sensor, similar to that described with reference to step 610. In steps 712A, 712B, one or more performance statistics of the first processing stack and the second processing stack, respectively, are analyzed. In the illustrated embodiment, a difference is calculated between the position and orientation determined from steps 708A, 708B and the relative movement of the handheld controller with respect to the headset, as determined using the movement data.

[0092] In step 714, the differences calculated in steps 712A, 712B are compared to each other. If the difference calculated in step 712A (i.e., the "first difference") exceeds the difference calculated in step 712B (i.e., the "second difference"), the first processing stack is stopped and the second processing stack is allowed to continue processing (e.g., continue determining the orientation of the handheld controller in the respective hemisphere based on the additional detected magnetic fields). On the other hand, if the first difference is less than the second difference, the second processing stack is stopped and the first processing stack is allowed to continue processing.

[0093] In some embodiments described herein, the term "EM tracking" may refer to the use of EM waves to determine the location of an object. In some embodiments described herein, the term "pose" may refer to the (x, y, z) definition of an object position relative to a world pose. In some embodiments described herein, the term "world pose" may refer to the position of an object relative to a reference point of (0, 0, 0). In some embodiments described herein, the term "head pose" may refer to the position of the head in the world. In some embodiments described herein, the term "totem pose" may refer to the position of the totem / handheld controller in the world. In some embodiments, the totem pose is calculated by adding the fusion pose to the head pose. In some embodiments described herein, the term "IMU pose" corresponds to the movement of the totem as measured by the IMU. In some embodiments described herein, the term "EM pose" corresponds to the movement of the totem as measured by EM tracking. In some embodiments described herein, the term "fused attitude" is calculated by taking the IMU attitude into account in the EM attitude, in addition to some additional filtering. In some embodiments, simultaneous attitudes are run through a tracking algorithm to obtain two versions of all of the totem attitudes, i.e., EM, IMU, and fused. Based on the method employed, the appropriate attitude rate (change in position) is compared to the expected change assumed depending on the observed conditions.

[0094] In some implementations, the totem pose hemisphere is established when no totem motion is detected in parallel with the changing head pose. The approach relies on the absence of totem motion detected by the totem IMU and also on the head pose changing at the same instant. The assumption is that if the totem pose were to be initialized in the wrong hemisphere, a change in head pose would result in a change in the totem pose, everything in the world frame, or vice versa, i.e., a totem pose initialized in the right hemisphere should not see a change in its pose even if the head pose changes.

[0095] In some embodiments, the totem pose is calculated by combining the EM sensor measured distances between the EM emitter and receiver and adding this to the head pose after passing the data through the necessary transformations so that the reference frames are aligned. If the totem is not moving and the head moves away from the totem, the EM distances should be changed by the same absolute value to ensure that the calculated totem pose remains the same. In some embodiments, the method provides an algebraic solution where an inequality in the calculated totem pose is an indicator of an incorrect hemisphere initialization. Multiple samples with the totem in a stationary state can be collected and if all of the samples result in a totem pose that is moving in the world frame, the hemisphere chosen when the head is moving can be discarded as being incorrect.

[0096] Figure 8 illustrates the results of a simulation showing the expected totem pose when the head is moving and the totem is stationary. The simulated data is supported by experimental data shown with reference to Figures 9-12, proving an embodiment of the present disclosure that is able to determine that the actual totem pose will not change in the world reference frame with the head moving and the totem stationary. The fused totem pose is shown in this example because EM totem poses tend to have significant noise and the fused pose includes processing to take into account the movement data and necessary filtering.

[0097] With reference to FIG. 8, the totem pose is unchanged in Examples 0 to 4 and is illustrated as position 8 in the world frame. In Example 0, the head pose is static at position 6 in the world frame and the afterimage totem pose is at position 4 in the world frame. With reference to Example 1, the head pose shifts from position 6 to approximately position 3 in the world frame. In response to this change in head pose, the actual totem position is static at position 8, but the afterimage totem pose shifts from position 4 to position 2 in the world frame. With reference to Example 2, the head pose shifts from position 6 to approximately position 4 and again the actual totem position is static while the afterimage totem pose shifts from position 4 to approximately position 0. With reference to Example 3, the head pose shifts from position 6 to position 8 and the afterimage totem pose also moves to position 8. With reference to Example 4, the head pose shifts from position 6 to approximately position 9, while the afterimage totem pose shifts from position 4 to approximately position 10. As can be observed in Examples 0 through 4, in some embodiments, the relative distance in the world frame between the head pose and each of the totem poses (actual and afterimage) can be the same over a wide range of head pose positions.

[0098] Figure 9 illustrates experimental data when the head is moving and the totem is stationary with the totem initialized in the correct hemisphere. As shown in Figure 9, the totem fusion posture remains largely unchanged despite head movement when the totem is initialized in the correct hemisphere, as theorized by the actual totem posture, as demonstrated by the simulated data in Figure 8, which showed no movement of the actual totem posture across Examples 0 to 4.

[0099] FIG. 10 illustrates experimental totem movement data when the totem is stationary, with the totem initialized in the correct hemisphere. The data shown in FIG. 10 may correspond to the attitude data shown in FIG. 9. The upper plot of FIG. 10 shows measurements of rotation (in degrees) from the IMU gyroscope as a function of time, and the lower plot of FIG. 10 shows measurements of rotation (in m / sec) from the IMU accelerometer as a function of time. 2 10 shows a measurement of acceleration (in degrees). As shown in FIG. 10, the IMU gyroscope data has noise around a zero degree rotation value. The IMU acceleration data shows no translational changes on the X, Y, and Z axes. Since the change in acceleration is measured relative to the acceleration of gravity, a constant offset on the Y and Z axes is expected even without totem motion.

[0100] FIG. 11 illustrates experimental data when the head is moving and the totem is stationary with the totem initialized in the inappropriate hemisphere. The experimental data shows that the totem posture changes with head movement even when the totem is stationary. It is clear that the totem fusion posture is an expanded version of the head posture when initialized in the inappropriate hemisphere as theorized by the simulated data of "afterimage totem posture" in FIG. 8, which showed significant movement of afterimage totem posture across Examples 0 to 4.

[0101] FIG. 12 illustrates experimental totem translation data without the totem motion corresponding to the attitude data displayed in FIG. 11. The top plot of FIG. 12 shows measurements of rotation (in degrees) from the IMU gyroscope as a function of time, and the bottom plot of FIG. 12 shows measurements of rotation (in m / s) from the IMU accelerometer as a function of time. 2 The IMU gyroscope data shows very little noise around the zero degree rotation value, and the IMU acceleration data shows no translational changes on the X, Y, and Z axes.

[0102] FIG. 13 illustrates a simplified computer system 1300 according to an embodiment described herein. A computer system 1300 as illustrated in FIG. 13 may be incorporated into a device described herein. FIG. 13 provides a schematic diagram of one embodiment of a computer system 1300 that may perform some or all of the steps of the methods provided by various embodiments. It should be noted that FIG. 13 is only meant to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. FIG. 13 thus broadly illustrates how individual system elements may be implemented in a relatively separated or relatively more integrated manner.

[0103] Computer system 1300 is shown to include hardware elements that may be electrically coupled or otherwise in communication via a bus 1305, as appropriate. The hardware elements may include one or more processors 1310, including one or more general purpose processors and / or one or more special purpose processors, such as, but not limited to, digital signal processing chips, graphic acceleration processors, and / or the like, one or more input devices 1315, which may include, but are not limited to, a mouse, a keyboard, a camera, and / or the like, and one or more output devices 1320, which may include, but are not limited to, a display device, a printer, and / or the like.

[0104] The computer system 1300 may further include and / or communicate with one or more non-transitory storage devices 1325, which may include, but are not limited to, local and / or network accessible storage devices, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices such as random access memory ("RAM"), and / or read-only memory ("ROM"), which may be programmable, flash updatable, and / or the like. Such storage devices may be configured to implement any suitable data store, including, but not limited to, various file systems, database structures, and / or the like.

[0105] The computer system 1300 may also include a communication subsystem 1319, which may include, but is not limited to, a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset, such as, but not limited to, a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication facility, and / or the like. The communication subsystem 1319 may include one or more input and / or output communication interfaces, allowing data to be exchanged with a network, such as a network described below to give an example, i.e., another computer system, a television, and / or any other device described herein. Depending on the desired functionality and / or other implementation concerns, a portable electronic device or similar device may communicate images and / or other information via the communication subsystem 1319. In other embodiments, a portable electronic device, e.g., a first electronic device, may be incorporated into the computer system 1300, e.g., an electronic device, as an input device 1315. In some embodiments, the computer system 1300 will further include a working memory 1335, which may include a RAM or ROM device as described above.

[0106] Computer system 1300 may also include software elements, shown as currently residing in working memory 1335, including an operating system 1340, device drivers, executable libraries, and / or other code, such as one or more application programs 1345, that may include computer programs provided by various embodiments and / or that may be designed to implement methods and / or configure systems provided by other embodiments as described herein. Merely by way of example, one or more procedures described with respect to the methods discussed above may be implemented as code and / or instructions executable by a computer and / or a processor within a computer, and in certain aspects such code and / or instructions may then be used to configure and / or adapt a general purpose computer or other device to perform one or more operations in accordance with the methods described.

[0107] A set of these instructions and / or code may be stored on a non-transitory computer-readable storage medium, such as storage device 1325 described above. In some cases, the storage medium may be incorporated within a computer system, such as computer system 1300. In other embodiments, the storage medium may be separate from the computer system, e.g., a removable medium such as a compact disc, and / or may be provided within an installation package such that the storage medium may be used to program, configure, and / or adapt a general-purpose computer with the instructions / code stored thereon. These instructions may take the form of executable code that is executable by computer system 1300, and / or may take the form of source and / or installable code that, upon compilation and / or installation on computer system 1300 using, for example, any of a variety of commonly available compilers, installation programs, compression / decompression utilities, and the like, then takes the form of executable code.

[0108] It will be apparent to those skilled in the art that substantial variations can be made in accordance with specific requirements. For example, customized hardware could also be used and / or particular elements could be implemented in hardware, software, including portable software such as applets, or both. Furthermore, connection to other computing devices, such as network input / output devices, could also be employed.

[0109] As noted above, in one aspect, some embodiments may employ a computer system, such as computer system 1300, to perform methods according to various embodiments of the present technology. According to one set of embodiments, some or all of the procedures of such methods are performed by computer system 1300 in response to processor 1310 executing one or more sequences of one or more instructions, which may be embedded in operating system 1340, and / or other code, such as application program 1345, contained in working memory 1335. Such instructions may be read into working memory 1335 from another computer-readable medium, such as one or more of storage devices 1325. Merely by way of example, execution of a sequence of instructions contained in working memory 1335 may cause processor 1310 to perform one or more procedures of the methods described herein. Additionally or alternatively, some of the methods described herein may be performed through specialized hardware.

[0110] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any medium that participates in providing data that causes a machine to operate in a tangible manner. In an embodiment implemented using computer system 1300, various computer-readable media may participate in providing instructions / code to processor 1310 for execution and / or may be used to store and / or carry such instructions / code. In many implementations, computer-readable media are physical and / or tangible storage media. Such media may take the form of non-volatile or volatile media. Non-volatile media include, for example, optical and / or magnetic disks, such as storage device 1325. Volatile media include dynamic memory, such as, but not limited to, working memory 1335.

[0111] Common forms of physical and / or tangible computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read instructions and / or code.

[0112] Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor 1310 for execution. By way of example only, the instructions may initially be carried on a magnetic and / or optical disk of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions as signals over a transmission medium to be received and / or executed by the computer system 1300.

[0113] The communications subsystem 1319 and / or its components generally receive signals and the bus 1305 may then convey the signals and / or the data, instructions, etc. carried by the signals to the working memory 1335, from which the processor 1310 retrieves and executes the instructions. The instructions received by the working memory 1335 may optionally be stored on a non-transitory storage device 1325 either before or after execution by the processor 1310.

[0114] The methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components, as appropriate. For example, in alternative configurations, the method may be performed in a different order than described, and / or various steps may be added, omitted, and / or combined. Also, features described with respect to one configuration may be combined in various other configurations. Different aspects and elements of the configurations may be combined in a similar manner. Also, technology evolves, and thus many of the elements are examples and do not limit the scope of the disclosure or claims.

[0115] Specific details are given in the description to provide a thorough understanding of the example configurations, including the implementation. However, the configurations may be practiced without these specific details. For example, well-known circuits, processes, algorithms, structures, and techniques are shown without unnecessary detail to avoid obscuring the configurations. This description provides only example configurations and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the configurations will provide those skilled in the art with an effective description for implementing the described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the present disclosure.

[0116] Also, the configurations may be described as processes, depicted as schematic flow charts or block diagrams. Although the operations may be described as sequential processes, many of the operations may be performed in parallel or simultaneously. In addition, the order of the operations may be rearranged. The processes may have additional steps not included in the figures. Furthermore, embodiments of the method may be implemented by hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments for performing the necessary tasks may be stored in a non-transitory computer-readable medium, such as a storage medium. A processor may perform the tasks described.

[0117] Although several example configurations have been described, various modifications, alternative constructions, and equivalents may be used without departing from the spirit of the disclosure. For example, the above elements may be components of a larger system, and other rules may take precedence over or otherwise modify the application of the technology. Also, some steps may take place before, during, or after the above elements are discussed. Thus, the above description is not intended to be binding on the scope of the claims.

[0118] As used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to a "user" includes one or more such users, and reference to a "processor" includes reference to one or more processors and equivalents thereof known to those skilled in the art, and so forth.

[0119] Additionally, the words "comprise," "comprising," "contains," "containing," "include," "including," and "includes," when used in this specification and the claims that follow, are intended to specify the presence of stated features, integers, components, or steps, but they do not exclude the presence or addition of one or more other features, integers, components, steps, acts, or groups.

[0120] It is also to be understood that the examples and embodiments described herein are for illustrative purposes only, and that various modifications or changes in light thereof will be suggested to those skilled in the art and are to be included within the spirit and scope of the present application and the appended claims.

Claims

1. 1. A method for resolving hemisphere ambiguity, the method comprising: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields with one or more sensors positioned relative to the headset; Detecting movement data corresponding to the handheld controller or the headset; executing a first process stack during a first time interval, the first process stack comprising: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference between said first position and said first orientation and said movement data; executing a second process stack during a second time interval, the second process stack comprising: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference between said second position and said second orientation and said movement data; and enabling the first process stack or the second process stack to continue execution based on one or both of the first difference and the second difference; A method comprising:

2. The method of claim 1 , further comprising: aborting either the first process stack or the second process stack based on one or both of the first difference and the second difference.

3. comparing the first difference to a threshold; determining that the first difference exceeds the threshold; in response to determining that the first difference exceeds the threshold, Aborting the first processing stack; allowing the second processing stack to continue; and The method of claim 2 , further comprising:

4. comparing the first difference with the second difference; and determining that the first difference exceeds the second difference; in response to determining that the first difference exceeds the second difference, Aborting the first processing stack; allowing the second processing stack to continue; and The method of claim 2 , further comprising:

5. The method of claim 1 , wherein the movement data corresponds to the handheld controller, and the movement data is detected by an inertial measurement unit (IMU) located within the handheld controller.

6. The method of claim 1 , wherein the movement data corresponds to the headset, and the movement data is detected by an inertial measurement unit (IMU) located within the headset.

7. The method of claim 1 , wherein the movement data is detected during one or both of the first time interval and the second time interval.

8. The method of claim 1 , wherein the first time interval occurs in parallel or simultaneously with the second time interval.

9. The first hemisphere is a front hemisphere relative to the headset, and the second hemisphere is The method of claim 1 , wherein the headset is a back hemisphere.

10. 1. A system comprising: a handheld controller comprising a magnetic field emitter configured to emit one or more magnetic fields; a headset comprising one or more magnetic field sensors configured to detect the one or more magnetic fields; a movement sensor configured to detect movement data corresponding to the handheld controller or the headset; one or more processors, the one or more processors comprising: executing a first process stack during a first time interval, the first process stack comprising: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference between said first position and said first orientation and said movement data; executing a second process stack during a second time interval, the second process stack comprising: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference between said second position and said second orientation and said movement data; and enabling the first process stack or the second process stack to continue execution based on one or both of the first difference and the second difference; one or more processors configured to perform operations including: A system comprising:

11. The operation further comprises: The system of claim 10 , further comprising aborting either the first process stack or the second process stack based on one or both of the first difference and the second difference.

12. The system of claim 10 , wherein the movement data corresponds to the handheld controller, and the movement data is detected by an inertial measurement unit (IMU) located within the handheld controller.

13. The system of claim 10 , wherein the first time interval occurs concurrently or simultaneously with the second time interval.

14. A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: emitting one or more magnetic fields at a handheld controller; detecting the one or more magnetic fields with one or more sensors positioned relative to the headset; Detecting movement data corresponding to the handheld controller or the headset; executing a first process stack during a first time interval, the first process stack comprising: determining a first position and a first orientation of the handheld controller within a first hemisphere relative to the headset based on the detected one or more magnetic fields; calculating a first difference between said first position and said first orientation and said movement data; executing a second process stack during a second time interval, the second process stack comprising: determining a second position and a second orientation of the handheld controller within a second hemisphere relative to the headset based on the detected one or more magnetic fields, the second hemisphere diametrically opposite the first hemisphere; and calculating a second difference between said second position and said second orientation and said movement data; and enabling the first process stack or the second process stack to continue execution based on one or both of the first difference and the second difference; A non-transitory computer readable medium for performing operations including:

15. The operation further comprises:

15. The non-transitory computer readable medium of claim 14, further comprising aborting either the first processing stack or the second processing stack based on one or both of the first discrepancy and the second discrepancy.

16. The operation further comprises: comparing the first difference to a threshold; determining that the first difference exceeds the threshold; in response to determining that the first difference exceeds the threshold, Aborting the first processing stack; allowing the second processing stack to continue; and 20. The non-transitory computer readable medium of claim 15, comprising:

17. The operation further comprises: comparing the first difference with the second difference; and determining that the first difference exceeds the second difference; in response to determining that the first difference exceeds the second difference, Aborting the first processing stack; allowing the second processing stack to continue; and 20. The non-transitory computer readable medium of claim 15, comprising:

Images