Electromagnetic Tracking Using Augmented Reality Systems

Abstract

To provide favorable electromagnetic tracking with augmented reality systems.SOLUTION: Head-mounted augmented reality (AR) devices can track the pose of a wearer's head to provide a three-dimensional virtual representation of objects in the wearer's environment. An electromagnetic (EM) tracking system can track the head or body pose. A handheld user input device can include an EM emitter that generates an EM field, and the head-mounted AR device can include an EM sensor that senses the EM field. EM information from the sensor can be analyzed to determine the location and / or orientation of the sensor and thereby the wearer's pose. The EM emitter and sensor may utilize time division multiplexing (TDM) or dynamic frequency tuning to operate at multiple frequencies.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Patent Application No. 62 / 328,003, filed April 26, 2016, entitled SYSTEMS AND METHODS FOR AUGMENTED REALITY, and U.S. Patent Application No. 62 / 479,111, filed March 30, 2017, entitled ELECTROMAGNETIC TRACKING WITH AUGMENTED REALITY SYSTEMS, all of which are incorporated herein by reference in their entireties.

[0002] The present disclosure relates to systems and methods for determining the position or orientation of one or more objects in the context of an augmented reality system. [Background technology]

[0003] Modern computing and display technology has facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images, or portions thereof, are presented to a user in a manner that appears or can be perceived as real. Virtual reality, or "VR," scenarios typically involve the presentation of digital or virtual image information without transparency to other actual, real-world visual input. Augmented reality, or "AR," scenarios typically involve the presentation of digital or virtual image information as an extension of a user's visualization of the real world around them. Summary of the Invention [Means for solving the problem]

[0004] Head-mounted augmented reality (AR) devices can track the pose of a wearer's head (or other body part) so as to provide a three-dimensional virtual representation of objects in the wearer's environment. Electromagnetic (EM) tracking system embodiments can be used to track head pose or body gestures. For example, a handheld user input device can include an EM emitter, and a head-mounted AR device can include an EM sensor. In some implementations, the EM emitter generates an EM field that can be sensed by the EM sensor. EM information from the sensor can be analyzed to determine the location and / or orientation of the sensor and, thereby, the wearer's head pose. The EM emitter and sensor may utilize time-division multiplexing (TDM) or dynamic frequency adjustment, which allows the tracking system to operate at multiple frequencies. Voltage gain control can be implemented in the transmitter rather than the sensor, allowing for smaller and lighter sensor designs. The EM sensor can implement noise cancellation to reduce the level of EM interference generated by nearby audio speakers.

[0005] An embodiment of a head-mounted display system includes a display positionable in front of a wearer's eyes, an electromagnetic (EM) field emitter configured to generate a magnetic field having a frequency, an EM sensor configured to sense the magnetic field at that frequency, and a processor programmed to receive a signal indicative of the sensed magnetic field from the EM sensor, analyze the received signal, and determine a position or orientation of the EM sensor.

[0006] An embodiment of an electromagnetic (EM) tracking system includes an EM field emitter comprising a first transmitter coil configured to generate a first magnetic field having a first frequency, a second transmitter coil configured to generate a second magnetic field having a second frequency, and a third transmitter coil configured to generate a third magnetic field having a third frequency, wherein the EM field emitter comprises a first time division multiplexing (TDM) circuit configured to switch power between the first transmitter coil, the second transmitter coil, and the third transmitter coil. A head-mounted augmented reality display device can include an embodiment of the EM tracking system.

[0007] An embodiment of an electromagnetic (EM) tracking system includes an EM field emitter including an automatic gain control (AGC) circuit and a transmitter coil, and an EM sensor without the AGC circuit, the EM sensor including a sensor coil. A head-mounted augmented reality display device can include an embodiment of the EM tracking system.

[0008] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter. The present invention provides, for example, the following. (Item 1) 1. A belt pack having a control and quick release module, comprising: a first outer housing component and a second outer housing component, wherein the first outer housing component and the second outer housing component are coupled together; one or more buttons located on the first outer housing, the one or more buttons being overlaid over an upper printed circuit board; a first end connected to the first outer housing; a second end connected to the second housing; a local processing and data module positioned between the first outer housing and the second outer housing; an electrical conductor positioned between the first outer housing and the second outer housing, the electrical conductor connecting the first end, the local processing and data module, and the second end; A belt pack equipped with (Item 2) Item 1. The belt pack of item 1, wherein the first outer housing component and the second outer housing component are coupled together with a magnetic coupling arrangement for improving mechanical latching. (Item 3) Item 1, wherein the one or more buttons comprise at least one of a circular button or a triangular button. (Item 4) Item 10. The belt pack of item 1, further comprising a display positioned between the first outer housing and the second outer housing, the electrical conductors further extending through the display. (Item 5) Item 1, wherein the upper printed circuit board is overlaid on top of the female contact pin array. (Item 6) Item 1, wherein the second housing covers a lower printed circuit board, and the lower printed circuit board is overlaid on top of the male contact pin array. (Item 7) 7. The belt pack of claim 6, wherein the male contact pin array mates with the female contact pin array. (Item 8) Item 8. The belt pack of item 7, wherein at least one pin of the male contact pin array or the female contact pin array is configured to be spring loaded so that it can be depressed along a longitudinal axis of each pin. (Item 9) Item 9. The belt pack of item 8, wherein the pins may comprise conductive material plated onto the male contact pins, and the width of the conductive material may be at least 25 μm. (Item 10) Item 8. The belt pack of item 7, wherein the male contact pin array comprises 46 male pins and the female contact pin array comprises 46 female pins. (Item 11) 8. The belt pack of claim 7, wherein the male contact pin array mates with the female contact pin array via a magnetic interface, the magnetic interface being generally rectangular, surrounding the pin array, and approximately 1 mm wide and 4.8 mm high. (Item 12) Item 12. The belt pack of item 11, wherein a first magnet surrounding the male pin array has a first polarity and a second magnet surrounding the female pin array has a second polarity opposite to the first polarity. (Item 13) 7. The belt pack according to item 6, wherein at least one of the male pin array or the female pin array has a length of about 42 to 50 mm, a width of about 7 to 10 mm, and a height of about 5 mm. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 depicts an illustration of an augmented reality scenario with a virtual reality object and a physical object viewed by a person. [Figure 2A] 2A-2D diagrammatically illustrate an example of a wearable system. [Figure 2B] 2A-2D diagrammatically illustrate an example of a wearable system. [Figure 2C] 2A-2D diagrammatically illustrate an example of a wearable system. [Figure 2D] 2A-2D diagrammatically illustrate an example of a wearable system. [Figure 3]FIG. 3 diagrammatically illustrates collaboration between cloud computing assets and local processing assets. [Figure 4] FIG. 4 diagrammatically illustrates an exemplary system diagram for an electromagnetic (EM) tracking system. [Figure 5] FIG. 5 is a flow chart illustrating an exemplary function of an embodiment of an electromagnetic tracking system. [Figure 6] FIG. 6 diagrammatically illustrates an example of an electromagnetic tracking system integrated with an AR system. [Figure 7] FIG. 7 is a flow chart illustrating the functioning of an embodiment of an electromagnetic tracking system in the context of an AR device. [Figure 8] FIG. 8 diagrammatically illustrates an example of components of an embodiment of an AR system. [Figure 9A] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 9B] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 9C] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 9D] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 9E] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 9F] 9A-9F diagrammatically illustrate an embodiment of a quick release module. [Figure 10] FIG. 10 illustrates a schematic diagram of a head mounted display system. [Figure 11A] 11A and 11B diagrammatically illustrate an example of an electromagnetic sensing coil coupled to a head-mounted display. [Figure 11B] 11A and 11B diagrammatically illustrate an example of an electromagnetic sensing coil coupled to a head-mounted display. [Figure 12A]12A-12E schematically illustrate example configurations of ferrite cores that may be coupled to electromagnetic sensors. [Figure 12B] 12A-12E schematically illustrate example configurations of ferrite cores that may be coupled to electromagnetic sensors. [Figure 12C] 12A-12E schematically illustrate example configurations of ferrite cores that may be coupled to electromagnetic sensors. [Figure 12D] 12A-12E schematically illustrate example configurations of ferrite cores that may be coupled to electromagnetic sensors. [Figure 12E] 12A-12E schematically illustrate example configurations of ferrite cores that may be coupled to electromagnetic sensors. [Figure 13A] FIG. 13A is a block diagram that schematically illustrates an embodiment of a frequency division multiplexed (FDM) EM transmitter circuit (EM emitter). [Figure 13B] FIG. 13B is a block diagram that schematically illustrates an embodiment of a frequency division multiplexed EM receiver circuit (EM sensor). [Figure 13C] FIG. 13C is a block diagram that schematically illustrates an embodiment of a time division multiplexed (TDM) EM transmitter circuit. [Figure 13D] FIG. 13D is a block diagram that schematically illustrates an embodiment of a dynamically tunable circuit for an EM transmitter. [Figure 13E] FIG. 13E is a graph showing an example of a resonance that can be achieved by dynamically adjusting the circuit shown in FIG. 13D. [Figure 13F] FIG. 13F illustrates an example of a timing diagram for a time-division multiplexed EM transmitter and receiver. [Figure 13G] FIG. 13G illustrates an example of scan timing for a time-division multiplexed EM transmitter and receiver. [Figure 13H] FIG. 13H is a block diagram that schematically illustrates an embodiment of a TDM receiver in an EM tracking system. [Figure 13I]FIG. 13I is a block diagram that schematically illustrates an embodiment of an EM receiver without automatic gain control (AGC). [Figure 13J] FIG. 13J is a block diagram that schematically illustrates an embodiment of an EM transmitter employing AGC. [Figure 14] 14 and 15 are flow charts illustrating an example of pose tracking using an electromagnetic tracking system in a head-mounted AR system. [Figure 15] 14 and 15 are flow charts illustrating an example of pose tracking using an electromagnetic tracking system in a head-mounted AR system. [Figure 16A] 16A and 16B diagrammatically illustrate examples of components of another embodiment of an AR system. [Figure 16B] 16A and 16B diagrammatically illustrate examples of components of another embodiment of an AR system. [Figure 17A] FIG. 17A diagrammatically illustrates an embodiment of a resonant circuit in a transmitter in an electromagnetic tracking system. [Figure 17B] FIG. 17B is a graph showing an example of resonance at 22 kHz in the resonant circuit of FIG. 17A. [Figure 17C] FIG. 17C is a graph showing an example of the current flowing through a resonant circuit. [Figure 17D] 17D and 17E diagrammatically illustrate an example of a dynamically adjustable configuration for a resonant circuit within an EM field transmitter of an electromagnetic tracking system. [Figure 17E] 17D and 17E diagrammatically illustrate an example of a dynamically adjustable configuration for a resonant circuit within an EM field transmitter of an electromagnetic tracking system. [Figure 17F] FIG. 17F is a graph showing an example of dynamically adjusted resonance by varying the capacitance value of capacitor C4 in the example circuit shown in FIG. 17E. [Figure 17G] FIG. 17G is a graph showing examples of maximum currents achieved at various resonant frequencies. [Figure 18A]FIG. 18A is a block diagram that schematically illustrates an example of an electromagnetic field sensor adjacent to an audio speaker. [Figure 18B] FIG. 18B is a block diagram that schematically illustrates an example of an electromagnetic field sensor with a noise cancellation system that receives input from both the sensor and an external audio speaker. [Figure 18C] FIG. 18C is a graph showing an example of how signals can be inverted and added to cancel magnetic interference caused by audio speakers. [Figure 18D] FIG. 18D is a flowchart illustrating an exemplary method for canceling interference received by an EM sensor in an EM tracking system. [Figure 19] FIG. 19 illustrates diagrammatically the use of light patterns to aid in the calibration of a vision system. [Figure 20A] 20A-20C are block diagrams of exemplary circuits that can be used with subsystems or components of a wearable display device. [Figure 20B] 20A-20C are block diagrams of exemplary circuits that can be used with subsystems or components of a wearable display device. [Figure 20C] 20A-20C are block diagrams of exemplary circuits that can be used with subsystems or components of a wearable display device. [Figure 21] FIG. 21 is a graph showing an example of merging the outputs from an IMU, an electromagnetic tracking sensor, and an optical sensor. [Figure 22A] 22A-22C diagrammatically illustrate additional embodiments of electromagnetic sensing coils coupled to a head-mounted display. [Figure 22B] 22A-22C diagrammatically illustrate additional embodiments of electromagnetic sensing coils coupled to a head-mounted display. [Figure 22C] 22A-22C diagrammatically illustrate additional embodiments of electromagnetic sensing coils coupled to a head-mounted display. [Figure 23A]23A-23C diagrammatically illustrate an embodiment of recalibrating a head mounted display using electromagnetic and acoustic signals. [Figure 23B] 23A-23C diagrammatically illustrate an embodiment of recalibrating a head mounted display using electromagnetic and acoustic signals. [Figure 23C] 23A-23C diagrammatically illustrate an embodiment of recalibrating a head mounted display using electromagnetic and acoustic signals. [Figure 24A] 24A-24D diagrammatically illustrate additional embodiments of recalibrating a head mounted display using a camera or depth sensor. [Figure 24B] 24A-24D diagrammatically illustrate additional embodiments of recalibrating a head mounted display using a camera or depth sensor. [Figure 24C] 24A-24D diagrammatically illustrate additional embodiments of recalibrating a head mounted display using a camera or depth sensor. [Figure 24D] 24A-24D diagrammatically illustrate additional embodiments of recalibrating a head mounted display using a camera or depth sensor. [Figure 25A] 25A and 25B diagrammatically illustrate techniques for resolving position ambiguity that may be associated with an electromagnetic tracking system. [Figure 25B] 25A and 25B diagrammatically illustrate techniques for resolving position ambiguity that may be associated with an electromagnetic tracking system. DETAILED DESCRIPTION OF THE INVENTION

[0010] Throughout the drawings, reference numbers may be reused to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the present disclosure. AR, VR, and Location Systems Overview

[0011] In Figure 1, an augmented reality scene (4) is depicted in which a user of the AR technology sees a real-world, park-like setting (6) featuring a concrete platform (1120) against a background of people, trees, and buildings. In addition to these items, the user of the AR technology also perceives as "seeing" a robotic figure (1110) standing on the real-world platform (1120) and a flying, cartoonish avatar character (2) that appears to be an anthropomorphic bumblebee, although these elements (2, 1110) do not exist in the real world. Consequently, the human visual perception system is highly complex, making it challenging to create VR or AR technologies that facilitate the comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0012] For example, a head-mounted AR display (or helmet-mounted display or smart glasses) is typically at least loosely coupled to a user's head and may therefore move as the user's head moves. When movement of the user's head is detected by the display system, the displayed data can be updated to account for changes in head pose.

[0013] As an example, when a user wearing a head-mounted display views a virtual representation of a three-dimensional (3-D) object on the display and walks around the area in which the 3-D object appears, the 3-D object can be re-rendered for each viewpoint, giving the user the perception that they are walking around objects that occupy real space. When a head-mounted display is used to present multiple objects in a virtual space (e.g., a rich virtual world), measurements of head pose (e.g., the user's head location and orientation) can be used to re-render the scene to match the user's dynamically changing head location and orientation, providing an increased sense of immersion within the virtual space.

[0014] In an AR system, detecting or calculating head pose can facilitate the display system rendering virtual objects so that they appear to occupy space in the real world in a manner that makes sense to the user. Additionally, detecting the position and / or orientation of real objects, such as a handheld device (which may also be referred to as a "totem"), tactile device, or other real physical object coupled with the user's head or the AR system, can also facilitate the display system presenting display information to the user and allowing the user to efficiently interact with certain aspects of the AR system. As the user's head moves around in the real world, virtual objects can be re-rendered according to head pose so that the virtual objects appear stable relative to the real world. At least for AR applications, placing virtual objects in spatial coordination with physical objects (e.g., presented to appear spatially proximate to the physical objects in two or three dimensions) may not be a trivial problem. For example, head movement can significantly complicate the placement of virtual objects from the perspective of the surrounding environment. This applies whether the viewpoint is captured as an image of the surrounding environment and then projected or displayed to the end user, or whether the end user directly perceives the viewpoint of the surrounding environment. For example, head movement will likely change the end user's field of view, which will likely require updates to where various virtual objects are displayed in the end user's field of view. Additionally, head movements can occur over a wide variety of ranges and speeds. Head movement speed may vary not only between different head movements, but also within or across the range of a single head movement. For example, head movement speed may increase (e.g., linearly or non-linearly) from a starting point and decrease as an end point is reached, achieving a maximum speed somewhere between the start and end points of the head movement. High-speed head movements may even exceed the capabilities of certain display or projection technologies, rendering images that appear to the end user as uniform and / or smooth motion.

[0015] Head tracking accuracy and latency (e.g., the elapsed time between a user moving their head and the image being updated and displayed to the user) present challenges for VR and AR systems. Particularly for display systems that fill a substantial portion of a user's field of view with virtual elements, it is advantageous if head tracking accuracy is high and overall system latency from the initial detection of head movement to the optical update delivered by the display to the user's visual system is very low. If latency is high, the system may introduce a mismatch between the user's vestibular and visual systems, creating user perception scenarios that may lead to motion sickness or 3D sickness. If system latency is high, the apparent location of virtual objects may appear unstable during fast head movements.

[0016] In addition to head-mounted display systems, other display systems can also benefit from accurate and low-latency head pose detection. These include head-tracked display systems, where the display is not attached to the user's body but is mounted, for example, on a wall or other surface. The head-tracked display can act like a window onto a scene; as the user moves their head relative to the "window," the scene is re-rendered to match the user's changing viewpoint. Other systems include head-mounted projection systems, where the head-mounted display projects light onto the real world.

[0017] Additionally, the AR system may be designed to interact with the user to provide a realistic augmented reality experience. For example, multiple users may play a ball game using a virtual ball and / or other virtual objects. One user may "catch" the virtual ball and throw it back to another user. In another embodiment, a first user may be provided with a totem (e.g., an actual bat communicatively coupled to the AR system) to hit the virtual ball. In other embodiments, a virtual user interface may be presented to the AR user to allow the user to select one of many options. The user may use a totem, a tactile device, a wearable component, or simply touch a virtual screen to interact with the system.

[0018] Detecting the pose and orientation of a user's head and the physical location of real objects in space allows AR systems to display virtual content in an effective and enjoyable manner. However, these capabilities, while important for AR systems, are difficult to achieve. In other words, AR systems must recognize the physical location of real objects (e.g., a user's head, a totem, a tactile device, a wearable component, a user's hand, etc.) and correlate the physical coordinates of the real objects with virtual coordinates corresponding to one or more virtual objects displayed to the user. This generally requires highly accurate sensors and sensor recognition systems that track the position and orientation of one or more objects at a fast rate. Current approaches do not perform localization at satisfactory speed or accuracy standards.

[0019] Therefore, there is a need for better localization systems in the context of AR and VR devices. Exemplary AR and VR Systems and Components

[0020] 2A-2D, several general component options are illustrated. In the Detailed Description section, following the discussion of Figures 2A-2D, various systems, subsystems, and components are presented to address the objective of providing a high-quality and comfortably perceived display system for human VR and / or AR.

[0021] 2A, an AR system user (60) is depicted wearing a head-mounted component (58) featuring a frame (64) structure coupled to a display system (62) positioned in front of the user's eyes. Speakers (66) are coupled to the frame (64) in the depicted configuration and positioned adjacent to the user's ear canals (in one embodiment, another speaker, not shown, is positioned adjacent to the user's other ear canal to provide spatial / shapeable sound control). The display (62) may be operatively coupled (68), such as by wired or wireless connectivity, to a local processing and data module (70), which may be mounted in a variety of configurations, such as fixedly attached to the frame (64), fixedly attached to a helmet or hat (80) as shown in the embodiment of FIG. 2B, embedded within headphones, removably attached to the torso (82) of the user (60) in a backpack-style configuration as shown in the embodiment of FIG. 2C, or removably attached to the hips (84) of the user (60) in a belt-coupled configuration as shown in the embodiment of FIG. 2D.

[0022] The local processing and data module (70) may include a power-efficient processor or controller and digital memory, such as flash memory, both of which may be utilized to a) assist in processing, caching, and storing data captured from sensors, such as image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, and / or gyros, that may be operatively coupled to the frame (64), and / or b) assist in processing, caching, and storing data that may be retrieved and / or processed using a remote processing module (72) and / or a remote data repository (74), possibly for passing to the display (62) after such processing or retrieval. The local processing and data module (70) may be operatively coupled (76, 78) to a remote processing module (72) and a remote data repository (74), such as via a wired or wireless communication link, such that the remote modules (72, 74) are operatively coupled to each other and available as resources to the local processing and data module (70).

[0023] In one embodiment, the remote processing module 72 may comprise one or more relatively powerful processors or controllers configured to analyze and process data and / or image information. In one embodiment, the remote data repository (74) may comprise a relatively large digital data storage facility, which may be available through the Internet or other networking configuration in a "cloud" resource configuration. In one embodiment, all data may be stored and all calculations may be performed within the local processing and data module, allowing for fully autonomous use from any remote module.

[0024] Referring now to Figure 3, a schematic diagram illustrates the coordination between cloud computing assets (46) and local processing assets, which may reside, for example, in a head-mounted component (58) coupled to a user's head (120) and a local processing and data module (70) coupled to the user's belt (308). Accordingly, the component 70 may also be referred to as a "belt pack" 70, as shown in Figure 3. In one embodiment, cloud (46) assets, such as one or more server systems (110), are operatively coupled (115) directly to one or both of the local computing assets (40, 42), such as processor and memory configurations coupled to the user's head (120) and belt (308), as previously described, such as via wired or wireless networking (wireless being preferred for mobile deployments, wired being preferred for certain high-bandwidth or high-data-volume transfers that may be desired). These computing assets local to the user may likewise be operatively coupled to one another via wired and / or wireless connectivity arrangements 44, such as the wired coupling 68 discussed below with reference to Figure 8. In one embodiment, to maintain the low inertia and small size of the user's head-mounted subsystem 120, the primary transfer between the user and the cloud 46 may be via a link between the belt-mounted subsystem 308 and the cloud, with the head-mounted subsystem 120 being primarily data-tethered to the belt-based subsystem 308 using wireless connectivity, such as, for example, ultra-wideband ("UWB") connectivity as currently employed in personal computing peripheral connectivity applications.

[0025] Using efficient local and remote processing coordination and an appropriate display device for the user, such as the user interface or user display system (62) shown in FIG. 2A or a variation thereof, aspects of the world related to the user's current real or virtual location can be transferred or "passed" to the user and updated in an efficient manner. In other words, a map of the world can be continuously updated in a storage location that may reside partially on the user's AR system and partially in cloud resources. The map (also referred to as a "passable world model") may be a large database comprising raster images, 3-D and 2-D points, parametric information, and other information about the real world. As more and more AR users continuously capture information about their real environment (e.g., through cameras, sensors, IMUs, etc.), the map becomes increasingly accurate and complete.

[0026] Using the above configuration, there is one world model that resides on and can be distributed from cloud computing resources, and such world can be “passable” to one or more users in a relatively low-bandwidth manner that is preferable to attempting to pass real-time video data or the like. The augmented experience of a person standing near the statue (i.e., as shown in FIG. 1 ) may be informed by the cloud-based world model, a subset of which may be passed to them and their local display device to complete the view. A person seated at a remote display device, which may be as simple as a personal computer on a desk, can efficiently download the same section of information from the cloud and have it rendered on their display. In fact, a person physically present in the park near the statue may stroll through the park with a remotely located friend, who participates through virtual and augmented reality. The system would need to know the location of streets, trees, and statues, but using that information on the cloud, a participating friend could download aspects of the scenario from the cloud and then begin walking along the augmented reality local to the person physically present in the park.

[0027] Three-dimensional (3-D) points may be captured from the environment, and the pose (e.g., vector and / or origin information relative to the world) of the camera capturing those images or points may be determined so that those points or images can be "tagged" or associated with this pose information. The points captured by the second camera may then be used to determine the pose of the second camera. In other words, the second camera can be oriented and / or located based on a comparison with the tagged image from the first camera. This knowledge may then be used to extract textures, create maps, and create virtual copies of the real world (as there are now two aligned cameras around).

[0028] Thus, at a basic level, in one embodiment, a person-worn system can be utilized to capture both 3-D points and the 2-D images that generated those points, and these points and images can be transmitted to cloud storage and processing resources. They can also be cached locally along with on-board pose information (i.e., caching tagged images). Thus, the cloud can have tagged 2-D images (i.e., tagged with 3-D pose) along with the 3-D points at its fingertips (i.e., in its available cache). If the user is observing something dynamic, they may also transmit additional information about their movement to the cloud (e.g., if looking at another person's face, the user can capture a texture map of the face and push it at an optimized frequency, even though the surrounding world is otherwise essentially static). Further information regarding object recognizers and passable world models may be found in U.S. Patent Publication No. 2014 / 0306866, entitled "System and method for augmented and virtual reality," which is incorporated herein by reference in its entirety, as well as the following additional disclosures related to augmented and virtual reality systems such as those developed by Magic Leap, Inc. (Plantation, Florida): U.S. Patent Publication No. 2015 / 0178939, U.S. Patent Publication No. 2015 / 0205126, U.S. Patent Publication No. 2014 / 0267420, U.S. Patent Publication No. 2015 / 0302652, U.S. Patent Publication No. 2013 / 0117377, and U.S. Patent Publication No. 2013 / 0128230, each of which is incorporated herein by reference in its entirety.

[0029] GPS and other location information may be utilized as input for such processing. Highly accurate location of the user's head, totem, hand gestures, tactile devices, etc. may be advantageous for displaying appropriate virtual content to the user.

[0030] The head-mounted device (58) may include a display positionable in front of the eyes of the wearer of the device. The display may comprise a light field display. The display may be configured to present images to the wearer at multiple depth planes. The display may comprise a planar waveguide with diffractive elements. Examples of displays, head-mounted devices, and other AR components usable with any of the embodiments disclosed herein are described in U.S. Patent Publication No. 2015 / 0016777, which is incorporated herein by reference in its entirety. Electromagnetic Location Example

[0031] One approach to achieving high-precision localization may involve the use of electromagnetic (EM) fields combined with electromagnetic sensors strategically placed on a user's AR headset, beltpack, and / or other auxiliary devices (e.g., totems, tactile devices, gaming equipment, etc.). Electromagnetic tracking systems typically include at least an electromagnetic field emitter and at least one electromagnetic field sensor. The sensors may measure electromagnetic fields with a known distribution. The electromagnetic field emitter generates an electromagnetic field with a known spatial (and / or temporal) distribution in the environment of the AR headset wearer. The electromagnetic field sensor measures the generated electromagnetic field at the sensor's location. Based on these measurements and knowledge of the distribution of the generated electromagnetic field, the pose (e.g., position and / or orientation) of the electromagnetic field sensor relative to the emitter can be determined. Thus, the pose of the object to which the sensor is attached can be determined.

[0032] 4, an exemplary system diagram of an electromagnetic tracking system (e.g., those developed by organizations such as Biosense, a subsidiary of Johnson & Johnson Corporation, Polhemus, Inc. (Colchester, Vermont), and those manufactured by Sixense Entertainment, Inc. (Los Gatos, California), as well as other tracking device manufacturers) is illustrated. In one or more embodiments, the electromagnetic tracking system comprises an electromagnetic field emitter 402 configured to emit a known magnetic field. As shown in FIG. 4, the electromagnetic field emitter may be coupled to a power source (e.g., electrical current, a battery, etc.) to provide power to the electromagnetic field emitter 402.

[0033] In one or more embodiments, the electromagnetic field emitter 402 comprises several coils (e.g., at least three coils positioned perpendicular to each other and generating fields in the X, Y, and Z directions) that generate a magnetic field. This magnetic field is used to establish a coordinate space (e.g., an XYZ Cartesian coordinate space). This allows the system to map the sensor's position (e.g., an (X, Y, Z) position) relative to a known magnetic field and helps determine the sensor's position and / or orientation. In one or more embodiments, the electromagnetic sensors 404a, 404b, etc. may be attached to one or more real objects. The electromagnetic sensors 404 may comprise smaller coils in which a current can be induced through the emitted electromagnetic field. Generally, the "sensor" component (404) may comprise small coils or loops, such as a set of three differently oriented (e.g., oriented orthogonally relative to each other) coils coupled together in a small structure, such as a cube or other container, positioned / oriented to capture magnetic flux entering from the magnetic field emitted by the emitter (402); by comparing the currents induced through these coils and knowing the relative positions and orientations of the coils relative to each other, the relative position and orientation of the sensor to the emitter can be calculated.

[0034] One or more parameters related to the behavior of the coil and inertial measurement unit ("IMU") components operatively coupled to the electromagnetic tracking sensor may be measured to detect the position and / or orientation of the sensor (and the object to which it is attached) relative to a coordinate system to which the electromagnetic field emitter is coupled. In one or more embodiments, multiple sensors may be used in conjunction with the electromagnetic emitter to detect the sensor's respective position and orientation within the coordinate space. The electromagnetic tracking system may provide position in three directions (i.e., X, Y, and Z directions) and also at two or three orientation angles. In one or more embodiments, the IMU measurements may be compared to the coil measurements to determine the sensor's position and orientation. In one or more embodiments, both the electromagnetic (EM) data and the IMU data may be combined with various other data sources, such as cameras, depth sensors, and other sensors, to determine position and orientation. This information may be transmitted to the controller 406 (e.g., via wireless communication, Bluetooth, etc.). In one or more embodiments, the attitude (or position and orientation) may be reported at a relatively high refresh rate in conventional systems. Traditionally, an electromagnetic field emitter is coupled to a relatively stable, large object such as a table, operating table, wall, or ceiling, and one or more sensors are coupled to a smaller object such as a medical device, handheld game component, or the like. Alternatively, as described below with reference to FIG. 6, various features of an electromagnetic tracking system may be employed to create a configuration in which changes or deltas in position and / or orientation between two objects moving in space relative to a more stable global coordinate system may be tracked. In other words, a configuration is shown in FIG. 6 in which a variation of the electromagnetic tracking system may be utilized to track position and orientation deltas between the head-mounted and handheld components, while the head pose relative to a global coordinate system (e.g., of the indoor environment local to the user) is determined otherwise, such as by simultaneous localization and mapping (“SLAM”) techniques, using an outward-facing capture camera that may be coupled to the system's head-mounted component.

[0035] The controller 406 may control the electromagnetic field emitters 402 and may also capture data from the various electromagnetic sensors 404. It should be understood that the various components of the system may be coupled to each other through any electromechanical or wireless / Bluetooth means. The controller 406 may also be equipped with data regarding the known magnetic fields and the coordinate space associated with the magnetic fields. This information may then be used to detect the position and orientation of the sensors relative to the coordinate space corresponding to the known electromagnetic fields.

[0036] One advantage of electromagnetic tracking systems is that they can produce highly accurate tracking results with minimal latency and high resolution. Additionally, electromagnetic tracking systems do not necessarily rely on optical trackers, and sensors / objects that are not within the user's line of sight can easily be tracked.

[0037] It should be understood that the strength of the electromagnetic field v decreases as a cubic function of the distance r from the coil transmitter (e.g., the electromagnetic field emitter 402). Accordingly, an algorithm may be used based on the distance from the electromagnetic field emitter. The controller 406 may be configured to use such an algorithm to determine the position and orientation of the sensor / object at variable distances from the electromagnetic field emitter. Given the rapid decrease in the strength of the electromagnetic field as the sensor moves away from the electromagnetic emitter, best results in terms of accuracy, efficiency, and low latency may be achieved at closer distances. In a typical electromagnetic tracking system, the electromagnetic field emitter is powered by an electric current (e.g., a plug-in power supply) and has the sensor located within a 20-foot radius of the electromagnetic field emitter. A shorter radius between the sensor and the field emitter may be more desirable in many applications, including AR applications.

[0038] Referring now to FIG. 5, an exemplary flow diagram illustrating the functionality of a typical electromagnetic tracking system is briefly described. At 502, a known electromagnetic field is emitted. In one or more embodiments, a magnetic field emitter may generate a magnetic field, and each coil may generate an electric field in one direction (e.g., X, Y, or Z). The magnetic field may be generated using any waveform. In one or more embodiments, the magnetic field components along each of the axes may oscillate at a slightly different frequency than other magnetic field components along other directions. At 504, a coordinate space corresponding to the electromagnetic field may be determined. For example, control 406 of FIG. 4 may automatically determine the coordinate space around the emitter based on the electromagnetic field. At 506, the behavior of the coil in a sensor (which may be attached to a known object) may be detected. For example, the current induced in the coil may be calculated. In other embodiments, the rotation of the coil or any other quantifiable behavior may be tracked and measured. At 508, this behavior may be used to detect the position or orientation of the sensor and / or the known object. For example, the controller 406 may consult a mapping table that correlates the behavior of the coils in the sensor to various positions or orientations. Based on these calculations, a position in coordinate space may be determined, along with the orientation of the sensor.

[0039] In the context of AR systems, one or more components of the electromagnetic tracking system may need to be modified to facilitate accurate tracking of the mobile component. As mentioned above, tracking the user's head pose and orientation may be desirable in many AR applications. Accurately determining the user's head pose and orientation allows the AR system to display the correct virtual content to the user. For example, a virtual scene may include a monster hidden behind a real building. Depending on the user's head pose and orientation relative to the building, the view of the virtual monster may need to be modified to provide a realistic AR experience. Alternatively, the position and / or orientation of a totem, tactile device, or some other means of interacting with the virtual content may be important in enabling the AR user to interact with the AR system. For example, in many gaming applications, the AR system must detect the position and orientation of real objects associated with the virtual content. Alternatively, when displaying a virtual interface, the position of a totem, the user's hand, tactile device, or any other real object configured for interaction with the AR system must be known in relation to the displayed virtual interface in order for the system to understand commands, etc. Conventional localization methods, including optical tracking and other methods, typically suffer from high latency and low resolution issues, making rendering virtual content difficult in many augmented reality applications.

[0040] In one or more embodiments, an electromagnetic tracking system, such as that discussed in connection with Figures 4 and 5, may be adapted to an AR system to detect the position and orientation of one or more objects relative to the emitted electromagnetic field. Typical electromagnetic systems tend to have large and bulky electromagnetic emitters (e.g., 402 in Figure 4), which presents a problem for head-mounted AR devices. However, smaller electromagnetic emitters (e.g., in the millimeter range) may be used to emit known electromagnetic fields in the context of an AR system.

[0041] Referring now to FIG. 6 , an electromagnetic tracking system may be incorporated with an AR system, as shown, with an electromagnetic field emitter 602 incorporated as part of a handheld controller 606. The controller 606 can be independently movable relative to the AR headset (or belt pack 70). For example, a user can hold the controller 606 in their hand, or the controller may be mounted on the user's hand or arm (e.g., as a ring or bracelet, or as part of a glove worn by the user). In one or more embodiments, the handheld controller may be a totem (e.g., a multi-degree-of-freedom controller) for use in gaming scenarios, or for providing a rich user experience within an AR environment, or for allowing a user to interact with the AR system. In other embodiments, the handheld controller may be a tactile device. In yet other embodiments, the electromagnetic field emitter may simply be incorporated as part of the belt pack 70. The handheld controller 606 may include a battery 610 or other power source that powers the electromagnetic field emitter 602. It should be appreciated that the electromagnetic field emitter 602 may also include or be coupled to an IMU component 650 configured to assist in determining the position and / or orientation of the electromagnetic field emitter 602 relative to other components. This may be particularly advantageous if both the electromagnetic field emitter 602 and the sensor (604) are mobile. Locating the electromagnetic field emitter 602 in the handheld controller rather than the beltpack, as shown in the embodiment of FIG. 6, ensures that the electromagnetic field emitter uses its own battery source in the handheld controller 606 rather than competing for resources at the beltpack. In yet other embodiments, the electromagnetic field emitter 602 may be located on the AR headset and the sensor 604 may be located on the controller 606 or the beltpack 70.

[0042] In one or more embodiments, the electromagnetic sensor (604) may be located in one or more locations on the user's headset, along with other sensing devices, such as one or more IMUs or additional flux capture coils (608). For example, as shown in FIG. 6 , the sensors (604, 608) may be located on one or both sides of the headset (58). Because these sensors are fabricated to be very small (and therefore, in some cases, may be insensitive), having multiple sensors may improve efficiency and accuracy. In one or more embodiments, one or more sensors may also be located on the belt pack 70 or any other part of the user's body. The sensors (604, 608) may communicate wirelessly or via Bluetooth with a computing device that determines the attitude and orientation of the sensor (and the AR headset to which it is attached). In one or more embodiments, the computing device may reside on the belt pack 70. In other embodiments, the computing device may reside on the headset itself or even on the handheld controller 606. The computing device may then have a mapping database (e.g., a passable world model, a coordinate space, etc.), detect pose, determine coordinates of real and virtual objects, and, in one or more embodiments, connect to cloud resources and a passable world model.

[0043] As previously mentioned, conventional electromagnetic emitters may be too bulky for use in AR devices. Therefore, the electromagnetic field emitter may be fabricated compactly using smaller coils compared to conventional systems. However, given that the strength of the electromagnetic field decreases as a cubic function of distance from the field emitter, a shorter radius (e.g., about 3-3.5 feet) between the electromagnetic sensor 604 and the electromagnetic field emitter 602 may reduce power consumption compared to conventional systems such as those detailed in FIG. 4.

[0044] This aspect may be utilized, in one or more embodiments, to extend the life of the battery 610, which may power the controller 606 and the electromagnetic field emitter 602. Or, in other embodiments, this aspect may be utilized to reduce the size of the coils that generate the magnetic field in the electromagnetic field emitter 602. However, to obtain the same strength of the magnetic field, the power may need to be increased. This allows for a compact electromagnetic field emitter unit 602 that may fit compactly in the handheld controller 606.

[0045] Some other modifications may be made when using an electromagnetic tracking system for an AR device. While this attitude reporting rate is quite good, AR systems may require an even more efficient attitude reporting rate. To this end, IMU-based attitude tracking may be used within the sensor (in addition to or instead of). Advantageously, the IMU should remain as stable as possible to increase the efficiency of the attitude detection process. The IMU may be engineered to remain stable for a maximum of 50-100 milliseconds. It should be understood that some embodiments may utilize an external attitude estimator module, which may allow attitude updates to be reported at a rate of 10-20 Hz (e.g., the IMU may drift over time). By keeping the IMU stable to a reasonable rate, the attitude update rate can be significantly reduced to 10-20 Hz (compared to the higher frequencies in conventional systems).

[0046] If the electromagnetic tracking system could be activated, for example, at a 10% duty cycle (e.g., pinging ground only every 100 milliseconds), this would be another way to save power in an AR system. This would mean that the electromagnetic tracking system would wake up for 10 milliseconds every 100 milliseconds to generate a pose estimate. This could directly translate into power consumption savings, which in turn could impact the size, battery life, and cost of the AR device.

[0047] In one or more embodiments, this reduction in duty cycle may be strategically utilized by providing two handheld controllers (not shown) instead of just one. For example, a user may play a game that requires two totems, etc. Or, in a multi-user game, two users may have their own totem / handheld controller and play the game. When two controllers (e.g., symmetrical controllers for each hand) are used instead of one, the controllers may operate with offset duty cycles. The same concept may also be applied to controllers utilized by, for example, two different users playing a multiplayer game.

[0048] Referring now to FIG. 7 , an exemplary flow diagram illustrating an electromagnetic tracking system in the context of an AR device is described. At 702, a portable (e.g., handheld) controller emits a magnetic field. At 704, an electromagnetic sensor (mounted on a headset, beltpack, etc.) detects the magnetic field. At 706, the attitude (e.g., position or orientation) of the headset / belt is determined based on the behavior of the coil / IMU in the sensor. At 708, the attitude information is communicated to a computing device (e.g., in the beltpack or headset). At 710, optionally, a mapping database (e.g., a passable world model) may be consulted to correlate real-world coordinates (e.g., determined with respect to the attitude of the headset / belt) with virtual world coordinates. At 712, virtual content may be delivered to the user in the AR headset and displayed to the user (e.g., via a light field display described herein). It should be understood that the foregoing flow diagram is for illustrative purposes only and should not be read as limiting.

[0049] Advantageously, the use of an electromagnetic tracking system similar to that outlined in Figure 6 enables pose tracking (e.g., head position and orientation, totem and other controller position and orientation), which allows the AR system to project virtual content (based at least in part on the determined pose) with greater accuracy and lower latency compared to optical tracking techniques.

[0050] Referring to FIG. 8 , a system configuration featuring multiple sensing components is illustrated. A head-mounted wearable component (58) is shown here operably coupled (68) to a local processing and data module (70), such as a beltpack, using physical multicore conductors that also feature a control and quick-release module (86), as described below with reference to FIGS. 9A-9F . The local processing and data module (70) may here be operably coupled (100) to a handheld component (606) via a wireless connection, such as low-power Bluetooth®. The handheld component (606) may also be operably coupled (94) directly to the head-mounted wearable component (58), such as via a wireless connection, such as low-power Bluetooth®. Generally, when IMU data is passed to coordinate attitude detection of various components, a high-frequency connection, such as in the range of hundreds or thousands of cycles per second or higher, is desirable. Tens of cycles per second may be adequate for electromagnetic localization sensing, such as by pairing a sensor 604 and a transmitter 602. Also shown is a global coordinate system 10 that represents fixed objects in the real world around the user, such as walls 8.

[0051] The cloud resources (46) may also be operatively coupled (42, 40, 88, 90) to resources that may be coupled to a local processing and data module (70), to a head-mounted wearable component (58), to a wall (8) or other item fixed relative to the global coordinate system (10), respectively. Resources coupled to a wall (8) or having a known position and / or orientation relative to the global coordinate system (10) may include a wireless transceiver (114), an electromagnetic emitter (602) and / or receiver (604), a beacon or reflector (112) configured to emit or reflect a given type of radiation, such as an infrared LED beacon, a cellular network transceiver (110), a RADAR emitter or detector (108), a LIDAR emitter or detector (106), a GPS transceiver (118), a poster or marker having a known detectable pattern (122), and a camera (124).

[0052] The head-mounted wearable component (58) features an optical emitter (130) configured to assist the camera (124) detector, such as an infrared emitter (130) for the infrared camera (124), as well as similar components as shown. The head-mounted wearable component (58) also features one or more strain gauges (116), which may be fixedly coupled to the frame or mechanical platform of the head-mounted wearable component (58) and configured to determine deflection of such platform between components such as the electromagnetic receiver sensor (604) or the display element (62), which may be important to understand when bending of the platform occurs, such as in thin portions of the platform, such as the portion above the protrusion on the eyeglass-like platform depicted in FIG. 8.

[0053] The head-mounted wearable component (58) also features a processor (128) and one or more IMUs (102). Each component is preferably operably coupled to the processor (128). The handheld component (606) and the local processing and data module (70) are shown to feature similar components. As shown in FIG. 8, with so many sensing and connectivity means, such a system is likely to be heavy, power-hungry, large, and relatively expensive. However, for illustrative purposes, such a system may be utilized to provide a very high level of connectivity, system component integration, and position / orientation tracking. For example, with such a configuration, the various primary mobile components (58, 70, 606) may be located in terms of position relative to a global coordinate system using Wi-Fi, GPS, or cellular signal triangulation, and beacons, electromagnetic tracking (as described herein), RADAR, and LIDIR systems may further provide location and / or orientation information and feedback. Markers and cameras may also be utilized to provide further information regarding relative and absolute position and orientation. For example, various camera components (124), such as the one shown coupled to the head-mounted wearable component (58), may be utilized to capture data that may be utilized in a simultaneous localization and mapping protocol, or "SLAM," to determine where and how the component (58) is oriented relative to other components.

[0054] 9A-9F, various aspects of the control and quick-release module 86 are depicted. With reference to FIG. 9A, two outer housing components (132, 134) are coupled together using a magnetic coupling arrangement, which may be augmented with a mechanical latch. Buttons (136) for operation of associated systems may be included, such as an on / off button (circular button) and an up / down volume control (triangular button). Opposite ends of the module 86 can be connected to electrical leads extending between the local processing and data module (70) and the display (62), as shown in FIG.

[0055] Figure 9B illustrates a partial cutaway view with the outer housing (132) removed to reveal the button (136) and underlying top printed circuit board (138). Referring to Figure 9C, the button (136) and underlying top printed circuit board (138) are removed to reveal the female contact pin array (140). Referring to Figure 9D, the opposite portion of the housing (134) is removed to reveal the lower printed circuit board (142). The lower printed circuit board (142) is removed to reveal the male contact pin array (144), as shown in Figure 9E.

[0056] Referring to the cross-sectional view of FIG. 9F, at least one of the male or female pins is configured to be spring-loaded so that it can be depressed along the longitudinal axis of each pin. The pins may be referred to as "pogo pins" and may generally comprise a highly conductive material such as copper or gold. The conductive material may be plated (e.g., dipped or electroplated) on the pins, and the width of the conductive material may be at least 25 μm, for example, in some cases for gold. When assembled, the illustrated configuration mates 46 male pins with 46 corresponding female pins, and the entire assembly may be quickly released by manually separating the two housings (132, 134) and overcoming the magnetic interface 146 load, which may be generated using north and south magnets oriented around the periphery of the pin arrays (140, 144). In one embodiment, the approximately 2 kg load from compressing the 46 pogo pins is countered by approximately 4 kg of closure retention force provided by the magnetic interface 146. The pins in the array may be separated by approximately 1.3 mm, and the pins may be operably coupled to various types of conductive lines, such as twisted pair or other combinations, to support interfaces such as USB 3.0, HDMI 2.0 (for digital video), I2S (for digital audio), transition minimized differential signaling (TMDS) for high-speed serial data, general-purpose input / output (GPIO), and mobile interface (e.g., MIPI) configurations, battery / power connections, and, in one embodiment, high-current analog lines and grounds configured for up to approximately 4 amps and 5 volts.

[0057] In one embodiment, the magnetic interface (146) is generally rectangular, surrounds the pin arrays (140, 144), and is approximately 1 mm wide and 4.8 mm high. The inner diameter of the rectangular magnet is approximately 14.6 mm. The magnet surrounding the male pin array (144) may have a first polarity (e.g., north), and the magnet surrounding the female pin array (140) may have a second (opposite) polarity (e.g., south). In some cases, each magnet has a mixed north and south polarity, with the opposing magnet having a corresponding opposite polarity, providing a magnetic attraction that helps hold the housings (132, 134) together.

[0058] The pogo pins in the arrays (140, 144) have heights ranging from 4.0 to 4.6 mm and diameters ranging from 0.6 to 0.8 mm. Different pins in the array can have different heights, diameters, and pitches. For example, in one implementation, the pin arrays (140, 144) have lengths of approximately 42 to 50 mm, widths of approximately 7 to 10 mm, and heights of approximately 5 mm. The pitch of the pin arrays for USB 2.0 and other signals can be approximately 1.3 mm, and the pitch of the pin arrays for high-speed signals can be approximately 2.0 to 2.5 mm.

[0059] With reference to Figure 10, it is useful to have a minimized component / feature set to reduce or minimize the weight and size of the various components, allowing for example to arrive at a relatively slim head-mounted component such as that (58) featured in Figure 10. Thus, various permutations and combinations of the various components shown in Figure 8 may be utilized. Exemplary Electromagnetic Sensing Components in an AR System

[0060] With reference to FIG. 11A , an electromagnetic sensing coil assembly (604, e.g., three individual coils coupled to a housing) is shown coupled to the head-mounted component (58). Such a configuration adds additional geometry to the overall assembly, which may be undesirable. With reference to FIG. 11B , rather than housing the coils in a box or single housing, as in the configuration of FIG. 11A , the individual coils may be integrated into various structures of the head-mounted component (58), as shown in FIG. 11B. FIG. 11B shows examples of locations on the head-mounted display 58 for the X-axis coil (148), Y-axis coil (150), and Z-axis coil (152). Thus, the sensing coils may be spatially distributed on or about the head-mounted display (58), providing a desired spatial resolution or accuracy of locating and / or orienting the display (58) via an electromagnetic tracking system.

[0061] Figures 12A-12E illustrate various configurations for increasing field sensitivity using ferrite cores 1200a-1200e coupled to electromagnetic sensors. Figure 12A illustrates a solid ferrite core 1200a in the shape of a cube, Figure 12B shows a ferrite core 1200b configured as multiple rectangular disks spaced apart from one another, Figure 12C shows a ferrite core 1200c with a uniaxial air core, Figure 12D shows a ferrite core 1200d with a triaxial air core, and Figure 12E shows a ferrite core 1200e comprising multiple ferrite rods within a housing (which may be made from plastic). The embodiments 1200b-1200e in Figures 12B-12E are lighter in weight than the solid-core embodiment 1200a in Figure 12A and may be utilized to conserve mass. While shown as a cube in Figures 12A-12E, the ferrite cores can be shaped differently in other embodiments. Frequency Division Multiplexing, Time Division Multiplexing, and Gain Control for EM Tracking Systems

[0062] Conventional EM tracking solutions typically employ either frequency division multiplexing (FDM) or time division multiplexing (TDM) circuit designs. However, FDM designs typically use more current, and TDM designs typically support only a limited number of users. As further described below, a circuit design that blends both FDM and TDM designs can achieve the benefits of both. The benefits of such a design can include savings in printed circuit board (PCB) area, material costs, number of components used, and / or current drain compared to conventional designs. The design can also provide improved or optimal performance for multiple users.

[0063] FIG. 13A is a block diagram that schematically illustrates an example of a frequency-division multiplexed EM transmitter (TX) circuit 1302. The EM transmitter circuit can drive three tuned quadrature coils in an EM tracking system. The time-varying EM field generated by the EM TX can be sensed by an EM receiver (e.g., as described with reference to FIG. 13B). The circuit uses a master control unit (MCU) to control three different synthesizers at three different radio frequency (RF) frequencies (f1, f2, and f3), the outputs of which are filtered (e.g., in band-pass filters (BPFs) and optional ferrite beads (FBs)), amplified (e.g., via preamplifiers (PAs)), and fed to the individual X, Y, and Z coils. The circuit also employs current-sense control circuits (R-sense and current control) that ensure the current into each coil remains constant. The circuit also has an RF wireless communication interface (eg, Bluetooth® Low Energy (BLE)) connected to the MCU, which communicates with the EM receiver unit described with reference to FIG. 13B.

[0064] FIG. 13B is a block diagram that schematically illustrates an example of a frequency-division multiplexed EM receiver (RX) circuit 1304. The EM receiver uses three quadrature coils (an X-coil operating at frequency f1, a Y-coil operating at frequency f2, and a Z-coil operating at frequency f3) to receive the time-varying EM signals generated by the EM TX circuit 1302 (see, e.g., FIG. 13A). The three signals are individually amplified (e.g., via a preamplifier (PA)) and filtered (e.g., by a bandpass filter (BPF)) in parallel. Optionally, the filter output may be further amplified. The amplified output is then fed into an analog-to-digital (ADC), and the digital signal is processed by a digital signal processor (DSP). The DSP can control the gain of the preamplifier to keep the ADC from saturating. The receiver design also has a radio frequency (RF) communication link connected to a DSP (or MCU) that communicates with the EM transmitter (e.g., as described with reference to FIG. 13B). The RF link can be configured to support any suitable wireless standard, including Bluetooth® Low Energy (BLE).

[0065] The EM TX and RX circuits 1302, 1304 shown in Figures 13A and 13B (and the TX and RX circuits described below with reference to Figures 13C-13J) can be used for EM tracking. For example, the EM TX circuit 1302 can be used in the EM field emitter 402 and the EM RX circuit 1304 can be used in the EM field sensor 404, described with reference to Figure 4. Additional embodiments of the EM TX and RX circuits will be described that can provide advantages such as reduced component count, reduced PCB area, lower material costs, etc., and can result in optimal performance for multiple users.

[0066] FIG. 13C is a block diagram that schematically illustrates an example of a time-division multiplexed EM transmitter circuit 1302. In this embodiment, the FDM circuit of FIG. 13A is modified to a time-division multiplexed circuit. The TDM circuit uses only one path, which is divided into three orthogonal coils. The X-, Y-, and Z-coils operate at frequencies f1, f2, and f3, respectively, and generate a time-varying EM field that is received by the EM receiver circuit. The TDM circuit can operate these coils at separate times t1, t2, and t3 according to a TDM timing protocol (see, e.g., FIGS. 13F and 13G). Automatic gain control (AGC) can be included within the transmitter circuit (described further below with reference to FIGS. 13I and 13J). Each coil can be dynamically frequency-tuned to a desired frequency assigned by the MCU. dynamic frequency adjustment

[0067] Dynamic frequency adjustment can be used to achieve resonance on each coil and obtain increased or maximum current flow in the EM TX circuit. Dynamic frequency adjustment can be used to accommodate multiple users. FIG. 13D is a block diagram that schematically illustrates an example of a dynamically adjustable circuit 1306. Other embodiments of the dynamically adjustable circuit 1306 are described with reference to FIGS. 17D-17G. In the circuit shown in FIG. 13D, the transmit coil is represented by inductor L1. A static capacitor (C2) is in parallel with the adjustable capacitor (C1). In this example, the frequencies generated by the coil by adjusting capacitor C1 cover the frequency range 16 kHz to 30 kHz. FIG. 13E is a graph showing an example of resonance at various frequencies (16 kHz to 30 kHz) that can be achieved by dynamically adjusting the circuit 1306 shown in FIG. 13D. To accommodate multiple users, an exemplary dynamic frequency adjustment circuit can employ one transmit (TX) frequency per user. An example of frequency allocation is shown in Table 1. [Table 1] Time division multiplexing

[0068] In some embodiments, synchronization between the transmitter and receiver circuitry may be utilized to achieve time division multiplexing on the transmitter. Two possible scenarios for synchronization are discussed below.

[0069] The first scenario uses synchronization over the RF air interface (e.g., BLE) of both the receiver and transmitter. The wireless RF link can be used to synchronize the clocks of both the transmitter and receiver. After synchronization is achieved, time division multiplexing can be referenced to an onboard real-time clock (RTC).

[0070] The second scenario uses synchronization through an electromagnetic pulse. The time of flight of the EM pulse is significantly shorter than the tolerances typically used in TDM circuits and may be ignored. A TX EM pulse is sent by the transmitter to the receiver, which calculates the time difference between the receiver clock and the transmitter clock. This time difference is communicated via the RF wireless link as a known offset or used to adjust the reference on the air interface (e.g., BLE) clock.

[0071] In some embodiments, one or both of these synchronization scenarios can be implemented. After synchronization is completed, a time sequence for TDM for the transmitter and receiver can be established. FIG. 13F illustrates an example of a TDM timing diagram 1308. The TX on the X-coil will remain on for a first period, allowing the X, Y, and Z coils of the receiver to receive the magnetic flux generated by the X-coil. During the first period, the TX on the Y-coil and Z-coil are substantially off (e.g., the coils are completely off or operate at a voltage well below (e.g., <10%, <5%, <1%, etc.) their normal operating voltage). Following the X-coil transmission, the TX on the Y-coil will turn on (and the X-coil will be substantially off while the Z-coil remains substantially off), and the X, Y, and Z coils on the receiver will receive the magnetic flux generated by the TX Y-coil. Following the Y-coil transmission, the TX on the Z-coil will turn on (and the Y-coil will be effectively turned off while the X-coil will remain effectively off), and the receiver's X, Y, and Z coils will receive the magnetic flux generated by the TX Z-coil. This timing sequence is then repeated continuously while the EM transmitter is operating.

[0072] The following describes a non-limiting illustrative example that accommodates multiple users. For example, it accommodates up to four users with two transmitters each requiring eight TX frequencies. Generally, it is advantageous if these frequencies do not overlap. In such an embodiment, a scanning process can be implemented by the EM receiver to determine whether a particular frequency is being used in close proximity. FIG. 13G illustrates an example of scan timing 1310. This scanning can be performed by the EM receiver 1304 at initialization and periodically during the user's session. The scanning can be performed by intentionally turning off the TX in the transmitter 1302 and cycling through the RX (in the receiver 1304) to measure the presence of unintentional interference. If energy at that frequency is determined to be present, an alternative frequency can be selected. This scanning can also be shortened by monitoring one or two (rather than all three) of the three quadrature coils, since position and orientation (PnO) is not required within that slot.

[0073] Figure 13H is a block diagram that schematically illustrates another embodiment of a receiver 1304 in an EM tracking system. As compared to the exemplary FDM receiver illustrated in Figure 13B, a TDM switch replaces the individual paths from the three quadrature coils. The TDM switch can be controlled by an RF wireless interface (e.g., BLE). The TDM switch can utilize a timing protocol 1308 illustrated in Figure 13F.

[0074] In various embodiments, the time-division multiplexed TX and / or RX circuits described with reference to Figures 13C-13H may provide one or more of the following advantages: (A) Current Drain and Battery Life. By time-multiplexing the transmitter and receiver, the amount of current used can be reduced. This reduction results from the fact that high-current circuits, such as the transmitter, are no longer utilized 100% of the time. The current drain of the system can be reduced by slightly more than one-third compared to the FDM circuit shown in Figures 13A and 13B. (B) Material Cost Bill. The number of components used to achieve the same result is reduced in the TDM embodiment described above (compared to the FDM circuit in Figures 13A and 13B). Multiplexing signals through the same path reduces part count, and in this case, component cost should also be reduced by slightly more than one-third compared to their FDM counterparts. (C) PCB Area. Another advantage of part reduction can be the savings obtained in PCB area. The component count is reduced by almost two-thirds, and therefore the space required on the PCB is also reduced.

[0075] Another potential advantage may be reduced mass of the TX and RX circuits. For example, the FDM TX and RX circuits shown in Figures 13A and 13B utilize separate filter and amplifier paths for each of the three quadrature coils. In contrast, the TDM TX and RX circuits shown in Figures 13C and 13H share filter and amplifier paths.

[0076] In addition to eliminating sensor housings and multiplexing, saving on-head hardware, the signal-to-noise ratio may be increased by having more than one set of electromagnetic sensors, each set being relatively small relative to a single, larger coil set. Also, the lower frequency limit typically required to have multiple sensing coils in close proximity may be improved to facilitate improved bandwidth requirements. There is generally a trade-off with TD multiplexing in that multiplexing typically spreads RF signal reception in time, resulting in a noisier signal. Therefore, larger coil diameters may be used for multiplexed systems. For example, a multiplexed system may utilize a cubic coil sensor box with 9 mm side dimensions, while a non-multiplexed system may utilize only a cubic coil box with 7 mm side dimensions for similar performance. Therefore, there may be trade-offs when minimizing geometry and mass and selecting between FDM and TDM circuitry embodiments. Exemplary Automatic Gain Control for an Electromagnetic Tracking System

[0077] 13A and 13B, the FDM receiver (FIG. 13B) implements closed-loop gain control, while the FDM transmitter (FIG. 13A) does not implement gain control and is left to transmit at its maximum output power regardless of the received level. The receiver gain can be set by a DSP. For example, the received voltage on the receiver coil is fed directly into a first stage with gain control. The large voltage can be determined in the DSP, which can automatically adjust the gain of the first stage. Placing the gain control in the receiver may utilize more power in the transmitter, even when it is not needed. Therefore, it may be advantageous to employ automatic gain control (AGC, sometimes also referred to as adaptive gain control) on the transmitter side (rather than the receiver side), which may save space in the receiver system (that would otherwise be used for AGC) and thereby enable a much smaller and more portable receiver.

[0078] FIG. 13I is a block diagram that schematically illustrates an embodiment of an EM receiver 1304 that does not utilize automatic gain control (AGC). The first stage is no longer an AGC circuit (compare FIG. 13B), and the receiver is simply designed to have a constant gain. The level of the voltage received on the coil is determined by a DSP, which provides that information to a wireless (e.g., BLE) link. This BLE link can provide that information to the transmitter to control the TX level (see FIG. 13J).

[0079] FIG. 13J is a block diagram that schematically illustrates an example of an EM transmitter 1302 employing AGC. The EM transmitter 1302 of FIG. 13J can communicate with the receiver 1304 of FIG. 13I. A wireless link (e.g., BLE) communicates the received voltage level (from the BLE link on the receiver) to the MCU. The amplifier stage can have an adjustable gain controlled by the MCU. This can allow for current savings on the transmitter when the required received voltage is small.

[0080] Thus, the RX and TX circuit embodiments in Figures 13I and 13J employ AGC in the EM transmitter 1302 instead of the EM receiver 1304. This change from the RX and TX circuit embodiments in Figures 13A and 13B can allow for a smaller RX design and a more power efficient design, as TX power will be allowed to be reduced when necessary. Example of EM tracking of user head or hand pose

[0081] Referring to FIG. 14 , in one embodiment, after a user powers on their wearable computing system (160), the head-mounted component assembly may capture a combination of IMU and camera data (the camera data is used for SLAM analysis, e.g., by a beltpack processor, where more raw processing power may be present) to determine and update the head's pose (e.g., position or orientation) relative to a real-world global coordinate system (162). The user may also activate a handheld component, e.g., to play an augmented reality game (164), which may include an electromagnetic transmitter operably coupled to one or both of the beltpack and head-mounted component (166). One or more electromagnetic field coil receiver sets (e.g., a set of three differently oriented individual coils) coupled to the head-mounted component capture magnetic flux from the transmitters, which may be utilized to determine the position or orientation difference (or “delta”) between the head-mounted component and the handheld component (168). The combination of a head-mounted component that assists in determining pose relative to a global coordinate system and a handheld that assists in determining the relative location and orientation of the handheld to the head-mounted component allows the system to generally determine where each component is located relative to the global coordinate system and therefore the pose of the user's head, and the handheld pose can preferably be tracked with relatively low latency for presentation of augmented reality image features and interaction using movement and rotation of the handheld component (170).

[0082] Referring to Figure 15, an embodiment is illustrated that is somewhat similar to that of Figure 14, but in which the system has more sensing devices and configurations available to assist in determining the pose of both the head-mounted component (172) and the handheld components (176, 178), so that the user's head pose and handheld pose can preferably be tracked with relatively low latency for presentation of augmented reality image features and interaction using movement and rotation of the handheld component (180). Exemplary Stereo and Time-of-Flight Depth Sensing

[0083] 16A and 16B, various aspects of a configuration similar to that of FIG. 8 are shown. The configuration of FIG. 16A differs from that of FIG. 8 in that, in addition to a LIDAR (106) type depth sensor, the configuration of FIG. 16A features a general-purpose depth camera or depth sensor (154), which, for illustrative purposes, can be, for example, either a stereo triangulation depth sensor (such as a passive stereo depth sensor, a texture projection stereo depth sensor, or a structured light stereo depth sensor) or a time-of-flight depth sensor (such as a LIDAR depth sensor or a modulated emission depth sensor). Furthermore, the configuration of FIG. 16A has an additional forward-looking “world” camera (124, which may be a grayscale camera with a sensor capable of resolution in the 720p range) and a relatively high-resolution “photo camera” (156, which may be a full-color camera with a sensor capable of high resolution, e.g., 2 megapixels or higher). FIG. 16B shows a partial orthogonal view of the configuration of FIG. 16A for illustrative purposes, as further described below with reference to FIG. 16B.

[0084] Referring back to FIG. 16A and the aforementioned stereo versus time-of-flight depth sensor, each of these depth sensor types can be employed with wearable computing solutions such as those disclosed herein, but each has various advantages and disadvantages. For example, many depth sensors have challenges with black surfaces and shiny or reflective surfaces. Passive stereo depth sensing is a relatively simple method of using a depth camera or sensor to obtain triangulation to calculate depth, but can be challenging when a wide field of view ("FOV") is required and can require relatively significant computing resources. Furthermore, such sensor types can have challenges with edge detection, which may be important for certain future use cases. Passive stereo can have challenges with textureless walls, low-light conditions, and repeating patterns. Passive stereo depth sensors are available from manufacturers such as Intel and Aquifi. Stereo with texture projection (also known as "active stereo") is similar to passive stereo, except that a texture projector broadcasts a projection pattern onto the environment; the more texture is broadcast, the more accuracy is available in triangulation for depth calculation. Active stereo also requires relatively high computational resources, presents challenges when a wide FOV is required, and can be somewhat suboptimal in detecting edges, but it addresses some of the challenges of passive stereo in that it is effective against textureless walls, performs well in low light, and generally does not have problems with repeating patterns. Active stereo depth sensors are available from manufacturers such as Intel and Aquifi.

[0085] Stereo with structured light, such as the system developed by Primesense, Inc. and available under the trademark Kinect, and systems available from Mantis Vision, Inc., which generally utilize a single camera / projector pairing and projector, is unique in that it is configured to broadcast a pattern of dots known a priori. Essentially, the system knows the pattern to be broadcast and knows that the variable to be determined is depth. Such configurations can be relatively efficient in terms of computational load and can be very effective and efficient in many scenarios, although they can pose challenges in scenarios with wide FOV requirements and scenarios involving ambient light and patterns broadcast from other nearby devices. Using modulated time-of-flight depth sensors, such as those available from PMD Technologies, AG and SoftKinetic Inc., the emitter may be configured to transmit a wave, such as a sine wave, of amplitude-modulated light. In some configurations, camera components, which may be positioned nearby or even overlapping, receive return signals onto each of the camera component's pixels, and a depth mapping may be determined / calculated. Such a configuration can have a relatively compact geometry, high accuracy, and low computational load, but can present challenges in terms of image resolution (such as at the edge of an object), multipath errors (such as when the sensor is aimed at a reflection or glare angle, resulting in the detector receiving more than one return path, such that there is some depth detection aliasing).

[0086] Direct time-of-flight sensors, also referred to as LIDAR, are available from suppliers such as LuminAR and Advanced Scientific Concepts, Inc. Using these time-of-flight configurations, generally, a pulse of light (such as a picosecond-, nanosecond-, or femtosecond-long pulse of light) is sent with this light ping to envelop the world oriented around it. Each pixel on the camera sensor then waits for that pulse to return, and by knowing the speed of light, the distance at each pixel can be calculated. Such configurations can have many of the advantages of modulated time-of-flight sensor configurations (no baseline, relatively wide FOV, high accuracy, relatively low computational load, etc.) as well as relatively high frame rates, such as millions of Hertz. They are also relatively expensive, have relatively low resolution, are sensitive to bright light, and are prone to multipath errors. They can also be relatively large and heavy.

[0087] Referring to FIG. 16B, a partial top view is shown for illustrative purposes, featuring a user's eye (12), a camera (14, such as an infrared camera) with fields of view (28, 30), and a light or radiation source (16, such as infrared) directed toward the eye (12) to facilitate eye tracking, observation, and / or image capture. Three outward-facing world capture cameras (124) are shown with their FOVs (18, 20, 22), as well as a depth camera (154) and its FOV (24) and a photo camera (156) and its FOV (26). Depth information gathered from the depth camera (154) may be enhanced by using overlapping FOVs and data from other forward-facing cameras. For example, the system may be configured with a sub-VGA image from the depth sensor (154), a 720p image from the world camera (124), and possibly a 2-megapixel color image from the photo camera (156). Such a configuration would have four cameras sharing a common FOV, two with disparate visible spectrum images, one with color, and a third with relatively low-resolution depth. The system may be configured to perform segmentation in the grayscale and color images, fuse them, create a relatively high-resolution image from them, obtain several stereo correspondences, use a depth sensor to provide hypotheses about stereo depth, and use the stereo correspondences to obtain a more refined depth map that may be significantly better than that available from the depth sensor alone. Such a process may be run on local mobile processing hardware, or possibly using cloud computing resources along with data from others in the area (such as two people sitting nearby across a table from each other), resulting in a highly refined mapping. In another embodiment, all of the aforementioned sensors may be combined into a single integrated sensor to perform such functionality. Exemplary Dynamic Tuning of a Transmit Coil for EM Tracking

[0088] Referring to FIGS. 17A-17G, aspects of a dynamic transmit coil tuning configuration are shown for electromagnetic tracking that facilitates transmit coils to optimally operate at multiple frequencies per orthogonal axis, allowing multiple users to operate on the same system. Typically, electromagnetic tracking transmitters will be designed to operate at a fixed frequency per orthogonal axis. Using such an approach, each transmit coil is tuned with a static series capacitance that creates a resonance only at the frequency of operation. Such a resonance allows the maximum possible current through the coil, which in turn maximizes the generated magnetic flux. FIG. 17A illustrates a typical resonant circuit 1305 used to create resonance at a fixed operating frequency. Inductor "L" represents a single-axis transmit coil with an inductance of 1 mH and capacitance set to 52 nF; resonance is created at 22 kHz, as shown in FIG. 17B. FIG. 17C shows the current through the circuit 1305 plotted in FIG. 17A versus frequency; it should be understood that the current is greatest at the resonant frequency. If the system is expected to operate at any other frequency, the operating circuit will not be at the maximum possible current (which occurs at the resonant frequency of 22 kHz).

[0089] FIG. 17D illustrates an embodiment of a dynamically adjustable configuration for the transmitter circuit 1306 of the transmitter 1302 of an electromagnetic tracking system. The example circuit 1306 shown in FIG. 17D may be used in embodiments of the EM field emitters 402, 602, 1302 described herein. The circuit in FIG. 17D includes an oscillating voltage source 1702, a transmit (TX) coil, a high-voltage (HV) capacitor, and multiple low-voltage (LV) capacitors in a capacitor bank 1704 that can be selected to provide tuning for a desired resonant frequency. The dynamic frequency tuning may be set to achieve resonance (at a desired dynamically adjustable frequency) and obtain maximum current on the coil. Another example of a dynamically adjustable circuit 1306 is shown in FIG. 17E, where the adjustable capacitor 1706 (“C4”) may be adjusted to produce resonance at different frequencies, as shown in the simulated data illustrated in FIG. 17F. Tuning the adjustable capacitor can include switching between multiple different capacitors, as illustrated diagrammatically in the circuit shown in FIG. 17D. As shown in the embodiment of FIG. 17E, one of the quadrature coils of the electromagnetic tracker is simulated as inductor "L," and static capacitor ("C5") is a fixed high-voltage capacitor. This high-voltage capacitor experiences higher voltages due to resonance, and therefore its package size will generally be larger. Capacitor C4 is a dynamically switched capacitor with a different value and therefore experiences a lower maximum voltage and generally has a smaller geometric package, allowing for space savings. Inductor L3 can also be utilized to fine-tune the resonant frequency.

[0090] FIG. 17F illustrates an example of a resonance that can be achieved by circuit 1306 of FIG. 17E. In FIG. 17F, the higher curve (248) shows the voltage Vmid-Vout across capacitor C5, and the lower curve (250) shows the voltage Vout across capacitor C4. Note that as the capacitance of C4 is varied, the resonant frequency is also changed (in this example, from about 22 kHz to 30 kHz), and the voltage across C5 (Vmid-Vout; curve 248) is higher than that across C4 (Vout; curve 250). This will generally allow for a smaller package area on C4, since multiple of this capacitor, e.g., one capacitor per resonant frequency of operation, will generally be used in the system (see, e.g., multiple LV capacitors in capacitor bank 1704 shown in FIG. 17D). FIG. 17G is a plot of current versus frequency illustrating that the maximum current achieved follows resonance regardless of the voltage across the capacitor. Thus, dynamically adjustable circuit embodiments can provide increased or maximum current in the transmitter coil across multiple frequencies, allowing improved or optimized performance for multiple users of a single EM tracking system. Exemplary Audio Noise Cancellation for EM Tracking Systems

[0091] Audio speakers (or any external magnets) can generate magnetic fields that can unintentionally interfere with the magnetic fields generated by the EM field emitters of an EM tracking system. Such interference can degrade the accuracy or reliability of the location estimates provided by the EM tracking system.

[0092] As AR devices evolve, they become more complex and incorporate more technologies that need to coexist and operate independently. EM tracking systems rely on the reception (by an EM sensor) of minute changes in magnetic flux (generated by an EM field emitter) to determine the 3-D position of the EM sensor (and thereby the 3-D position of the object to which the sensor is attached or embedded). Audio speakers residing in close proximity to the EM tracking sensor coil can emit magnetic flux that can interfere with the EM tracking system's ability to calculate true position.

[0093] 18A-18C, the electromagnetic tracking system may be bounded to function below approximately 30 kHz, slightly above the audible range for human hearing. FIG. 18A shows a configuration in which an audio speaker 1820 is in close proximity to the EM sensor 604. The audio speaker 1820 is driven by a time-varying voltage source 1822 and an amplifier 1824. The magnetic field of the speaker 1820 can cause unintentional magnetic interference to the EM tracking system because the speaker generates noise in the magnetic field sensed by the coil of the EM sensor 604. In some implementations, the distance between the audio speaker 1820 and the EM sensor 604 can be increased to reduce the received interference. However, the magnetic flux from the speaker increases as the cube of the distance from the sensor (1 / r 3 ), there will be a point where a large distance provides very little attenuation to the interference. Audio speakers (e.g., speaker 66 shown in FIGS. 2A-2D ) will typically be used within AR devices to provide an audio experience to the wearer of the AR device. Thus, it may be common for the audio speaker to be relatively close to an EM sensor that is also located on the AR device (see, e.g., EM sensor 604 located near speaker 66 in exemplary wearable display device 58 shown in FIG. 11A ). The magnetic field from the audio speaker may interfere with the EM field sensed by the EM sensor of the EM tracking system.

[0094] Referring to FIG. 18A, there may be some audio systems that generate noise within the usable frequencies for such an electromagnetic tracking system. Furthermore, audio speakers typically have a magnetic field and one or more coils that may also interfere with the electromagnetic tracking system. Referring to FIG. 18B, a block diagram is shown for an example of a noise cancellation system 1830 for an electromagnetic tracking system. Because unintentional EM interference is a known entity (because the signal provided to the audio speaker 1820 by the voltage source 1822 is known or can be measured), this knowledge can be used to cancel the EM interference from the audio speaker 1820 and improve the performance of the EM tracking system. In other words, the audio signal generated by the system may be utilized to eliminate magnetic interference from the speaker received by the coil of the EM sensor 604. As shown diagrammatically in FIG. 18B, the noise cancellation circuit 1830 may be configured to receive a corrupted signal 1850a from the EM sensor 604 and a signal 1850b from the audio system. A noise cancellation system can combine the signals 1850a, 1850b and cancel the interference received from the audio speaker 1820 to provide an uncorrupted sensor signal 1850c.

[0095] FIG. 18C is a plot showing an illustrative, non-limiting example of how an audio signal 1850b may be added to an inverted, corrupted sensor signal 1850a to cancel interference and provide a substantially uncorrupted sensor signal 1850c. The top plot, V(noise), is the noise signal 1850b added to the EM tracking system by the audio speaker 1820. The bottom plot, V(cancel), is an inverted audio signal (e.g., −V(noise)), which, when added together, has the effect of eliminating noise corruption from the audio. In other words, the noise cancellation system receives a corrupted signal 1850a, which is the sum, V(sensor)+V(noise), of the true EM sensor signal V(sensor), which represents the signal from the EM transmitter coil, and a noise signal. By adding the inverted audio signal −V(noise) to the corrupted signal 1850a, the uncorrupted signal V(sensor) 1850c is restored. The uncorrupted signal 1850c reflects the response of the sensor 604 as if the audio speaker 604 were not present, and thus reflects the EM transmitter field at the location of the sensor 604. Equivalently, the noise signal 1850b can be subtracted from the corrupted signal 1850a to restore the uncorrupted signal V(sensor) 1850c. The noise cancellation can result in cancellation of substantially all (e.g., >80%, >90%, >95%, or more) of the noise signal (e.g., from the audio speaker). The present noise cancellation technique is not limited to canceling only audio speaker noise, but can also be applied to other sources of noise interference to the EM sensor signal, provided that a measurement (or estimate) of the noise signal can be determined (which can then be removed from the EM sensor signal, as described above).

[0096] FIG. 18D is a flowchart showing an example method 1800 for canceling interference received by an EM sensor in an EM tracking system. Method 1800 can be implemented, for example, by a hardware processor in an AR device, such as local processing and data module 70, or by a hardware processor in the EM tracking system. In block 1802, the method receives a noise signal from an electromagnetic sensor. As described above, the noise signal can be caused by interference from a nearby audio speaker that generates electromagnetic interference. In block 1804, the method receives a signal from an EM interference source. For example, the signal can be signal 1850b used to drive the audio speaker (see, for example, FIG. 18B). In block 1806, the noise signal and the interference signal are combined to obtain a de-noised EM signal. For example, the interference signal can be inverted and added to the noise signal, or the interference signal can be subtracted from the noise signal. In block 1808, the de-noised signal can be used to determine the location of the EM sensor. Locations obtained using the denoised signal (compared to using the noisy signal) are more accurate and reliable.

[0097] Thus, the foregoing provides a method for removing unintentional noise generated by an audio speaker in proximity to an EM tracker sensor. The method employs a noise cancellation method that uses known information about the audio and removes it from the EM tracking signal. The system may be used when sufficient physical separation of the audio speaker and the EM sensor coil cannot be achieved (so that interference is sufficiently low). While the foregoing describes interference noise as being generated by an audio speaker, this is for purposes of illustration and not limitation. The foregoing embodiments can be applied to any interference signal that can be measured and then subtracted from a corrupted sensor signal. Example Calibration of a Vision System

[0098] 19 , in one embodiment, a known pattern 1900 of light or other emitters (such as a circular pattern) may be utilized to aid in the calibration of a vision system. For example, the circular pattern may be utilized as a reference. That is, the orientation of an object, such as the handheld totem device 606, may be determined as a camera or other capture device with a known orientation captures the shape of the pattern while an object coupled to the pattern is reoriented. Such orientation may be compared to that resulting from an associated IMU on the object (e.g., the totem) for use in error determination and calibration. Still referring to FIG. 19 , the pattern of light 1900 may be produced by light emitters (e.g., multiple LEDs) on the handheld totem 606 (schematically represented in FIG. 19 as a cylinder). As shown in FIG. 19 , when the totem is viewed directly on by a camera on the AR headset 58, the pattern of light 1900 appears circular. When the totem 606 is tilted in other orientations, the pattern 1900 appears elliptical. The pattern of light 1900 can be identified using computer vision techniques and the orientation of the totem 606 can be determined.

[0099] In various implementations, the augmented reality device may include a computer vision system configured to implement one or more computer vision techniques and identify patterns of light (or perform other computer vision procedures used or described herein). Non-limiting examples of computer vision techniques include Scale Invariant Feature Transform (SIFT), Speed-Up Robust Features (SURF), Orientation FAST and Rotation BRIEF (ORB), Binary Robust Invariant Scalable Keypoints (BRISK), Fast Retinal Keypoints (FREAK), Viola-Jones algorithm, Eigenfaces approach, Lucas-Kanade algorithm, Horn-Schunk algorithm, Mean-shift algorithm, visual simultaneous localization and mapping (vSLAM) techniques, sequential Bayes estimators (e.g., Kalman filter, extended Kalman filter, etc.), bundle adjustment, adaptive thresholding (and other thresholding techniques), iterative nearest neighbor (ICP), semi-global matching (SGM), semi-global block matching (SGBM), feature point histograms, various machine learning algorithms (e.g., support vector machines, k-nearest neighbor algorithms, naive Bayes, neural networks (including convolutional or deep neural networks), or other supervised / unsupervised models, etc.), etc. Exemplary Circuits for a Subsystem of a Wearable Display Device

[0100] 20A-20C, a configuration is shown with a summing amplifier 2002 to simplify the circuitry between two subsystems or components of a wearable computing configuration, such as a head-mounted component and a beltpack component. In a conventional configuration, each coil 2004 (left in FIG. 20A) of the electromagnetic tracking sensor 604 is associated with an amplifier 2006, and three distinct amplified signals can be transmitted through cabling to the summing amplifier 2002 and other components (e.g., processing circuitry as shown in FIG. 20B). In the illustrated embodiment, the three distinct amplified signals may be directed to the summing amplifier 2002, which advantageously produces a single amplified signal directed along a simplified cable 2008, each signal being at a different frequency. The summing amplifier 2002 may be configured to amplify all three signals received by the amplifier, and then, after analog-to-digital conversion, to a receiving digital signal processor (as shown in FIG. 20B) and separate the signals at the other end. Gain control may be used. 20C illustrates filters for each frequency (F1, F2, and F3), so that the signals may be separated back and output in such stages. The three signals may be analyzed by a computational algorithm (e.g., to determine the sensor pose), and the position or orientation results may be used by the AR system (e.g., to appropriately display virtual content to the user based on the user's instantaneous head pose). Exemplary EM Tracking System Update

[0101] 21, electromagnetic ("EM") tracking updates may be relatively "expensive" in terms of power for a portable system and may not allow for very high frequency updates. In a "sensor fusion" configuration, more frequently updated localization information from another sensor, such as an IMU, may be combined with data from another sensor, such as an optical sensor (e.g., a camera or depth camera), which may or may not be relatively high frequency. The net effect of fusing all of these inputs places lower demands on the EM system and provides faster updates.

[0102] Referring back to FIG. 11B , a distributed sensor coil configuration is shown for an AR device 58. Referring to FIG. 22A , an AR device 58 with a single electromagnetic sensor device (604), such as a housing containing three orthogonal sensing coils, one for each of the X, Y, and Z directions, may be coupled to a wearable component (58) for six-degree-of-freedom tracking, as described above. Also, as previously mentioned, such a device may be de-integrated, with three sub-portions (e.g., coils) attached to different locations on the wearable component (58), as shown in FIGS. 22B and 22C . Referring to FIG. 22C , to provide further design alternatives, each individual sensor coil may be replaced with a group of similarly oriented coils such that the overall magnetic flux for any given orthogonal direction is captured by a group (148, 150, 152) rather than by a single coil per orthogonal direction. In other words, rather than one coil per orthogonal direction, a group of smaller coils may be utilized, and their signals may be aggregated to form a signal for that orthogonal direction. In another embodiment where a particular system component, such as the head-mounted component (58), features two or more electromagnetic coil sensor sets, the system may be configured to selectively utilize sensor and emitter pairs that are closest to each other (e.g., within 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, or 10 cm) to improve or optimize system performance. Example of Recalibrating a Wearable Display System

[0103] Referring to Figures 23A-23C, it may be useful to recalibrate a wearable computing system such as those discussed herein. In one embodiment, acoustic (e.g., ultrasound) signals generated at the transmitter, along with an acoustic sensor (e.g., microphone) at the receiver and acoustic time-of-flight calculations, may be utilized to determine the acoustic propagation delay between the transmitter and receiver, and thereby the distance between the transmitter and receiver (since the speed of sound is known). Figure 23A shows that, in one embodiment, three coils on the transmitter are excited with bursts of sine waves, and simultaneously, the ultrasound transducer may be excited with bursts of sine waves, preferably at the same frequency as one of the coils. Figure 23B illustrates that the EM receiver may be configured to receive three EM waves using the X, Y, and Z sensor coils and acoustic ultrasound using a microphone (MIC). The total distance may be calculated from the amplitudes of the three EM signals. The time of flight (sound propagation delay time 2300) may be calculated by comparing the timing of the acoustic (microphone) response 2302 with the response of the EM coil 2304 (see, e.g., FIG. 23C). This may also be used to calculate distance. A comparison of the electromagnetically calculated distance and acoustic delay time 2300 can be used to calibrate the EM TX or RX circuitry (e.g., by a correction factor).

[0104] Referring to FIG. 24A, in another embodiment, in an augmented reality system featuring a camera, distance may be calculated by measuring the size in pixels of a known size-matching feature (depicted as an arrow in FIG. 24A) on another device, such as a handheld controller (e.g., controller 606).

[0105] Referring to FIG. 24B, in another embodiment, in an augmented reality system featuring a depth sensor, such as an infrared (“IR”) depth sensor, distance may be calculated by such depth sensor and reported directly to the controller.

[0106] Referring to Figures 24C and 24D, once the total distance is known, either a camera or a depth sensor can be used to determine location in space. The augmented reality system may be configured to project one or more virtual alignment targets to the user. The user may align the controller with the target, and the system can calculate the position from both the EM response and the direction of the virtual target plus the previously calculated distance. Roll angle calibration may be performed by aligning the projected virtual target from the user with a known feature on the controller. Yaw and pitch angles may be calibrated by presenting the virtual target to the user and having the user align two features on the controller with the target (as with aiming a rifle).

[0107] 25A and 25B, there may be inherent ambiguities associated with EM tracking systems. That is, the receiver will generate similar responses at two diagonally opposite locations around the transmitter. For example, FIG. 25A shows handheld device 606 and residual image device 606a generating similar responses. Such challenges are particularly relevant in systems where both the transmitter and receiver may be mobile with respect to each other.

[0108] In one embodiment, the system may use an IMU sensor to determine whether the user is on the positive or negative side of a reference (e.g., symmetry) axis. In an embodiment, such as those described above, featuring a world camera and a depth camera, the system can use that information to detect whether the handheld component (e.g., handheld 2500 in FIG. 25B) is on the positive or negative side of the reference axis. If the handheld 2500 is outside the field of view of the camera and / or depth sensor, the system may be configured to determine (or the user may determine) that the handheld component 2500 is within a 180-degree zone directly behind the user, e.g., at afterglow position 2500a, as shown in FIG. 25B. Additional Aspects and Benefits

[0109] In a first aspect, a head-mounted display system includes a display positionable in front of a wearer's eyes, an electromagnetic (EM) field emitter configured to generate a magnetic field having a frequency, an EM sensor configured to sense the magnetic field at that frequency, and a processor programmed to receive a signal indicative of the sensed magnetic field from the EM sensor, analyze the received signal, and determine a position or orientation of the EM sensor.

[0110] In a second aspect, the head-mounted display system of aspect 1, wherein the display comprises a light field display.

[0111] In a third aspect, the head-mounted display system of aspect 1 or aspect 2, wherein the EM field emitter comprises time division multiplexing (TDM) circuitry.

[0112] In a fourth aspect, the head-mounted display system of aspect 3, wherein the TDM circuitry comprises a single amplifier circuit that is TDM switched to each of the multiple radio frequency (RF) transmitter coils.

[0113] In a fifth aspect, the head-mounted display system of aspect 3 or aspect 4, wherein the EM field emitter is configured to dynamically adjust frequency.

[0114] In a sixth aspect, the head-mounted display system of aspect 5, wherein to dynamically adjust the frequency, the EM field emitter is configured to vary the capacitance of a capacitor or select between multiple capacitors in a capacitor bank.

[0115] In a seventh aspect, a head-mounted display system described in any one of aspects 3-6, wherein the EM field emitter comprises a first wireless interface, the EM sensor comprises a second wireless interface and a TDM circuit, and the EM field emitter and the EM sensor are configured to establish a wireless link between the first wireless interface and the second wireless interface and synchronize via the wireless link.

[0116] In an eighth aspect, a head-mounted display system described in any one of aspects 3-7, wherein the EM field emitter is configured to transmit an EM pulse to the EM sensor, and the EM field emitter or the EM sensor is configured to determine a timing difference between the EM field emitter clock and the EM sensor clock.

[0117] In a ninth aspect, the head-mounted display system of aspect 7 or aspect 8, wherein the EM field emitter comprises a first coil and a second coil, and the EM field emitter is configured to apply a TDM timing protocol, wherein the first coil transmits for a first period while the second coil does not substantially transmit during the first period, and the second coil transmits for a second period different from the first period while the first coil does not substantially transmit during the second period.

[0118] In a tenth aspect, a head-mounted display system described in any one of aspects 7-9, wherein the EM sensor is configured to scan for unintentional RF interference at the frequency for a third period of time and, in response to determining the presence of unintentional RF interference at the frequency, switch to an alternative frequency different from the frequency.

[0119] In an eleventh aspect, the head-mounted display system of aspect 10 is configured such that the EM field emitter substantially ceases transmission during a third period of time.

[0120] In a twelfth aspect, the head-mounted display system of any one of aspects 1-11, wherein the EM field emitter includes an automatic gain control (AGC) circuit.

[0121] In a thirteenth aspect, the head-mounted display system of aspect 12, wherein the EM sensor does not include an AGC circuit.

[0122] In a fourteenth aspect, a head-mounted display system as described in aspect 12 or aspect 13, wherein the AGC circuit of the EM field emitter is configured to receive a voltage level for the coil within the EM sensor and adjust a gain for the amplification stage of the EM field emitter based, at least in part, on the received voltage level.

[0123] In a fifteenth aspect, the head-mounted display system is a head-mounted display system described in any one of aspects 1-14, wherein the head-mounted display system further comprises an audio speaker and the EM sensor comprises a noise cancellation circuit.

[0124] In a sixteenth aspect, the head-mounted display system described in aspect 15 is configured to receive a first signal from the EM sensor and a second signal from the audio speaker, combine the first signal and the second signal, and provide a noise cancellation signal.

[0125] In a seventeenth aspect, the head-mounted display system described in aspect 16 is configured such that, to combine the first signal and the second signal, the noise cancellation circuit (a) inverts the second signal and adds the inverted second signal to the first signal, or (b) subtracts the second signal from the first signal.

[0126] In an eighteenth aspect, the head-mounted display system of any one of aspects 1-17 further comprises a user input totem, the user input totem comprising an EM field emitter.

[0127] In a nineteenth aspect, an electromagnetic (EM) tracking system includes an EM field emitter including a first transmitter coil configured to generate a first magnetic field having a first frequency, a second transmitter coil configured to generate a second magnetic field having a second frequency, and a third transmitter coil configured to generate a third magnetic field having a third frequency, wherein the EM field emitter includes a first time division multiplexing (TDM) circuit configured to switch power between the first transmitter coil, the second transmitter coil, and the third transmitter coil.

[0128] In a twentieth aspect, the EM tracking system of aspect 19, wherein the first transmitter coil, the second transmitter coil, and the third transmitter coil are co-located along mutually orthogonal axes.

[0129] In a twenty-first aspect, the EM tracking system of aspect 19 or aspect 20 is configured to dynamically adjust the first frequency, the second frequency, or the third frequency.

[0130] In a 22nd aspect, the EM tracking system of any one of aspects 19-21, wherein the first, second, or third frequency is dynamically adjusted and the EM field emitter is configured to vary the capacitance of a capacitor or select between multiple capacitors in a capacitor bank.

[0131] In a twenty-third aspect, the EM tracking system of any one of aspects 19-22, wherein the EM field emitter is configured with an automatic gain control (AGC) circuit.

[0132] In a twenty-fourth aspect, the EM tracking system of aspect 23, wherein the AGC circuit comprises a control loop between a digital signal processor (DSP) and the amplification stage.

[0133] In a twenty-fifth aspect, an EM tracking system as described in aspect 23 or aspect 24, wherein the EM field emitter is configured to receive a voltage level associated with a coil in the EM sensor and adjust a gain associated with an amplification stage of the EM field emitter based, at least in part, on the received voltage level.

[0134] In a 26th aspect, the EM tracking system of any one of aspects 19-25 further includes an EM sensor having a first receiver coil configured to sense a first magnetic field having a first frequency, a second receiver coil configured to sense a second magnetic field having a second frequency, and a third receiver coil configured to sense a third magnetic field having a third frequency, wherein the EM field sensor includes a second time division multiplexing (TDM) circuit configured to switch power between the first receiver coil, the second receiver coil, and the third receiver coil.

[0135] In a twenty-seventh aspect, the EM tracking system is configured to synchronize the EM field emitter and the EM sensor via a wireless link between the EM field emitter and the EM sensor, as described in aspect 26.

[0136] In a twenty-eighth aspect, an EM tracking system as described in aspect 27, wherein the EM field emitter is configured to transmit an EM pulse to the EM sensor, and the EM field emitter or the EM sensor is configured to determine a timing difference between the EM field emitter clock and the EM sensor clock.

[0137] In a 29th aspect, an EM tracking system described in any one of aspects 26-28, wherein during a first period, the first transmitter coil is configured to generate a first magnetic field having a first frequency, during which the second transmitter coil and the third transmitter coil do not substantially transmit respective second magnetic fields and third magnetic fields, and the first receiver coil, the second receiver coil of the EM sensor, and the third receiver coil are configured to be activated sequentially.

[0138] In a 30th aspect, the EM tracking system described in aspect 29, wherein during a second period following the first period, the second transmitter coil is configured to generate a second magnetic field having a second frequency, during which the first transmitter coil and the third transmitter coil are configured to substantially not transmit separate first and third magnetic fields, and the first receiver coil, second receiver coil, and third receiver coil of the EM sensor are configured to be activated sequentially.

[0139] In a thirty-first aspect, the EM tracking system of any one of aspects 26-30 is configured to scan frequencies when in use.

[0140] In a thirty-second aspect, the EM tracking system described in aspect 31, wherein during a third period, the first transmitter coil is configured to substantially not transmit the first magnetic field, and the first receiver coil is activated and configured to measure the presence of interference at the first frequency.

[0141] In a thirty-third aspect, the EM tracking system of aspect 32 is configured to change the first frequency to a different frequency in response to detecting interference at the first frequency.

[0142] In a thirty-fourth aspect, an EM tracking system described in any one of aspects 26-33, wherein the EM sensor is configured to receive an interference signal representative of a magnetic interference source, at least partially cancel the interference signal, and output a sensor signal that is substantially free of the magnetic interference source.

[0143] In a thirty-fifth aspect, an electromagnetic (EM) tracking system includes an EM field emitter including an automatic gain control (AGC) circuit and a transmitter coil, and an EM sensor without the AGC circuit, the EM sensor including a sensor coil.

[0144] In a thirty-sixth aspect, the EM sensor is configured to wirelessly communicate the sensor coil signal level to the EM field emitter, and the EM field emitter is configured to adjust the gain of the transmitter coil based, at least in part, on the sensor coil signal level, in an EM tracking system described in aspect 35.

[0145] In a thirty-seventh aspect, the EM tracking system of aspect 35 or aspect 36 is configured such that the EM field emitter is configured to dynamically adjust the radio frequency emitted by the transmitter coil.

[0146] In a thirty-eighth aspect, the EM tracking system of any one of aspects 35-37, wherein the EM field emitter and the EM sensor are configured to operate using time division multiplexing.

[0147] In a 39th aspect, an EM tracking system described in any one of aspects 35-38, wherein the EM sensor is configured to receive an interference signal representative of a magnetic interference source, at least partially cancel the interference signal, and output a sensor signal that is substantially free of the magnetic interference source.

[0148] In a fortieth aspect, a head-mounted augmented reality (AR) display device is provided, comprising the EM tracking system described in any one of aspects 35-39.

[0149] In a forty-first aspect, the head-mounted AR display device described in aspect 40, wherein the EM sensor is arranged on a frame of the AR display device.

[0150] In a forty-second aspect, the head-mounted AR display device of aspect 40 or aspect 41, wherein the EM field emitter is disposed within a handheld user input totem.

[0151] In a forty-third aspect, an augmented reality display system includes a display configured to project a virtual image onto a wearer's eye, a frame configured to mount the display in front of the wearer's eye, an electromagnetic (EM) field emitter configured to generate a magnetic field, an EM sensor configured to sense the magnetic field, one of the EM field emitter or the EM sensor mechanically coupled to the frame and the other of the EM field emitter or the EM sensor mechanically coupled to a component of the augmented reality display system that is independently movable relative to the frame, and a hardware processor programmed to receive a signal indicative of the sensed magnetic field from the EM sensor, analyze the received signal, and determine a position or orientation of the EM sensor.

[0152] In a forty-fourth aspect, the augmented reality display system of aspect 43, wherein the display comprises a light field display.

[0153] In a forty-fifth aspect, the augmented reality display system of aspect 43 or aspect 44, wherein the component comprises a user input totem or belt pack, the EM sensor is mechanically coupled to the frame, and the EM field emitter is mechanically coupled to the user input totem or belt pack.

[0154] In a forty-sixth aspect, the augmented reality display system of any one of aspects 43-45 further comprises an audio speaker, wherein the augmented reality display system comprises a noise cancellation circuit configured to cancel magnetic interference in a sensed magnetic field generated by the audio speaker. The audio speaker may be mechanically coupled to the frame.

[0155] In aspect 47, the augmented reality display system described in aspect 46, wherein the noise cancellation circuit is configured to receive a first signal from the EM sensor, receive a second signal from the audio speaker, combine the first signal and the second signal, and provide a noise cancellation signal.

[0156] In aspect 48, the augmented reality display system described in aspect 47, wherein to combine the first signal and the second signal, the noise cancellation circuit is configured to (a) invert the second signal and add the inverted second signal to the first signal, or (b) subtract the second signal from the first signal.

[0157] In a forty-ninth aspect, the EM field emitter comprises a first transmitter coil configured to generate a first magnetic field having a first frequency, a second transmitter coil configured to generate a second magnetic field having a second frequency, and a time division multiplexing (TDM) circuit configured to switch power between the first transmitter coil and the second transmitter coil, respectively, in an augmented reality display system described in any one of aspects 43-48.

[0158] In a 50th aspect, the augmented reality display system described in aspect 49, wherein the TDM circuitry comprises a single amplifier circuit that is TDM switched to each of the first and second transmitter coils.

[0159] In aspect 51, the augmented reality display system of aspect 49 or aspect 50 is configured such that the first transmitter coil and the second transmitter coil are arranged along mutually orthogonal axes.

[0160] In aspect 52, the augmented reality display system of any one of aspects 43-51 is configured to dynamically adjust the first frequency or the second frequency.

[0161] In a 53rd aspect, the augmented reality display system described in aspect 52, wherein the EM field emitter is configured to vary the capacitance of a capacitor or select between multiple capacitors in a capacitor bank to dynamically adjust the first frequency or the second frequency.

[0162] In aspect 54, the augmented reality display system of any one of aspects 43-53 is configured such that the EM field emitter includes a first wireless interface, the EM sensor includes a second wireless interface and a second TDM circuit, and the EM field emitter and the EM sensor are configured to establish a wireless link between the first wireless interface and the second wireless interface and synchronize timing of the EM field emitter clock and the EM sensor clock via the wireless link.

[0163] In a fifty-fifth aspect, the augmented reality display system of any one of aspects 43-54 is configured such that the EM field emitter is configured to transmit an EM pulse to the EM sensor, and the EM field emitter or the EM sensor is configured to determine a timing difference between an EM field emitter clock and an EM sensor clock. In another aspect, the AR display system of any one of aspects 43-54 may be configured such that the EM field emitter includes an acoustic generator and the EM sensor includes an acoustic sensor. The AR display system may be configured to determine a time of flight of a first distance between the emitter and the sensor based, at least in part, on a measured delay between the acoustic signal and the electromagnetic signal. The AR display system may further be configured to determine a second distance between the emitter and the sensor based, at least in part, on an amplitude of the electromagnetic signal. The AR display system may further be configured to calibrate the system based, at least in part, on a comparison of the first distance and the second distance.

[0164] In aspect 56, the EM field emitter is configured to apply a TDM timing protocol, wherein the first transmitter coil transmits for a first period while the second transmitter coil does not substantially transmit during the first period, and the second transmitter coil transmits for a second period different from the first period while the first transmitter coil does not substantially transmit during the second period, an augmented reality display system described in any one of aspects 49-55.

[0165] In aspect 57, an augmented reality display system described in any one of aspects 43-56, wherein the EM field emitter is configured to generate a magnetic field at a first frequency, and the EM sensor is configured to scan for unintentional radio frequency (RF) interference at the first frequency and, in response to determining the presence of unintentional RF interference at the first frequency, switch to a second frequency different from the first frequency.

[0166] In aspect 58, an augmented reality display system described in any one of aspects 43-57, wherein the EM field emitter includes an automatic gain control (AGC) circuit configured to receive a voltage level for the coil within the EM sensor and adjust a gain for an amplification stage of the EM field emitter based, at least in part, on the received voltage level.

[0167] In a fifty-ninth aspect, a method of operating an electromagnetic (EM) tracking system for an augmented reality (AR) display system is provided, wherein the AR display system comprises a head-mounted AR display, an EM emitter, and a portable user input device comprising an EM sensor. The method includes emitting a time-varying magnetic field by the EM emitter in the portable user input device, detecting the time-varying magnetic field by the EM sensor, determining a posture of the EM sensor based at least in part on the detected magnetic field, determining virtual content for display to a user of the AR display system based at least in part on the determined posture, and displaying the virtual content by the head-mounted AR display. The head-mounted AR display may comprise a light field display.

[0168] In a 60th aspect, the method of aspect 59 further comprises the step of time-synchronizing the EM emitter and the EM sensor.

[0169] In a 61st aspect, the method described in aspect 59 or aspect 60 further includes a step of canceling magnetic interference from the detected magnetic field.

[0170] In aspect 62, a method according to any one of aspects 59-61 further comprises correlating real world coordinates associated with the pose of the EM sensor with virtual world coordinates associated with the virtual content.

[0171] In a 63rd aspect, an AR display system is operated according to the method of any one of aspects 59-62. The AR display system may include a light field display. Additional Considerations

[0172] Each of the processes, methods, and algorithms described herein and / or depicted in the accompanying figures may be embodied in code modules executed by one or more physical computing systems, hardware computer processors, application-specific circuits, and / or electronic hardware configured to execute specific computer instructions, and thereby may be fully or partially automated. For example, a computing system may include a general-purpose computer (e.g., a server) or a special-purpose computer programmed with specific computer instructions, special-purpose circuitry, etc. Code modules may be compiled and linked into an executable program, installed in a dynamic link library, or written in an interpreted programming language. In some implementations, particular operations and methods may be performed by circuitry specific to a given function.

[0173] Furthermore, certain implementations of the functionality of the present disclosure may be sufficiently mathematically, computationally, or technically complex that special-purpose hardware (utilizing appropriate specialized executable instructions) or one or more physical computing devices may be required to perform the functionality, e.g., due to the amount or complexity of the calculations involved or to provide results in substantially real time. For example, a video may contain many frames, each frame may have millions of pixels, and specifically programmed computer hardware may be required to process the video data to provide the desired image processing task or application in a commercially reasonable amount of time.

[0174] Code modules or any type of data may be stored on any type of non-transitory computer-readable medium, such as physical computer storage devices, including hard drives, solid-state memory, random-access memory (RAM), read-only memory (ROM), optical disks, volatile or non-volatile storage devices, combinations of the same, and / or the like. The methods and modules (or data) may also be transmitted as data signals (e.g., as part of a carrier wave or other analog or digital propagated signal) generated over various computer-readable transmission media, including wireless-based and wired / cable-based media, and may take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps may be stored, persistently or otherwise, in any type of non-transitory tangible computer storage device or communicated via a computer-readable transmission medium.

[0175] Any process, block, state, step, or functionality in the flow diagrams described herein and / or depicted in the accompanying figures should be understood as potentially representing a code module, segment, or portion of code, comprising one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in the process. Various processes, blocks, states, steps, or functionality can be combined, rearranged, added, deleted, modified, or otherwise changed from the illustrative examples provided herein. In some embodiments, additional or different computing systems or code modules may perform some or all of the functionality described herein. The methods and processes described herein are also not limited to any particular sequence, and the blocks, steps, or states associated therewith can be performed in other suitable sequences, e.g., serially, in parallel, or in some other manner. Tasks or events may be added to or removed from the disclosed exemplary embodiments. Furthermore, the separation of various system components in the implementations described herein is for illustrative purposes and should not be understood as requiring such separation in all implementations. It should be understood that the described program components, methods, and systems may generally be integrated together in a single computer product or packaged in multiple computer products. Many implementation variations are possible.

[0176] The processes, methods, and systems can be implemented in a network (or distributed) computing environment. Network environments include enterprise-wide computer networks, intranets, local area networks (LANs), wide area networks (WANs), personal area networks (PANs), cloud computing networks, crowdsourced computing networks, the Internet, and the World Wide Web. The network can be a wired or wireless network or any other type of communication network.

[0177] The present invention includes methods that may be performed using the subject devices. The methods may include the act of providing such a suitable device. Such provision may be performed by an end user. In other words, the act of "providing" merely requires the end user to obtain, access, access, locate, configure, activate, power on, or otherwise act upon the requisite device in the subject method, so as to provide the requisite device. Methods described herein may be carried out in any order of the recited events, and in the recited order of events, that is logically possible.

[0178] The systems and methods of the present disclosure each have several innovative aspects, none of which is solely responsible for or required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of one another or combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Various modifications of the implementations described in the present disclosure will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations without departing from the spirit or scope of the present disclosure. Therefore, the claims are not intended to be limited to the implementations shown herein, but are to be accorded the widest scope consistent with the present disclosure, the principles, and novel features disclosed herein.

[0179] Certain features described herein in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation can also be implemented separately in multiple implementations or in any suitable subcombination. Furthermore, while features may be described above as operative in a combination and may even be initially claimed as such, one or more features from the claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or variation of the subcombination. No single feature or group of features is required or essential to every embodiment.

[0180] Conditional statements used herein, such as "can," "could," "might," "may," "e.g.," and the like, among others, are intended to generally convey that certain embodiments include certain features, elements, and / or steps, while other embodiments do not, unless specifically stated otherwise or understood otherwise within the context as used. Thus, such conditional statements are generally not intended to imply that features, elements, and / or steps are in any way required for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether those features, elements, and / or steps are to be included or performed in any particular embodiment, with or without authorial input or prompting. The terms "comprise," "include," "have," and the like are synonymous and used inclusively in a non-limiting manner and do not exclude additional elements, features, acts, operations, etc. Also, the term "or" is used in its inclusive sense (and not its exclusive sense); thus, for example, when used to connect a list of elements, the term "or" refers to one, some, or all of the elements in the list. In addition, the articles "a," "an," and "the," as used in this application and the appended claims, should be construed to mean "one or more" or "at least one," unless otherwise specified. Unless specifically defined herein, all technical and scientific terms used herein should be given the broadest possible commonly understood meaning while maintaining validity of the claims.

[0181] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single elements. As an example, "at least one of A, B, or C" is intended to cover A, B, C, A and B, A and C, B and C, and A, B, and C. Transitional phrases such as "at least one of X, Y, and Z," unless specifically stated otherwise, are generally understood differently in the context in which they are used to convey that an item, term, etc. may be at least one of X, Y, or Z. Thus, such transitional phrases generally are not intended to suggest that an embodiment requires that at least one of X, at least one of Y, and at least one of Z, respectively, be present.

[0182] Similarly, while operations may be depicted in the figures in a particular order, it should be recognized that such operations need not be performed in the particular order shown, or in sequential order, or that all of the depicted operations need not be performed, to achieve desirable results. Additionally, the figures may diagrammatically depict one or more example processes in the form of a flowchart. However, other operations not depicted may be incorporated within the diagrammatically depicted example methods and processes. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the depicted operations. Additionally, operations may be rearranged or reordered in other implementations. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged in multiple software products. Additionally, other implementations are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. (Item 1) 1. An augmented reality display system, comprising: a display configured to project a virtual image onto the wearer's eye; a frame configured to mount the display in front of the wearer's eyes; an electromagnetic (EM) field emitter configured to generate a magnetic field; an EM sensor configured to sense the magnetic field, wherein one of the EM field emitter or the EM sensor is mechanically coupled to the frame and the other of the EM field emitter or the EM sensor is mechanically coupled to a component of the augmented reality display system that is independently movable relative to the frame; and 1. A hardware processor, comprising: receiving a signal indicative of a sensed magnetic field from the EM sensor; analyzing the received signal to determine a position or orientation of the EM sensor; a hardware processor programmed to perform the An augmented reality display system comprising: (Item 2) Item 1. The augmented reality display system of item 1, wherein the display comprises a light field display. (Item 3) the component comprises a user input totem or beltpack; the EM sensor is mechanically coupled to the frame; the EM field emitter is mechanically coupled to the user input totem or the beltpack; Item 1. The augmented reality display system of item 1. (Item 4) It also has an audio speaker, the augmented reality display system comprising a noise cancellation circuit configured to cancel magnetic interference in a sensed magnetic field generated by the audio speaker; Item 1. The augmented reality display system of item 1. (Item 5) The noise cancellation circuit is receiving a first signal from the EM sensor; receiving a second signal from the audio speaker; combining the first signal and the second signal to provide a noise cancellation signal; Item 5. The augmented reality display system of item 4, configured to perform the following: (Item 6) Item 6. The augmented reality display system of item 5, wherein, to combine the first signal and the second signal, the noise cancellation circuit is configured to (a) invert the second signal and add the inverted second signal to the first signal, or (b) subtract the second signal from the first signal. (Item 7) The EM field emitter is a first transmitter coil configured to generate a first magnetic field having a first frequency; a second transmitter coil configured to generate a second magnetic field having a second frequency; a time division multiplexing (TDM) circuit configured to switch power between the first transmitter coil and the second transmitter coil, respectively; Item 1. The augmented reality display system of item 1, comprising: (Item 8) 8. The augmented reality display system of item 7, wherein the TDM circuitry comprises a single amplifier circuit that is TDM switched to each of the first and second transmitter coils. (Item 9) Item 8. The augmented reality display system of item 7, wherein the first transmitter coil and the second transmitter coil are arranged along mutually orthogonal axes. (Item 10) Item 8. The augmented reality display system of item 7, wherein the EM field emitter is configured to dynamically adjust the first frequency or the second frequency. (Item 11) Item 11. The augmented reality display system of item 10, wherein the EM field emitter is configured to vary the capacitance of a capacitor or select between multiple capacitors in a capacitor bank to dynamically adjust the first frequency or the second frequency. (Item 12) the EM field emitter comprises a first wireless interface, the EM sensor comprises a second wireless interface and a second TDM circuit, the EM field emitter and the EM sensor comprising: establishing a wireless link between the first wireless interface and the second wireless interface; Synchronizing the timing of the EM field emitter clock and the EM sensor clock via the wireless link. Item 8. The augmented reality display system of item 7, configured to: (Item 13) 8. The augmented reality display system of item 7, wherein the EM field emitter is configured to transmit an EM pulse to the EM sensor, and the EM field emitter or the EM sensor is configured to determine a timing difference between an EM field emitter clock and an EM sensor clock. (Item 14) the EM field emitter is configured to apply a TDM timing protocol; the first transmitter coil transmits during a first time period while the second transmitter coil does not substantially transmit during the first time period; 8. The augmented reality display system of item 7, wherein the second transmitter coil transmits for a second period different from the first period, while the first transmitter coil does not substantially transmit during the second period. (Item 15) The EM field emitter is configured to generate the magnetic field at a first frequency, and the EM sensor comprises: scanning for unintentional radio frequency (RF) interference at the first frequency; switching to a second frequency different from the first frequency in response to determining the presence of unintentional RF interference at the first frequency; Item 1. The augmented reality display system of item 1, configured to perform the following: (Item 16) The EM field emitter is An automatic gain control (AGC) circuit, receiving a voltage level for a coil in the EM sensor; adjusting a gain for an amplification stage of the EM field emitter based at least in part on the received voltage level; and an automatic gain control (AGC) circuit configured to: Item 1. The augmented reality display system of item 1, comprising: (Item 17) 1. A method of operating an electromagnetic (EM) tracking system for an augmented reality (AR) display system, the AR display system comprising a head-mounted AR display, an EM emitter, and a portable user input device comprising an EM sensor, the method comprising: emitting a time-varying magnetic field by an EM emitter within the portable user input device; detecting the time-varying magnetic field with the EM sensor; determining an attitude of the EM sensor based at least in part on the detected magnetic field; determining virtual content for display to a user of the AR display system based at least in part on the determined pose; and Displaying the virtual content by the head-mounted AR display. A method comprising: (Item 18) Item 18. The method of item 17, further comprising time-synchronizing the EM emitter and the EM sensor. (Item 19) 18. The method of claim 17, further comprising canceling magnetic interference from the detected magnetic field. (Item 20) Item 18. The method of item 17, further comprising correlating real world coordinates associated with the pose of the EM sensor with virtual world coordinates associated with the virtual content.

Claims

1. 1. A head mounted display system, comprising: a display positionable in front of the wearer's eyes; an electromagnetic (EM) field emitter configured to generate a magnetic field having a frequency, the EM field emitter configured to dynamically adjust the frequency by varying a capacitance of a capacitor or selecting between multiple capacitors in a capacitor bank; an EM sensor configured to sense the magnetic field at the frequency; Processor and Equipped with The processor: receiving a signal indicative of a sensed magnetic field from the EM sensor; determining a position or orientation of the EM sensor by analyzing the received signal; receiving an image of an environment in front of the wearer from one or more cameras; analyzing the received image to determine if the EM sensor is outside a field of view (FOV) of the one or more cameras; determining the position or the orientation of the EM sensor based on the sensed magnetic field and whether the EM sensor is determined to be outside the FOV of the one or more cameras; A head-mounted display system that is programmed to perform

2. The EM field emitter comprises: a first transmitter coil configured to generate a first magnetic field having a first frequency; a second transmitter coil configured to generate a second magnetic field having a second frequency; a third transmitter coil configured to generate a third magnetic field having a third frequency; and The head mounted display system of claim 1 , comprising:

3. 3. The head-mounted display system of claim 2, wherein the EM field emitter comprises a first time division multiplexing (TDM) circuit configured to switch power between the first transmitter coil, the second transmitter coil, and the third transmitter coil.

4. The head mounted display system of claim 2 , wherein the first transmitter coil, the second transmitter coil, and the third transmitter coil are arranged along mutually orthogonal axes.

5. The head mounted display system of claim 2 , wherein the EM field emitter is configured to dynamically adjust the first frequency, the second frequency, or the third frequency.

6. The head mounted display system of claim 3 , wherein the TDM circuitry comprises a single amplifier circuit TDM switched to each of a plurality of radio frequency (RF) transmitter coils.

7. The head mounted display system of claim 1 , wherein the EM field emitter is configured to dynamically adjust the frequency.

8. The head mounted display system of claim 1 , wherein the EM field emitter is located in a handheld controller.

9. The head mounted display system of claim 8 , wherein the EM sensor is located within the head mounted display or within a belt pack.

10. The head mounted display system of claim 1 , wherein the EM field emitter is positioned within the head mounted display or within a belt pack.

11. The head mounted display system of claim 10 , wherein the EM sensor is located in a first handheld controller.

12. A head-mounted display system, comprising: a display positionable in front of the wearer's eyes; an electromagnetic (EM) field emitter configured to generate a magnetic field having a first frequency, the EM field emitter configured to dynamically adjust the first frequency by varying a capacitance of a capacitor or selecting between multiple capacitors in a capacitor bank, the EM field emitter being positioned within the head mounted display or a belt pack; an EM sensor configured to sense the magnetic field at the first frequency; a second EM sensor configured to sense the magnetic field at a second frequency; and Processor and Equipped with the EM field emitter is configured to generate the magnetic field at the second frequency; The processor: receiving a signal indicative of a sensed magnetic field from the EM sensor; determining a position or orientation of the EM sensor by analyzing the received signal; A head-mounted display system that is programmed to perform

13. 13. The head mounted display system of claim 12, wherein the second frequency is generated by selection of a second capacitor in the capacitor bank.

14. The head mounted display system of claim 12 , wherein the second EM sensor is located in a second handheld controller.

15. 15. The head mounted display system of claim 14, further comprising a third EM sensor configured to sense the magnetic field at a third frequency, the EM field emitter configured to generate the magnetic field at the third frequency.

16. The head mounted display system of claim 15 , wherein the third EM sensor is located in a third handheld controller.

17. 17. The head mounted display system of claim 16, wherein the EM field emitter is configured to dynamically switch between the first frequency, the second frequency, and the third frequency.

18. The head-mounted display system of claim 16, wherein the EM sensor is positioned within a first handheld controller, the first handheld controller is operated by a first user, the second handheld controller is operated by a second user, and the third handheld controller is operated by a third user.

Images