Method and system that utilize phased array beam formation for six degree-of-freedom tracking for emitter in augmented reality system

Abstract

To provide a method and a system for performing six degree-of-freedom (DoF) tracking in a VR and / or an AR system.SOLUTION: An electromagnetic tracking system includes a handheld controller including a first phased array element characterized by a first phase and a second phased array element characterized by a second phase different from the first phase. The first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern. The electromagnetic tracking system also includes a head-mounted augmented reality display that includes an electromagnetic sensor configured to sense the magnetic field pattern.SELECTED DRAWING: Figure 5A

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 811,914, filed February 28, 2019, and entitled “METHOD AND SYSTEM UTILIZING PHASED ARRAY BEAMFORMING FOR SIX DEGREE OF FREEDOM TRACKING FOR AN EMITTER IN AUGMENTED REALITY SYSTEMS,” the contents of which are incorporated herein by reference in their entirety. [Background technology]

[0002] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or may be perceived as real. Virtual reality, or "VR", scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality, or "AR", scenarios typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user.

[0003] Despite the advances made in these display technologies, there remains a need in the art for improved methods, systems, and devices relating to augmented reality systems, and particularly display systems. Summary of the Invention [Means for solving the problem]

[0004] The present disclosure relates to virtual reality (VR) and / or augmented reality (AR) imaging and visualization systems. The present disclosure generally relates to methods and systems for implementing six degrees of freedom (DoF) tracking in VR and / or AR systems. More specifically, embodiments of the present disclosure provide methods and systems for directing energy transmitted by an emitter (also referred to as a transmitter) to calculate the position and orientation of the emitter relative to a sensor. In some embodiments, beamforming using phased array elements of the emitter is utilized to enable beam steering as described more fully herein. The present disclosure is applicable to various applications in computer vision and image display systems.

[0005] According to an embodiment of the present disclosure, an electromagnetic tracking system is provided. The electromagnetic tracking system includes a handheld controller including an electromagnetic emitter. The electromagnetic emitter includes a first phased array element characterized by a first phase and a second phased array element characterized by a second phase different from the first phase. The first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern. The electromagnetic tracking system also includes a head-mounted AR display including an electromagnetic sensor configured to sense the magnetic field pattern.

[0006] According to another embodiment of the present disclosure, a method for determining a 6DoF attitude of a handheld controller is provided. The method includes determining an attitude of an electromagnetic sensor and generating an electromagnetic beam from the handheld controller. The electromagnetic beam is characterized by an electromagnetic field pattern. The method also includes steering the electromagnetic beam and increasing the received power at the electromagnetic sensor, and determining a beam angle associated with the electromagnetic beam. The method further includes determining a distance between the handheld controller and the electromagnetic sensor, and determining the 6DoF attitude of the handheld controller using the attitude, beam angle, and distance of the electromagnetic sensor.

[0007] Numerous benefits are achieved by the methods of the present disclosure over conventional techniques. For example, embodiments of the present disclosure provide methods and systems that increase the electromagnetic field strength at the sensor in a predetermined manner. Embodiments of the present disclosure enable active tracking of emitter position, thereby increasing the accuracy of attitude determination. These and other embodiments of the present disclosure, along with many of its advantages and features, are described in further detail in conjunction with the following text and accompanying figures. [Brief description of the drawings]

[0008] The accompanying drawings, which are included to provide a further understanding of the present disclosure, are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure, and together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than may be necessary for a fundamental understanding of the present disclosure and the various ways in which it may be practiced.

[0009] [Figure 1] FIG. 1 diagrammatically illustrates a system diagram of an electromagnetic (EM) tracking system, according to some embodiments.

[0010] [Diagram 2] FIG. 2 is a flow chart illustrating the functionality of the electromagnetic tracking system, according to some embodiments.

[0011] [Diagram 3] FIG. 3 diagrammatically illustrates an electromagnetic tracking system integrated with an augmented reality (AR) system, according to some embodiments.

[0012] [Figure 4] FIG. 4 diagrammatically illustrates electromagnetic tracking using steered beams in the context of an AR device, in accordance with some embodiments.

[0013] [Figure 5A]FIG. 5A diagrammatically illustrates a set of three electromagnetic sources generating steered beams at a predetermined angle, according to some embodiments.

[0014] [Figure 5B] FIG. 5B diagrammatically illustrates a set of four electromagnetic sources in a tetrahedral array, according to some embodiments.

[0015] [Figure 6] FIG. 6 is a flow chart illustrating a method for tracking a handheld controller position according to some embodiments.

[0016] [Figure 7A] FIG. 7A diagrammatically illustrates the variation of a received signal as a function of two emitter positions according to some embodiments.

[0017] [Figure 7B] FIG. 7B diagrammatically illustrates the variation of the received signal as a function of two emitter positions according to another embodiment.

[0018] [Figure 8] FIG. 8 diagrammatically illustrates an augmented reality system in communication with a 5G cellular mobile communication system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Augmented reality (AR) systems can be designed to be interactive with a user. As an example, a user may be provided with a handheld controller, also referred to as a totem, that the user can utilize to interact with the AR system. It is therefore useful to be able to determine the position and orientation (e.g., six degrees of freedom (DoF) pose) of the handheld controller relative to other elements of the AR system, including a head-mounted display system, also referred to as an AR headset or headset, worn by the user.

[0020] One approach to achieving high-precision localization may involve the transmission and detection of electromagnetic fields. For example, an electromagnetic field may be emitted by at least one electromagnetic field emitter (generally referred to as an "emitter") and received by at least one electromagnetic field sensor (generally referred to as a "sensor"). The emitter and sensor may be strategically placed on the user's AR headset, beltpack, and / or other auxiliary devices (e.g., totems, tactile devices, gaming equipment, etc.). The emitter generates an electromagnetic field having a known spatial (and / or temporal) distribution in the environment of the user of the AR headset. The sensor measures the generated electromagnetic field at the sensor's location. Based on these measurements and knowledge of the distribution of the generated electromagnetic field, the pose (e.g., position and / or orientation) of the sensor relative to the emitter can be determined. Thus, the pose of the object to which the sensor and / or emitter are attached can be determined. That is, the relative positions of the sensor and emitter can be determined.

[0021] 1 illustrates an example system diagram of an electromagnetic tracking system 100. In some embodiments, the electromagnetic tracking system 100 includes one or more electromagnetic field emitters 102 (generally referred to as "emitters 102") configured to emit a known electromagnetic field. As shown in FIG. 1, the emitters 102 may be coupled to a power supply 110 (e.g., electrical current, a battery, etc.) to provide power to the emitters 102.

[0022] In some embodiments, the emitter 102 includes several coils (e.g., at least three coils positioned perpendicular to each other and generating electromagnetic fields in the X, Y, and Z directions) that generate an electromagnetic field. The coils can be operated at a predetermined AC frequency (e.g., in the range of 20 kHz to 40 kHz) such that each coil generates an AC dipole-shaped electromagnetic field. This electromagnetic field is used to establish a coordinate space (e.g., an XYZ Cartesian coordinate space). This allows the system to map the positions (e.g., (X, Y, Z) positions) of the electromagnetic sensors 104a, 104b in relation to a known electromagnetic field and determine the position and / or orientation of the electromagnetic sensors 104a, 104b. In some embodiments, the electromagnetic sensors 104a, 104b (generally referred to as "sensors 104") may be attached to one or more real objects. The sensors 104 may include coils in which a current can be induced through an electromagnetic field, e.g., an electromagnetic field emitted by the emitter 102. The sensor 104 may include a coil or loop (e.g., at least three coils positioned perpendicular to each other) positioned / oriented to capture incident electromagnetic flux from an electromagnetic field, e.g., the electromagnetic field emitted by the emitter 102, and by comparing the currents induced through these coils and knowing the relative positions and orientations of the coils with respect to each other, the relative position and orientation of the sensor 104 with respect to the emitter 102 can be calculated.

[0023] One or more parameters relating to the behavior of the coil and inertial measurement unit ("IMU") components operatively coupled to the sensor 104 may be measured to detect the position and / or orientation of the sensor 104 (and the object to which it is attached) relative to a coordinate system to which the emitter 102 is coupled. In some embodiments, multiple sensors 104 may be used in conjunction with the emitter 102 to detect the position and orientation of each of the sensors 104 in the coordinate space. The electromagnetic tracking system 100 may provide position in three directions (e.g., X, Y, and Z directions) and further two or three orientation angles. In some embodiments, the IMU measurements may be compared to the coil measurements to determine the position and orientation of the sensor 104. In some embodiments, both the electromagnetic (EM) data and the IMU data may be combined with various other data sources, such as cameras, depth sensors, and other sensors, to determine the position and orientation. This information may be transmitted (e.g., wireless communication, Bluetooth, etc.) to the controller 106. In some embodiments, pose (or position and orientation) may be reported at a relatively high refresh rate in conventional systems.

[0024] Traditionally, an electromagnetic field emitter is coupled to a relatively stable, large object such as a table, operating table, wall, or ceiling, and one or more sensors are coupled to a smaller object such as a medical device, a handheld gaming component, or the like. Alternatively, as described below with reference to FIG. 3, various features of an electromagnetic tracking system may be employed to generate a configuration in which changes or deltas in position and / or orientation between two objects moving in space relative to a more stable global coordinate system may be tracked. In other words, a configuration is shown in FIG. 3 in which fluctuations of the electromagnetic tracking system may be utilized to track position and orientation deltas (changes) between a head-mounted component and a handheld component, while head pose relative to a global coordinate system (e.g., of a room environment local to the user) is otherwise determined by simultaneous localization and mapping ("SLAM") techniques, or the like, using an outward-facing capture camera that may be coupled to the head-mounted component of the system.

[0025] The controller 106 may control the emitters 102 and may also capture data from the sensors 104. It should be understood that the various components of the system may be coupled to one another through any electromechanical or wireless / Bluetooth means. The controller 106 may also include data regarding the known electromagnetic field and the coordinate space associated with the electromagnetic field. This information is then used to detect the position and orientation of the sensors 104 in relation to the coordinate space corresponding to the known electromagnetic field.

[0026] One advantage of electromagnetic tracking systems is that they produce high resolution, highly repeatable tracking results with minimal latency. In addition, electromagnetic tracking systems do not necessarily rely on optical trackers, and sensors / objects that are not within the user's line of sight can easily be tracked.

[0027] It should be appreciated that the strength V of the electromagnetic field drops off as a cubic function of the distance r from the coil emitter (e.g., emitter 102). Thus, an algorithm may be used based on the distance away from the emitter 102. The controller 106, which may also be referred to as a processor, may be configured to use such algorithms to determine the position and orientation of the sensor 104 at variable distances away from the emitter 102. Given the steep decrease in the strength of the electromagnetic field as the sensor 104 moves farther away from the emitter 102, best results in terms of accuracy, efficiency, and low latency may be achieved at closer distances. In a typical electromagnetic tracking system, the emitter is powered by an electric current (e.g., a plug-in power supply) and the sensor is located within a 20-foot radius of the emitter. A smaller radius between the sensor and the emitter may be more desirable in many applications, including AR applications.

[0028] FIG. 2 illustrates an example flow chart for a method 200 describing the function of an electromagnetic tracking system, such as the electromagnetic tracking system 100. At 202, a known electromagnetic field is emitted. In some embodiments, an electromagnetic field emitter may generate the electromagnetic field. For example, each coil of the electromagnetic field emitter may generate an electromagnetic field in one direction (e.g., X, Y, or Z). The electromagnetic field may be generated with an arbitrary waveform. In some embodiments, the electromagnetic field components along each of the axes may oscillate at a slightly different frequency than other electromagnetic field components along other directions. At 204, a coordinate space corresponding to the electromagnetic field may optionally be determined. For example, the controller may automatically determine a coordinate space around the emitter and / or sensor based on the electromagnetic field. In some embodiments, the coordinate space may not be determined at this stage of the method. At 206, the behavior of the coil in the sensor (which may be attached to a known object) may be detected. For example, the current induced in the coil may be calculated. In some embodiments, the rotation or any other quantifiable behavior of the coil may be tracked and measured. At 208, this behavior may be used to detect the position or orientation of the sensor and / or a known object (e.g., an AR headset that includes the sensor) relative to the emitter or vice versa. For example, the controller 106 may consult a mapping table that correlates various positions or orientations with the behavior of the coils in the sensor. Based on these calculations, a position in coordinate space may be determined, along with the orientation of the sensor and / or emitter.

[0029] In the context of an AR system, one or more components of the electromagnetic tracking system may need to be modified to facilitate accurate tracking of the mobile components (e.g., emitters and sensors). As described above, tracking the user's head pose and orientation may be desirable in many AR applications. Accurate determination of the user's head pose and orientation allows the AR system to display appropriate / relevant virtual content to the user. For example, a virtual scene may include a virtual monster hiding behind a real building. Depending on the user's head pose and orientation relative to the building, the view of the virtual monster may need to be modified so that a realistic AR experience is provided. Alternatively, the position and / or orientation of a totem, tactile device, or some other means of interacting with the virtual content may be important in allowing the user to interact with the AR system. For example, in many gaming applications, the AR system may detect the position and orientation of a real object in relation to the virtual content. Alternatively, when displaying a virtual interface, the position of a totem, a user's hand, a tactile device, or any other real object configured for interaction with the AR system may be known in relation to the displayed virtual interface in order for the system to understand commands, interactions, and the like. Some localization methods, such as optical tracking, can suffer from high latency and low resolution issues, which make rendering virtual content difficult in many AR applications.

[0030] In some embodiments, an electromagnetic tracking system, discussed in connection with Figures 1 and 2, may be adapted for an AR system to detect the position and orientation of one or more objects relative to the emitted electromagnetic field. Typical electromagnetic tracking systems tend to have large and bulky electromagnetic emitters (e.g., 102 in Figure 1), which is problematic for head-mounted AR devices, such as with totems. However, smaller electromagnetic emitters (e.g., in the millimeter range) may be used to emit a known electromagnetic field in the context of an AR system.

[0031] FIG. 3 illustrates an electromagnetic tracking system integrated with an AR system, with an electromagnetic field phased array emitter 302 (generally referred to as a "phased array emitter 302") integrated as part of a handheld controller 306. The handheld controller 306 can be independently movable relative to the AR headset 301 (or belt pack 370). For example, the handheld controller 306 can be held in the user's hand, or the handheld controller 306 can be mounted on the user's hand or arm (e.g., as a ring or bracelet, or as part of a glove worn by the user). In some embodiments, the handheld controller 306 can be a totem, for example, used in gaming scenarios (e.g., a multi-degree-of-freedom controller), or providing a rich user experience within the AR environment, or allowing the user to interact with the AR system. In some embodiments, the controller 306 can be a tactile device. In some embodiments, the phased array emitter 302 can be integrated as part of the belt pack 370.

[0032] The handheld controller may include a battery 310 or other power source that powers the phased array emitter 302. It should be appreciated that the phased array emitter 302 may also include or be coupled to an IMU 350 component configured to assist in determining the positioning and / or orientation of the phased array emitter 302 relative to other components. This may be advantageous, particularly if both the phased array emitter 302 and the electromagnetic field sensor 304 (generally referred to as “sensor 304”) are mobile. Locating the phased array emitter 302 in the handheld controller 306 rather than in the belt pack 307 as shown in the embodiment of FIG. 3 helps ensure that the phased array emitter 302 does not compete for resources in the belt pack 370, but rather uses its own battery source in the handheld controller 306. In some embodiments, the phased array emitter 302 can be located on the AR headset 301 and the sensor 304 can be located on the handheld controller 306 or the beltpack 370. Thus, embodiments of the present disclosure provide implementations where the handheld controller 306 is implemented as a handheld unit, while in other embodiments the handheld controller is implemented within the AR headset 301, while additional embodiments provide implementations where the handheld controller is implemented within an auxiliary unit, such as the beltpack 307. Furthermore, in addition to implementations where the handheld controller 306 is implemented within a single device, the functionality of the handheld controller and associated physical components can be distributed across multiple devices, such as the handheld controller 306, the AR headset 301, and / or auxiliary units such as the beltpack 307.

[0033] In some embodiments, the sensor 304 may be installed on one or more locations on the AR headset 301 along with other sensing devices or sensors 308, such as one or more IMUs or additional electromagnetic flux capture coils. For example, as shown in FIG. 3, the sensors 304, 308 may be installed on one or both sides of the AR headset 301. Having multiple sensors 304, 308 may improve efficiency and accuracy, since the sensors 304, 308 can be engineered to be fairly small (and in some cases may not be very sensitive). In some embodiments, one or more sensors may also be installed on the belt pack 370 or any other part of the user's body. The sensors 304, 308 may communicate wirelessly, for example, through Bluetooth, to a computing device that determines the attitude and orientation of the sensors 304, 308 (and the AR headset 301 to which it is attached). In some embodiments, the computing device may reside on the belt pack 370. In some embodiments, the computing device may reside on the AR headset 301 or the controller 306. In some embodiments, the computing device may then include a mapping database 330 (e.g., a mapping database, cloud resources, a passable world model, a coordinate space, and the like), detect poses, determine coordinates of real and / or virtual objects, and even connect to cloud resources and passable world models. The handheld controller 306, in some embodiments, is capable of controlling the timing of electromagnetic emissions by the electromagnetic emitters and sensing by the electromagnetic sensors such that the positions and orientations of the electromagnetic emitters and sensors are calculated based on fields from the modified electromagnetic field pattern. In some embodiments, the positions and orientations of the electromagnetic emitters are calculated relative to the electromagnetic sensors. In other embodiments, the positions and orientations of the electromagnetic sensors are calculated relative to the electromagnetic emitters. In some embodiments, the positions and orientations of the electromagnetic emitters and sensors are calculated.

[0034] As explained above, some electromagnetic emitters may be too bulky for AR devices. Thus, the emitter may be engineered to be compact, using smaller components (e.g., coils) than conventional systems. However, a smaller radius between the sensor 304 and the phased array emitter 302 (e.g., about 3 to 3.5 feet) may reduce power consumption compared to conventional systems such as those detailed in FIG. 1, provided that the strength of the electromagnetic field decreases as a cubic function of distance away from the emitter.

[0035] In some embodiments, this aspect may be utilized to extend the life of the battery 310, which in one or more embodiments may power the handheld controller 306 and the phased array emitter 302. In some embodiments, this aspect may be utilized to reduce the size of the coils that generate the electromagnetic field in the phased array emitter 302. However, to obtain the same strength of the electromagnetic field, the power may need to be increased. This allows for a compact phased array emitter 302 that may fit compactly in the handheld controller 306.

[0036] Some other modifications may be made when using an electromagnetic tracking system for an AR device. While the present attitude reporting rate is quite good, AR systems may require an even more efficient attitude reporting rate. To this end, IMU-based attitude tracking may be used (in addition or as an alternative). Advantageously, the IMU may remain as stable as possible to increase the efficiency of the attitude detection process. IMUs may be engineered such that they remain stable for a maximum of 50-100 milliseconds. It should be appreciated that some embodiments may utilize an outer attitude estimator module that may allow attitude updates to be reported at a rate of 10-20 Hz (e.g., the IMU may drift over time). By keeping the IMU stable at a reasonable rate, the rate of attitude updates may be dramatically reduced to 10-20 Hz (compared to higher frequencies in conventional systems).

[0037] If the electromagnetic tracking system could be woken up, for example, on a 10% duty cycle (e.g., only pinging for ground truth every 100 ms), the AR system could save power. This could mean that the electromagnetic tracking system wakes up and generates pose estimation 10 ms out of 100 ms. This could translate directly into power consumption savings, which in turn could impact the size, battery life, and cost of the AR device (e.g., AR headset 301 and / or controller 306).

[0038] In some embodiments, this reduction in duty cycle may be strategically utilized by providing a second handheld controller (not shown) rather than only one handheld controller 306 as illustrated in FIG. 3. For example, a user may be playing a game that requires two controllers and the like. Or, in a multi-user game, two users may have their own controllers to play the game. When two controllers (e.g., symmetrical controllers per hand) are used rather than one, the controllers may operate with an offset duty cycle. The same concept may also be applied to controllers utilized by two different users playing a multiplayer game.

[0039] Although some embodiments of the present disclosure are discussed in the context of using a phased array of electromagnetic elements within an electromagnetic emitter, embodiments of the present disclosure are not limited to the use of electromagnetic radiation; other forms of energy can also be used to perform beamforming, including acoustic energy (i.e., sound), which results in an energy distribution characterized by regions of constructive and destructive interference.

[0040] Magnetic tracking can be utilized to determine the 6DoF attitude (i.e., position and orientation) of the handheld controller 306 relative to the AR headset 301. Using emitters located in the handheld controller, a magnetic field of known geometry can be generated. One or more sensors in the headset can then be used to determine the 6DoF relationship between the handheld controller and the AR headset based on measurements of the location of the AR headset in the known geometry. In some cases, measurement data provided by an IMU can be utilized to calculate or refine measurements of the 6DoF attitude of the handheld controller, for example, in the context of sensor fusion. However, while IMUs may be suitable for measuring relative motion, they may have a level of bias associated with their measurement data. As this bias is integrated, drift in the IMU's position can accumulate and reduce measurement accuracy. Thus, embodiments of the present disclosure provide measurement data characterized by reduced bias. In some embodiments, the measurement data may include a noise component, but the reduction in bias allows for averaging of the noise and provides a signal with improved accuracy.

[0041] FIG. 4 diagrammatically illustrates electromagnetic tracking using a steered beam in the context of an AR device, according to some embodiments. As described more fully below, an emitted beam 402 emitted by a handheld controller 410 is steered at a beam angle θ. Knowing the phase delay relationship between the individual phased array elements in the phased array emitter 412, the beam angle θ can be calculated. A feedback loop implemented using a bidirectional communication path 430 (e.g., Bluetooth) can then be utilized such that the phase delays associated with each of the phased array elements in the phased array emitter 412 are adjusted to steer the beam and achieve / maintain a maximum value of received power at the electromagnetic field sensor 422 of the AR headset 420. As described more fully herein, the communication path 430 allows data related to the handheld controller 410, such as handheld controller IMU data, phase delays of the individual phased array elements in the phased array emitter 412, beam angle θ, and the like, to be transmitted to the AR headset 420. Additionally, communication path 430 allows data related to AR headset 420, such as AR headset IMU data, received power at electromagnetic field sensor 422, and the like, to be transmitted to handheld controller 410. The beam angle θ of emitted beam 402 may be swept and steered to maximize the energy received at the AR headset 420, and more specifically, at the electromagnetic field sensor 422.

[0042] Given the phase delays of the individual phased array elements in the phased array emitter 412, which may be calculated based on the characteristics of the emitted beam 402, and the distance between the handheld controller 410 and the AR headset 420, the 6DoF pose of the handheld controller 410 can be determined relative to the known pose of the AR headset 420, as will be described more fully below.

[0043] FIG. 5A diagrammatically illustrates a set of three electromagnetic sources (e.g., individual phased array elements) generating a steered beam at a predetermined angle, according to some embodiments. As illustrated in FIG. 5A, three phased array elements 510, 512, and 514 are included as elements of a phased array emitter 505 (e.g., phased array emitter 412 of FIG. 4) disposed within a handheld controller. In some embodiments, the phased array elements 510, 512, and 514 may each be a coil operated at a predetermined AC frequency (e.g., in the range of 20 kHz to 40 kHz) such that each coil generates an AC dipole-shaped electromagnetic field. In the exemplary phased array emitter 505, all of the phased array elements 510, 512, 514 align their coils with a general direction, e.g., the z-direction. In general, the relative orientation and arrangement of the phased array elements 510, 512, 514 are selected such that the main lobe generated by the phased array emitter 505 is oriented along the most common direction that an AR headset (e.g., AR headset 420 of FIG. 4) would be relative to a handheld controller (e.g., handheld controller 410 of FIG. 4) during normal operation (e.g., a handheld controller held in a user's hand at approximately waist / mid-torso height with the palm facing left or right, depending on which hand the user is using to hold the handheld controller). This most common direction that an AR headset (e.g., AR headset 301) would be relative to a handheld controller (e.g., handheld controller 306) during normal operation is illustrated in FIG.

[0044] As electromagnetic radiation is emitted by each of the phased array elements 510, 512, 514, the wave nature of the emitted electromagnetic radiation results in interference between the electromagnetic radiation emitted by the phased array elements 510, 512, and 514. This interference creates regions of constructive and destructive interference. For the phased array elements 510, 512, and 514 illustrated in FIG. 5A, if the phase of each element is matched (i.e., there is no delay between the elements), a main lobe will be generated along the x-direction, which is orthogonal to the plane of the figure. If a fixed phase delay relationship is implemented between the phased array elements (i.e., phased array element 510 with phase delay φ, which may be zero, phased array element 512 with phase delay φ1, and phased array element 514 with phase delay φ2), the interference between the phased array elements 510, 512, and 514 will result in the generation of a main lobe 525 and side lobes (not shown for purposes of clarity). In some embodiments, the phased array elements 510, 512, and 514 will generally be positioned such that the yz plane illustrated in Figure 5A would be orthogonal to the beam angle θ illustrated in Figure 4 to align the main lobe for no phase delay with the most common orientation between the handheld controller and the AR headset. Although three phased array elements are illustrated in Figure 5A, embodiments of the disclosure are not limited to this particular number and other numbers, including two phased array elements and more than three phased array elements, can be utilized by embodiments of the disclosure.

[0045] In some embodiments, a single coil can be utilized per phased array element. In some embodiments, multiple coils can be utilized per phased array element. As an example, to improve performance when the handheld controller is rotated 90° relative to the most common orientation that an AR headset would be relative to the handheld controller, a second coil per phased array element can be utilized to increase the strength of the main lobe in the direction pointing into / out of the plane of the figure (i.e., ±x direction). In addition to the phased array elements of the handheld controller, the emitter can be implemented as a three coil emitter, as illustrated by the electromagnetic field emitter 102 in FIG. 1.

[0046] Thus, the three phased array elements 510, 512, and 514 are static (i.e., their positions within the handheld controller are fixed), but control of the phase delay associated with each phased array element can enable steering of the main lobe 525. As illustrated in FIG. 5A, the main lobe 525 has components θ x , θ y , and θ z φ1 and φ2. The vector 520 can be referred to as a center vector because it is aligned with the center of the main lobe. By modifying the phase delays φ1 and / or φ2, the main lobe 525 can be steered such that the vector 520 can be oriented at an arbitrary beam angle θ. Although only the phase delays φ1 and / or φ2 corresponding to the phased array elements 512 and 514 are modified in this embodiment, it should be understood that an additional phase delay φ can be associated with the phased array element 510 in conjunction with control over the beam angle θ associated with the main lobe 525. When an additional phase delay φ is utilized, all three phase delays can be controlled to achieve a desired phase delay between the phased array elements 510, 512, 514. One skilled in the art would recognize many variations, modifications, and alternatives.

[0047] Beam steering of the main lobe 525 can be used to align the vector 520 with a vector directed from a phased array emitter 505 located in the handheld controller to an electromagnetic field sensor (e.g., electromagnetic field sensor 422) in the AR headset. Once the vector 520 directed from the phased array emitter 505 located in the handheld controller and the electromagnetic field sensor in the AR headset are aligned, the energy received at the electromagnetic field sensor will be maximized since the center of the main lobe 525 will be directed to the electromagnetic field sensor. As will be more fully described in connection with FIG. 6, beam steering of the main lobe 525 will be utilized to determine the orientation of the handheld controller relative to the AR headset.

[0048] Although a description of the installation of the phased array emitter in the handheld controller and the installation of the electromagnetic field sensor in the AR headset is provided herein, this is not required by the present disclosure, and the phased array emitter can be installed in the AR headset with a corresponding electromagnetic field sensor installed in the handheld controller. In addition, in some embodiments, the phased array emitter or the electromagnetic field sensor can be installed in the auxiliary unit. Typically, the handheld controller can support the power requirements and weighting associated with the operation of the phased array emitter, and as a result, the phased array emitter will be implemented as a component installed in the handheld controller. With reference to FIG. 3, the computational work can be performed in the handheld controller 306, the AR headset 301, the belt pack 370, or a small cell (not shown), or distributed among these elements as appropriate. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0049] FIG. 5B diagrammatically illustrates a set of four electromagnetic sources in a tetrahedral array, according to some embodiments. As illustrated in FIG. 5B, four phased array elements 510, 512, 514, and 516 are positioned such that they are on the four vertices of a tetrahedron. Phased array elements 510, 512, and 514 are in the yz plane, while phased array element 516 is positioned at a predetermined height above the yz plane measured along the x-axis. By utilizing the tetrahedral array illustrated in FIG. 5B, additional control is provided over the main lobe beam shape, as well as increased control for beam steering in the yz plane.

[0050] FIG. 6 illustrates an example flowchart illustrating a method 600 of tracking a handheld controller position according to some embodiments. The method includes determining a pose (i.e., position and orientation) of an electromagnetic field sensor (610). In some embodiments, determining the pose of the electromagnetic field sensor may include determining a head pose of an AR headset that includes the electromagnetic field sensor. In some embodiments, only the position of the electromagnetic field sensor may be determined. In some embodiments, one or more IMUs in the AR headset may be used to provide this head pose determination. The method also includes an optional process of determining an approximate position of the electromagnetic field emitter (e.g., a handheld controller) (620), e.g., using one or more IMUs in the electromagnetic field emitter. In some implementations, this optional process also determines an approximate orientation (i.e., approximate pose) of the electromagnetic field emitter (e.g., a handheld controller), e.g., using one or more IMUs in the electromagnetic field emitter. In some embodiments, the approximate orientation can be determined based on acceleration due to gravity in excess of acceleration in a direction perpendicular to gravity as a user holds the handheld controller and calibrates or initializes the handheld controller.

[0051] Beam steering is then commenced (630) to steer the main lobe of the electromagnetic beam, for example main lobe 525 in FIG. 5. Using an optimization algorithm, also referred to as a search algorithm, the beam is steered until the energy received at the electromagnetic sensor is maximized. One of several optimization algorithms known to those skilled in the art can be utilized in this beam steering process. Typically, a fitness function F is maximized. [ka] where Δφ1=φ1-φ and Δφ2=φ2-φ, φ is the phase delay associated with phased array element 510, φ1 is the phase delay associated with phased array element 512, and φ2 is the phase delay associated with phased array element 514. The phased array elements can be associated with operating frequencies as follows: phased array element 510: ωt+φ, where φ can be zero, phased array element 512: ωt+φ1, and phased array element 514: ωt+φ2. As discussed above, beam steering can be accomplished through control of the phase delays φ1 and φ2, or control of all three phases φ, φ1, and φ2.

[0052] According to various embodiments, the beam steering process discussed herein can be implemented using beam steering information determined in the AR headset or in the handheld controller. For example, the following process flow can be implemented when the beam steering information is determined in the AR headset: The handheld controller transmits an electromagnetic beam via a first communication path and transmits sensor data (e.g., sensor data collected at the handheld controller) via a second communication path. The AR headset receives the electromagnetic beam via the first communication path and receives the handheld controller sensor data via the second communication path.

[0053] The AR headset then calculates the power associated with the electromagnetic beam and determines beam steering information based on the calculated power associated with the electromagnetic beam, the handheld device sensor data, and the AR headset sensor data. Given this beam steering information, the AR headset transmits the beam steering information over the second communication path, and the handheld controller receives the beam steering information. Thus, the handheld controller can change the beam steering properties of the electromagnetic beam based on the received beam steering information. For example, the received beam steering information may indicate that the handheld controller is to increase or decrease one or more of the phase delays φ1 and φ2. As another example, the AR headset may determine, based on the handheld device sensor data and the AR headset sensor data, that the handheld controller is moving relative to the AR headset, and therefore may generate beam steering information that causes the handheld controller to increase or decrease one or more of the phase delays φ1 and φ2 so that a main beam (e.g., main beam 525) and its corresponding vector (e.g., vector 520) may remain aligned with an electromagnetic sensor in the AR headset.

[0054] Additionally, the following process flow can be implemented when the beam steering information is determined at the handheld controller: the handheld controller transmits an electromagnetic beam via a first communication path, and the AR headset receives the electromagnetic beam via the first communication path. The AR headset then calculates a power associated with the electromagnetic beam and transmits the calculated power and the AR headset sensor data to the handheld controller via a second communication path.

[0055] The handheld controller receives the calculated power and the AR headset sensor data and determines beam steering information based on the calculated power, the AR headset sensor data, and the handheld controller sensor data associated with the electromagnetic beam. Thus, the handheld controller can vary the beam steering properties of the electromagnetic beam based on the received beam steering information.

[0056] In some embodiments, a calibration process can be utilized to account for system characteristics and improve the accuracy of the beam steering algorithm. Additionally, a variety of control and optimization algorithms are suitable for use with embodiments of the present disclosure. Exemplary control and optimization algorithms that can be used to maximize the fitness function include gradient descent methods, stochastic gradient descent methods, gradient descent methods with momentum, deep learning algorithms, and the like. As an example, in an initialization phase, the beam can be steered by varying the phase delays φ1 and φ2 by many amounts that result in large angular deviations (e.g., 10° increments) for the steered beam as the area around the handheld controller is analyzed. Returning to the beam angle (and associated phase delay) that provided the maximum value of received power, smaller increments can be utilized progressively (e.g., 5°, 2°, 1°, 0.5°) to maximize the received power.

[0057] The initialization process may be informed by ergonomic studies that provide information related to the probability that the handheld controller will be oriented in a given direction relative to the AR headset. As an example, if the handheld controller is typically held at approximately waist / mid-torso height with the palm facing left, the initialization process may begin near an angle that is aligned with the direction between the handheld controller and the AR headset in this orientation. This most common direction between the handheld controller and the AR headset is illustrated in FIG. 4 as beam angle θ. As would be apparent to one skilled in the art, optimization algorithms that may provide faster convergence are included within the scope of this disclosure. Once beam lock is achieved, the angular adjustment of the beam steering angle may be decreased or increased as appropriate for a particular application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0058] In running these optimization algorithms, a communication path is provided between the handheld controller and the AR headset, for example, to exchange data collected during the operation of the optimization algorithm. An example of the communication path 430 in FIG. 4 is a Bluetooth® wireless link, which provides bidirectional communication between the handheld controller and the AR headset. This communication path can be used, for example, to transmit received power data from the AR headset to the handheld controller. In addition, this communication path can be used to transmit information about beam angles (e.g., beam angles based on phase delays or phase delays for calculation of beam angles in the AR headset) from the handheld controller to the AR headset. Thus, the communication path 430 allows control signals and data to be transmitted in a bidirectional manner between the handheld controller and the AR headset. For example, IMU data generated in either the AR headset or the handheld controller can be transmitted through the communication path 430. Although a Bluetooth® wireless link is described as an example of the communication path 430, other communication technologies are included within the scope of this disclosure, including WiFi wireless links, wired links such as USB links, and the like.

[0059] In operation, beam steering due to modification of the phase delay results in an increase / decrease in received power at the AR headset. In response to a change in received power at the electromagnetic field sensor in the AR headset, the electromagnetic field emitter in the handheld controller can vary the phase delay between the phased array elements to steer the beam in a manner that increases the received power.

[0060] As explained above, once the beam is steered to align the vector passing through the center of the main lobe and the vector directed from the phased array emitter in the handheld controller to the electromagnetic field sensor, which may also be referred to as the receiver in the AR headset, the energy received at the electromagnetic field sensor will be maximized. Thus, using feedback transmitted over the communication path from the electromagnetic field sensor in the AR headset to the phased array emitter in the handheld controller, beam steering to maximize (and in some embodiments to maintain) the received power can be accomplished.

[0061] Referring back to FIG. 6, once the beam is steered to maximize the received power, an indicator can be generated to indicate the achievement of beam lock (632). Achieving beam lock will be associated with a given direction between the handheld controller and the AR headset at a given time. As the handheld controller (and / or AR headset) moves as a function of time, the tracking algorithm will continually run to maintain beam lock through beam steering. Data provided by the IMU in the handheld controller and / or the IMU in the AR headset may be utilized by the initialization and / or tracking algorithm, for example, to reduce the angular step size deviation in response to small changes in the handheld controller and AR headset position and / or orientation, velocity, acceleration, or the like, or to increase the angular step size deviation in response to large changes in the handheld controller and AR headset position and / or orientation, velocity, acceleration, or the like. In some embodiments, the angular step size deviation is modified in response to the angular velocity and / or acceleration of the handheld controller, since rotation of the handheld controller can result in rapid steering of the emitted beam.

[0062] The beam angle θ at which the main lobe is oriented when the energy at the electromagnetic field sensor achieves a maximum value can be determined (634) as a function of the phase delays φ1 and φ2 associated with the phased array elements 512 and 514, respectively. In other words, given the phase delays φ1 and φ2 and the geometric arrangement of the phased array elements 510, 512, and 514, the beam angle θ can be calculated as a function of the phase delays and the geometric arrangement. Given the beam angle θ between the handheld controller and the AR headset, it is known that the handheld controller and the AR headset are positioned along a vector connecting the handheld controller and the AR headset. To this information, the distance between the handheld controller and the AR headset is determined and added (634), as described more fully below.

[0063] The embodiments of the present disclosure utilize one of several techniques to determine the distance between the handheld controller and the AR headset. For example, in some embodiments, the measured power can be compared to a calibrated table or function that relates received power to distance. Using a communication path, IMU data from the handheld controller can be fused with the measured power at the AR headset to refine the distance calculation. In some embodiments, a known electromagnetic field pattern can be used to determine the distance between the handheld controller and the AR headset given the received power. As an example, for a given electromagnetic field pattern and a given distance, steering the main lobe a given angle (e.g., 5°) will result in a reduction in received power by a given amount (e.g., 10%). At a larger distance (e.g., twice the distance), steering the main lobe a given angle will result in a smaller reduction in received power (e.g., 20%). Once the received power is maximized, the phase delay can be varied to steer the beam by a given angle (e.g., through a series of angles), and the reduction in received power can be used to calculate the distance. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0064] In some embodiments, measurements of received power as a function of handheld controller position can be utilized to determine the distance between the handheld controller and the AR headset, as described in connection with Figures 7A and 7B.

[0065] 7A diagrammatically illustrates an example of the variation of a received signal as a function of two emitter positions, according to some embodiments. In FIG. 7A, a phased array emitter 710 is positioned at position x1 and is characterized by a transmit pattern 711 having a central angle aligned with vector 715. For purposes of clarity, only the center or main lobe of the transmit pattern is illustrated. An electromagnetic field sensor 720 is positioned at a given distance from the phased array emitter 710 equal to the length of vector 715, and beam lock is achieved with respect to the phased array emitter at position x1 (Tx(x1)).

[0066] Referring to FIG. 7A, the phased array emitter 712 is shown after being translated to position x2 (Tx(x2)). Position translation information can be obtained by analyzing IMU data generated by the handheld controller and transmitted to the AR headset, as this IMU data can be used to track the handheld controller attitude. In some embodiments, the IMU data related to the handheld controller attitude is fused with electromagnetic force measurements received by the AR headset to provide a handheld controller attitude value. After the translation, if the transmission pattern is maintained given the new phased array emitter position (i.e., x2), the power received at the electromagnetic field sensor 720 will decrease, as illustrated by vector 717 in FIG. 7A intersecting the main lobe at position 718. Because the main lobe amplitude is significantly reduced at position 718, a function can be generated that correlates the translation distance with the received power for a given distance.

[0067] FIG. 7B diagrammatically illustrates an example of the variation of the received signal as a function of two emitter positions, according to some embodiments. In FIG. 7B, a distance equal to the length of vector 735 exceeds a distance equal to the length of vector 715 illustrated in FIG. 7A. In FIG. 7B, a phased array emitter 730 is positioned at position x1 and is characterized by a transmission pattern 731 having a central angle aligned with vector 735. For purposes of clarity, only the center or main lobe of the transmission pattern is illustrated. An electromagnetic field sensor 740 is positioned at a distance equal to the length of vector 735 from the phased array emitter 730, and beam lock is achieved with respect to the phased array emitter 730 at position x1 (Tx(x1)).

[0068] As illustrated in FIG. 7B, the phased array emitter 732 is shown after being translated to position x2 (Tx(x2)). Position translation information can be obtained by analyzing IMU data generated by the handheld controller. After the translation, if the transmission pattern is maintained given the new phased array emitter position (i.e., x2), the power received at the electromagnetic field sensor 740 will decrease as illustrated by vector 737 in FIG. 7B intersecting the main lobe at position 738. Due to the larger distance between the electromagnetic emitter and the electromagnetic sensor in FIG. 7B compared to FIG. 7A, the decrease in the received power for a given translation (i.e., from position x1 to position x2) is reduced compared to the decrease in the received power for the same translation in FIG. 7A. Thus, an additional function can be generated that correlates the translation distance with the received power for a given distance greater than that illustrated in FIG. 7B. By measuring the received power at various translation distances, it is possible to generate a lookup table, equation, or other mathematical construct that maps the increase or decrease in received power as a function of (1) the translation distance and (2) the distance between the phased array emitter and the electromagnetic sensor. As a result, given the received power increase or decrease and the translation distance, the distance between the phased array emitter and the electromagnetic field sensor can be calculated. In some embodiments, the values ​​discussed above can be utilized to map the power derivative. [ka] This can be used to determine the distance between the phased array emitter and the electromagnetic field sensor as a function of received power and translation distance. Thus, embodiments of the present disclosure can utilize changes in the position of the handheld controller, as well as changes in the orientation of the handheld controller, to determine the distance between the emitter and the sensor.

[0069] It should be noted that in some embodiments, the increase or decrease in received power as a function of change in handheld controller position utilized to determine emitter-sensor distance is performed over a short period of time as compared to an active tracking loop that may re-point a steerable. Thus, embodiments enable distance measurement in conjunction with active tracking, e.g., as discussed with respect to FIG.

[0070] In some embodiments, the functionality of the handheld controller is enhanced utilizing information collected and / or available at the handheld controller. For example, if a change in IMU data is determined at the handheld controller, e.g., a determination that the handheld controller has been translated in a given direction by a given amount or rotated about a given axis by a given amount, beam steering can be initiated at the handheld controller prior to feedback being received from an electromagnetic field sensor in the AR headset. Thus, within the scope of the present disclosure are embodiments in which the handheld controller performs beam steering in response to feedback from the AR headset, as well as embodiments in which the handheld controller initiates beam steering in response to data measured and / or available at the handheld controller. Furthermore, combinations of these approaches are within the scope of the present disclosure. As an example, in an implementation in which IMU measurements at the handheld controller result in beam steering initiated at the handheld controller, feedback received from the AR headset can be utilized after the initiation of beam steering as an element of the control system for the beam steering process.

[0071] In some embodiments, in addition to the beam angle and distance between the handheld controller and the AR headset calculated using beam steering as described above, an IMU integrated into the handheld controller can be utilized to provide information about the handheld controller pose that can be integrated with the beam angle and distance between the handheld controller and the AR headset calculated above. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0072] Returning to FIG. 6, using the position of the AR headset, specifically the position of the electromagnetic field sensor in the xyz coordinate system, the angle θ between the handheld controller and the AR headset, and the distance from the handheld controller to the AR headset, the 6DoF pose of the handheld controller can be determined (640). The position of the AR headset can be determined from the head pose of the AR headset, which is typically tracked using an IMU in the AR headset. Based on the known position of the AR headset, the position of the handheld controller can be determined given the beam angle and the distance between the handheld controller and the AR headset. Furthermore, the angle of the beam with respect to the plane of the phased array elements (i.e., the yz plane as illustrated in FIG. 5A) can be used to determine the orientation of the handheld controller, resulting in the determination of the 6DoF pose (i.e., position and orientation). In some embodiments, data from the handheld controller IMU can be utilized to disambiguate the handheld controller orientation in situations where the rotation of the AR headset and totem around the z axis may result in a given distance and beam angle that does not distinguish a unique handheld controller pose. In some embodiments, the distance and beam angle are sufficient to determine the handheld controller pose, given the head pose (i.e., the pose of the AR headset).

[0073] Given the handheld controller position, active tracking of the handheld controller position and orientation (i.e., pose) can be performed (650). Thus, once the 6DoF pose of the handheld controller is determined, active tracking of the handheld controller 6DoF pose can be performed to update the 6DoF pose as a function of time. As an example, using a communication path between the AR headset and the handheld controller, movement of the handheld controller and / or the AR headset will result in a decrease in received power as the vector aligned with the main lobe and the vector between the handheld controller and the AR headset become misaligned. Based on this measured decrease in received power communicated to the handheld controller through the communication path between the AR headset and the handheld controller, the phase delay can be adjusted to steer the beam and realign the vector aligned with the main lobe and the vector between the handheld controller and the AR headset.

[0074] In an iterative manner, active tracking can be performed by determining the beam angle and distance and calculating the 6DoF pose of the handheld controller. As discussed above, the step size of the beam deviation can be adjusted during this active tracking process. Thus, as either the AR headset or the handheld controller moves, the active tracking can maintain the beam angle in real time based on the feedback so that the beam angle is aligned with the direction between the AR headset and the handheld controller.

[0075] FIG. 8 diagrammatically illustrates an augmented reality system in communication with a 5G cellular mobile communication system. As illustrated in FIG. 8, a 5G transmitter 810, also referred to as a small cell (e.g., femtocell, picocell, or microcell), communicates with an AR system 805 including an AR headset 820, an electromagnetic handheld controller 822, and an auxiliary unit 824. A communication device, e.g., a 5G device, is thus included in the AR system in some embodiments of the present disclosure. With reference to FIGS. 3 and 8, the AR headset 820, the electromagnetic handheld controller 822, and the auxiliary unit 824 illustrated in FIG. 8 may correspond to the AR headset 301, the electromagnetic field emitter 302, and the beltpack 370, respectively, illustrated in FIG. 3.

[0076] 8, beamforming can be utilized by the 5G transmitter 810 to direct a communication path from the 5G transmitter to the AR system 805, for example, along beam path 812 for communication with an AR headset 820, or along beam path 814 for communication with an auxiliary unit 824. In some implementations, the spatial proximity of various elements of the AR system 805 is sufficiently close to one another such that the beamforming performed by the 5G transmitter 810 is characterized by a single beam path connecting the 5G transmitter and the AR system without specificity associated with any particular element of the AR system.

[0077] By implementing beamforming in the communication path between the 5G transmitter 810 and the AR system 805, increased data rates, lower latency, and other benefits can be provided. Additionally, beamforming can be implemented in the AR system 805 to communicate with the 5G transmitter 810, as illustrated by beam path 830. Although beamforming in the handheld controller is illustrated by beam path 830, embodiments of the present disclosure can also utilize beamforming in the auxiliary unit or AR headset to facilitate communication with the 5G transmitter. Thus, in addition to beamforming implemented in the 5G transmitter to facilitate communication with the AR system, beamforming can be implemented in one or more elements of the AR system to facilitate communication with the 5G transmitter. Additionally, beamforming can be implemented in one or more of the elements of the AR system, including the AR headset, the handheld controller, and / or the auxiliary unit. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0078] In some embodiments, the location of the 5G transmitter or other suitable communication hub is known in a predetermined coordinate space (e.g., XYZ Cartesian coordinate space). Given this known location of the 5G transmitter, the AR system can utilize a communication path between the 5G transmitter and the AR system, for example, using beam steering, to determine the location of the AR system (or an element of the AR system) relative to the known location of the 5G transmitter. As an example, when communication is established between the 5G transmitter and an auxiliary unit of the AR system, the location of the auxiliary unit relative to the known location of the 5G transmitter can be determined. Thus, for example, during an initialization process for the AR system, the location of the AR system can be determined. Although 5G is illustrated in FIG. 8, embodiments of the present disclosure are not limited to this particular communication standard, and other communication hubs with known locations can be utilized within the scope of the present disclosure. Conversely, once the 5G transmitter and the AR system have established their relative positions, the AR system may use independently collected information about its changing position or attitude to more precisely perform beam steering along beam path 830, and may communicate such information about its changing position or attitude to the 5G transmitter to more precisely perform beam steering along beam paths 812 and 814.

[0079] It should also be understood that the examples and embodiments described herein are for illustrative purposes only, and that various modifications or changes in light thereof will be suggested to those skilled in the art and are to be included within the spirit and scope of the present application and the appended claims.

Claims

1. 1. An electromagnetic tracking system comprising:

1. A handheld controller including an electromagnetic emitter, the electromagnetic emitter comprising: a first phased array element characterized by a first phase; a second phased array element characterized by a second phase different from the first phase; and Including, a handheld controller, the first phased array element and the second phased array element configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern; and a head mounted augmented reality display including an electromagnetic sensor configured to sense the electromagnetic field pattern; An electromagnetic tracking system comprising:

2. A processor, the processor comprising: Controlling the first phase and the second phase; digitally calculating a position and orientation of the handheld controller based on the electromagnetic field pattern; a processor configured to: The electromagnetic tracking system of claim 1 further comprising:

3. The electromagnetic tracking system of claim 2 , wherein the processor is located within the handheld controller.

4. The electromagnetic tracking system of claim 2 further comprising an auxiliary unit including the handheld controller.

5. The electromagnetic tracking system of claim 4 , wherein the auxiliary unit comprises a beltpack.

6. The electromagnetic tracking system of claim 1 , wherein the electromagnetic field pattern is steered by modifying the first phase and the second phase.

7. 2. The electromagnetic tracking system of claim 1, wherein the electromagnetic emitter further comprises a third phased array element, the first phased array element, the second phased array element, and the third phased array element being arranged in a plane.

8. 8. The electromagnetic tracking system of claim 7, wherein the electromagnetic emitter further comprises a fourth phased array element, the first phased array element, the second phased array element, and the third phased array element forming three corners of a tetrahedron, and the fourth phased array element forming a fourth corner of the tetrahedron.

9. 1. A method for determining six degrees of freedom (DoF) pose of a handheld controller, the method comprising: Determining an attitude of an electromagnetic sensor; generating an electromagnetic beam from the handheld controller, the electromagnetic beam being characterized by an electromagnetic field pattern; steering the electromagnetic beam to increase received power at the electromagnetic sensor; determining a beam angle associated with the electromagnetic beam; determining a distance between the handheld controller and the electromagnetic sensor; determining a 6DoF attitude of the handheld controller using the attitude of the electromagnetic sensor, the beam angle, and the distance; A method comprising:

10. The handheld controller includes an electromagnetic emitter, the electromagnetic emitter comprising: a first phased array element configured to operate with a first phase; a second phased array element configured to operate with a second phase; and 10. The method of claim 9, comprising:

11. 11. The method of claim 10, wherein the electromagnetic emitter further comprises a third phased array element, the first phased array element, the second phased array element, and the third phased array element being arranged in a plane.

12. The method of claim 9 , wherein the pose of the electromagnetic sensor comprises a head pose, and the electromagnetic sensor is an element of an AR headset.

13. The method of claim 9 , wherein steering the electromagnetic beam includes aligning a central vector of the electromagnetic beam with a vector from the handheld controller to the electromagnetic sensor.

14. The method of claim 9 , wherein steering the electromagnetic beam to increase received power at the electromagnetic sensor comprises maximizing received power at the electromagnetic sensor.

15. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to: Determining an attitude of an electromagnetic sensor; generating an electromagnetic beam from a handheld controller, the electromagnetic beam being characterized by an electromagnetic field pattern; steering the electromagnetic beam to increase received power at the electromagnetic sensor; determining a beam angle associated with the electromagnetic beam; determining a distance between the handheld controller and the electromagnetic sensor; determining a six degree of freedom (DoF) attitude of the handheld controller using the attitude of the electromagnetic sensor, the beam angle, and the distance; 23. A non-transitory computer readable medium for performing operations including:

16. The handheld controller includes an electromagnetic emitter, the electromagnetic emitter comprising: a first phased array element configured to operate with a first phase; a second phased array element configured to operate with a second phase; and 20. The non-transitory computer readable medium of claim 15, comprising:

17. 17. The non-transitory computer-readable medium of claim 16, wherein the electromagnetic emitter further comprises a third phased array element, the first phased array element, the second phased array element, and the third phased array element being arranged in a plane.

18. 16. The non-transitory computer-readable medium of claim 15, wherein the pose of the electromagnetic sensor comprises a head pose, and the electromagnetic sensor is an element of an AR headset.

19. 16. The non-transitory computer-readable medium of claim 15, wherein steering the electromagnetic beam comprises aligning a central vector of the electromagnetic beam with a vector from the handheld controller to the electromagnetic sensor.

20. 16. The non-transitory computer-readable medium of claim 15, wherein steering the electromagnetic beam to increase received power at the electromagnetic sensor comprises maximizing received power at the electromagnetic sensor.

Images