Method and system for six-degree-of-freedom tracking of a transmitter in an augmented reality system using phased array beamforming

Abstract

An electromagnetic tracking system includes a handheld controller comprising a first phased array element characterized by a first phase and a second phased array element characterized by a second phase different from the first phase. The first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern. The electromagnetic tracking system also includes a head-mounted augmented reality display (HMD) comprising an electromagnetic sensor configured to sense the electromagnetic field pattern.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 811,914, filed on February 28, 2019, entitled “METHOD AND SYSTEM UTILIZING PHASED ARRAY BEAMFORMING FOR SIX DEGREE OFFREEDOM TRACKING FOR AN EMITTER IN AUGMENTED REALITY SYSTEMS,” the contents of which are incorporated herein by reference in their entirety. Background Art

[0003] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images, or portions thereof, are presented to a user in such a way that they appear to be real or can be perceived as real. Virtual reality or "VR" scenarios generally involve the presentation of digital or virtual image information that is opaque to other actual real-world visual input; augmented reality or "AR" scenarios generally involve the presentation of digital or virtual image information as an enhancement to the visualization of the actual world around the user.

[0004] Despite the advances in these display technologies, there remains a need in the art for improved methods, systems, and apparatus related to augmented reality systems, and particularly display systems. Summary of the Invention

[0005] The present disclosure relates to virtual reality (VR) and / or augmented reality (AR) imaging and visualization systems. The present disclosure generally relates to methods and systems for performing six degrees of freedom (DoF) tracking in VR and / or AR systems. More specifically, embodiments of the present disclosure provide methods and systems for directing energy emitted by an emitter (also referred to as a transmitter) in order to calculate the position and orientation of the emitter relative to a sensor. In some embodiments, beamforming using phased array elements of the emitter is used to achieve beam steering as more fully described herein. The present disclosure is applicable to a variety of applications in computer vision and image display systems.

[0006] According to an embodiment of the present disclosure, an electromagnetic tracking system is provided. The electromagnetic tracking system includes a handheld controller including an electromagnetic transmitter. The electromagnetic transmitter includes a first phased array element characterized by a first phase, and a second phased array element characterized by a second phase different from the first phase. The first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern. The electromagnetic tracking system also includes a head-mounted augmented reality display (ARD) including an electromagnetic sensor configured to sense the electromagnetic field pattern.

[0007] According to another embodiment of the present disclosure, a method for determining a six-degree-of-freedom pose of a handheld controller is provided. The method includes determining a pose of an electromagnetic sensor and generating an electromagnetic beam from the handheld controller. The electromagnetic beam is characterized by an electromagnetic field pattern. The method also includes steering the electromagnetic beam to increase received power at the electromagnetic sensor and determining a beam angle associated with the electromagnetic beam. The method further includes determining a distance between the handheld controller and the electromagnetic sensor, and using the pose of the electromagnetic sensor, the beam angle, and the distance to determine the six-degree-of-freedom pose of the handheld controller.

[0008] The present disclosure achieves numerous benefits over conventional techniques. For example, embodiments of the present disclosure provide methods and systems for increasing electromagnetic field strength at a sensor in a predetermined manner. Embodiments of the present disclosure can actively track the position of a transmitter, thereby improving the accuracy of attitude determination. These and other embodiments of the present disclosure, as well as their numerous advantages and features, are described in greater detail below in conjunction with the accompanying figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the detailed description, serve to explain the principles of the present disclosure. No attempt is made to show structural details of the present disclosure in more detail than is necessary for a basic understanding of the present disclosure and the various ways in which it can be implemented.

[0010] Figure 1 A system diagram schematically illustrates an electromagnetic (EM) tracking system according to some embodiments.

[0011] Figure 2 is a flow chart describing the functionality of an electromagnetic tracking system according to some embodiments.

[0012] Figure 3 An electromagnetic tracking system in combination with an augmented reality (AR) system is schematically illustrated, according to some embodiments.

[0013] Figure 4 Schematically illustrates electromagnetic tracking using a steered beam in the context of an AR device, according to some embodiments.

[0014] Figure 5A Schematically illustrated is a set of three electromagnetic sources generating a steered beam at a predetermined angle, according to some embodiments.

[0015] Figure 5B Schematically illustrating a set of four electromagnetic sources in a tetrahedral arrangement according to some embodiments.

[0016] Figure 6 is a flow chart illustrating a method of tracking the position of a handheld controller according to some embodiments.

[0017] Figure 7A Schematically illustrates the variation of the received signal as a function of the position of two transmitters according to some embodiments.

[0018] Figure 7B The variation of the received signal as a function of the position of two transmitters according to further embodiments is schematically shown.

[0019] Figure 8 Schematically illustrates an augmented reality system communicating with a 5G cellular mobile communication system. DETAILED DESCRIPTION

[0020] Augmented reality (AR) systems can be designed to interact with users. As an example, a user can be provided with a handheld controller, also known as a totem, which the user can use to interact with the AR system. Therefore, it is useful to be able to determine the position and orientation (e.g., 6 degrees of freedom (DoF) pose) of the handheld controller relative to other elements of the AR system (including a head-mounted display system worn by the user, also known as an AR headset or headphones).

[0021] One approach to achieving high-accuracy positioning may involve the transmission and detection of electromagnetic fields. For example, an electromagnetic field may be emitted by at least one electromagnetic field transmitter (commonly referred to as a "transmitter") and received by at least one electromagnetic field sensor (commonly referred to as a "sensor"). The transmitter(s) and sensor(s) may be strategically placed on the user's AR headset, waist pack, and / or other auxiliary devices (e.g., totems, tactile devices, gaming instruments, etc.). The transmitter(s) generate an electromagnetic field with a known spatial (and / or temporal) distribution in the environment of the AR headset user. The sensor(s) measure the electromagnetic field generated at the location(s) of the sensor(s). Based on these measurements and knowledge of the generated electromagnetic field distribution, the pose (e.g., position and / or orientation) of the sensor(s) relative to the transmitter(s) may be determined. Thus, the pose of the object to which the sensor(s) and / or transmitter(s) are attached may be determined. That is, the relative position of the sensor(s) and transmitter(s) may be determined.

[0022] Figure 1 An example system diagram of an electromagnetic tracking system 100 is shown. In some embodiments, the electromagnetic tracking system 100 includes one or more electromagnetic field transmitters (generally referred to as "transmitters 102") configured to transmit a known electromagnetic field. Figure 1 As shown in , the transmitter 102 can be coupled to a power source 110 (eg, a current source, a battery, etc.) to provide power to the transmitter 102 .

[0023] In some embodiments, the transmitter 102 includes several coils that generate an electromagnetic field (e.g., at least three coils positioned perpendicular to each other to generate fields in the X, Y, and Z directions). The coils can be operated at a predetermined AC frequency (e.g., in the range of 20 kHz-40 kHz) such that each coil generates an AC dipole-shaped electromagnetic field. This electromagnetic field is used to establish a coordinate space (e.g., an XYZ Cartesian coordinate space). This allows the system to map the position (e.g., (X, Y, Z) position) of the electromagnetic sensors 104a, 104b relative to a known electromagnetic field and determine the position and / or orientation of the electromagnetic sensors 104a, 104b. In some embodiments, the electromagnetic sensors 104a, 104b (generally referred to as "sensors") can be attached to one or more real-world objects. The sensors can include coils in which currents can be induced by an electromagnetic field (e.g., the electromagnetic field emitted by the transmitter 102). The sensor may include coils or loops (e.g., at least three coils positioned perpendicular to one another) positioned / oriented to capture incoming electromagnetic flux from an electromagnetic field (e.g., the electromagnetic field emitted by transmitter 102), and by comparing the currents induced through these coils, and knowing the relative positioning and orientation of the coils with respect to one another, the relative position and orientation of the sensor with respect to transmitter 102 may be calculated.

[0024] One or more parameters related to the behavior of the coils and inertial measurement unit (IMU) components operably coupled to the sensor can be measured to detect the position and / or orientation of the sensor (and the object to which it is attached) relative to the coordinate system to which the transmitter 102 is coupled. In some embodiments, multiple sensors can be used relative to the transmitter 102 to detect the position and orientation of each of the sensors in the coordinate space. The electromagnetic tracking system 100 can provide position in three directions (e.g., X, Y, and Z directions) and further in two or three orientation angles. In some embodiments, the IMU measurements can be compared with the coil measurements to determine the position and orientation of the sensor. In some embodiments, both electromagnetic (EM) data and IMU data, as well as various other data sources (such as cameras, depth sensors, and other sensors), can be combined to determine position and orientation. This information can be transmitted (e.g., wirelessly, via Bluetooth, etc.) to the controller 106. In some embodiments, the pose (or position and orientation) can be reported at a relatively high refresh rate in conventional systems.

[0025] Typically, the electromagnetic field transmitter is coupled to a relatively stable and large object, such as a table, console, wall, or ceiling, and one or more sensors are coupled to a small object, such as a medical device, handheld game component, etc. Alternatively, as described below with reference to Figure 3As described above, various features of an electromagnetic tracking system can be employed to produce a configuration in which changes or deltas in position and / or orientation between two objects moving in space relative to a more stable global coordinate system can be tracked. In other words, Figure 3 , a configuration is shown in which changes in the electromagnetic tracking system can be used to track position and orientation deltas (changes) between a head-mounted component and a handheld component, while otherwise determining the head pose relative to a global coordinate system (with respect to the user's local room environment), such as by using simultaneous localization and mapping (SLAM) techniques using an outward-facing capture camera that can be coupled to the head-mounted component of the system.

[0026] Controller 106 can control transmitter 102 and can also capture data from the sensor. It should be understood that the various components of the system can be coupled to each other through any electromechanical or wireless / Bluetooth means. Controller 106 can also include data regarding a known electromagnetic field and a coordinate space relative to the electromagnetic field. This information is then used to detect the position and orientation of the sensor relative to the coordinate space corresponding to the known electromagnetic field.

[0027] One of the advantages of electromagnetic tracking systems is that they produce high-resolution, highly repeatable tracking results with minimal latency. Additionally, electromagnetic tracking systems do not necessarily rely on optical trackers and can easily track sensors / objects that are not within the user's line of sight.

[0028] It will be appreciated that the strength V of the electromagnetic field decreases as a cubic function of the distance r from the coil transmitter (e.g., transmitter 102). Therefore, an algorithm may be used based on the distance from the transmitter 102. The controller 106 (which may also be referred to as a processor) may be configured with such an algorithm to determine the position and orientation of the sensor at different distances from the transmitter 102. Given that the strength of the electromagnetic field decreases rapidly as the sensor moves away from the transmitter 102, optimal results in terms of accuracy, efficiency, and low latency may be achieved at closer distances. In a typical electromagnetic tracking system, the transmitter is powered by an electric current (e.g., a plug-in power supply) and the sensor is located within a 20 foot radius of the transmitter. In many applications, including AR applications, a shorter radius between the sensor and the transmitter may be more desirable.

[0029] Figure 2An example flow chart of a method 200 describing the functionality of an electromagnetic tracking system (such as electromagnetic tracking system 100) is shown. At 202, a known electromagnetic field is emitted. In some embodiments, an electromagnetic field transmitter can generate the electromagnetic field. For example, each coil of the electromagnetic field transmitter can generate an electromagnetic field in one direction (e.g., X, Y, or Z). The electromagnetic field can be generated with an arbitrary waveform. In some embodiments, the electromagnetic field component along each axis can oscillate at a slightly different frequency than other electromagnetic field components along other directions. At 204, a coordinate space corresponding to the electromagnetic field can optionally be determined. For example, a controller can automatically determine the coordinate space around the transmitter and / or sensor based on the electromagnetic field. In some embodiments, the coordinate space may not be determined at this stage of the method. At 206, the behavior of the coil at the sensor (which can be attached to a known object) can be detected. For example, the current induced in the coil can be calculated. In some embodiments, the rotation of the coil or any other quantifiable behavior can be tracked and measured. At 208, this behavior can be used to detect the position or orientation of the sensor and / or known object (e.g., an AR headset including the sensor) relative to the transmitter, or vice versa. For example, the controller 106 may consult a mapping table that relates the behavior of the coils at the sensor to various positions or orientations. Based on these calculations, the position in coordinate space and the orientation of the sensor and / or transmitter may be determined.

[0030] In the context of AR systems, it may be necessary to modify one or more components of the electromagnetic tracking system to facilitate accurate tracking of mobile components (e.g., emitters and sensors). As mentioned above, in many AR applications, it may be necessary to track the user's head pose and orientation. Accurately determining the user's head pose and orientation allows the AR system to display appropriate / relevant virtual content to the user. For example, a virtual scene may include a virtual monster hidden behind a real building. Depending on the pose and orientation of the user's head relative to the building, the view of the virtual monster may need to be modified to provide a realistic AR experience. Alternatively, the position and / or orientation of a totem, haptic device, or some other device that interacts with virtual content may be important in enabling the user to interact with the AR system. For example, in many gaming applications, the AR system can detect the position and orientation of real objects relative to virtual content. Alternatively, when a virtual interface is displayed, the position of a totem, the user's hand, a haptic device, or any other real object configured to interact with the AR system can be known relative to the displayed virtual interface so that the system can understand commands, interactions, etc. Some positioning methods (such as optical tracking) may suffer from high latency and low resolution, making rendering virtual content challenging in many AR applications.

[0031] In some embodiments, relative to Figure 1 and Figure 2The electromagnetic tracking system discussed can be applied to AR systems to detect the position and orientation of one or more objects relative to the emitted electromagnetic field. Typical electromagnetic tracking systems tend to have large and bulky electromagnetic transmitters (e.g., Figure 1 This is problematic for head-mounted AR devices (e.g., with totems). However, in the context of AR systems, smaller electromagnetic transmitters (e.g., in the millimeter range) can be used to emit known electromagnetic fields.

[0032] Figure 3 An electromagnetic tracking system is shown in conjunction with an AR system, wherein an electromagnetic field phased array transmitter (generally referred to as "phased array transmitter 302") is incorporated as part of a handheld controller 306. Handheld controller 306 can move independently relative to AR headset 301 (or belt pack 370). For example, handheld controller 306 can be held in the hand of the user, or handheld controller 306 can be mounted to the hand or arm of the user (e.g., as a ring or bracelet or as part of a glove worn by the user). In some embodiments, handheld controller 306 can be a totem, for example, for use in a gaming environment (e.g., a multi-degree-of-freedom controller) or to provide a rich user experience in an AR environment or to allow the user to interact with the AR system. In some embodiments, handheld controller 306 can be a haptic device. In some embodiments, phased array transmitter 302 can be incorporated as part of belt pack 370.

[0033] The handheld controller may include a battery 310 or other power source for powering the phased array transmitter 302. It should be understood that the phased array transmitter 302 may also include or be coupled to an IMU 350 assembly configured to assist in determining the position and / or orientation of the phased array transmitter 302 relative to other components. This is particularly advantageous where both the phased array transmitter 302 and the electromagnetic field sensor 304 (generally referred to as "sensor 304") are movable. Placing the phased array transmitter 302 in the handheld controller 306 rather than the belt pack 307, as shown, may be advantageous. Figure 3, helps ensure that the phased array transmitter 302 does not compete for resources at the belt pack 370, but instead uses its own battery source at the handheld controller 306. In some embodiments, the phased array transmitter 302 can be provided on the AR headset 301, and the sensor 304 can be provided on the handheld controller 306 or the belt pack 370. Thus, embodiments of the present disclosure provide implementations in which the handheld controller 306 is implemented as a handheld unit, while in other embodiments, the handheld controller is implemented in the AR headset 301, and in additional embodiments, the handheld controller is implemented in an auxiliary unit (e.g., a belt pack 307). Furthermore, in addition to implementations in which the handheld controller 306 is implemented in a single device, the functionality of the handheld controller and accompanying physical components can be distributed across multiple devices (e.g., the handheld controller 306, the AR headset 301, and / or an auxiliary unit such as a belt pack 307).

[0034] In some embodiments, sensor 304 may be placed at one or more locations on AR headset 301 along with other sensing devices or sensors 308 (such as one or more IMUs or additional electromagnetic flux capture coils). Figure 3 As shown in FIG, sensors 304, 308 can be placed on one or both sides of the AR headset 301. Because sensors 304, 308 can be designed to be quite small (and in some cases, less sensitive), having multiple sensors 304, 308 can improve efficiency and accuracy. In some embodiments, one or more sensors can also be placed on a waist pack 370 or any other part of the user's body. Sensors 304, 308 can wirelessly communicate, for example, via Bluetooth, to a computing device that determines the pose and orientation of sensors 304, 308 (and the AR headset 301 to which they are attached). In some embodiments, the computing device can be located at the waist pack 370. In some embodiments, the computing device can be located at the AR headset 301 or the handheld controller 306. In some embodiments, the computing device can further include a mapping database 330 (e.g., a mapping database, a cloud resource, a navigable world model, a coordinate space, etc.) to detect poses to determine the coordinates of real and / or virtual objects, and can even connect to cloud resources and a navigable world model. In some embodiments, the handheld controller 306 can control the timing of electromagnetic transmission by the electromagnetic transmitter and sensing by the electromagnetic sensor such that the position and orientation of the electromagnetic transmitter and the electromagnetic sensor are calculated based on the field from the modified electromagnetic field pattern. In some embodiments, the position and orientation of the electromagnetic transmitter is calculated relative to the electromagnetic sensor. In other embodiments, the position and orientation of the electromagnetic sensor is calculated relative to the electromagnetic transmitter. In some embodiments, the position and orientation of the electromagnetic transmitter and the electromagnetic sensor are calculated.

[0035] As mentioned above, some electromagnetic transmitters may be too bulky for AR devices. Therefore, the transmitter can be designed to be compact using smaller components (e.g., coils) than traditional systems. However, given that the strength of the electromagnetic field decreases as a cubic function of the distance from the transmitter, when compared to, for example, Figure 1 A shorter radius (eg, approximately 3 to 3.5 feet) between the sensor 304 and the phased array transmitter 302 may reduce power consumption when compared to conventional systems as described in detail in the system.

[0036] In some embodiments, this aspect can be used to extend the life of the battery 310 that can power the handheld controller 306 and the phased array transmitter 302 in one or more embodiments. In some embodiments, this aspect can be used to reduce the size of the coils that generate the electromagnetic field at the phased array transmitter 302. However, to achieve the same strength of the electromagnetic field, the power may need to be increased. This allows for a compact phased array transmitter 302 that can be compactly mounted in the handheld controller 306.

[0037] When using an AR device for an electromagnetic tracking system, some other changes may be made. While this attitude reporting rate is quite good, an AR system may benefit from a more efficient attitude reporting rate. To this end, IMU-based attitude tracking may be used (in addition or alternatively). Advantageously, the IMU may be kept as stable as possible in order to improve the efficiency of the attitude detection process. The IMU may be designed so that it remains stable for 50-100 milliseconds. It will be appreciated that some embodiments may utilize an external attitude estimator module (e.g., the IM may drift over time) which may enable attitude updates to be reported at a rate of 10 to 20 Hz. By keeping the IMU stable at a reasonable rate, the rate of attitude updates may be significantly reduced to 10 to 20 Hz (compared to the higher frequencies in conventional systems).

[0038] If the electromagnetic tracking system can be run at, for example, a 10% duty cycle (e.g., only every 100 milliseconds for a ground truth ping), the AR system can save power. This can mean that the electromagnetic tracking system wakes up for 10 milliseconds out of every 100 milliseconds to generate a pose estimate. This can directly translate into power savings, which in turn can impact the size, battery life, and cost of the AR device (e.g., AR headset 301 and / or handheld controller 306).

[0039] In some embodiments, this can be accomplished by providing a second handheld controller (not shown) instead of Figure 3This reduction in duty cycle can be strategically exploited by using only one handheld controller 306 as shown in FIG. For example, a user may be playing a game that requires two controllers, etc. Or, in a multi-user game, two users may be using their own controllers to play the game. When two controllers are used (e.g., symmetrical controllers for each hand) instead of one controller, the controllers can operate with an offset duty cycle. The same concept can also be applied to controllers used by two different users playing a multi-player game.

[0040] Although some embodiments of the present disclosure are discussed in the context of using phased arrays of electromagnetic elements in electromagnetic transmitters, embodiments of the present disclosure are not limited to the use of electromagnetic radiation and other forms of energy may be used to perform beamforming, including acoustic energy (i.e., sound), resulting in an energy distribution characterized by regions of constructive and destructive interference.

[0041] In order to determine the six DoF pose (i.e., position and orientation) of the handheld controller 306 relative to the AR headset 301, magnetic tracking can be utilized. Using a transmitter provided in the handheld controller, a magnetic field of a known geometry can be created. One or more sensors in the headset can then be used to determine the six DoF relationship between the handheld controller and the AR headset based on measurements of the position of the AR headset in the known geometry. In some cases, the measurement data provided by the IMU can be used, for example, in the context of sensor fusion, to calculate or improve the measurement of the six DoF pose of the handheld controller. However, although IMUs can be suitable for measuring relative motion, they may have a level of bias associated with their measurement data. Due to the integration of this bias, drift in the IMU position may accumulate, thereby degrading the measurement accuracy. Therefore, embodiments of the present disclosure provide measurement data characterized by reduced bias. In some embodiments, although the measurement data may include a noise component, the reduction in bias enables the noise to be averaged to provide a signal with improved accuracy.

[0042] Figure 4Schematic diagram illustrating electromagnetic tracking using a steered beam in the context of an AR device, according to some embodiments. As described more fully below, a transmit beam 402 transmitted by a handheld controller 410 is oriented at a beam angle θ. Knowing the phase delay relationship between the individual phased array elements in a phased array transmitter 412, the beam angle θ can be calculated. A feedback loop implemented using a bidirectional communication path 430 (e.g., Bluetooth) can then be utilized to adjust the phase delay associated with each phased array element in the phased array transmitter 412 to steer the beam and achieve / maintain a maximum received power at an electromagnetic field sensor 422 of an AR headset 420. As described more fully herein, the communication path 430 enables data related to the handheld controller 410 to be transmitted to the AR headset 420, such as handheld controller IMU data, phase delays of the individual phased array elements in the phased array transmitter 412, beam angle θ, and the like. In addition, communication path 430 enables data related to AR headset 420 to be sent to handheld controller 410, such as AR headset IMU data, received power at electromagnetic field sensor 422, etc. The beam angle θ of transmit beam 402 can be swept and steered to maximize the energy received at AR headset 420, and more specifically at electromagnetic field sensor 422.

[0043] Given the phase delays of the individual phased array elements in phased array transmitter 412 and the distance between handheld controller 410 and AR headset 420 (which can be calculated based on the characteristics of transmit beam 402, as described more fully below), the six-DoF pose of handheld controller 410 can be determined relative to the known pose of AR headset 420.

[0044] Figure 5A Schematically illustrates a set of three electromagnetic sources (e.g., individual phased array elements) generating a steered beam at a predetermined angle in accordance with some embodiments. Figure 5A As shown in FIG, three phased array elements 510, 512 and 514 are included as elements of a phased array transmitter 505 provided in a handheld controller (e.g., Figure 4 In some embodiments, each of the phased array elements 510, 512, and 514 can be a coil operated at a predetermined AC frequency (e.g., in the range of 20 KHz-40 KHz) such that each coil generates an AC dipole-shaped electromagnetic field. In the exemplary phased array transmitter 505, all of the phased array elements 510, 512, 514 have their coils aligned with a common direction (e.g., the z-direction). Typically, the relative orientation and arrangement of the phased array elements 510, 512, 514 are selected so that the main lobe generated by the phased array transmitter 505 is along the direction of the AR headset (e.g., Figure 4 The AR headset 420 in FIG. 4 will be positioned relative to the handheld controller (e.g. Figure 4The AR headset (e.g., AR headset 301) will orient the handheld controller (e.g., handheld controller 306) in the most common orientation relative to the handheld controller (e.g., handheld controller 410) during normal operation (e.g., the handheld controller is held in the user's hand at approximately waist / mid-torso level, with the palm facing left or right, depending on which hand the user is holding the handheld controller). Figure 3 Shown in.

[0045] Because each of the phased array elements 510, 512, 514 emits electromagnetic radiation, the wave nature of the emitted electromagnetic radiation causes interference between the electromagnetic radiation emitted by the phased array elements 510, 512, and 514. This interference creates regions of constructive interference and regions of destructive interference. Figure 5A The phased array elements 510, 512, and 514 shown in FIG, if the phase of each element is aligned (i.e., there is no delay between the elements), will produce a main lobe along the x-direction orthogonal to the plane of the drawing. If a fixed phase delay relationship is achieved between the phased array elements (i.e., phased array element 510 has a phase delay (p) which can be zero, phased array element 512 has a phase delay (pi), and phased array element 514 has a phase delay (p2), interference between the phased array elements 510, 512, and 514 will result in the generation of a main lobe 525 as well as side lobes (not shown for clarity). In some embodiments, the phased array elements 510, 512, and 514 will be positioned so that Figure 5A The yz plane shown in will be Figure 4 The beam angles Θ shown in are orthogonal so that the main lobe is aligned with the most common direction between the handheld controller and the AR headset, with no phase delay. Figure 5A Three phased array elements are shown in FIG, but embodiments of the present disclosure are not limited to this particular number and embodiments of the present disclosure may utilize other numbers including two phased array elements and more than three phased array elements.

[0046] In some embodiments, a single coil may be used for each phased array element. In some embodiments, multiple coils may be used for each phased array element. As an example, to improve performance when the handheld controller is rotated 90° with respect to the most common orientation of the handheld controller with respect to the AR headset, a second coil for each phased array element may be used to increase the strength of the main lobe in directions pointing in / out of the graphics plane (i.e., ±x directions). In addition to the phased array elements of the handheld controller, the transmitter may be implemented as a Figure 1 The transmitter 102 in FIG. 1 is a three-coil transmitter.

[0047] Thus, even though the three phased array elements 510, 512, and 514 are static (i.e., their positions in the handheld controller are fixed), control of the phase delay associated with each phased array element can achieve steering of the main lobe 525. Figure 5A As shown in FIG, the main lobe 525 is centered about a vector 520 oriented at a beam angle θ having components θ along the x-axis, the y-axis, and the z-axis, respectively. x ,θ y and θ z Vector 520 can be called the center vector because it is aligned with the center of the main lobe. By modifying the phase delay and / or The main lobe 525 can be steered so that the vector 520 can be oriented at an arbitrary beam angle Θ. Although only the phase delays corresponding to phased array elements 512 and 514 are modified in this example and / or However, it will be appreciated that, in conjunction with the control of the beam angle Θ associated with the main lobe 525, the additional phase delay can be associated with phased array element 510. When utilizing additional phase delay When , all three phase delays can be controlled to achieve the desired phase delay between the phased array elements 510, 512, 514. Many variations, modifications, and alternatives will be recognized in the art.

[0048] The beam steering of the main lobe 525 can be used to align the vector 520 with the vector pointing from the phased array transmitter 505 provided in the handheld controller to the electromagnetic field sensor in the AR headset (e.g., electromagnetic field sensor 422). Once the vector 520 pointing from the phased array transmitter 505 provided in the handheld controller is aligned with the electromagnetic field sensor in the AR headset, the energy received at the electromagnetic field sensor will be maximized because the center of the main lobe 525 will be pointing towards the electromagnetic field sensor. Figure 6 As described more fully, beam steering of the main lobe 525 will be used to determine the orientation of the handheld controller relative to the AR headset.

[0049] While a description of placement of a phased array transmitter in a handheld controller and placement of an electromagnetic field sensor in an AR headset is provided herein, this is not required by the present disclosure, and the phased array transmitter may be placed in the AR headset with the corresponding electromagnetic field sensor placed in the handheld controller. Additionally, in some embodiments, the phased array transmitter or the electromagnetic field sensor may be placed in an auxiliary unit. Typically, the handheld controller can support the power requirements and weight associated with operation of the phased array transmitter, and therefore, the phased array transmitter will be implemented as a component disposed in the handheld controller. Reference Figure 3 , the computing work can be performed in the handheld controller 306, the AR headset 301, the belt pack 370, or the small cell (not shown), or distributed among these elements as appropriate. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0050] Figure 5B Schematically illustrates a set of four electromagnetic sources in a tetrahedral arrangement according to some embodiments. Figure 5B As shown in FIG, four phased array elements 510, 512, 514, and 516 are positioned to be located at the four vertices of a tetrahedron. Phased array elements 510, 512, and 514 are located in the yz plane, while phased array element 516 is located at a predetermined height above the yz plane measured along the x-axis. By utilizing Figure 5B The tetrahedral arrangement shown in , provides additional control over the beam shape of the main lobe and increased control over beam steering in the yz plane.

[0051] Figure 6 An example flow chart illustrating a method 600 for tracking the position of a handheld controller in accordance with some embodiments is shown. The method includes determining a pose (i.e., position and orientation) of an electromagnetic field sensor (step 610). In some embodiments, determining the pose of the electromagnetic field sensor may include determining a head pose of an AR headset that includes the electromagnetic field sensor. In some embodiments, only the position of the electromagnetic field sensor is determined. In some embodiments, one or more IMUs in the AR headset may be used to provide this head pose determination. The method also includes an optional process of determining an approximate position of the electromagnetic field transmitter (e.g., a handheld controller) using, for example, one or more IMUs in the electromagnetic field transmitter (step 620). In some implementations, the optional process also determines an approximate orientation (i.e., an approximate pose) of the electromagnetic field transmitter (e.g., a handheld controller) using, for example, one or more IMUs in the electromagnetic field transmitter. In some embodiments, the approximate pose may be determined based on acceleration due to gravity that exceeds acceleration in a direction orthogonal to gravity when the user holds the handheld controller and calibrates or initializes the handheld controller.

[0052] Beam steering is then initiated (step 630) to control the main lobe of the electromagnetic beam, such as main lobe 525 in FIG. 5 . Using an optimization algorithm, also known as a search algorithm, the beam is steered until the energy received by the electromagnetic sensor is maximized. One of several optimization algorithms known to those skilled in the art can be used for this beam steering process. Typically, the fitness function F is maximized:

[0053]

[0054] in and in is the phase delay associated with phased array element 510, is the phase delay associated with phased array element 512, and is the phase delay associated with phased array element 514. The phased array elements may be associated with the following operating frequencies: Phased array element 510: in Can be zero; Phased array element 512: And the phased array element 514: As mentioned above, beam steering can be achieved by controlling the phase delay and or control all three phases and to achieve.

[0055] According to various embodiments, the beam steering process discussed herein can be implemented using beam steering information determined at an AR headset or a handheld controller. For example, when beam steering information is determined at the AR headset, the following process can be implemented. The handheld controller transmits an electromagnetic beam via a first communication path and transmits sensor data (e.g., sensor data collected at the handheld controller) via a second communication path. The AR headset receives the electromagnetic beam via the first communication path and receives the handheld controller sensor data via the second communication path.

[0056] The AR headset then calculates the power associated with the electromagnetic beam and determines beam steering information based on the calculated power associated with the electromagnetic beam, the handset sensor data, and the AR headset sensor data. Given the beam steering information, the AR headset transmits the beam steering information via a second communication path, and the handheld controller receives the beam steering information. Thus, the handheld controller is able to change the beam steering characteristics of the electromagnetic beam based on the received beam steering information. For example, the received beam steering information may instruct the handheld controller to increase or decrease the phase delay. and As another example, the AR headset may determine that a handheld controller is moving relative to the AR headset based on the handset sensor data and the AR headset sensor data, and may accordingly generate beam steering information that causes the handheld controller to increase or decrease phase delay. and , so that the main beam (e.g., main lobe 525) and its corresponding vector (e.g., vector 520) can continue to be aligned with the electromagnetic sensor in the AR headset.

[0057] Furthermore, when beam steering information is determined at a handheld controller, the following process flow may be implemented: the handheld controller transmits an electromagnetic beam via a first communication path, and the AR headset receives the electromagnetic beam via the first communication path. The AR headset then calculates the power associated with the electromagnetic beam and transmits the calculated power and AR headset sensor data to the handheld controller via a second communication path.

[0058] The handheld controller receives the calculated power and the AR headset sensor data and determines beam steering information based on the calculated power associated with the electromagnetic beam, the AR headset sensor data, and the handheld controller sensor data. Thus, the handheld controller is capable of changing the beam steering characteristics of the electromagnetic beam based on the received beam steering information.

[0059] In some embodiments, a calibration process can be used to account for system characteristics and improve the accuracy of the beam steering algorithm. In addition, a variety of control and optimization algorithms are applicable to embodiments of the present disclosure. Exemplary control and optimization algorithms that can be used to maximize the fitness function include gradient descent, stochastic gradient descent, gradient descent with momentum, deep learning algorithms, etc. As an example, during the initialization phase, the beam can be stabilized by changing the phase delay by a large amount. and To steer, this larger amount results in a large angular deviation of the steered beam (e.g., 10° increments) as the area around the handheld controller is analyzed. Returning to the beam angle (and associated phase delay) that provides the maximum received power value, progressively smaller increments (e.g., 5°, 2°, 1°, 0.5°) can be utilized to maximize the received power.

[0060] The initialization process can be informed by ergonomic studies that provide information related to the probability that the handheld controller will be oriented in a given orientation relative to the AR headset. As an example, if the handheld controller is typically held at waist / mid-torso with the palm facing left, the initialization process can be started near an angle that aligns the handheld controller and the AR headset in that orientation. The most common orientations between the handheld controller and the AR headset are as follows: Figure 4 θ is shown as the beam angle θ. It will be apparent to those skilled in the art that optimization algorithms that can provide faster convergence are within the scope of this disclosure. Once beam lock is achieved, the angular adjustment of the beam steering angle can be appropriately reduced or increased depending on the specific application. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0061] When operating these optimization algorithms, a communication path is provided between the handheld controller and the AR headset to exchange data collected during operation of, for example, the optimization algorithm. Figure 4An example of a communication path 430 in is a Bluetooth wireless link that provides two-way communication between a handheld controller and an AR headset. For example, the communication path can be used to send power data received from the AR headset to the handheld controller. In addition, the communication path can be used to send information about the beam angle from the handheld controller to the AR headset (e.g., a beam angle based on a phase delay or a phase delay for calculating the beam angle at the AR headset). Thus, the communication path 430 enables control signals and data to be sent in a bidirectional manner between the handheld controller and the AR headset. For example, IMU data generated in the AR headset or handheld controller can be sent via the communication path 430. Although a Bluetooth wireless link is described as an example of a communication path 430, other communication technologies are also included within the scope of the present disclosure, including a WiFi wireless link, a wired link such as a USB link, and the like.

[0062] In operation, beam steering due to modification of the phase delay results in an increase / decrease in received power at the AR headset. In response to changes in received power at the electromagnetic field sensor in the AR headset, the electromagnetic field transmitter in the handheld controller can change the phase delay between the phased array elements to steer the beam in a manner that increases received power.

[0063] As described above, once the beam has been steered to align the vector passing through the center of the main lobe with the vector pointing from the phased array transmitter in the handheld controller to the electromagnetic field sensor (which may also be referred to as a receiver), the energy received at the electromagnetic field sensor in the AR headset is maximized. Thus, beam steering can be implemented to maximize received power (and, in some embodiments, maintain received power) using feedback sent via the communication path from the electromagnetic field sensor in the AR headset to the phased array transmitter in the handheld controller.

[0064] Reference again Figure 6 Once the beam is steered to maximize received power, an indicator may be generated to indicate that beam lock is achieved (step 632). Achieving beam lock will be associated with a given orientation between the handheld controller and the AR headset at a given time. As the handheld controller (and / or AR headset) moves over time, the tracking algorithm will continue to run to maintain beam lock through beam steering. Data provided by the IMU in the handheld controller and / or the IMU in the AR headset may be utilized by the initialization and / or tracking algorithm, for example, to reduce the angular step deviation in response to small changes in the position and / or orientation, velocity, acceleration, etc. of the handheld controller and the AR headset, or to increase the angular step deviation in response to large changes in the position and / or orientation, velocity, acceleration, etc. of the handheld controller and the AR headset. In some embodiments, the angular step deviation is modified in response to the angular velocity and / or acceleration of the handheld controller because rotation of the handheld controller can result in rapid steering of the transmit beam.

[0065] When the energy at the electromagnetic field sensor reaches a maximum, the beam angle θ of the main lobe orientation may be based on the phase delays associated with phased array elements 512 and 514, respectively. and In other words, given the phase delay and As well as the geometric arrangement of phased array elements 510, 512, and 514, a beam angle θ can be calculated based on the phase delays and the geometric arrangement. Given the beam angle θ between the handheld controller and the AR headset, it is known that the handheld controller and the AR headset are arranged along a vector connecting the handheld controller and the AR headset. With this information, as described in more detail below, the distance between the handheld controller and the AR headset can be determined and added at step 634.

[0066] Embodiments of the present disclosure utilize one of several techniques to determine the distance between a handheld controller and an AR headset. For example, in some embodiments, the measured power can be compared to a calibration table or function that associates received power with distance. Utilizing a communication path, IMU data from the handheld controller can be fused with the measured power at the AR headset to optimize distance calculations. In some embodiments, a known electromagnetic field pattern can be used to determine the distance between the handheld controller and the AR headset given a given received power. As an example, for a given electromagnetic field pattern and a given distance, steering the main lobe to a predetermined angle (e.g., 5°) will result in a predetermined amount (e.g., 10%) reduction in received power. At a greater distance (e.g., twice the distance), steering the main lobe to a predetermined angle will result in a smaller reduction (e.g., 20%) in received power. Once the received power is maximized, the phase delay can be changed to steer the beam to a predetermined angle (e.g., through a range of angles), and the reduction in received power can be used to calculate the distance. One of ordinary skill in the art will recognize many variations, modifications, and alternatives.

[0067] In some embodiments, the measurement of received power based on the position of the handheld controller can be used to determine the distance between the handheld controller and the AR headset. Figure 7A and 7B describe.

[0068] Figure 7A Schematically illustrates an example of how the received signal varies depending on the positions of two transmitters according to some embodiments. Figure 7AIn FIG, a phased array transmitter 710 is located at position x1 and is characterized by a transmit pattern 711 having a center angle aligned with a vector 715. For clarity, only the center or main lobe of the transmit pattern is shown. An electromagnetic field sensor 720 is located at a given distance from the phased array transmitter 710, which is equal to the length of the vector 715, and beam locking of the phased array transmitter has been achieved at position x1 (Tx(x1)).

[0069] refer to Figure 7A , showing the phased array transmitter 712 after having been translated to position x2 (Tx(x2)). The position translation information can be obtained by analyzing the IMU data generated by the handheld controller and sent to the AR headset, as this IMU data can be used to track the pose of the handheld controller. In some embodiments, the IMU data related to the pose of the handheld controller is fused with measurements of electromagnetic power received by the AR headset to provide a handheld controller pose value. After the translation, if the transmit mode is maintained given the new phased array transmitter position (i.e., x2), the power received at the electromagnetic field sensor 720 will decrease, as Figure 7A As shown by vector 717 in , it intersects the main lobe at position 718. Because the main lobe amplitude is significantly reduced at position 718, a function can be generated that relates the translation distance to the received power for a given distance.

[0070] Figure 7B Schematically illustrates an example of how the received signal varies depending on the positions of two transmitters according to some embodiments. Figure 7B The distance equal to the length of vector 735 is greater than the distance equal to Figure 7A The length of the vector 715 shown in is equal to the distance. Figure 7B , a phased array transmitter 730 is located at position x1 and is characterized by a transmit pattern 731 having a center angle aligned with a vector 735. For clarity, only the center or main lobe of the transmit pattern is shown. An electromagnetic sensor 740 is located at a distance from the phased array transmitter 730 equal to the length of the vector 735, and the phased array transmitter 730 achieves beam lock at position x1 (Tx(x1)).

[0071] Also there Figure 7B , the phased array transmitter 732 after it has been translated to position x2 (Tx(x2)). The position translation information can be obtained by analyzing the IMU data generated by the handheld controller. After the translation, if the transmit mode is maintained given the new phased array transmitter position (i.e., x2), the power received at the electromagnetic sensor 740 will decrease, as shown in FIG. Figure 7B As shown by vector 737 in , it intersects the main lobe at position 738. Figure 7A In comparison, due to Figure 7BFor larger distances between the electromagnetic transmitter and the electromagnetic sensor, for a given translation (i.e., from position x1 to position x2), the received power decreases with Figure 7A Therefore, an additional function can be generated that combines the translation distance with the received power for Figure 7B By measuring the received power at various translation distances, a lookup table, equation, or other mathematical structure can be generated that maps an increase or decrease in received power as a function of (1) the translation distance and (2) the distance from the phased array transmitter to the electromagnetic sensor. As a result, given an increase or decrease in received power and the translation distance, the distance between the phased array transmitter and the electromagnetic field sensor can be calculated. In some embodiments, the values ​​discussed above can be used to map the power derivative function:

[0072]

[0073] It can be used to determine the distance between the phased array transmitter and the electromagnetic field sensor based on the received power and the translation distance. Therefore, embodiments of the present disclosure can use the position change of the handheld controller and the orientation change of the handheld controller to determine the distance from the transmitter to the sensor.

[0074] It should be noted that in some embodiments, the increase or decrease in received power based on changes in the position of the handheld controller (which is used to determine the distance from the transmitter to the sensor) is performed in a short period of time, which is compared to the active tracking cycle that can redirect the steerable. Thus, embodiments can be used as, for example, with respect to Figure 6 The discussed combination with active tracking is used for distance measurement.

[0075] In some embodiments, information collected and / or available at the handheld controller is utilized to enhance functionality of the handheld controller. For example, if a change in IMU data is determined at the handheld controller, e.g., a determination that the handheld controller has translated a given amount in a given direction or rotated a given amount about a given axis, beam steering can be initiated at the handheld controller before feedback is received from an electromagnetic field sensor in the AR headset. Thus, embodiments of the present disclosure in which the handheld controller performs beam steering in response to feedback from the AR headset, as well as embodiments in which the handheld controller initiates beam steering in response to data measured and / or available at the handheld controller, are included within the scope of the present disclosure. Furthermore, combinations of these methods are included within the scope of the present disclosure. As an example, in implementations in which IMU measurements at the handheld controller result in the initiation of beam steering at the handheld controller, feedback received from the AR headset after beam steering is initiated can be utilized as an element of a control system for the beam steering process.

[0076] In some embodiments, in addition to the beam angle and the distance between the handheld controller and the AR headset calculated using beam steering as described above, an IMU incorporated into the handheld controller can also be used to provide information about the handheld controller's pose, which can be combined with the beam angle and the distance between the handheld controller and the AR headset calculated above. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0077] return Figure 6 , using the position of the AR headset, specifically the position of the electromagnetic field sensor in the xyz coordinate system, the angle Θ between the handheld controller and the AR headset, and the distance from the handheld controller to the AR headset, the six DoF pose of the handheld controller can be determined (step 640). The position of the AR headset can be determined from the head pose of the AR headset, which is typically tracked using an IMU in the AR headset. Based on the known position of the AR headset, the position of the handheld controller can be determined given the beam angle and the distance between the handheld controller and the AR headset. In addition, the beam is relative to the plane of the phased array elements (i.e., Figure 5A The angles of the handheld controller (shown in the yz plane) can be used to determine the orientation of the handheld controller, and thus determine the six-DoF pose (i.e., position and orientation). In some embodiments, data from the handheld controller IMU can be used to remove uncertainty in the handheld controller's orientation where rotation of the AR headset and totem about the z-axis may render it impossible to distinguish a unique handheld controller pose given a given distance and beam angle. In some embodiments, given a head pose (i.e., the pose of the AR headset), distance and beam angle are sufficient to determine the handheld controller pose.

[0078] Given the handheld controller position, active tracking of the handheld controller position and orientation (i.e., pose) can be performed (step 650). Therefore, once the six DoF pose of the handheld controller is determined, active tracking of the six DoF pose of the handheld controller can be performed to update the six DoF pose as a function of time. As an example, using the communication path between the AR headset and the handheld controller, movement of the handheld controller and / or the AR headset will result in a decrease in received power because the vector aligned with the main lobe and the vector between the handheld controller and the AR headset are not aligned. Based on this measured decrease in received power transmitted to the handheld controller via the communication path between the AR headset and the handheld controller, the phase delay can be adjusted to steer the beam and realign the vector aligned with the main lobe and the vector between the handheld controller and the AR headset.

[0079] Active tracking can be performed iteratively by determining the beam angle and distance and calculating the six-DoF pose of the handheld controller. As discussed above, the step size of the beam deviation can be adjusted during this active tracking process. Therefore, as the AR headset or handheld controller moves, active tracking can maintain the beam angle in real time based on feedback so that the beam angle is aligned with the direction between the AR headset and the handheld controller.

[0080] Figure 8 Schematically shows an augmented reality system communicating with a 5G cellular mobile communication system. Figure 8 As shown in FIG, a 5G transmitter 810, also known as a small cell (e.g., a femtocell, picocell, or microcell), communicates with an AR system 805 including an AR headset 820, an electromagnetic handheld controller 822, and an auxiliary unit 824. Therefore, in some embodiments of the present disclosure, a communication device, such as a 5G device, is included in the AR system. Figure 3 and Figure 8 , Figure 8 The AR headset 820, electromagnetic handheld controller 822, and auxiliary unit 824 shown in FIG. 8 may correspond to Figure 3 FIG3 shows an AR headset 301, a phased array transmitter 302, and a belt pack 370.

[0081] like Figure 8 , 5G transmitter 810 can utilize beamforming to steer a communication path from the 5G transmitter to AR system 805, e.g., along beam path 812 for communicating with AR headset 820 or along beam path 814 for communicating with auxiliary unit 824. In some implementations, the spatial proximity of various elements of AR system 805 are sufficiently close to one another that the beamforming performed by 5G transmitter 810 can be characterized as a single beam path connecting the 5G transmitter and the AR system, without specificity associated with a particular element of the AR system.

[0082] By implementing beamforming in the communication path between the 5G transmitter 810 and the AR system 805, increased data rates, lower latency, and other benefits can be provided. In addition, as shown by beam path 830, beamforming can be performed at the AR system 805 to communicate with the 5G transmitter 810. Although beamforming at the handheld controller is shown by beam path 830, embodiments of the present disclosure can also utilize beamforming at the auxiliary unit or AR headset to facilitate communication with the 5G transmitter. Therefore, in addition to performing beamforming at the 5G transmitter to facilitate communication with the AR system, beamforming can also be performed at one or more elements of the AR system to facilitate communication with the 5G transmitter. In addition, beamforming can be performed at one or more elements of the AR system, including the AR headset, the handheld controller, and / or the auxiliary unit. Those of ordinary skill in the art will recognize many variations, modifications, and alternatives.

[0083] In some embodiments, the location of the 5G transmitter or other suitable communication hub is known in a predetermined coordinate space (e.g., an XYZ Cartesian coordinate space). Given this known location of the 5G transmitter, the AR system can utilize the communication path between the 5G transmitter and the AR system, for example, using beam steering, to determine the location of the AR system (or an element of the AR system) relative to the known location of the 5G transmitter. As an example, when communication is established between the 5G transmitter and an auxiliary unit of the AR system, the location of the auxiliary unit relative to the known location of the 5G transmitter can be determined. Thus, for example, during the initialization process of the AR system, the location of the AR system can be determined. Although in Figure 8 5G is shown in FIG, but embodiments of the present disclosure are not limited to this particular communication standard and other communication hubs with known locations may be utilized within the scope of the present disclosure. Instead, once the 5G transmitter and the AR system establish their relative positions, the AR system can use information it has independently collected about its changing position or attitude to more accurately perform beam steering along beam path 830, and can communicate such information about its changing position or attitude to the 5G transmitter so that the 5G transmitter can more accurately perform beam steering along beam paths 812 and 814.

[0084] It should also be understood that the examples and embodiments described herein are for illustrative purposes only, and various modifications or changes made thereto by those skilled in the art will be included within the spirit and purview of this application and the scope of the appended claims.

Claims

1. An electromagnetic tracking system comprising: A handheld controller comprising an electromagnetic transmitter, wherein the electromagnetic transmitter comprises: a first phased array element characterized by a first phase; and a second phased array element characterized by a second phase different from the first phase; wherein the first phased array element and the second phased array element are configured to generate a steerable electromagnetic beam characterized by an electromagnetic field pattern; A head-mounted augmented reality display comprising an electromagnetic sensor configured to sense the electromagnetic field pattern, and One or more processors configured to: determining a posture of the electromagnetic sensor; controlling the first phase and the second phase to steer the electromagnetic beam to increase received power at the electromagnetic sensor, wherein the electromagnetic beam is steered based on beam steering information sent from the head-mounted augmented reality display to the handheld controller; and The pose of the handheld controller is determined using the pose of the electromagnetic sensor and the electromagnetic field pattern transmitted at the electromagnetic sensor.

2. The electromagnetic tracking system according to claim 1, wherein: The one or more processors are further configured to: The position and orientation of the handheld controller are digitally calculated based on the electromagnetic field pattern.

3. The electromagnetic tracking system according to claim 1, wherein: At least one of the one or more processors is disposed in the handheld controller.

4. The electromagnetic tracking system of claim 1, further comprising an auxiliary unit comprising the handheld controller.

5. The electromagnetic tracking system according to claim 4, wherein: The auxiliary unit includes a waist pack.

6. The electromagnetic tracking system according to claim 1, wherein: The electromagnetic transmitter further includes a third phased array element, wherein the first phased array element, the second phased array element, and the third phased array element are arranged in a plane.

7. The electromagnetic tracking system according to claim 6, wherein: The electromagnetic transmitter further includes a fourth phased array element, wherein the first phased array element, the second phased array element, and the third phased array element form three corners of a tetrahedron, and the fourth phased array element forms a fourth corner of the tetrahedron.

8. A method for determining a six-degree-of-freedom (DoF) pose of a handheld controller, the method comprising: Determine the attitude of the electromagnetic sensor; generating an electromagnetic beam from the handheld controller, wherein the electromagnetic beam is characterized by an electromagnetic field pattern; steering the electromagnetic beam to increase received power at the electromagnetic sensor, wherein the electromagnetic beam is steered based on beam steering information sent from a head-mounted augmented reality display to the handheld controller; determining a beam angle associated with the electromagnetic beam; determining a distance between the handheld controller and the electromagnetic sensor; and The six degrees of freedom (DoF) pose of the handheld controller is determined using the pose of the electromagnetic sensor, the beam angle, and the distance.

9. The method according to claim 8, wherein The handheld controller includes an electromagnetic transmitter, and the electromagnetic transmitter includes: a first phased array element configured to operate in a first phase; and A second phased array element is configured to operate with a second phase.

10. The method according to claim 9, wherein: The electromagnetic transmitter further includes a third phased array element, wherein the first phased array element, the second phased array element, and the third phased array element are arranged in a plane.

11. The method according to claim 8, wherein The pose of the electromagnetic sensor includes a head pose, and the electromagnetic sensor is an element of the head-mounted augmented reality display.

12. The method according to claim 8, wherein Steering the electromagnetic beam includes aligning a center vector of the electromagnetic beam with a vector from the handheld controller to the electromagnetic sensor.

13. The method according to claim 8, wherein Steering the electromagnetic beam to increase received power at the electromagnetic sensor includes maximizing received power at the electromagnetic sensor.

14. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: Determine the attitude of the electromagnetic sensor; Generates an electromagnetic beam from a handheld controller, where The electromagnetic beam is characterized by an electromagnetic field pattern; steering the electromagnetic beam to increase received power at the electromagnetic sensor, wherein the electromagnetic beam is steered based on beam steering information sent from a head-mounted augmented reality display to the handheld controller; determining a beam angle associated with the electromagnetic beam; determining a distance between the handheld controller and the electromagnetic sensor; and A six degree-of-freedom (DoF) pose of the handheld controller is determined using the pose of the electromagnetic sensor, the beam angle, and the distance.

15. The non-transitory computer readable medium of claim 14, wherein: The handheld controller includes an electromagnetic transmitter, and the electromagnetic transmitter includes: a first phased array element configured to operate in a first phase; and A second phased array element is configured to operate with a second phase.

16. The non-transitory computer readable medium of claim 15, wherein: The electromagnetic transmitter further includes a third phased array element, wherein the first phased array element, the second phased array element, and the third phased array element are arranged in a plane.

17. The non-transitory computer readable medium of claim 14, wherein: The pose of the electromagnetic sensor includes a head pose, and the electromagnetic sensor is an element of the head-mounted augmented reality display.

18. The non-transitory computer readable medium of claim 14, wherein: Steering the electromagnetic beam includes aligning a center vector of the electromagnetic beam with a vector from the handheld controller to the electromagnetic sensor.

19. The non-transitory computer readable medium of claim 14, wherein: Steering the electromagnetic beam to increase received power at the electromagnetic sensor includes maximizing received power at the electromagnetic sensor.

Images