Directed emitter / sensor for electromagnetic tracking in augmented reality systems

Abstract

To provide a directed emitter / sensor for electromagnetic tracking in suitable augmented reality systems.SOLUTION: An electromagnetic tracking system includes a hand held controller including an electromagnetic emitter configured to generate an electromagnetic field characterized by an electromagnetic field pattern and a first electromagnetic reflector positioned adjacent the electromagnetic emitter and configured to form a modified electromagnetic field pattern. The electromagnetic tracking system also includes a head mounted augmented reality display including an electromagnetic sensor configured to sense the electromagnetic field and a second electromagnetic reflector adjacent to sensor configured to optimally sense electromagnetic field pattern in a region of interest.SELECTED DRAWING: Figure 6

Description

[Background technology]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 727,489, filed September 5, 2018, and entitled "DIRECTED EMITTER / SENSOR FOR ELECTROMAGNETIC TRACKING IN AUGMENTED REALITY SYSTEMS," the contents of which are incorporated herein by reference in their entirety for all purposes.

[0002] BACKGROUND OF THEINVENTION Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality (VR)" or "augmented reality (AR)" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or can be perceived as real. Virtual reality, i.e., VR scenarios, typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality, i.e., AR scenarios, typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user.

[0003] Despite the advances made in these display technologies, there remains a need in the art for improved methods, systems, and devices relating to augmented reality systems, and particularly display systems. Summary of the Invention [Means for solving the problem]

[0004] SUMMARY OF THEINVENTION The present disclosure relates to virtual reality and / or augmented reality imaging and visualization systems. The present disclosure generally relates to methods and systems related to electromagnetic tracking in virtual reality and / or augmented reality systems. More specifically, embodiments of the present disclosure provide methods and systems for directing energy transmitted by an emitter (also referred to as a transmitter) and / or received by a sensor (also referred to as a receiver) to improve the performance of the localization process. In some embodiments, a shaped electromagnetic (EM) reflector is utilized to modify the emission pattern generated using the EM emitter / received pattern received by the EM sensor. In some embodiments, a non-shaped EM pattern may be subject to distortion, which may affect its ability to accurately determine position and orientation. In some embodiments, a shaped EM field may minimize distortion and increase field strength. As a result, the field strength near the electromagnetic sensor is increased. Similarly, the receiving capability of the electromagnetic sensor in the direction of the electromagnetic emitter is also increased. These modifications may result in improved localization information, improved efficiency in power consumption, a reduction in EM distortion, and a reduction in the size of the electromagnetic emitter and / or electromagnetic sensor.The present disclosure is applicable to a variety of applications in computer vision and image display systems.

[0005] According to an embodiment of the present invention, an electromagnetic tracking system is provided. The electromagnetic tracking system includes a handheld controller including an electromagnetic emitter configured to generate an electromagnetic field characterized by an electromagnetic field pattern, and a first electromagnetic reflector positioned adjacent to the electromagnetic emitter and configured to form a modified electromagnetic field pattern. The electromagnetic tracking system also includes a head-mounted augmented reality display including an electromagnetic sensor configured to sense the electromagnetic field, and a second electromagnetic reflector adjacent to the electromagnetic sensor configured to optimally sense the electromagnetic field pattern within a region of interest.

[0006] According to a specific embodiment of the present invention, a method of operating an electromagnetic tracking system is provided. The method includes generating an electromagnetic field using an electromagnetic emitter and reflecting the electromagnetic field using a first electromagnetic reflector to form a modified electromagnetic field pattern. The method also includes reflecting a portion of the modified electromagnetic field pattern using a second electromagnetic reflector and sensing the reflected portion of the modified electromagnetic field pattern using an electromagnetic sensor adjacent to the second electromagnetic reflector.

[0007] Numerous advantages are achieved by the methods of the present disclosure over conventional techniques. For example, embodiments of the present disclosure provide methods and systems that increase electromagnetic field strength in a predetermined manner. Thus, the systems can achieve desired functionality while reducing transmission power, reducing component size, reducing or avoiding EM distortion, and the like. These and other embodiments of the present disclosure, along with many of its advantages and features, are described in further detail in conjunction with the following text and accompanying figures. The present invention provides, for example, the following: (Item 1) 1. An electromagnetic tracking system comprising: A handheld controller, an electromagnetic emitter configured to generate an electromagnetic field characterized by an electromagnetic field pattern; a first electromagnetic reflector positioned adjacent to the electromagnetic emitter and configured to form a modified electromagnetic field pattern; a handheld controller including: 1. A head-mounted augmented reality display, comprising: an electromagnetic sensor configured to sense the modified electromagnetic field pattern; a second electromagnetic reflector adjacent to the electromagnetic sensor configured to sense the modified electromagnetic field pattern within a region of interest; Head-mounted augmented reality displays, including An electromagnetic tracking system comprising: (Item 2) Controlling the timing of electromagnetic emission and sensing; digitally calculating positions and orientations of the electromagnetic emitters and the electromagnetic sensors based on the modified electromagnetic field pattern; Item 1 , wherein the electromagnetic tracking system further comprises a controller operable to: (Item 3) 3. The electromagnetic tracking system of item 2, further comprising an auxiliary unit including the controller. (Item 4) 4. The electromagnetic tracking system of claim 3, wherein the auxiliary unit comprises a belt pack. (Item 5) 4. The electromagnetic tracking system of item 3, wherein the controller is distributed among the handheld controller, the head-mounted augmented reality display, and the auxiliary unit. (Item 6) 2. The electromagnetic tracking system of claim 1, wherein the electromagnetic field pattern is characterized by an initial full-width-half-maximum width and the modified electromagnetic field pattern is characterized by a modified full-width-half-maximum width that is less than the initial full-width-half-maximum width. (Item 7) 7. The electromagnetic tracking system of claim 6, wherein the modified electromagnetic field pattern is characterized by a field strength at a lobe center that is higher than the electromagnetic field pattern. (Item 8) 9. The electromagnetic tracking system of claim 1, wherein the electromagnetic emitter and the electromagnetic sensor include parallel plates aligned with an x-axis, the electromagnetic field extends along a negative z-axis and a positive z-axis, and the modified electromagnetic field pattern extends only along the positive z-axis. 2. The electromagnetic tracking system of claim 1, wherein at least one of the first electromagnetic reflector or the second electromagnetic reflector comprises two or more reflector plates. (Item 10) 10. The electromagnetic tracking system of claim 9, wherein the two or more reflector plates are joined at vertices. (Item 11) 10. The electromagnetic tracking system of claim 9, wherein the two or more reflector plates comprise three reflector plates defining corner vertices of a cube. (Item 12) 2. The electromagnetic tracking system of claim 1, wherein at least one of the first electromagnetic reflector or the second electromagnetic reflector comprises a single reflector element. (Item 13) 1. A method of operating an electromagnetic tracking system, the method comprising: generating an electromagnetic field using an electromagnetic emitter; reflecting the electromagnetic field using a first electromagnetic reflector to form a modified electromagnetic field pattern; reflecting a portion of the modified electromagnetic field pattern using a second electromagnetic reflector; sensing the reflected portion of the modified electromagnetic field pattern using an electromagnetic sensor adjacent to the second electromagnetic reflector; A method comprising: (Item 14) Item 14. The method of item 13, wherein the electromagnetic emitter is disposed in a handheld controller and the electromagnetic sensor is disposed in a head-mounted augmented reality display. (Item 15) Item 14. The method of item 13, wherein the first electromagnetic reflector is positioned adjacent to the electromagnetic emitter. (Item 16) controlling the timing of generating the electromagnetic field and sensing the reflected portion of the modified electromagnetic field pattern; digitally calculating positions and orientations of the electromagnetic emitters and the electromagnetic sensors based on the modified electromagnetic field pattern; Item 14. The method of item 13, further comprising: (Item 17) Item 14. The method of item 13, wherein at least one of the first electromagnetic reflector or the second electromagnetic reflector comprises two or more reflector plates. (Item 18) Item 18. The method of item 17, wherein the two or more reflector plates are joined at vertices. (Item 19) Item 18. The method of item 17, wherein the two or more reflector plates comprise three reflector plates defining corner vertices of a cube. (Item 20) Item 14. The method of item 13, wherein at least one of the first electromagnetic reflector or the second electromagnetic reflector comprises a single reflector element. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 diagrammatically illustrates a system diagram of an electromagnetic (EM) tracking system, according to some embodiments.

[0009] [Diagram 2] FIG. 2 is a flow chart illustrating the functionality of the electromagnetic tracking system, according to some embodiments.

[0010] [Diagram 3] FIG. 3 diagrammatically illustrates an electromagnetic tracking system integrated with an augmented reality (AR) system, according to some embodiments.

[0011] [Figure 4] FIG. 4 is a flowchart illustrating functionality of an electromagnetic tracking system in the context of an AR device, according to some embodiments.

[0012] [Diagram 5] FIG. 5 is a plan view of an electromagnetic emitter and corresponding electromagnetic field lines according to some embodiments.

[0013] [Figure 6] FIG. 6 is a plan view of an electromagnetic emitter incorporating a two-sided reflector and corresponding electromagnetic field lines according to some embodiments.

[0014] [Figure 7]FIG. 7 is a plan view of an electromagnetic emitter and corresponding electromagnetic field lines incorporating a segmented reflector according to some embodiments.

[0015] [Figure 8] FIG. 8 is a plan view of an electromagnetic emitter and corresponding electromagnetic field lines incorporating a three-sided reflector according to some embodiments.

[0016] [Figure 9] FIG. 9 is a plan view of an electromagnetic emitter and corresponding electromagnetic field lines incorporating a hemispherical reflector according to some embodiments.

[0017] [Figure 10A] FIG. 10A is a perspective view illustrating the integration of an electromagnetic emitter incorporating a three-sided reflector with a handheld controller, according to some embodiments.

[0018] [Figure 10B] FIG. 10B is a perspective view illustrating the integration of an electromagnetic sensor with a headset incorporating a three-sided reflector, according to some embodiments.

[0019] [Figure 10C] FIG. 10C is a perspective view illustrating an expanded view of the sensor housing shown in FIG. 10B.

[0020] [Figure 11] FIG. 11 is a simplified flow chart illustrating a method of operating an electromagnetic tracking system incorporating an integrated reflector, in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS In an augmented reality (AR) system, the AR system can be designed to be interactive with the user. As an example, the user may be provided with a handheld controller, also called a totem, that the user can use to interact with the AR system. It is therefore useful to be able to determine the position and orientation (e.g., six degrees of freedom (DOF) pose) of the totem relative to other elements of the AR system, including a head-mounted display system, also called a headset or AR headset, worn by the user.

[0022] One approach to achieving high-precision localization may involve the use of electromagnetic (EM) fields emitted by electromagnetic field emitters coupled with electromagnetic field sensors strategically placed on the user's AR headset, beltpack, and / or other auxiliary devices (e.g., totems, tactile devices, gaming equipment, etc.). An electromagnetic tracking system typically includes at least one electromagnetic field emitter (generally referred to as an "emitter") and at least one electromagnetic field sensor (generally referred to as a "sensor"). The emitter generates an electromagnetic field having a known spatial (and / or temporal) distribution in the environment of the user of the AR headset. The sensor measures the generated electromagnetic field at the sensor's location. Based on these measurements and knowledge of the distribution of the generated electromagnetic field, the pose (e.g., position and / or orientation) of the sensor relative to the emitter can be determined. Thus, the pose of the object to which the sensor and / or emitter are attached can be determined. That is, the relative positions of the sensor and the emitter can be determined.

[0023] 1, an example system diagram of an electromagnetic tracking system is illustrated. In some embodiments, the electromagnetic tracking system includes one or more electromagnetic field emitters 102 (generally referred to as "emitters 102") configured to emit a known electromagnetic field. As shown in FIG. 1, the emitters 102 may be coupled to a power supply 110 (e.g., electrical current, a battery, etc.) to provide power to the emitters 102.

[0024] In some embodiments, the emitter 102 includes several coils (e.g., at least three coils positioned perpendicular to each other and generating fields in the X, Y, and Z directions) that generate an electromagnetic field. This electromagnetic field is used to establish a coordinate space (e.g., an XYZ Cartesian coordinate space). This allows the system to map the positions (e.g., (X, Y, Z) positions) of the electromagnetic sensors 104a, 104b in relation to the known electromagnetic field and to determine the position and / or orientation of the electromagnetic sensors 104a, 104b. In some embodiments, the electromagnetic sensors 104a, 104b (generally referred to as "sensors" 104) may be attached to one or more real objects. The sensors 104 may include coils in which a current may be induced through an electromagnetic field, e.g., an electromagnetic field emitted by the emitter 102. The sensor 104 may include coils or loops (e.g., at least three coils positioned perpendicular to each other) coupled together within a small structure such as a cube or other container that are positioned / oriented to capture incident electromagnetic flux from an electromagnetic field, e.g., the electromagnetic field emitted by the emitter 102; by comparing the currents induced through these coils and knowing the relative positions and orientations of the coils with respect to each other, the relative position and orientation of the sensor 104 with respect to the emitter 102 can be calculated.

[0025] One or more parameters relating to the behavior of the coil and inertial measurement unit ("IMU") components operatively coupled to the sensor 104 may be measured to detect the position and / or orientation of the sensor 104 (and the object to which it is attached) relative to a coordinate system to which the emitter 102 is coupled. In some embodiments, multiple sensors 104 may be used in conjunction with the emitter 102 to detect the position and orientation of each of the sensors 104 in the coordinate space. An electromagnetic tracking system may provide position in three directions (e.g., X, Y, and Z directions) and also two or three orientation angles. In some embodiments, the IMU measurements may be compared to the coil measurements to determine the position and orientation of the sensor 104. In some embodiments, both the electromagnetic (EM) data and the IMU data may be combined with various other data sources, such as cameras, depth sensors, and other sensors, to determine the position and orientation. This information may be transmitted to the controller 106 (e.g., wireless communication, Bluetooth, etc.). In some embodiments, the attitude (or position and orientation) may be reported at a relatively high refresh rate in conventional systems. Traditionally, an electromagnetic field emitter is coupled to a relatively stable, large object, such as a table, operating table, wall, or ceiling, and one or more sensors are coupled to a smaller object, such as a medical device, a component for a handheld game, or the like. Alternatively, as described below with reference to Figure 3, various features of an electromagnetic tracking system may be employed to create a configuration in which changes or deltas in position and / or orientation between two objects moving in space relative to a more stable global coordinate system can be tracked.In other words, the configuration is as shown in FIG. 3, where fluctuations of the electromagnetic tracking system may be utilized to track position and orientation deltas (changes) between the head-mounted and handheld components, while the head pose relative to a global coordinate system (e.g., of the room environment local to the user) is determined otherwise, such as by simultaneous localization and mapping ("SLAM") techniques using an outward-facing capture camera that may be coupled to the head-mounted components of the system.

[0026] The controller 106 may control the emitters 102 and may also capture data from the sensors 104. It should be understood that the various components of the system may be coupled to each other through any electromechanical or wireless / Bluetooth means. The controller 106 may also include data regarding the known electromagnetic field and the coordinate space associated with the electromagnetic field. This information is then used to detect the position and orientation of the sensors 104 in relation to the coordinate space corresponding to the known electromagnetic field.

[0027] One advantage of electromagnetic tracking systems is that they produce high resolution, highly repeatable tracking results with minimal latency. In addition, electromagnetic tracking systems do not necessarily rely on optical trackers, and sensors / objects that are not within the user's line of sight can easily be tracked.

[0028] It should be appreciated that the strength of the electromagnetic field v drops off as a cubic function of the distance r from the coil emitter (e.g., emitter 102). Accordingly, an algorithm may be used based on the distance away from the emitter 102. The controller 106 may be configured to use such an algorithm to determine the position and orientation of the sensor 104 at variable distances away from the emitter 102. Given the steep decrease in the strength of the electromagnetic field as the sensor 104 moves farther away from the emitter 102, best results in terms of accuracy, efficiency, and low latency may be achieved at closer distances. In a typical electromagnetic tracking system, the emitter is powered by an electric current (e.g., a plug-in power supply) and the sensor is located within a 20-foot radius of the emitter. A smaller radius between the sensor and the emitter may be more desirable in many applications, including AR applications.

[0029] Referring now to FIG. 2, an exemplary flow chart illustrating the functionality of an electromagnetic tracking system according to some embodiments is briefly described. At 202, a known electromagnetic field is emitted. In some embodiments, an electromagnetic field emitter may generate an electromagnetic field. For example, each coil of the electromagnetic field emitter may generate an electromagnetic field in one direction (e.g., X, Y, or Z). The electromagnetic field may be generated with an arbitrary waveform. In some embodiments, the electromagnetic field components along each of the axes may oscillate at a slightly different frequency than other electromagnetic field components along other directions. At 204, a coordinate space corresponding to the electromagnetic field may be optionally determined. For example, the controller may automatically determine the coordinate space around the emitter and / or sensor based on the electromagnetic field. In some embodiments, the coordinate space may not be determined at this stage of the method. At 206, the behavior of the coil in the sensor (which may be attached to a known object) may be detected. For example, the current induced in the coil may be calculated. In some embodiments, the rotation or any other quantifiable behavior of the coil may be tracked and measured. At 208, this behavior may be used to detect the position or orientation of the sensor and / or a known object (e.g., an AR headset that includes the sensor) relative to the emitter or vice versa. For example, the controller 106 may consult a mapping table that correlates various positions or orientations with the behavior of the coils in the sensor. Based on these calculations, a position in coordinate space may be determined along with the orientation of the sensor and / or emitter.

[0030] In the context of an AR system, one or more components of the electromagnetic tracking system may need to be modified to facilitate accurate tracking of the mobile components (e.g., emitters and sensors). As described above, tracking the user's head pose and orientation may be desirable in many AR applications. Accurate determination of the user's head pose and orientation allows the AR system to display appropriate / relevant virtual content to the user. For example, a virtual scene may include a virtual monster hiding behind a real building. Depending on the user's head pose and orientation relative to the building, the view of the virtual monster may need to be modified so that a realistic AR experience is provided. Alternatively, the position and / or orientation of a totem, tactile device, or some other means of interacting with the virtual content may be important in allowing the user to interact with the AR system. For example, in many gaming applications, the AR system may detect the position and orientation of a real object in relation to the virtual content. Alternatively, when displaying a virtual interface, the position of a totem, a user's hand, a tactile device, or any other real object configured for interaction with the AR system can be understood in relation to the displayed virtual interface in order for the system to understand commands, interactions, and the like. Some localization methods, including optical tracking and other methods, can suffer from high latency and low resolution issues, which make rendering virtual content difficult in many AR applications.

[0031] In some embodiments, an electromagnetic tracking system, discussed in connection with Figures 1 and 2, may be adapted for an AR system to detect the position and orientation of one or more objects relative to the emitted electromagnetic field. Typical electromagnetic tracking systems tend to have large and bulky electromagnetic emitters (e.g., 102 in Figure 1), which is problematic for head-mounted AR devices, such as with totems. However, smaller electromagnetic emitters (e.g., in the millimeter range) may be used to emit a known electromagnetic field in the context of an AR system.

[0032] 3, an electromagnetic tracking system may be incorporated with an AR system, with an electromagnetic field emitter 302 (generally referred to as "emitter 302") incorporated as part of a handheld controller 306 (generally referred to as "controller 306") as shown. The controller 306 may be independently movable relative to the AR headset 301 (or belt pack 370). For example, the controller 306 may be held in the user's hand, or the controller 306 may be mounted on the user's hand or arm (e.g., as a ring or bracelet, or as part of a glove worn by the user). In some embodiments, the controller 306 may be a totem, for example, used in gaming scenarios (e.g., a multi-degree-of-freedom controller), or providing a rich user experience in the AR environment or allowing the user to interact with the AR system. In some embodiments, the controller 306 may be a tactile device. In some embodiments, the emitter 302 may be incorporated as part of the belt pack 370. The controller 306 may include a battery 310 or other power source that powers its emitter 302. It should be appreciated that the emitter 302 may also include or be coupled to an IMU 350 component configured to assist in determining the positioning and / or orientation of the emitter 302 relative to other components. This may be advantageous, particularly if both the emitter 302 and the electromagnetic field sensor 304 (generally referred to as the "sensor 304") are mobile. Locating the emitter 302 in the controller 306 rather than the belt pack 307 as shown in the embodiment of FIG. 3 helps ensure that the emitter 302 does not compete for resources in the belt pack 370, but rather uses its own battery source in the controller 306. In some embodiments, the emitter 302 may be located on the AR headset 301, and the sensor 304 may be located on the controller 306 or the belt pack 370.Accordingly, embodiments of the present invention provide implementations where the controller 306 is implemented as a handheld unit, in other embodiments the controller is implemented within the AR headset 301, and in additional embodiments the controller is implemented within an auxiliary unit, such as the beltpack 307. Furthermore, in addition to implementations where the controller 306 is implemented within a single device, the functionality of the controller and associated physical components can be distributed across multiple devices, such as the controller 306, the AR headset 301, and / or auxiliary units, such as the beltpack 307.

[0033] In some embodiments, the sensor 304 may be installed on one or more locations on the AR headset 301 along with other sensing devices such as one or more IMUs or additional electromagnetic flux capture coils 308. For example, as shown in FIG. 3, the sensors 304, 308 may be installed on one or both sides of the AR headset 301. Having multiple sensors 304, 308 may improve efficiency and accuracy, since the sensors 304, 308 can be engineered to be fairly small (and in some cases may not be very sensitive). In some embodiments, one or more sensors may also be installed on the belt pack 370 or any other part of the user's body. The sensors 304, 308 may communicate wirelessly, for example, through Bluetooth, to a computing device that determines the attitude and orientation of the sensors 304, 308 (and the AR headset 301 to which it is attached). In some embodiments, the computing device may reside on the belt pack 370. In some embodiments, the computing device may reside on the AR headset 301 or the controller 306. In some embodiments, the computing device may then include a mapping database 330 (e.g., a mapping database, cloud resources, a passable world model, a coordinate space, and the like), detect poses, determine coordinates of real and / or virtual objects, and even connect to cloud resources and passable world models. The controller 306, in an embodiment, may control the timing of electromagnetic emission by the electromagnetic emitters and sensing by the electromagnetic sensors such that the positions and orientations of the electromagnetic emitters and sensors are calculated based on fields from the modified electromagnetic field pattern. In some embodiments, the positions and orientations of the electromagnetic emitters are calculated relative to the electromagnetic sensors. In other embodiments, the positions and orientations of the electromagnetic sensors are calculated relative to the electromagnetic emitters. In some embodiments, the positions and orientations of the electromagnetic emitters and sensors are calculated.

[0034] As explained above, some electromagnetic emitters may be too bulky for an AR device. Thus, the emitter may be engineered to be compact, using smaller components (e.g., coils) than conventional systems. However, given that the strength of the electromagnetic field decreases as a cubic function of distance away from the emitter, a smaller radius between the sensor 304 and the emitter 302 (e.g., about 3 to 3.5 feet) may reduce power consumption compared to conventional systems such as those detailed in FIG. 1.

[0035] In some embodiments, this aspect may be utilized to extend the life of the battery 310, which in one or more embodiments may power the controller 306 and the emitter 302. In some embodiments, this aspect may be utilized to reduce the size of the coil that generates the electromagnetic field in the emitter 302. However, to obtain the same strength of the electromagnetic field, the power may need to be increased. This allows for a compact emitter 302 that may fit compactly into the controller 306.

[0036] Some other modifications may be made when using an electromagnetic tracking system for an AR device. While the present attitude reporting rate is quite good, AR systems may require an even more efficient attitude reporting rate. To this end, IMU-based attitude tracking may be used (in addition or as an alternative). Advantageously, the IMU may remain as stable as possible to increase the efficiency of the attitude detection process. The IMU may be engineered such that they remain stable for a maximum of 50-100 milliseconds. It should be appreciated that some embodiments may utilize an outer attitude estimator module, which may allow attitude updates to be reported at a rate of 10-20 Hz (e.g., the IMU may drift over time). By keeping the IMU stable at a reasonable rate, the rate of attitude updates may be dramatically reduced to 10-20 Hz (compared to the higher frequencies in conventional systems).

[0037] If the electromagnetic tracking system can be woken up, for example, on a 10% duty cycle (e.g., only pinging for ground truth every 100 ms), the AR system can conserve power. This can mean that the electromagnetic tracking system wakes up and generates pose estimation 10 ms out of 100 ms. This can translate directly into power consumption savings, which in turn can impact the size, battery life, and cost of the AR device (e.g., AR headset 301 and / or controller 306).

[0038] In some embodiments, this reduction in duty cycle may be strategically utilized by providing two controllers 306 (not shown) rather than only one controller 306 as illustrated in FIG. 3. For example, a user may be playing a game that requires two controllers 306 and the like. Or, in a multi-user game, two users may have their own controllers 306 to play the game. When two controllers 306 (e.g., symmetrical controllers per hand) are used rather than one, the controllers 306 may operate with offset duty cycles. The same concept may also be applied to controllers 306 utilized by two different users playing a multiplayer game.

[0039] 4, an exemplary flow chart illustrating an electromagnetic tracking system in the context of an AR device is described. At 402, a portable (e.g., handheld) controller (e.g., controller 306) emits an electromagnetic field. For example, emitter 302 emits an electromagnetic field. At 404, an electromagnetic sensor (located on a headset, beltpack, etc.) detects the electromagnetic field. For example, sensors 304, 308 detect the electromagnetic field. At 406, a posture (e.g., position or orientation) of the AR headset / beltpack is determined based on the behavior of the coil / IMU in the sensor. For example, AR headset 301 / beltpack 370 determines the posture of the AR headset 301 / beltpack 370 based on the behavior of sensor 304 and / or IMU and coil 308. At 408, the posture information is communicated to a computing device. For example, the posture information is communicated to a computing device in AR headset 301 and / or beltpack 370. At 410, optionally, a mapping database may be consulted to correlate real world coordinates (e.g., determined with respect to the headset / belt pose) and virtual world coordinates. For example, the mapping database 330 may be consulted to correlate real world coordinates and virtual world coordinates. At 412, the virtual content may be delivered to the AR headset and displayed to the user (e.g., via a light field display as described herein). For example, the virtual content may be delivered to the AR headset 301 and displayed to the user. It should be understood that the flow chart described above is for illustrative purposes only and should not be read as limiting.

[0040] Advantageously, using an electromagnetic tracking system similar to that outlined in FIG. 3 enables pose tracking (e.g., head position and orientation, totem and other controller position and orientation). This allows the AR system to project virtual content (based at least in part on the determined pose) with greater accuracy and very low latency compared to optical tracking techniques. Furthermore, this allows the AR system to track user input devices (e.g., controller 306) with high accuracy, low power consumption (e.g., by battery 210), low latency, and the like.

[0041] Figure 5 is a plan view of an electromagnetic emitter and corresponding electromagnetic field lines, according to some embodiments. As illustrated in Figure 5, the electromagnetic field lines 520 emitted by an electromagnetic emitter 510 (generally referred to as "emitter 510") form a closed loop that passes through an interior region of the emitter 510 along an x-direction that is generally parallel to the line connecting the poles created by the emitter 510. As illustrated in Figure 5, the electromagnetic field generated by the emitter 510 extends equally in both the positive and negative z-directions.

[0042] As described above with respect to Figure 3, during use, an electromagnetic field established by an emitter 510 (e.g., emitter 302 in Figure 3) will be detected at a sensor 304 (e.g., sensor 304 in Figure 3) to provide the desired localization information. Because the transmitted electromagnetic field extends away from the emitter 510 in both the positive and negative z-directions, energy directed in a direction opposite to the direction from the emitter 510 to the sensor will not be utilized and will not impair system efficiency.

[0043] In some embodiments, the schematics of Figures 6-9 may be based on finite element analysis of time-varying electromagnetic fields. In some embodiments, the addition of reflectors of various shapes and configurations at or near the coils of the emitter and / or sensor may increase the intensity of electromagnetic radiation from the emitter and / or at the sensor within a region of interest (ROI). The ROI may be a location where motion in an AR or VR system is primarily active. Although the description refers to the emitter, a similar reflector may be applied to the sensor to increase sensor reception within the ROI. Figure 6 is a plan view of an electromagnetic emitter incorporating a two-sided reflector and corresponding electromagnetic field lines, according to some embodiments. In Figure 6, only electromagnetic field lines in the xz plane are illustrated for purposes of clarity, but it will be understood by those skilled in the art that three-dimensional lobe patterns will exist. As discussed more fully below in connection with Figure 8, the design illustrated in Figure 6 can be extended into three dimensions.

[0044] Referring to FIG. 6, two reflective elements, a first reflective element 620 and a second reflective element 622, are positioned adjacent to the emitter 510, thereby providing an integrated electromagnetic reflector, also referred to as an integrated reflector. The emitter 510 is oriented such that the electromagnetic field passing through the coil of the emitter is aligned with the x-axis. The first reflective element 620 is oriented at a predetermined angle to the x-axis, for example, an angle of 135°, and the second reflective element 622 is oriented at a predetermined angle to the x-axis, for example, an angle of 45°. In other words, the first reflective element 620 is aligned with a diagonal having a slope of −1 measured in the xz plane, and the second reflective element 622 is aligned with a diagonal having a slope of +1 measured in the xz plane. As illustrated in FIG. 6, the first reflective element 620 and the second reflective element 622 are joined at an apex 630 , which is located at the midpoint of the emitter 510 .

[0045] The first reflective element 620 and the second reflective element 622 are fabricated using materials that are highly conductive at the frequencies at which the emitter 510 operates (e.g., 27 kHz to 40 kHz, e.g., 35 kHz). In some embodiments, highly conductive metal plates, e.g., 2 mm thick copper plates, can be utilized to form the reflective elements 620, 622. In some embodiments, a substrate coated with a highly conductive material may be employed to utilize the mechanical properties of the substrate (e.g., plastic) in conjunction with the electrical properties of the conductive material coated on the substrate. As would be apparent to one skilled in the art, the materials utilized to fabricate the first reflective element 620 and the second reflective element 622 are applicable to the other reflective elements described herein, if desired. One skilled in the art would recognize many variations, modifications, and alternatives.

[0046] Because the first reflective element 620 and the second reflective element 622 reflect the electromagnetic field that would be established in the negative z-direction, the electromagnetic field lines 610 form a single lobe oriented along the positive z-direction. As a result of the electromagnetic energy being in a single lobe, an electromagnetic field sensor (e.g., sensor 304) positioned along the positive z-axis relative to the emitter 510 will detect a stronger electromagnetic field at a given distance from the emitter 510, thereby improving system performance.

[0047] Furthermore, the electromagnetic field line 610 is characterized by a higher intensity field for a given lobe width. With reference to FIG. 6, the presence of the reflective elements 620 and 622 results in a compression of the electromagnetic field line 610, thereby generating a higher intensity field. The width of the electromagnetic field line 610, characterized by half the maximum field intensity, is defined as the full width at half maximum of the electromagnetic field. As illustrated in FIG. 6, for the electromagnetic field line 610 having an intensity of half the maximum field intensity, the width is equal to W. Thus, the full width at half maximum of the lobe pattern illustrated in FIG. 6, which may be referred to as a modified electromagnetic field pattern, is W. This can be compared to the electromagnetic field generated by the emitter in the absence of the first reflective element 620 and the second reflective element 622. In the absence of reflective elements 620 and 622, the electromagnetic field lines 610 extend over a large area and are characterized by a full width at half maximum greater than W, which corresponds to a weaker intensity field within the ROI 640 as well as undesired EM energy within the non-ROI area. Thus, while a conventional electromagnetic field pattern generated by a conventional emitter would be characterized by an initial full width at half maximum width, in some embodiments, utilizing reflective structures such as reflective elements 620 and 622 would generate a modified electromagnetic field pattern that would be characterized by a modified full width at half maximum width that is less than the initial full width at half maximum width. In other words, the electromagnetic field lines within the ROI 640 have a higher intensity field. Referring to FIG. 6, the presence of reflective elements 620 and 622 results in the reflection of the electromagnetic field lines in addition to the original ones when reflective elements 620 and 622 are not present. This effectively creates a modified field pattern with a much higher intensity field than the original field in the ROI 640 without the reflective elements 620 and 622. This can be compared to the electromagnetic field generated by the emitter in the absence of the reflective elements 620, 622 as illustrated in FIG. 5. In the absence of the reflective elements 620, 622, the electromagnetic field lines 610 would extend over a large area and the field intensity in the ROI 640 would be significantly reduced. Thus, a conventional electromagnetic field pattern generated by a conventional emitter would distribute its energy field not only within the ROI 640, but also within non-ROI areas, as shown in FIG. 5.Utilizing reflective elements such as reflective elements 620 and 622 will generate a modified electromagnetic field pattern that will be concentrated within ROI 640. With the modified pattern, distortion caused by metal in areas where electromagnetic radiation is minimal will be minimal. Thus, reflective elements 620 and 622 shield the emitter coil and the sensor coil from distortion caused by metal on the other side of reflective elements 620 and 622. For such configurations as shown in FIG. 6, as well as those on FIGS. 7-9, controller 106 may be able to calculate position and orientation based on the modified field pattern.

[0048] 7 is a plan view of an electromagnetic emitter incorporating a segmented reflector and corresponding electromagnetic field lines, according to some embodiments. The segmented reflector 720 is placed adjacent to one side of the emitter 510. The segmented reflector 720 includes a first distal reflective element 722, a central reflective element 724, and a second distal reflective element 726. The elements of the segmented reflector 720 reflect an electromagnetic field that would be established in the negative z-direction, so that the electromagnetic field lines 710 form a single lobe oriented along the positive z-direction. The dimensions, e.g., the lengths of the reflective elements 720, 724, 726, as well as the angle between the first distal reflective element 722 and the central reflective element 724, and the angle between the second distal reflective element 726 and the central reflective element 724, can be selected to control the distribution of the electromagnetic field lines 710.

[0049] In some embodiments, the length of the central reflective element 724 is equal to the length of the emitter 510 in the x-direction, and the angles between the distal reflective elements 722, 726 and the central reflective element 724 are both 45°. The length of the distal reflective elements 722, 726 can be selected as a function of the length of the central reflective element 724. As would be apparent to one skilled in the art, increasing the length of the distal reflective elements 722, 726 may result in less of the electromagnetic field being present in the area behind the distal reflective elements 722, 726 that faces the centerline of the electromagnetic field pattern. However, increasing the length of the distal reflective elements 722, 726 may result in increased system weight and cost. Similarly, the angle between the central reflective element 724 and the paired distal reflective elements 722, 726 can be varied as needed for a particular application. Thus, although an iso-angle of 45° is illustrated in FIG. 7, the embodiment is not limited to this implementation, and configurations with other angles can also be utilized. Additionally, the angles do not have to be equal but can be different. One of ordinary skill in the art would recognize many variations, modifications, and alternatives.

[0050] 8 is a plan view of an electromagnetic emitter incorporating a three-sided reflector and corresponding electromagnetic field lines according to some embodiments. In FIG. 8, only the electromagnetic field lines lying in the xz plane are illustrated for purposes of clarity, but it will be understood by one skilled in the art that a three-dimensional lobe pattern would exist with the centers of the lobes extending away from the electromagnetic emitter in directions perpendicular to the x-axis and at angles of 45° to the xz plane.

[0051] With reference to FIG. 8, three reflective elements are illustrated. A first reflective element 812 and a second reflective element 814 are positioned adjacent to the emitter 510. A third reflective element 816, which is in the xz plane, is illustrated by a dashed line. In the embodiment illustrated in FIG. 8, the three reflective elements 812, 814, 816 are mutually orthogonal and form one half of a cube structure, with the intersections of the three reflective elements forming the corner vertices of the cube. The emitter 510 is oriented at 45° with respect to the orientation illustrated in FIG. 6. Thus, the plates of the emitter 510 are oriented at 45° with respect to the x-axis in this embodiment.

[0052] The first reflective element 620, the second reflective element 622, and the third reflective element 816 reflect the electromagnetic field that would be established in the negative z-direction and the positive y-direction, so that the electromagnetic field lines 810 form a single lobe that is oriented out of the plane of the figure along the positive z-direction and the negative y-direction. In some embodiments, the presence of the reflective elements 812, 814, 816 in the half-cube configuration results in an efficiency increase of up to eight times and a power consumption reduction of up to one-eighth. Alternatively, the size of the emitter 510 can be reduced for a given efficiency / power consumption. Furthermore, in some embodiments, the size of the emitter 510 is reduced while achieving improved efficiency / power consumption performance. Those skilled in the art will recognize many variations, modifications, and alternatives.

[0053] 9 is a plan view of an electromagnetic emitter incorporating a hemispherical reflector and corresponding electromagnetic field lines, according to some embodiments. In FIG. 9, a portion of the hemispherical reflector 920 that lies in the xz plane is illustrated as a circular arc. It should be understood that rotation of the illustrated arc about the z-axis would define the hemispherical shape of the hemispherical reflector. Because the hemispherical reflector 920 reflects an electromagnetic field that would be established in the negative z-direction, the electromagnetic field lines 910 form a single lobe that is oriented along the positive z-direction.

[0054] 10A is a perspective view illustrating the integration of an electromagnetic emitter incorporating a three-sided reflector with a handheld controller, according to some embodiments. Referring to FIG. 10A, a handheld controller 1000 includes an electromagnetic emitter 1010 (generally referred to as "emitter 1010") integrated with a three-sided cube reflector 1020. In this example, the three sides of the reflector lie in the following planes, respectively: a first side 1022 in the xz plane, a second side 1024 in the xy plane, and a third side 1026 in the yz plane. The reflection of the electromagnetic field generated by the emitter 1010 by the three-sided cube reflector 1020 results in the center of the three-dimensional lobe pattern generated by the emitter 1010 being aligned with a vector 1030 that is aligned with a direction away from the intersection of the three reflector planes in the direction of a principal line equidistant to the three axes of the three-sided cube reflector, each axis corresponding to the intersection of the two reflectors. As illustrated in FIG. 10A, the vector 1030 is directed from the origin of the xyz coordinate space along a line directed to a point (1,1,1) in the xyz coordinate space, which corresponds to the center of the three-dimensional lobe pattern. When applied to a sensor, such a three-sided cube reflector would enable the sensor to sense most effectively along the principal line, which would be directed toward the ROI.

[0055] In typical use, the handheld controller 1000 is positioned in front of the user, with the surface 1050 being approximately orthogonal to a line pointed toward the user's head and the AR headset worn by the user. The line connecting the surface 1050 and the user's head, as well as the normal to the surface 1050, are approximately parallel to the vector 1030. As a result, the enhanced directivity of the electromagnetic field generated by the emitter 1010 creates a stronger field along the vector 1030, and therefore a stronger field is created in the vicinity of the user's head and the AR headset worn by the user. Similarly, as described in connection with FIG. 10B below, the headset can include a sensor with a corresponding three-sided cube reflector (or other suitable integrated reflector as described herein) configured such that maximum reception will occur within the ROI in which the emitter is expected to be present. As a result, improved system performance is provided relative to implementations that do not utilize reflective elements.

[0056] FIG. 10B is a perspective view illustrating the integration of an electromagnetic sensor and a headset incorporating a three-sided reflector, according to some embodiments. As illustrated in FIG. 10B, the AR headset 301 includes a sensor housing 1075, which in some embodiments is mounted under the right temple of the AR headset 301. FIG. 10C is a perspective view illustrating a close-up view of the sensor housing 1075 illustrated in FIG. 10B. The sensor housing 1075 includes an electromagnetic sensor 1070 (generally referred to as "sensor 1070") integrated with a three-sided cube reflector 1072. The reflection of the electromagnetic field received by the sensor 1070 by the three-sided cube reflector results in the center of the three-dimensional lobe pattern received by the sensor 1070 being aligned with a vector 1074 oriented along a direction that points away from the intersection of the three reflector planes in the direction of a principal line equidistant to the three axes of the three-sided cube reflector, each axis corresponding to the intersection of the two reflectors. As illustrated in Figure 10B, a vector 1074 is directed from the origin of the xyz coordinate space along a line directed to a point (1,1,1) in the xyz coordinate space, which corresponds to the center of the three-dimensional lobe pattern. Note that the xyz coordinate space illustrated in Figure 10B differs from that illustrated in Figure 10A for purposes of clarity. As illustrated in Figure 10B, applying a three-sided cube reflector 1072 to the sensor 1070 allows the sensor 1070 to sense most effectively along a principal line that would be pointed toward the emitter.

[0057] In typical use, the handheld controller 1000 illustrated in FIG. 10A is positioned in front of and below the user's head, so that the electromagnetic field received from the emitter at the sensor housing 1075 will be enhanced by the presence of the three-sided cube reflector 1072. The location of the three-sided cube reflector 1072 adjacent to the sensor 1070 results in enhanced directionality of the electromagnetic field received by the sensor 1070. As a result, increased sensor sensitivity is achieved along the line connecting the handheld controller 1000 and the AR headset 301. Thus, the sensor 1070 has increased sensitivity to electromagnetic fields generated in the vicinity of the handheld controller. As a result, improved system performance is provided relative to implementations that do not utilize reflective elements.

[0058] 11 is a simplified flow chart illustrating a method of operating an electromagnetic tracking system incorporating an integrated reflector, according to an embodiment of the present invention. The method 1100, in which the electromagnetic tracking system incorporates one or more integrated electromagnetic reflectors, includes generating (1100) an electromagnetic field using an electromagnetic emitter. The electromagnetic emitter can be located in a handheld controller, which is one element of an electromagnetic tracking system that includes a handheld controller, an auxiliary unit that may include a controller, and a head-mounted augmented reality display. The integrated electromagnetic reflector can be utilized with any of the emitters and / or sensors, and can be any of the integrated electromagnetic reflectors illustrated in FIGS. 6-9 herein.

[0059] The method also includes reflecting the electromagnetic field using a first electromagnetic reflector to form a modified electromagnetic field pattern (1112). The first electromagnetic reflector can be positioned adjacent to the electromagnetic emitter. The first electromagnetic reflector can include reflective elements with various geometries, including two or more reflector plates that can be joined at vertices. In other embodiments, three reflector plates are utilized and arranged to define the corner vertices of a cube. In alternative embodiments, the first electromagnetic reflector is formed as a single reflector element. By way of example, the single reflector element can be a segmented reflector as discussed and illustrated in connection with FIG. 7, or a hemispherical reflector as discussed and illustrated in connection with FIG. 9.

[0060] The method further includes reflecting 1114 a portion of the modified electromagnetic field pattern using a second electromagnetic reflector and sensing 1116 the reflected portion of the modified electromagnetic field pattern using an electromagnetic sensor adjacent to the second electromagnetic reflector. Utilizing a controller, the method may further include controlling the timing of the generation of the electromagnetic field and the sensing of the reflected portion of the modified electromagnetic field, and digitally calculating positions and orientations of the electromagnetic emitter and the electromagnetic sensor based on the modified electromagnetic field pattern.

[0061] It should be understood that the specific steps illustrated in FIG. 11 provide a particular method of operating an electromagnetic emitter incorporating an integrated reflector in accordance with an embodiment of the present invention. Other sequences of steps may also be performed in accordance with alternative embodiments. For example, alternative embodiments of the present invention may perform the steps outlined above in a different order. Additionally, individual steps illustrated in FIG. 11 may include multiple sub-steps that may be performed in various sequences depending on the needs of the individual step. Additionally, additional steps may be added or removed depending on the particular application. Those skilled in the art will recognize numerous variations, modifications, and alternatives.

[0062] It should also be understood that the examples and embodiments described herein are for illustrative purposes only, and that various modifications or changes in light thereof will be suggested to those skilled in the art and are within the spirit and scope of the present application and the appended claims.

Claims

1. 1. A head-mounted augmented reality display system, comprising:

1. A handheld controller comprising an emitter housing, the emitter housing comprising: a first electromagnetic reflector including three first reflective elements forming a first vertex of a first cube, the first electromagnetic reflector defining a first coordinate space, the first vertex of the first cube being at an origin of the first coordinate space; and an electromagnetic emitter configured to generate an electromagnetic field, the electromagnetic emitter being positioned along a line directed to a point (1,1,1) in the first coordinate space; a handheld controller including:

1. A headset, comprising: Right temple, Left temple, and A sensor housing mounted on at least one of the right temple or the left temple, the sensor housing comprising: An electromagnetic sensor; a second electromagnetic reflector including three second reflective elements forming second vertices of a second cube; A sensor housing comprising: A headset comprising: A head-mounted augmented reality display system comprising:

2. 2. The head-mounted augmented reality display system of claim 1, wherein the first electromagnetic reflector generates a modified electromagnetic field having a lobe along the line that is directed toward a point (1,1,1) in the first coordinate space.

3. 3. The head-mounted augmented reality display system of claim 2, wherein the electromagnetic field is characterized by an initial full-width-half-maximum width and the modified electromagnetic field is characterized by a modified full-width-half-maximum width that is less than the initial full-width-half-maximum width.

4. The head mounted augmented reality display system of claim 2 , wherein the modified electromagnetic field is characterized by a higher field strength at a lobe center than the electromagnetic field.

5. Controlling the timing of electromagnetic emission and sensing; digitally calculating positions and orientations of the electromagnetic emitters and the electromagnetic sensors based on the modified electromagnetic fields; The head mounted augmented reality display system of claim 2 , further comprising a controller operable to:

6. The head-mounted augmented reality display system of claim 5 , further comprising an auxiliary unit including the controller.

7. The head mounted augmented reality display system of claim 6 , wherein the auxiliary unit comprises a beltpack.

8. The head-mounted augmented reality display system of claim 6 , wherein the controller is distributed among the handheld controller, the headset, and the auxiliary unit.

9. The head-mounted augmented reality display system of claim 1 , wherein the sensor housing is mounted on the right temple.

10. The head-mounted augmented reality display system of claim 1 , wherein the sensor housing is mounted on the left temple.

11. 2. The head-mounted augmented reality display system of claim 1, wherein the second electromagnetic reflector defines a second coordinate space, and the second vertex of the second cube is at the origin of the second coordinate space.

12. The head mounted augmented reality display system of claim 11 , wherein the three second reflective elements define a quadrant of the second coordinate space in which the electromagnetic emitter is positioned.

13. The head-mounted augmented reality display system of claim 1 , wherein the three first reflective elements comprise three reflector plates and the three second reflective elements comprise three additional reflector plates.

Images