Tracking the optical flow of backscattered laser speckle patterns.

Abstract

To provide a suitable tracking of an optical flow of a backscattered laser speckle pattern.SOLUTION: Augmented reality headgear includes: a transparent display that allows a user to view the real world and virtual content located in the real world at the same time; at least one coherent light source; and at least one sensor array for sensing a speckle pattern created when the coherent light collides with an environmental surface in chronological order. A circuit is provided for sensing a shift in the speckle pattern, determining a motion that caused the shift in the speckle pattern, adjusting a display of a virtual object displayed by the augmented reality headgear, and compensating for the motion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on U.S. Provisional Patent Application No. 62 / 489,339, filed April 24, 2017. The present invention relates to augmented reality headgear. [Background technology]

[0002] Recently, virtual reality devices have been introduced that immerse users in computer-generated virtual worlds. One virtual reality device includes a pair of displays placed in close proximity to the user's eyes and corrective optics interposed between the displays and the user's eyes, the purpose of which is to allow the user to focus on the images displayed on the displays despite the close proximity of the displays. The primary application for such virtual reality devices is for gaming, although other applications, such as scientific data visualization, are also being considered.

[0003] A related but more advanced technology under development is augmented reality. Augmented reality wearables (i.e., headgear with an eyeglass form factor) would allow a user to simultaneously view the real world and virtual computer-generated content superimposed on the real world. To improve the illusion that the virtual content is real and / or to more seamlessly integrate the virtual content and the real world, it would be desirable for the virtual content to appear to exist in an inertial reference frame that is fixed to the user's real environment, even as the user rotates their head with the headgear or moves around in that environment. Thus, for example, if the virtual content were to include a virtual book resting on the corner of a real-world desk, the book should remain fixed in the real world, e.g., on the desk corner, even as the user rotates their head with the augmented reality headgear that projects an image of the book onto the desk corner. To accomplish this, the movement of the headgear carried on the user's head would need to be carefully tracked, and the image being generated by the augmented reality headgear would need to be rapidly shifted in the field of view to offset the effects of the headgear movement. One method of tracking the orientation of an object is to use one or more gyroscopes. However, gyroscopes inherently suffer from drift, and therefore attempts to anchor virtual objects in the real world based on gyroscope output would lead to virtual objects that slowly drift relative to the real world when they are intended to maintain a fixed position relative to the real world. Summary of the Invention [Means for solving the problem]

[0004] According to certain embodiments disclosed herein, the augmented reality headgear is equipped with at least one laser and at least one optical sensor array. Each laser emits a beam that is partially reflected from room surfaces (e.g., walls, ceiling, or floor) and creates a speckle pattern that is detected by an associated optical sensor array. The optical sensor arrays can be of the type that can be used in digital cameras, for example; however, in this application, 2D optical sensor arrays need not be used to capture focused images of objects; rather, they can capture patterned speckles generated by the laser reflected from room surfaces. The movement of the speckle pattern across the sensor array (sometimes appropriately referred to as "optical flow") is used to calculate the movement of the augmented reality headgear within an inertial reference frame fixed to the room. The image displayed using the augmented reality eyepiece is shifted based on the calculated movement to maintain its position or velocity within the inertial reference frame fixed to the room.

[0005] One aspect of the present disclosure is an augmented reality headgear comprising at least one per-image modulated light source, at least one transparent eyepiece configured to couple the per-image modulated light into a user's eye while allowing the user to view a real world, at least a first coherent light source aimed outward from the augmented reality headgear in a first direction to project coherent light onto at least one environmental surface, at least a first sensor array configured to receive light diffracted by and reflected from the at least one environmental surface, the light forming a first speckle pattern on the first sensor array, and an electronic circuit coupled to the per-image modulated light source and the first sensor array, the electronic circuit configured to operate the per-image modulated light source and the augmented reality headgear. a first sensor array configured to receive light reflected from at least one environmental surface and to adjust the per-image modulated light based on the movement of the augmented reality headgear, and to maintain the virtual object at the set of coordinates defined in the inertial reference frame. Additionally, the augmented reality headgear may also include a second coherent light source aimed outward from the augmented reality headgear in a second direction to project coherent light onto the at least one environmental surface. Additionally, the augmented reality headgear may also include at least one pupil diaphragm configured to substantially eliminate light from the second coherent light source reflected by the at least one environmental surface from reaching the first sensor array.In addition, the augmented reality headgear may further include at least one optical component configured to establish mutually exclusive emission solid angle ranges of the first coherent light source and the second coherent light source. In addition, the first sensor array may be mounted to have a first field of view, and the second sensor array may be mounted to have a second field of view, and the first field of view may partially overlap with the second field of view.

[0006] One aspect of the present disclosure is a method for sensing and distinguishing between translational movement of a structure along a set of three independent axes and rotation of the structure about one of the set of three independent axes, the method comprising the steps of: providing at least one coherent light source that emits light over a predetermined solid angle range and that is coupled to the structure; providing a first 2D optical sensor array that is coupled to the structure, the first 2D optical sensor array having a first normal vector pointing in a first direction; and providing a second 2D optical sensor array that is coupled to the structure. and a second 2D optical sensor array having a second normal vector pointing in a second direction, the first normal vector and the second normal vector defining a plane, the first normal vector being angled with respect to the second normal vector in the plane; illuminating a non-specular environment surface using at least one coherent illumination source, whereby a first speckle pattern is produced on the first 2D optical sensor array and a second speckle pattern is produced on the 2D optical sensor array; sensing a translation of the structure along a first of a set of three independent axes, the axis including a non-zero projection onto a plane between a first normal vector and a second normal vector, by sensing a first optical flow of a first speckle pattern on the first 2D optical sensor array and sensing a second optical flow of a second speckle pattern on a second 2D optical sensor array, the first optical flow and the second optical flow having opposite projections onto a difference vector between the first normal vector and the second normal vector; sensing a translation of the structure along a second one of the set of three independent axes, the second one having a non-zero projection onto a plane outside an angular range between the first normal vector and the second normal vector, by sensing a third optical flow of the first speckle pattern on the sensor array and sensing a fourth optical flow of the second speckle pattern on a second 2D optical sensor array, wherein the third optical flow and the fourth optical flow have a common directional projection onto a difference vector between the first normal vector and the second normal vector;sensing translation of the structure along a third of the set of three independent axes, the third of the set including a non-zero component perpendicular to the plane, by sensing identically sensed vertical optical flows of the first speckle pattern on the first 2D optical sensor array and the second speckle pattern on the second 2D optical sensor array; and sensing rotation of the structure about a first of the three independent axes by sensing oppositely sensed vertical optical flows of the first speckle pattern on the first 2D optical sensor array and the second speckle pattern on the second 2D optical sensor array.

[0007] One aspect of the present disclosure includes a method for sensing and distinguishing translational movement of a structure along and rotation about a set of three independent axes, the method including the steps of providing a first 2D optical sensor array coupled to the structure, the first 2D optical sensor array having a first surface normal oriented in a first direction and a first field of view, and providing a second 2D optical sensor array coupled to the structure, the second 2D optical sensor array having a second surface normal oriented in a second direction and a second field of view. providing a third 2D optical sensor array coupled to the structure, the third 2D optical sensor array having a third surface normal oriented in a third direction and a third field of view, the first direction, the second direction, and the third direction being independent; and providing at least one coherent light source that projects light into the first field of view, the second field of view, and the third field of view, wherein light diffracted by and reflected from a non-specular surrounding surface projects a first speckle pattern onto the first 2D optical sensor array and a second speckle pattern onto the second 2D optical sensor array. forming a speckle pattern on a second 2D optical sensor array and a third speckle pattern on the third 2D optical sensor array; using the second 2D optical sensor array to sense rotation about the first direction and to sense translation of the second speckle pattern in a direction that is an azimuth angle relative to the first direction; and using the third 2D optical sensor array to sense translation of the third speckle pattern in a direction that is an azimuth angle relative to the first direction. sensing a translation of the second speckle pattern in the first direction and using a third 2D optical sensor array to sense the translation of the third speckle pattern in the third direction; sensing a rotation about the second direction by using the first 2D optical sensor array and sensing the translation of the first speckle pattern in a direction that is an azimuth angle to the second direction; and sensing the translation of the third speckle pattern in a direction that is an azimuth angle to the second direction by using the third 2D optical sensor array;using a first 2D optical sensor array to sense translation in the second direction, sensing translation of the first speckle pattern in the second direction, and using a third 2D optical sensor array to sense translation of the third speckle pattern in the second direction; using a second 2D optical sensor array to sense rotation about the third direction, sensing translation of the second speckle pattern in a direction that is an azimuth angle to the third direction, and using the first 2D optical sensor array to sense translation of the first speckle pattern in a direction that is an azimuth angle to the third direction; using the first 2D optical sensor array to sense translation in the third direction, sensing translation of the first speckle pattern in the third direction, and using the second 2D optical sensor array to sense translation of the second speckle pattern in the third direction. The present invention provides, for example, the following. (Item 1) 1. Augmented reality headgear, comprising: at least one imagewise modulated light source; at least one transparent eyepiece configured to couple the image-by-image modulated light into the user's eye while allowing the user to view the real world; at least a first coherent light source, the at least first coherent light source being aimed outward from the augmented reality headgear in a first direction to project coherent light onto at least one environmental surface; at least a first sensor array configured to receive light reflected from the at least one environmental surface, the light forming a first speckle pattern on the first sensor array; an electronic circuit coupled to the image-wise modulated light source and the first sensor array, the electronic circuit comprising: operating the per-image modulated light source to display a virtual object at a set of coordinates defined within an inertial reference frame fixed to a physical space occupied by a user wearing the augmented reality headgear; receiving a first copy of the first speckle pattern at a first time; receiving a second copy of the first speckle pattern at a second time; determining a shift in a second copy of the first speckle pattern relative to a first copy of the first speckle pattern; determining a movement of the augmented reality headgear within the physical space occupied by a user based on a shift in a second copy of the first speckle pattern relative to a first copy of the first speckle pattern; an electronic circuit configured to perform Augmented reality headgear. (Item 2) The electronic circuit further comprises: adjusting the per-image modulated light based on movement of the augmented reality headgear to compensate for movement of the augmented reality headgear and maintain the virtual object at the set of coordinates defined within the inertial reference frame. Item 1. The augmented reality headgear according to item 1, configured to perform the following: (Item 3) Item 1, the augmented reality headgear further comprising a second sensor array configured to receive light from the at least one environmental surface. (Item 4) Item 4. The augmented reality headgear of item 3, further comprising a second coherent light source aimed outward from the augmented reality headgear in a second direction to project coherent light onto the at least one environmental surface. (Item 5) Item 5. The augmented reality headgear of item 4, further comprising at least one aperture stop configured to substantially eliminate light from the second coherent light source reflected by the at least one environmental surface from reaching the first sensor array. (Item 6) Item 5. The augmented reality headgear of item 4, further comprising at least one optical component configured to establish mutually exclusive emission solid angle ranges of the first coherent light source and the second coherent light source. (Item 7) 1. A method for sensing and distinguishing between translational movement of a structure along a set of three independent axes and rotation of the structure about one of the set of three independent axes, the method comprising: providing at least one coherent light source that emits light over a predetermined solid angle range and is coupled into said structure; providing a first 2D optical sensor array coupled to the structure, the first 2D optical sensor array having a first normal vector pointing in a first direction; providing a second 2D optical sensor array coupled to the structure, the second 2D optical sensor array having a second normal vector pointing in a second direction, the first normal vector and the second normal vector defining a plane, the first normal vector being angled with respect to the second normal vector in the plane; illuminating a non-specular environmental surface using at least one coherent illumination source, whereby a first speckle pattern is produced on the first 2D optical sensor array and a second speckle pattern is produced on the 2D optical sensor array; sensing a translation of the structure along a first one of the set of three independent axes having a non-zero projection onto a plane between the first normal vector and the second normal vector by sensing a first optical flow of the first speckle pattern on the first 2D optical sensor array and sensing a second optical flow of a second speckle pattern on the second 2D optical sensor array, wherein the first optical flow and the second optical flow have opposite projections onto a difference vector between the first normal vector and the second normal vector; sensing a translation of the structure along a second one of the set of three independent axes having a non-zero projection onto a plane outside an angular range between the first normal vector and the second normal vector by sensing a third optical flow of the first speckle pattern on the first 2D optical sensor array and sensing a fourth optical flow of the second speckle pattern on the second 2D optical sensor array, wherein the third optical flow and the fourth optical flow have a common directional projection onto a difference vector between the first normal vector and the second normal vector; sensing a translation of the structure along a third of the set of three independent axes that includes a non-zero component perpendicular to the plane by sensing a same sensed vertical optical flow of the first speckle pattern on the first 2D optical sensor array and the second speckle pattern on the second 2D optical sensor array; sensing a rotation of the structure about a first of the three independent axes by sensing opposite vertical optical flows of the first speckle pattern on the first 2D optical sensor array and the second speckle pattern on the second 2D optical sensor array; and A method comprising: (Item 8) 1. A method for sensing and distinguishing between translational movement of a structure along a set of three independent axes and rotation about said set of three independent axes, said method comprising: providing a first 2D optical sensor array coupled to the structure, the first 2D optical sensor array having a first surface normal oriented in a first direction and a first field of view; providing a second 2D optical sensor array coupled to the structure, the second 2D optical sensor array having a second surface normal oriented in a second direction and a second field of view; providing a third 2D optical sensor array coupled to the structure, the third 2D optical sensor array having a third surface normal oriented in a third direction and a third field of view; the first direction, the second direction, and the third direction are independent; and providing at least one coherent light source that projects light into the first field of view, the second field of view, and the third field of view, wherein light reflected from non-specular surrounding surfaces forms a first speckle pattern on the first 2D optical sensor array, a second speckle pattern on the second 2D optical sensor array, and a third speckle pattern on the third 2D optical sensor array; using the second 2D optical sensor array to sense rotation about the first direction, sensing translation of the second speckle pattern in a direction that is an azimuthal angle relative to the first direction, and using the third 2D optical sensor array to sense translation of the third speckle pattern in a direction that is an azimuthal angle relative to the first direction; using the second 2D optical sensor array to sense translation in the first direction, sensing translation of the second speckle pattern in the first direction, and using the third 2D optical sensor array to sense translation of the third speckle pattern in the first direction; using the first 2D optical sensor array to sense rotation about the second direction and to sense translation of the first speckle pattern in a direction that is an azimuth angle relative to the second direction, and using the third 2D optical sensor array to sense translation of the third speckle pattern in a direction that is an azimuth angle relative to the second direction; using the first 2D optical sensor array to sense translation in the second direction, sensing translation of the first speckle pattern in the second direction, and using the third 2D optical sensor array to sense translation of the third speckle pattern in the second direction; using the second 2D optical sensor array to sense rotation about the third direction, sensing translation of the second speckle pattern in a direction that is an azimuth angle relative to the third direction, and using the first 2D optical sensor array to sense translation of the first speckle pattern in a direction that is an azimuth angle relative to the third direction; using the first 2D optical sensor array to sense translation in the third direction, sensing translation of the first speckle pattern in the third direction, and using the second 2D optical sensor array to sense translation of the second speckle pattern in the third direction; A method comprising: [Brief explanation of the drawings]

[0008] The drawings illustrate the design and utility of preferred embodiments of the present invention, where like elements are referred to by common reference numerals. To better understand how the above and other advantages and objects of the present invention are obtained, a more particular description of the invention described above will be given by reference to specific embodiments thereof, which are illustrated in the accompanying drawings. With the understanding that these drawings depict only exemplary embodiments of the invention and are not to be considered as limiting its scope, the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings.

[0009] [Figure 1] FIG. 1 is a front view of a user's head wearing augmented reality headgear according to an embodiment disclosed herein.

[0010] [Figure 2] FIG. 2 is a schematic diagram of a motion detection system included within the augmented reality headgear shown in FIG.

[0011] [Figure 3]FIG. 3 is a schematic diagram of a portion of the motion detection system shown in FIG. 2 illustrating optical isolation between two coherent light sources, according to an embodiment.

[0012] [Figure 4-1] 4a-4f show a sequence of speckle patterns corresponding to a sequence of translation steps. [Figure 4-2] 4a-4f show a sequence of speckle patterns corresponding to a sequence of translation steps. [Figure 4-3] 4a-4f show a sequence of speckle patterns corresponding to a sequence of translation steps.

[0013] [Figure 5] FIG. 5 is a schematic diagram of a motion detection system included within augmented reality headgear, according to an alternative embodiment.

[0014] [Figure 6] FIG. 6 is a perspective view of augmented reality headgear with an eyeglass form factor according to another embodiment.

[0015] [Figure 7] FIG. 7 is a schematic diagram of a motion detection system included within the augmented reality headgear shown in FIG.

[0016] [Figure 8] FIG. 8 is a block diagram of an augmented reality headgear, applicable to the augmented reality headgear shown in FIGS. 1 and 6, according to an embodiment.

[0017] [Figure 9] FIG. 9 is a method flowchart for operating the augmented reality headgear shown in FIG. 1, according to one embodiment.

[0018] [Figure 10] FIG. 10 is a diagram of an example of the use of augmented reality headgear, according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0019] 1 is a front view of a user's head 102 wearing augmented reality headgear 104, according to one embodiment. The augmented reality headgear 104 has an eyeglass form factor and includes a frame 106 on which a left (user's left) eyepiece 108 and a right eyepiece 110 are mounted, taking the form of lenses (eyepieces) of the headgear 104. The eyepieces 108, 110 may include surface-relief reflective and transmissive gratings that control the redirection of light toward the user's eyes, or may include tilted partially reflective mirrors integrated into the bulk eyepieces 108, 110. For example, each eyepiece 108, 110 may include an internal coupling grating (ICG), which receives per-image modulated light and diffracts the per-image modulated light to angles above the critical angle for total internal reflection (TIR) ​​within each eyepiece. In addition to the ICG, each eyepiece may further include an orthogonal pupil expansion grating (OPE), which progressively deflects light directed by the ICG to the OPE toward an exit pupil expander (EPE), which progressively deflects the light outward toward the user's eye position. Alternatively, the eyepieces 108, 110 may include a partially reflective front surface coating that redirects light toward the user's eye. In the latter case, the eyepieces need not be flat, but may include curved front and back (near-eye) surfaces with finite optical power. Referring again to FIG. 1 , the image-by-image modulated left light source 112 is optically coupled to the left eyepiece 108, and the image-by-image modulated right light source 114 is optically coupled to the right eyepiece 110. The image-by-image modulated light sources 112, 114 may include, for example, a liquid crystal on silicon (LCoS) image projector, a fiber scanner, or an emissive (e.g., micro-light-emitting diode, micro-organic light-emitting diode) display panel. The eyepieces 108, 110 serve to optically couple the image-modulated light to the user's eyes. The eyepieces 108, 110 are also transparent, thereby allowing the user to view the real world. The image-modulated light sources 112, 114 and the eyepieces 108, 110 are used to display images, which in this context are referred to as "virtual content." The virtual content augments the real world visible through the eyepieces 108, 110.The left eyepiece 108 is combined with an image-by-image modulated left light source 112 to together form the left display, and the right eyepiece 110 is combined with an image-by-image modulated right light source 114 to together form the right display.

[0020] For many applications, it is desirable to have virtual content (e.g., a walking virtual person) located within an inertial reference frame that is fixed to the environment (e.g., a room) in which the user of the augmented reality headgear is located, despite the fact that the augmented reality headgear 104 moves and rotates with the user as the user moves around the environment and turns their head 102 to look in different directions. To achieve this latter goal, the images displayed through the eyepieces 108, 110 and the image-by-image modulated light sources 112, 114 must be shifted to compensate for the user's motion. Determining the correct shift requires careful tracking of the user's motion (translation and rotation). To achieve this goal, the augmented reality headgear 104 is equipped with an upward-pointing laser (coherent illumination source) 116, a side-pointing laser 118, and a forward-pointing laser 120, all of which are mechanically coupled to the frame 106. The pointing directions of the lasers 116, 118, and 120 can also be modified relative to the aforementioned directions, but this is merely a simple example of a preferred direction. An upward-facing 2D optical sensor array 122, a side-facing 2D optical sensor array 124, and a front-facing 2D optical sensor array 126 are also provided and mechanically coupled to the frame 106. The 2D optical sensor arrays 122, 124, and 126 can comprise, for example, complementary metal-oxide semiconductor (CMOS) pixel arrays or charge-coupled device (CCD) pixel arrays. Light emitted by the upward-facing laser 116 is angularly within the field of view of the upward-facing 2D optical sensor array 122, light emitted by the side-facing laser 118 is angularly within the field of view of the side-facing 2D optical sensor array 124, and light emitted by the front-facing laser 120 is angularly within the field of view of the front-facing 2D optical sensor array 126.Light from each laser 116, 118, 120, to the extent that the light propagates away from the emitting laser and associated 2D optical sensor arrays 122, 124, 126, is angularly within the field of view of a particular one of the 2D optical sensor arrays 122, 124, 126, but will not be detected by the associated 2D optical sensor arrays 122, 124, 126 to any significant extent unless it is backscattered by a surface it strikes, such as a wall, ceiling, or furniture. With the exception of windows and mirrors, such surfaces are generally diffuse, such that the backscattered light will take the form of a speckle pattern that fills the space between the scattering surface and the 2D optical sensor arrays 122, 124, 126. The speckle pattern, in this case, is a diffraction pattern of small-scale surface roughness. The speckle pattern is detected by the 2D optical sensor arrays 122, 124, 126. Furthermore, as the user moves their head 102 along with the augmented reality headgear 104, the 2D optical sensor arrays 122, 124, 126 move through space filling the speckle pattern, and such movement is detectable by reading from the 2D optical sensor arrays 122, 124, 126 in continuous time. Because the lasers 116, 118, 120 move along with the 2D optical sensor arrays 122, 124, 126, the detected movement of the speckle pattern across the 2D optical sensor arrays will be twice the physical movement of the 2D optical sensor arrays 122, 124, 126. With rigid body mechanics, there are three translational degrees of freedom (e.g., translation along the Cartesian X, Y, and Z axes) and three rotational degrees of freedom (e.g., yaw, pitch, and roll). For convenience, the variables Rx, Ry, and Rz may alternatively be used to refer to rotations about the X, Y, and Z axes.

[0021] Figure 2 is a schematic diagram of a motion detection system 200 included within the augmented reality headgear shown in Figure 1. The motion detection system 200 includes an upward-facing laser 116, a side-facing laser 118, a front-facing laser 120, an upward-facing 2D optical sensor array 122, a side-facing 2D optical sensor array 124, and a front-facing 2D optical sensor array 126. The aforementioned components are shown oriented with respect to a 3D Cartesian coordinate system 202, which includes Z (upward), Y (side, to the user's left), and X (frontward) axes. Coherent light from the upward (+Z)-facing laser 116 is incident on a ceiling surface 204, which is non-specular and contains some small-scale surface roughness, which diffracts the coherent light, forming a speckle pattern, and impinges on the upward (+Z)-facing 2D optical sensor array 122. Similarly, coherent light from the side (+Y)-facing laser 118 is incident on the side-facing patch 206 of the wall, which scatters light that forms a speckle pattern onto the side (+Y)-facing 2D optical sensor array 124. Similarly, coherent light from the front (+X)-facing laser 120 is incident on the rear-facing patch 208 of the wall, which scatters light that forms a speckle pattern onto the front (+X)-facing 2D optical sensor array 126. In FIG. 2, rotations about the X, Y, and Z axes are indicated by the notations Rx, Ry, and Rz. A component of translational motion of the augmented reality headgear 104 along a particular Cartesian axis (X, Y, or Z) produces a speckle shift that is parallel to the component on the two 2D optical sensor arrays 122, 124, and / or 126 that face in a direction different from the component of translational motion. Thus, for example, a translation component in the +X direction causes a +X translation of the speckle pattern on the upward (+Z) facing 2D optical sensor array 122 and the side (+Y) facing 2D optical sensor array 124. The induced translation of the speckle pattern is equal to twice the physical translation of the headgear 104.Each 2D optical sensor array 122, 124, 126 is operated at a sufficiently high frame rate so that the laser spot being sensed by the 2D sensor array 122, 124, 126 moves a distance that is a fraction of the size of the spot (i.e., full width at half maximum, FWHM) between successive frames.

[0022] As shown in the schematic diagram in Figure 2, the 2D optical sensor arrays 122, 124, 126 are displaced from the Cartesian coordinate system origin. In the augmented reality headgear 104, the 2D optical sensor arrays are displaced from a virtual center of rotation for head movement located near the back of the user's neck. Rotation about the X, Y, and Z axes is sensed by sensing shifts in the speckle patterns produced on the 2D optical sensor arrays 122, 124, 126. A component of rotation about a given axis (X, Y, or Z) will induce a speckle shift that is azimuthal with respect to the given axis on the 2D optical sensor arrays 122, 124, or 126 other than the 2D optical sensor arrays 122, 124, or 126 oriented parallel to the given axis. (For present purposes, the positive sense of rotation about a positive Cartesian axis is defined using the right-hand rule.) Thus, for example, the -X direction is the azimuthal angle relative to the Z-axis on the side (+Y)-facing 2D optical sensor array 124, and the +Y direction is also the azimuthal angle relative to the Z-axis on the front (+X)-facing 2D optical sensor array 126. Thus, a positive Rz rotation component about the Z-axis will induce an X-direction speckle shift on the side (+Y)-facing 2D optical sensor array 124 and a -Y-direction speckle shift on the front (+X)-facing 2D optical sensor array 126. The matrix equation EQU. 1 below relates incremental translations along and rotations about the Cartesian X, Y, and Z axes to the shifts of the speckle patterns on the 2D optical sensor arrays 122, 124, 126.

number

[0023] Non-zero coefficients in the coefficient matrix to the left of EQU. 1 have three subscripts. The first subscript identifies one of the six degrees of freedom of movement of the augmented reality headgear 104 between X, Y, and Z translations and Rx, Ry, and Rz rotations about the X, Y, and Z axes. The second subscript identifies one of the 2D optical sensor arrays 122, 124, and 126 by the direction it faces (X, Y, or Z), which is equal to the direction of the normal vector to the front (light-receiving) surface of the 2D optical sensor arrays 122, 124, and 126. The third subscript identifies the direction (X, Y, or Z) of speckle shift on the particular 2D optical sensor array 122, 124, and 126 identified by the second subscript. Non-zero coefficients in the first three columns in the coefficient matrix for translation degrees of freedom have a value of 2. Non-zero coefficients in the third through sixth columns of the translation matrix have a value of 2 / R sensor and R sensor is the distance between the 2D optical sensor array 122, 124, or 126 identified by the second subscript and the effective center of rotation when the user is wearing the augmented reality headgear 104 (e.g., behind the user's neck).

[0024] The left column vector of EQU.1 includes incremental translations Δx, Δy, Δz as the first three elements, and incremental rotations ΔRx, ΔRy, ΔRz about the X, Y, and Z axes as the last three elements. Each element of the right column vector of EQU.1 is a speckle shift on one of the 2D optical sensor arrays 122, 124, 126. Each speckle shift element is denoted by a subscript and a superscript S. The subscript identifies one of the 2D optical sensor arrays 122, 124, 126 by the direction its normal vector is oriented (the direction it faces). The superscript identifies the direction of the speckle shift on the 2D optical sensor arrays 122, 124, 126. Illustratively, the first row of the coefficient matrix C represents the translations in the y direction (C y,x,y ) and rotation around the Z axis (C Rz,x,y(as shown by ) will cause a speckle shift in the y direction (azimuthal angle relative to the z-axis) on the front (+X) facing 2D optical sensor array 126). The coefficient matrix can be easily inverted, leading to the matrix equation EQU. 2 given below:

number

[0025] In the formula, C -1is the inverse of the matrix in EQU.1. EQU.2 is used to determine the incremental translation and rotation of the augmented reality headgear 104 based on the speckle shift vector appearing on the left side of EQU.2. The incremental speckle shift can be obtained by reading the speckle patterns formed on the 2D optical sensor arrays 122, 124, 126 at two consecutive times and determining the relative shift of the speckle patterns at the two consecutive times. An optical flow tracking method, such as the Farneback method or normalized cross-correlation, may be used to determine the relative shift of the speckle patterns. Based on the determined incremental translation and rotation, the virtual content being output via the per-image modulated light sources 112, 114 and the eyepieces 108, 110 is adjusted to maintain position and / or motion within an inertial coordinate system that is fixed to the environment in which the augmented reality headgear 104 is being used. The 2D optical sensor arrays 122, 124, 126 are read at a rate sufficiently high compared to the maximum expected rate of the augmented reality headgear so that the frame-to-frame change in the pointing angle of the lasers 116, 118, 120 is a percentage of the FWHM beam divergence of the lasers 116, 118, 120 (including the effect of the diffusers 310, 312 in FIG. 3 ). Therefore, the frame-to-frame change in the speckle pattern sensed by the 2D optical sensor arrays 122, 124, 126 is primarily a shift (optical flow). In addition, the lasers 116, 118, 120 can be operated in pulsed mode, at a pulse rate equal to the frame rate of the 2D optical sensor arrays 122, 124, 126 and a pulse width substantially shorter than the frame period (1 / frame rate) of the 2D optical sensor arrays 122, 124, 126. Using such short laser pulse widths will help avoid motion-induced blurring of the speckle pattern.

[0026] FIG. 3 is a schematic diagram of a portion of the motion detection system shown in FIG. 2 , illustrating optical isolation between two coherent light sources, according to an embodiment. The diagram shown in FIG. 3 includes a side-pointing laser 118 and a front-pointing laser 120. As shown, the side-pointing laser 118 includes a first laser diode 302 optically coupled to a first collimating lens 304, and similarly, the front-pointing laser 120 includes a second laser diode 306 optically coupled to a second collimating lens 308. The collimating lenses 304, 308 establish mutually exclusive solid angle ranges of coverage (emission) of the side-pointing laser 118 and the front-pointing laser 120. The first collimating lens 304 forms a side-propagating light beam, and the second collimating lens 308 forms a front-propagating light beam. Alternatively, lenses that form a controlled divergence beam may be used instead of collimating lenses. Furthermore, beam-shaping lenses that establish a radially (e.g., flat-top) or non-axially symmetric beam profile may be used instead of collimating lenses. A first low-angle diffuser 310 is positioned in front of the first collimating lens 304, and a second low-angle diffuser 312 is positioned in front of the second collimating lens 308. The low-angle diffusers 310, 312 reduce the brightness of the light beams formed by the collimating lenses 304, 308, which is useful for eye safety. Note that the laser diodes 302, 306 may emit infrared light, in which case a user would be unable to see the emitted light. The low-angle diffusers 310, 312 can be characterized by a diffusion FWHM of 2° to 20°, as a non-limiting example. Side-facing 2D optical sensor array 124 and front-facing 2D optical sensor array 126 are also shown in Figure 3. Side-facing pupil diaphragm 314 is positioned in front of side-facing 2D optical sensor array 124 and serves to limit the field of view of side-facing 2D optical sensor array 124. Similarly, front-facing pupil diaphragm 316 is positioned in front of front-facing 2D optical sensor array 126 and serves to limit the field of view of front-facing 2D optical sensor array 126.The side-facing pupil diaphragm 314 establishes a field of view for the side-facing 2D optical sensor array 124 that substantially overlaps the solid angular range of the side-facing laser 118's emission as extended by the first low-angle diffuser 310 and substantially excludes the solid angular range of the forward-facing laser 120's emission as extended by the second low-angle diffuser 312. Similarly, the front-facing pupil diaphragm 316 establishes a field of view for the front-facing 2D optical sensor array 126 that substantially overlaps the solid angular range of the front-facing laser 120's emission as extended by the second low-angle diffuser 312 and substantially excludes the solid angular range of the side-facing laser 118's emission as extended by the first low-angle diffuser 310. Thus, each 2D optical sensor array will receive only a single speckle pattern produced by the light emitted by its associated laser. Another purpose of the pupil stops 314, 316 is to magnify the size of the speckles in the speckle pattern incident on each 2D optical sensor array 124, 126. The characteristic size of the speckles in the speckle pattern should be equal to or larger than the size of the individual sensor elements (pixels) that make up the 2D sensor arrays 124, 126. Although not shown in Figure 3, the upward-facing laser 116 has the same internal design as shown in Figure 3 for the side-facing laser 118 and the front-facing laser 120 and may also be equipped with a low-angle diffuser, and the upward-facing 2D optical sensor array 122 may also be equipped with a pupil stop.

[0027] According to an alternative embodiment, the sensor arrays 122, 124, 126 are spectrally isolated from the emissions of lasers 116, 118, 120 other than those with which they are associated. In one implementation, the upward-pointing laser 116 emits a first spectral line having a first peak wavelength, the side-pointing laser 118 emits a second spectral line having a second peak wavelength, and the front-pointing laser 120 emits a third spectral line having a third peak wavelength. A first spectrally selective filter that transmits the first spectral line but not the second or third spectral line is positioned across the upward-facing 2D optical sensor array 122, a second spectrally selective filter that transmits the second spectral line but not the first or third spectral line is positioned across the side-facing 2D optical sensor array 124, and a third spectrally selective filter that transmits the third spectral line but not the first or second spectral line is positioned across the front-facing 2D optical sensor array 126.

[0028] FIG. 4a is an initial image of a speckle pattern 400 that can be used to track motion within the systems shown in FIGS. 1-3. FIGS. 4b-4f show a sequence of images 408, 410, 412, 414, and 416 of the speckle pattern 400 shown in FIG. 4a after a sequence of translation increments. An initial arbitrary position 402 within the speckle pattern 400 is marked with a vertical line 404 and a horizontal line 406. Optical flow tracking methods, such as those described above, can be used to track the movement of the speckle pattern. The translation distance of the speckle pattern is equal to twice the physical translational augmented reality headgear 104, which includes the motion detection system 200. The location of the arbitrary position 402 is shown in each of the successive FIGS. 4b-4f.

[0029] FIG. 5 is a schematic diagram of a motion detection system 500 included within the augmented reality headgear 104 according to an alternative embodiment. The system 300 includes a first laser 506 oriented to point in a first direction, a second laser 508 oriented to point in a second direction, and a third laser oriented to point in a third direction. The system 300 also includes a first 2D optical sensor array oriented in the first direction, a second 2D optical sensor array oriented in the second direction, and a third 2D optical sensor array oriented in the third direction. In the motion detection system 300 shown in FIG. 3, the lasers 116, 118, and 120 point in a set of three orthogonal directions, and the 2D optical sensor arrays 122, 124, and 126 are oriented to point in the same set of three orthogonal directions, whereas in the motion detection system 500 shown in FIG. 5, the first, second, and third directions are not orthogonal.

[0030] Figure 6 is a perspective view of augmented reality headgear 600 with an eyeglass form factor according to another embodiment, and Figure 7 is a schematic diagram of a motion detection system 700 included within the augmented reality headgear 600. The augmented reality headgear 600 includes a frame 602 that supports a left (user's left) transparent eyepiece 604 and a right transparent eyepiece 606. The left transparent eyepiece 604 includes a left ICG 604A, a left OPE 604B, and a left EPE 604C; similarly, the right transparent eyepiece 606 includes a right ICG 606A, a right OPE 606B, and a right EPE 606C. Each of the eyepieces 604, 606 can alternatively include multiple waveguides to handle multiple color channels and / or output images with different wavefront curvatures (corresponding to different virtual image distances). A per-image modulated left light source 608 is optically coupled to the left transparent eyepiece 604, and a per-image modulated right light source 610 is optically coupled to the right transparent eyepiece 606. The transparent eyepieces 604, 606 serve to optically couple the per-image modulated light to the user's eye.

[0031] The augmented reality headgear 600 is further equipped with a right laser 611, a left laser 612, a left 2D optical sensor array 614, and a right 2D optical sensor array 616. The forward direction corresponds to the +X axis of the three axes of the Cartesian coordinate system shown in Figure 6. The left laser 612 and the left 2D optical sensor array 614 face in a direction tilted azimuthally to the left with respect to the forward direction (they rotate about a vertically oriented Z axis), and the right laser 611 and the right 2D optical sensor array 616 face in a direction tilted azimuthally to the right with respect to the forward direction. The directions in which the left laser 612 and the left optical sensor 614 face are determined by a left normal vector N normal to the light-receiving surface of the left optical sensor 614. L The direction in which the right laser 611 and the right optical sensor 616 are directed is defined by a right normal vector N R Alternatively, there may be a difference between the direction that each laser and associated sensor points. The left normal vector N L and the right normal vector N R The difference vector D between is shown in FIG. 7. The left 2D optical sensor array 614 and the right 2D optical sensor array 616 have separate fields of view. The field of view of the left 2D optical sensor array 614 includes at least a substantial portion of the range of emission of the left laser 612, and the field of view of the right 2D optical sensor array 616 includes at least a substantial portion of the stereoscopic range of emission of the right laser 611. The fields of view of the left and right 2D optical sensor arrays 614, 616 may be limited by pupil diaphragms or other field-limiting optical components (not shown in FIG. 6). For example, the field of view of the left 2D optical sensor array 614 may be limited to exclude the angular range of the emitting right laser 611, or vice versa.

[0032] Optionally, the left 2D optical sensor array 614 may be equipped with a left imaging lens 618, and the right 2D optical sensor array 616 may be equipped with a right imaging lens 620. The imaging lenses 618, 620 focus and magnify or demagnify speckle light from a left focal plane 622 and a right focal plane 624, respectively, positioned in the space in front of the imaging lenses 618, 620, onto the 2D optical sensor arrays 614, 616.

[0033] Note that the Y-axis of the Cartesian coordinate system extends laterally from left to right. The motion detection system 700 incorporated within the augmented reality headgear 600 is capable of sensing and distinguishing four degrees of freedom, including translational components along the X-, Y-, and Z-axes, and rotation about the forward-facing X-axis. The left normal vector N L and the right normal vector N R defines an imaginary plane, and N L and N R On the plane defined by L and N R There is an angle range between N L and N R N in the angle range between L and N R With a projection onto the plane defined by, the translation component is N L and N R The optical flow can be sensed by sensing opposite direction speckle pattern shifts on the left and right 2D optical sensor arrays 614, 616 parallel to the plane defined by (x, y, y). In the above case, the first optical flow on the left 2D optical sensor array 614 and the second optical flow on the right 2D optical sensor array 616 have opposite direction projections onto the difference vector D.

[0034] On the other hand, N L and N R N outside the angle range between L and N R With a projection onto the plane defined by, the translation component is N L and N R The optical flow can be sensed by sensing the same directional speckle pattern shift on the left and right 2D optical sensor arrays 614, 616 parallel to the plane defined by (N). In the latter case, the first optical flow on the left 2D optical sensor array 614 and the second optical flow on the right 2D optical sensor array 616 have a common directional projection onto the difference vector D. Furthermore, N L and N RThe translation component perpendicular to the plane defined by is N L and N R The rotation about the +X axis extending forward can be sensed by sensing the same directional speckle pattern shift on the left and right 2D optical sensor arrays 614, 616 perpendicular to the plane defined by N. L and N R The speckle pattern can be sensed by sensing opposite vertical speckle pattern shifts on the left and right 2D optical sensor arrays 614, 616 perpendicular to the plane defined by

[0035] Figure 8 is a block diagram of an augmented reality headgear 900, according to one embodiment. The design shown in Figure 8 can be used for the augmented reality headgear shown in Figures 1 and 6, according to one embodiment. The augmented reality headgear 900 includes a left optically transmissive (see-through) eyepiece 902, a right optically transmissive eyepiece 904, a first laser diode 906, a first 2D optical sensor array 908, an Nth laser diode 910, and an Nth 2D optical sensor array 912, all mechanically coupled to an augmented reality headgear frame 914. It is understood that the use of the identifier "Nth" indicates that the number of similar components, including the one identified by "Nth," is variable. For example, in the augmented reality headgear 104 shown in FIG. 1, three laser diodes are utilized in the three lasers 116, 118, 120, while in the augmented reality headgear 600 shown in FIG. 6, a single laser diode (not shown in FIG. 6) is provided in the forward-pointing laser 612.

[0036] A per-image modulated left light source 916 is optically coupled to the left optically transmissive eyepiece 902, and a per-image modulated right light source 918 is optically coupled to the right optically transmissive eyepiece 904. The per-image modulated light sources 916, 918 may comprise, for example, a fiber scanner, an LCoS projector or MEMS light beam scanner, or a micro-emissive display. A left image data source 920 is coupled to the per-image modulated left light source 916, and a right image data source 922 is coupled to the per-image modulated right light source 918. The image data sources 920, 922 may take the form of, for example, display drivers. The per-image modulated left light source 916 in combination with the left optically transmissive eyepiece 902 forms a left display 948, and the per-image modulated right light source 918 in combination with the right optically transmissive eyepiece 904 forms a right display 950. Left and per-image modulated right light sources 916, 918 modulate light according to data provided by left and right image data sources 920, 922, respectively. The left and right image data sources 920, 922 may take the form of frame buffers fed by a graphics processing unit (GPU), which, together with a microprocessor, runs the game engine program.

[0037] The first laser diode 906 is optically coupled to a first diffuser 926 through a first collimating lens 924, and the Nth laser diode 912 is optically coupled to an Nth diffuser 930 through an Nth collimating lens 928. Coherent light from the first laser diode 906, coupled through the first collimating lens 924 and the first diffuser 926, is incident on a first surface patch 932 (e.g., wall, ceiling, floor, furniture) in the environment of the augmented reality headgear 900, which forms a first speckle pattern (diffraction pattern of small-scale surface roughness) that is incident on the first 2D optical sensor array 908. Similarly, coherent light from the Nth laser diode 910, coupled through the Nth collimating lens 928 and the Nth diffuser 930, is incident on the Nth surface patch 934 in the environment of the augmented reality headgear 900, forming an Nth speckle pattern, which is incident on the Nth 2D optical sensor array 912.

[0038] The first sensor readout circuit 936 is coupled to the first 2D optical sensor array 908, and the Nth sensor readout circuit 938 is coupled to the Nth 2D optical sensor array 912. An inertial measurement unit (IMU) 952 is mechanically coupled to the frame 914. The first sensor readout circuit 936, the Nth sensor readout circuit 938, the left image data source 920, the first image data source 922, the IMU 952, at least one processor 940, at least one program memory 942, and at least one workspace memory 944 are coupled together via at least one bus 946. The at least one processor 940 may include, for example, a microprocessor, a graphics processing unit, a digital signal processor, and / or a microcontroller. The IMU 952 can be used in conjunction with the components described above to detect motion via speckle optical flow. For example, the IMU 952 can be used as an additional redundant source of motion information to improve accuracy, or information from the IMU can be combined with information obtained via speckle flow monitoring to fully determine the 6 DoF of the headgear 900.

[0039] FIG. 9 is a flowchart of a method 1000 of operating the augmented reality headgear 104 shown in FIG. 1 , according to one embodiment. The augmented reality headgear 600, 900 shown in FIGS. 6, 8 can be operated in a similar manner. In block 1002, surfaces (e.g., 204, 206, 208) in the headgear 104's local environment are illuminated with lasers 116, 118, 120 fixed to the augmented reality headgear 104. In block 1004, the augmented reality headgear's display is operated to display at least one virtual object at a location (defined by a set of coordinates) in the local environment. The eyepieces 108, 110, in combination with the image-by-image modulated light sources 112, 114, together form the display. FIG. 10 is a schematic diagram of an example of use of the augmented reality 104 headgear shown in FIG. 1 being used according to the method shown in FIG. 10. In FIG. 10 , headgear 1100 is shown on a user's head 102. The user is viewing a virtual object 1102 in the form of a book, which is displayed using the augmented reality headgear's display. The virtual object 1102 is located on a real table 1104. Other examples of virtual content may include, for example, people, animals, imaginary creatures, and / or moving objects. In the case of moving virtual content, the movement of the virtual content is defined within an inertial reference frame fixed to the physical environment, and the motion of the headgear is tracked so that the movement of the headgear can be compensated (cancelled) so that it adds velocity to the intended movement of the virtual content relative to the physical environment. Referring again to FIG. 9 , in block 1006, optical sensor arrays 122, 124, 126 are used to sense speckle patterns created by lasers 116, 118, 120 being scattered by environmental surfaces (e.g., 204, 206, 208). In executing blocks 1006 and 1010 described below, the speckle pattern may be received, for example, from sensor readout circuits 936, 938 (FIG. 8) in workspace memory 944 (FIG. 8) under the control of at least one processor 940 (FIG. 8). Block 1008 marks the start of a loop that is executed every successive time.In block 1010, the optical sensor array is again used to sense speckle patterns created by the lasers 116, 118, 120 being scattered by environmental surfaces (e.g., 204, 206, 208). In block 1012, a shift in each particular speckle pattern is determined compared to at least one previous measurement of the particular speckle pattern. An optical flow determination method such as that discussed above may be used in performing block 1012. In block 1014, changes in translation and rotation coordinates of the augmented reality headgear are determined based on the step of determining the shift in the speckle pattern in block 1012. EQU. 2, described above, may be used in performing block 1014. Next, in block 1016, the position (coordinates) of the virtual object on the display of the augmented reality headgear 104 are shifted according to the change in at least one coordinate as determined in block 1014 to maintain the position of the virtual object in the local environment, e.g., to maintain the virtual book 1102 in a fixed position on the real table 1104. The position of the virtual object is shifted by adjusting per-image modulated light coupled into the user's eye through the eyepiece. Method 1100 can be performed under control of a program stored in at least one memory 942 and executed by at least one processor 940 using at least one workspace memory 944. More generally, the headgear 100, 600, 900 comprises some form of electronic circuitry, which may alternatively include, by way of non-limiting example, an application specific integrated circuit (ASIC) and / or FPGA that determines speckle shifts between successive times, determines incremental motions (translation and rotation) between successive times based on the speckle shifts, and adjusts the position of the displayed virtual object to compensate for the incremental motions.

[0040] While the embodiments described above include augmented reality glasses that include clear eyepieces through which a user can view the real world while also viewing virtual content, the 6DoF tracking system described above may alternatively be incorporated into virtual reality goggles in which the user's view of the real world is occluded and the user can only see the virtual content. The 6DoF system described above may also be applied to a type of augmented reality in which the user cannot directly view the real world but can view images of the real world that are captured by one or more cameras and displayed to the user along with the virtual content.

[0041] Various exemplary embodiments of the present invention are described herein. Reference is made to these examples in a non-limiting sense. They are provided to illustrate the more broadly applicable aspects of the present invention. Various changes may be made to the invention described, and equivalents may be substituted without departing from the true spirit and scope of the invention. In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process acts, or steps to the objective, spirit, or scope of the present invention. Moreover, as will be understood by those skilled in the art, each of the individual variations described and illustrated herein has discrete components and features that can be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the invention. All such modifications are intended to be within the scope of the claims associated with this disclosure.

[0042] The present invention includes methods that may be performed using the subject devices. The methods may include the act of providing such a suitable device. Such provision may be performed by an end user. In other words, the act of "providing" merely requires the end user to obtain, access, access, locate, configure, activate, power on, or otherwise act upon the requisite device in the subject method, so as to provide the requisite device. Methods described herein may be carried out in any order of the recited events, and in the recited order of events, that is logically possible.

[0043] Exemplary aspects of the invention, along with details regarding material selection and manufacturing, have been described above. As for other details of the invention, these will be understood in connection with the above-referenced patents and publications and will generally be known or understood by those skilled in the art. The same may be true with respect to method-based aspects of the invention in terms of additional acts as may be commonly or logically employed.

[0044] Additionally, while the present invention has been described with reference to several embodiments that optionally incorporate various features, the present invention is not limited to those described and indicated as being considered with respect to each variation of the present invention. Various modifications may be made to the invention as described, and equivalents (whether described herein or not included for some simplicity) may be substituted without departing from the true spirit and scope of the invention. Additionally, when a range of values ​​is provided, it is understood that all intervening values ​​between the upper and lower limits of that range, and any other stated or intervening value within that stated range, are encompassed within the invention.

[0045] It is also contemplated that any optional features of the described inventive variations may be described and claimed independently or in combination with any one or more of the features described herein. Reference to a singular item includes the possibility that the same items are present in plural. More specifically, as used herein and in the claims associated herewith, the singular forms "a," "an," "said," and "the" include plural referents unless otherwise specified. In other words, the use of articles in the above description and in the claims associated with this disclosure allows for "at least one" of the subject item. Furthermore, it should be noted that such claims may be drafted to exclude any optional element. Accordingly, this statement is intended to serve as a predicate for the use of exclusive terminology such as "solely," "only," and equivalents, or the use of a "negative" limitation, in connection with the recitation of claim elements.

[0046] Without using such exclusive terminology, the term "comprising" in the claims associated with this disclosure shall be construed as allowing for the inclusion of any additional elements, regardless of whether a given number of elements are recited in such claims or whether the addition of features can be considered as a transformation of the nature of the elements recited in such claims. Except as specifically defined herein, all technical and scientific terms used herein shall be given the broadest possible commonly understood meaning while maintaining the validity of the claims.

[0047] The scope of the present invention is not to be limited by the exemplary and / or subject specification provided, but rather is to be limited only by the scope of the language of the claims associated with this disclosure.

Claims

1. 1. A method for detecting movement of an augmented reality (AR) headgear, the method comprising: projecting light from a first coherent light source onto a first surface; projecting light from a second coherent light source onto a second surface; receiving light reflected from the first surface and forming a first speckle pattern on a first optical sensor characterized by a first surface normal oriented in a first direction defined within a coordinate system; receiving light reflected from the second surface and forming a second speckle pattern on a second optical sensor characterized by a second surface normal oriented in a second direction defined within the coordinate system, the second direction being different from the first direction and a third direction orthogonal to the first direction and the second direction; using the first optical sensor to sense a translation of the first speckle pattern in the second direction; using the second optical sensor to sense a translation of the second speckle pattern in the first direction; sensing a positive rotation of the AR headgear about the third direction based on a translation of the first speckle pattern in a +X direction and a translation of the second speckle pattern in a −Y direction, or sensing a negative rotation of the AR headgear about the third direction based on a translation of the first speckle pattern in a −X direction and a translation of the second speckle pattern in a +Y direction; A method comprising:

2. using the second optical sensor to sense translation of the second speckle pattern in the third direction; sensing translation of the AR headgear in the third direction based on translation of the second speckle pattern in the third direction; The method of claim 1 further comprising:

3. using the first optical sensor to sense a translation of the first speckle pattern in the third direction; sensing translation of the AR headgear in the third direction based on translation of the first speckle pattern in the third direction; The method of claim 1 further comprising:

4. projecting light from a third coherent light source onto a third surface; receiving light reflected from the third surface and forming a third speckle pattern on a third optical sensor characterized by a surface normal oriented in the third direction; using the third optical sensor to sense a translation of the third speckle pattern in the second direction; using the second optical sensor to sense translation of the second speckle pattern in the third direction; sensing a rotation of the AR headgear about the first direction based on a translation of the second speckle pattern and a translation of the third speckle pattern; The method of claim 1 further comprising:

5. using the third optical sensor to sense a translation of the third speckle pattern in the first direction; using the first optical sensor to sense a translation of the first speckle pattern in the third direction; sensing a rotation of the AR headgear about the second direction based on a translation of the first speckle pattern and a translation of the third speckle pattern; The method of claim 4 further comprising:

6. using the third optical sensor to sense a translation of the third speckle pattern in the first direction; sensing translation of the AR headgear in the first direction based on translation of the third speckle pattern in the first direction; The method of claim 4 further comprising:

7. using the third optical sensor to sense a translation of the third speckle pattern in the second direction; sensing translation of the AR headgear in the second direction based on translation of the third speckle pattern in the second direction; The method of claim 4 further comprising:

8. 5. The method of claim 4, wherein the AR headgear further comprises an aperture stop positioned in front of the third optical sensor and operable to prevent light reflected from the first surface from reaching the third optical sensor.

9. 1. An augmented reality (AR) headgear, comprising: a first optical sensor having a first surface normal oriented in a first direction and operable to receive light reflected from the first surface, the first optical sensor being characterized by a first field of view; a second optical sensor having a second surface normal oriented in a second direction different from the first direction and operable to receive light reflected from the second surface, the second optical sensor being characterized by a second field of view; and a coherent light source operable to project light onto the first surface and onto the second surface; Electronic circuits and Equipped with The electronic circuit configured to sense translational movement of a first speckle pattern formed on the first optical sensor by light reflected from the first surface, and to sense translational movement of a second speckle pattern formed on the second optical sensor by light reflected from the second surface; the translation of the first speckle pattern is along the second direction; the translation of the second speckle pattern is along the first direction; Detecting a positive rotation of the AR headgear about a third direction orthogonal to the first direction and the second direction based on a translation of the first speckle pattern in the +X direction and a translation of the second speckle pattern in the −Y direction, or detecting a negative rotation of the AR headgear about the third direction based on a translation of the first speckle pattern in the −X direction and a translation of the second speckle pattern in the +Y direction. AR headgear.

10. 10. The AR headgear of claim 9, further comprising an aperture stop disposed in front of the first optical sensor and operable to prevent light reflected from the second surface from reaching the first optical sensor.

11. 10. The AR headgear of claim 9, further comprising an aperture stop disposed in front of the second optical sensor and operable to prevent light reflected from the first surface from reaching the second optical sensor.

12. at least one imagewise modulated light source; at least one transparent eyepiece configured to couple image-by-image modulated light into a user's eye while allowing the user to view the real world; and The AR headgear of claim 9 further comprising:

Images