Determining the center of eye rotation using one or more eye-tracking cameras

The display system uses eye-tracking cameras to estimate corneal parameters, enabling precise rendering of virtual image content at varying depths and orientations, addressing challenges in VR, AR, and MR technologies and enhancing user experience.

JP7591551B2Active Publication Date: 2024-11-28MAGIC LEAP INC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2022502459
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-07-16
Filing Date
2020-07-15
Publication Date
2024-11-28
Estimated Expiration
2040-07-15

AI Technical Summary

Technical Problem

Existing virtual, augmented, and mixed reality technologies face challenges in accurately presenting virtual image elements at different depths and orientations relative to the user's gaze, leading to discomfort and reduced immersion.

Method used

A display system that projects light into the user's eyes using a head-mounted setup with eye-tracking cameras and processing electronics to estimate the center of rotation and curvature of the cornea, allowing for precise rendering of virtual image content at varying depths and orientations based on eye movements.

Benefits of technology

Enhances the immersion and comfort of virtual, augmented, and mixed reality experiences by accurately presenting virtual image content at different depths and orientations relative to the user's gaze, improving the overall user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591551000001
    Figure 0007591551000001
  • Figure 0007591551000002
    Figure 0007591551000002
  • Figure 0007591551000003
    Figure 0007591551000003
Patent Text Reader

Abstract

The display system can include a head-mounted display configured to project light to the user's eyes to display virtual image content at different divergences and amounts of collimation. The display system can possibly include an inward-facing imaging system comprising multiple cameras that image the user's eyes and the phosphenes thereon, and processing electronics in communication with the inward-facing imaging system and configured to use corneal data derived from the phosphenes images to obtain an estimate of the center of rotation of the user's eyes. The display system can render the virtual image content using a rendering camera positioned at the determined location of the center of rotation of the eyes.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 62 / 874,867, filed July 16, 2019, and entitled "EYE CENTER OF ROTATION DETERMINATION WITH ONE OR MORE EYE TRACKING CAMERAS," the contents of which are incorporated herein by reference in their entireties. This application is related to U.S. Application No. 16 / 250,931, filed January 17, 2019, and entitled "EYE CENTER OF ROTATION DETERMINATION, DEPTH PLANE SELECTION, AND RENDER CAMERA POSITIONING IN DISPLAY SYSTEMS," and U.S. Patent Publication No. 2018 / 0018515, published January 18, 2018, and entitled "IRIS BOUNDARY ESTIMATION USING CORNEA CURVATURE," all of which are incorporated herein by reference in their entireties.

[0002] (Field) The present disclosure relates to display systems, virtual reality, and augmented reality imaging and visualization systems, and more particularly to eye tracking using an eye's center of rotation calculated using corneal data. [Background technology]

[0003] (background) Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality," "augmented reality," or "mixed reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or may be perceived as real. Virtual reality, or "VR," scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality, or "AR," scenarios typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user. Mixed reality, or "MR," relates to the merging of real and virtual worlds to generate new environments in which physical and virtual objects coexist and interact in real time. In conclusion, the human visual perception system is highly complex, making it difficult to produce VR, AR, or MR technologies that facilitate comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements. The systems and methods disclosed herein address various challenges associated with VR, AR, and MR technologies. Summary of the Invention [Means for solving the problem]

[0004] (summary) Various embodiments of depth plane selection in a mixed reality system are disclosed.

[0005] The display system can be configured to project light to the user's eye to display virtual image content within the user's field of view. The user's eye can have a cornea, an iris, a pupil, a lens, a retina, and an optical axis extending through the lens, the pupil, and the cornea. The display system can include a frame configured to be supported on the user's head, a head-mounted display disposed on the frame, one or more eye tracking cameras configured to image the user's eye, and processing electronics in communication with the display and the one or more eye tracking cameras, the processing electronics configured to obtain estimates of parameters of the eye based on images of the eye acquired with the one or more eye tracking cameras. In some implementations, the eye parameters include a center of curvature of the cornea (e.g., the center of curvature as measured at the corneal apex). In some implementations, the center of curvature of the cornea or the center of the cornea refers to the center of curvature of a portion of the cornea or the center of curvature of a spherical surface that coincides with a portion of the surface of the cornea. For example, in some implementations, the center of curvature of the cornea or the center of the cornea refers to the center of curvature of the corneal apex or the center of curvature of a spherical surface that coincides with a portion of the surface of the corneal apex. In some implementations, the ocular parameters include the center of rotation of the eye. Other parameters and information may be determined as well.

[0006] In some implementations, the display is configured to project light into the user's eye and display virtual image content in the user's field of view at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths. In some implementations, the displayed virtual image content appears to originate from different depths at different time periods.

[0007] Described herein are various embodiments of display systems that project light to one or both of a user's eyes to display virtual image content within the user's field of view, such as the embodiments listed below. Part I

[0008] Example 1: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content within the user's field of view; first and second eye tracking cameras configured to image the user's eye; a plurality of light emitters; and processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to receive images of the user's eye captured by the first and second eye tracking cameras, where flash reflexes of different light emitters are observable in the images of the eye captured by the first and second eye tracking cameras, and to estimate a location of the center of corneal curvature of the user's eye based on locations of the flash reflexes in the images produced by both the first and second eye tracking cameras, and based on the locations of both the first and second eye tracking cameras and the locations of the emitters that produced the individual flash reflexes.

[0009] Example 2: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eye; a plurality of light emitters; and processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics configured to receive images of the user's eye captured by the first and second eye tracking cameras, where flash reflexes of different light emitters are observable in the images of the eye captured by the first and second eye tracking cameras, and to estimate a location of the center of rotation of the user's eye based on locations of the flash reflexes in the images produced by both the first and second eye tracking cameras and based on locations of both the first and second eye tracking cameras and locations of the emitter that produced the flash reflex with respect to a plurality of eye poses.

[0010] Example 3: A method for determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to a user's eye for displaying virtual image content within the user's field of view, the eye having a cornea, the method comprising: capturing multiple images of the user's eye using multiple eye tracking cameras configured to image the user's eye and multiple light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising the multiple flashes of light; and obtaining an estimate of a center of rotation of the eye based on the multiple flashes of light, the obtaining an estimate of a center of rotation of the eye comprising: determining multiple estimates of a center of corneal curvature of the user's eye based on the multiple flashes of light; generating a three-dimensional surface from the multiple estimates of the center of corneal curvature; and determining an estimate of the center of rotation of the user's eye using the three-dimensional surface.

[0011] Example 4: A display system configured to project light to a user's eyes to display virtual image content within a field of view of the user, the display system including a frame configured to be supported on a user's head, a head mounted display disposed on the frame, the display configured to project light into the user's eyes and display virtual image content, first and second eye tracking cameras configured to image the user's eyes, and processing electronics in communication with the display and the first and second eye tracking cameras to capture images of multiple pairs of the user's eyes. and processing electronics configured to receive images from the first and second eye tracking cameras, for paired images received from the first and second eye tracking cameras, respectively, obtain an estimate of a center of corneal curvature of the user's eye based at least in part on respective paired captured images, and determine a three-dimensional surface, identify a center of curvature of the 3D surface, and obtain an estimate of a center of rotation of the user's eye based on the estimated center of corneal curvature of the user's eye obtained based on multiple paired captured images of the user's eye received from the respective first and second eye tracking cameras.

[0012] Example 5: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye to display virtual image content within the user's field of view; an eye tracking camera configured to image the user's eye; a plurality of light emitters; and processing electronics in communication with the display and the eye tracking camera, the processing electronics configured to receive images of the user's eye captured by the eye tracking camera at first and second locations, where flash reflexes of different light emitters are observable in the images of the eye captured by the eye tracking camera, and to estimate a location of the center of corneal curvature of the user's eye based on the location of the flash reflexes in the images produced by the eye tracking camera, and based on the location of the eye tracking camera and the location of the emitter that produced the individual flash reflexes.

[0013] Example 6: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; an eye tracking camera configured to image the user's eye; a plurality of light emitters; and processing electronics in communication with the display and the eye tracking camera, the processing electronics configured to receive images of the user's eye captured by the eye tracking camera at a first location and a second location, where flash reflexes of different light emitters are observable in the images of the eye captured by the eye tracking camera, and to estimate a location of the center of rotation of the user's eye based on the location of the flash reflexes in the images produced by the eye tracking camera and based on the location of the emitter that produced the flash reflex for the first and second locations of the eye tracking camera and a plurality of eye poses.

[0014] Example 7: A method for determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to a user's eye for displaying virtual image content within the user's field of view, the eye having a cornea, comprising: capturing multiple images of the user's eye using an eye tracking camera configured to image the user's eye and multiple light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising the multiple flashes of light; and obtaining an estimate of a center of rotation of the eye based on the multiple flashes of light, the obtaining an estimate of a center of rotation of the eye comprising: determining multiple estimates of a center of corneal curvature of the user's eye based on the multiple flashes of light; generating a three-dimensional surface from the multiple estimates of the center of corneal curvature; and determining an estimate of the center of rotation of the user's eye using the three-dimensional surface.

[0015] Example 8: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content; an eye tracking camera configured to image the user's eye; and processing electronics in communication with the display and the eye tracking camera, the processing electronics configured to receive captured images of multiple pairs of the user's eye from the eye tracking camera, obtain an estimate of the center of corneal curvature of the user's eye based, at least in part, on each of the paired images received from the eye tracking camera, and determine a three-dimensional surface based on the estimated center of corneal curvature of the user's eye obtained based on the captured images of the multiple pairs of the user's eye received from the eye tracking camera, identify a center of curvature of the 3D surface, and obtain an estimate of a center of rotation of the user's eye.

[0016] Example 9: A display system configured to project light into a user's eye to display virtual image content within the user's field of view, the display system comprising: a frame configured to be supported on a user's head; a head-mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; at least one eye tracking camera configured to image the user's eye; a plurality of light emitters; and processing electronics in communication with the display and the eye tracking camera, the processing electronics configured to receive images of the user's eye captured by the at least one eye tracking camera at first and second locations, where flash reflexes of different light emitters are observable in the images of the eye captured by the eye tracking camera, and to estimate a location of the center of corneal curvature of the user's eye based on the location of the flash reflexes in the images produced by the at least one eye tracking camera and based on the location of the at least one eye tracking camera and the location of the emitter that produced the individual flash reflexes. Part II

[0017] Example 1: A display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, comprising: a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eyes; A plurality of light emitters; processing electronics in communication with the display and the first and second eye tracking cameras, receiving images of the user's eyes captured by the first and second eye tracking cameras, where flashing reflections of different light emitters are observable in the images of the eyes captured by the first and second eye tracking cameras; estimating a location of a center of corneal curvature of the user's eye based on locations of the flash reflections in the images produced by both the first and second eye tracking cameras and based on the locations of both the first and second eye tracking cameras and the location of the emitter that produced the respective flash reflections; processing electronics configured to A display system comprising:

[0018] Example 2: The processing electronics comprises: determining a first direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the first eye tracking camera and based on a location of the first eye tracking camera and a location of an emitter that produced the flash of light; determining a second direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the second eye tracking camera and based on the location of the second eye tracking camera and the location of the emitter that produced the flash of light; The display system according to Example 1, configured as follows:

[0019] Example 3: The processing electronics comprises: defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 3. The display system of claim 2, wherein the display system is configured to determine the first direction by:

[0020] Example 4: The processing electronics comprises: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 4. The display system of claim 3, wherein the display system is configured to determine the second direction by:

[0021] Example 5: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate a location of the center of the corneal curvature of the user's eye based on the first and second directions toward the center of the corneal curvature of the user's eye.

[0022] Example 6: The processing electronics comprises: determining a first direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from a first eye tracking camera; determining a second direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from a second eye tracking camera, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0023] Example 7: A display system as described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the center of corneal curvature of the user's eye based on the convergence of the first and second directions.

[0024] Example 8: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate the location of the center of the corneal curvature of the user's eye by identifying an area of ​​convergence of the first and second directions toward the center of the corneal curvature of the user's eye.

[0025] Example 9: A display system as described in any of the above examples, wherein the processing electronics is configured to obtain an estimate of the center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for different eye postures.

[0026] Example 10: A display system as described in any of the above examples, wherein the processing electronics are configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye postures.

[0027] Example 11: A display system as described in Example 10, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye based on the locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye postures.

[0028] Example 12: A display system as described in Example 10 or 11, wherein the processing electronics is configured to determine a surface and obtain an estimate of a center of rotation of the user's eyes based on the locus of points.

[0029] Example 13: A display system as described in Example 10 or 11, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eyes by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0030] Example 14: A display system as described in Example 10 or 11, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eyes by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0031] Example 15: A display system as described in any of Examples 12, 13, or 14, wherein the processing electronics is configured to fit the surface to the locus of points to obtain the surface.

[0032] Example 16: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0033] Example 17: A display system as described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0034] Example 18: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eyes; A plurality of light emitters; processing electronics in communication with the display and the first and second eye tracking cameras, receiving images of the user's eyes captured by the first and second eye tracking cameras, where flashing reflections of different light emitters are observable in the images of the eyes captured by the first and second eye tracking cameras; estimating a location of the center of rotation of the user's eye based on a location of the flash reflection in the image produced by both the first and second eye tracking cameras and based on a location of both the first and second eye tracking cameras and a location of an emitter that produced the flash reflection for a plurality of eye poses; processing electronics configured to A display system comprising:

[0035] Example 19: To obtain an estimate of the center of rotation of the eye, the processing electronics: determining a plurality of estimates of a center of corneal curvature of the user's eye based on a plurality of flash reflexes for a plurality of eye postures; determining an estimate of a center of rotation of the user's eye based on a plurality of estimates of centers of corneal curvature of the user's eye for the plurality of eye postures; The display system according to Example 18, configured as follows:

[0036] Example 20: To determine the plurality of estimates of corneal curvature of the user's eye, the processing electronics: determining a first direction toward a center of corneal curvature based on respective locations of at least a portion of the plurality of emitters and a first one of the eye tracking cameras; determining a second direction toward a center of corneal curvature based on respective locations of at least a portion of the plurality of emitters and a second one of the eye tracking cameras; determining an estimate of a center of corneal curvature of the user's eye based on the first and second orientations; The display system of Example 19, configured as follows:

[0037] Example 21: The processing electronics comprises: defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 21. The display system of claim 20, configured to determine the first direction by:

[0038] Example 22: The processing electronics comprises: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 22. The display system of claim 21, configured to determine the second direction by:

[0039] Example 23: A display system described in any of Examples 20-22, wherein to determine the multiple estimates of the corneal curvature of the user's eye, the processing electronics are configured to determine an area of ​​convergence between a first direction and a second direction and determine an estimate of the center of the corneal curvature of the user's eye.

[0040] Example 24: To obtain an estimate of the center of rotation of the eye, the processing electronics: generating a three-dimensional surface associated with the plurality of estimates of the center of corneal curvature; determining an estimate of a center of rotation of the user's eyes based on the three-dimensional surface; A display system according to any one of Examples 19-23, configured as follows:

[0041] Example 25: A display system as described in Example 24, wherein the processing electronics are configured to fit a surface to multiple estimates of the center of corneal curvature to generate a three-dimensional surface associated with multiple estimates of the center of corneal curvature.

[0042] Example 26: A display system as described in Example 24, wherein the processing electronics is configured to fit a spherical surface to multiple estimates of the center of corneal curvature to generate a three-dimensional surface associated with multiple estimates of the center of corneal curvature.

[0043] Example 27: To determine an estimate of the center of rotation of the user's eyes, the processing electronics: Determining two or more normals to a three-dimensional surface; Determine the area of ​​convergence of two or more normals, A display system as described in any of Examples 24-26, configured so that the area of ​​convergence comprises an estimate of the center of rotation of the user's eyes.

[0044] Example 28: A display system described in any of Examples 21-27, wherein the one or more images of the user's eye comprise one or more images associated with different gaze vectors of the user's eye.

[0045] Example 29: A display system described in any of Examples 21-28, wherein the processing electronics is configured to map the cornea of ​​the user's eye using a gaze target.

[0046] Example 30: A display system described in any of Examples 18-29, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0047] Example 31: A display system described in any of Examples 18-30, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0048] Example 32: A method of determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, the eye having a cornea, the cornea having a center of curvature, capturing a plurality of images of the user's eye with a plurality of eye tracking cameras configured to image the user's eye and a plurality of light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising a plurality of flashes of light; obtaining an estimate of a center of rotation of the eye based on a plurality of flashes of light; and obtaining an estimate of the eye's center of rotation comprises: determining a plurality of estimates of a center of corneal curvature of the user's eye based on the plurality of flashes; generating a three-dimensional surface from the multiple estimates of the center of corneal curvature; determining an estimate of a center of rotation of the user's eyes using the three-dimensional surface; A method comprising:

[0049] Example 33: The step of determining a plurality of estimates of corneal curvature of an eye of a user includes: determining a first vector directed toward a center of corneal curvature based on locations of at least some of the plurality of light emitters and a location of a first camera of the plurality of eye tracking cameras; determining a second vector based on locations of at least some of the plurality of light emitters and a location of a second camera of the plurality of eye tracking cameras, the second vector being directed toward a center of corneal curvature; determining a region of convergence between the first vector and the second vector to determine an estimate of the center of corneal curvature of the user's eye; The method of Example 32, comprising:

[0050] Example 34: The first direction is defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; The method of Example 33, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 29, 32, 33, 34, 35, 36, 37, 38, 39, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,

[0051] Example 35: The second direction is defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; The method of Example 33, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 29, 32, 33, 34, 35, 36, 37, 38, 39, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,

[0052] Example 36: A method described in any of Examples 32-35, wherein the step of generating a three-dimensional surface from multiple estimates of the center of corneal curvature includes a step of fitting a surface to the multiple estimates of the center of corneal curvature.

[0053] Example 37: A method described in any of Examples 32-35, wherein the step of generating a three-dimensional surface from multiple estimates of the center of corneal curvature includes a step of fitting a sphere to the multiple estimates of the center of corneal curvature.

[0054] Example 38: The step of determining an estimate of a center of rotation of the user's eye includes: determining two or more vectors normal to a three-dimensional surface; determining a region of convergence of two or more vectors normal to a three-dimensional surface, the region of convergence comprising an estimate of a center of rotation of the user's eyes; The method of any of Examples 32-37, comprising:

[0055] Example 39: The method of any of Examples 32-38, wherein the multiple images of the user's eye are provided with images associated with different gaze directions of the user's eye.

[0056] Example 40: The method described in any of Examples 32-39, further comprising mapping the cornea of ​​the user's eye using a gaze target.

[0057] Example 41: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content; first and second eye tracking cameras configured to image the user's eyes; processing electronics in communication with the display and the first and second eye tracking cameras, receiving captured images of a plurality of pairs of the user's eyes from the first and second eye tracking cameras; obtaining an estimate of a center of corneal curvature of the user's eye based at least in part on each pair of captured images for each pair of images received from the first and second eye tracking cameras, respectively; determining a three-dimensional surface based on an estimated center of corneal curvature of the user's eye obtained based on a plurality of pairs of captured images of the user's eye received from the respective first and second eye tracking cameras; Identify the center of curvature of the 3D surface and obtain an estimate of the center of rotation of the user's eye, processing electronics configured to A display system comprising:

[0058] Example 42: A display system as described in Example 41, wherein the processing electronics are configured to fit a three-dimensional surface to an estimated center of corneal curvature of the user's eye obtained based on multiple pairs of captured images of the user's eyes received from separate first and second eye tracking cameras.

[0059] Example 43: To obtain an estimate of the center of corneal curvature of the user's eye based, at least in part, on the individual pairs of captured images, the processing electronics: determining a first vector along which a center of corneal curvature of the user's eye is estimated to be located based on a first image received from the first eye tracking camera; determining a second vector along which a center of corneal curvature of the user's eye is estimated to be located based on a second image received from a second eye tracking camera, the first and second images corresponding to one of the pair of images; identifying a region of convergence between paths extending in the direction of the first vector and the second vector to obtain an estimate of a center of corneal curvature of the user's eye; The display system according to Example 41 or 42, configured as follows:

[0060] Example 44: A method for detecting a glare from a user's eye, comprising: To determine a first vector based on a first image of the pair of captured images, the processing electronics defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; identifying a region of convergence of the first plane and the second plane, the region of convergence extending along a direction of the first vector; The display system of Example 43, configured as follows:

[0061] Example 45: To determine a second vector based on a second image in each pair of captured images, the processing electronics: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along a direction of the second vector; The display system of Example 44, configured as follows:

[0062] Example 46: A display system described in any of Examples 41-45, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0063] Example 47: A display system described in any of Examples 41-46, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0064] Example 48: A display system described in any of the above examples, wherein at least a portion of the display is transparent and positioned in front of the user's eyes when the user wears the head mounted display so as to transmit light from a portion of the environment in front of the user and the head mounted display to the user's eyes and provide a view of that portion of the environment in front of the user and the head mounted display.

[0065] Example 49: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; an eye tracking camera configured to image the eye of a user; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving an image of a user's eye captured by an eye tracking camera, where a flash of light reflection of a different light emitter is observable in the image of the eye captured by the eye tracking camera; estimating a location of a center of corneal curvature of the user's eye based on a location of the glint in the image produced by the eye tracking camera and based on a location of the eye tracking camera and a location of the emitter that produced the individual glint; processing electronics configured to A display system comprising:

[0066] Example 50: The processing electronics comprises: determining a first direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the eye tracking camera and based on a location of the eye tracking camera and a location of an emitter that produced the flash of light; determining a second direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the eye tracking camera and based on the location of the eye tracking camera and the location of the emitter that produced the flash of light; The display system of Example 49, configured as follows:

[0067] Example 51: The processing electronics comprises: defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; A display system as described in Example 50, configured to determine a first direction by:

[0068] Example 52: The processing electronics comprises: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 52. A display system as described in Example 51, configured to determine a second direction by:

[0069] Example 53: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate the location of the center of the corneal curvature of the user's eye based on the first and second directions toward the center of the corneal curvature of the user's eye.

[0070] Example 54: The processing electronics comprises: determining a first direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from an eye tracking camera location; determining a second direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from an eye tracking camera location, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0071] Example 55: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the center of corneal curvature of the user's eye based on the convergence of the first and second directions.

[0072] Example 56: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate the location of the center of the corneal curvature of the user's eye by identifying an area of ​​convergence of the first and second directions toward the center of the corneal curvature of the user's eye.

[0073] Example 57: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for different eye postures.

[0074] Example 58: A display system described in any of the above examples, wherein the processing electronics are configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye postures.

[0075] Example 59: A display system as described in Example 58, wherein the processing electronics is configured to obtain an estimate of the center of rotation of the user's eye based on the locus of points corresponding to an estimate of the center of corneal curvature of the user's eye for different eye postures.

[0076] Example 60: A display system as described in Example 58 or 59, wherein the processing electronics are configured to determine a surface and obtain an estimate of the center of rotation of the user's eyes based on the locus of points.

[0077] Example 61: A display system as described in Example 58 or 59, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0078] Example 62: A display system as described in Example 58 or 59, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eyes by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0079] Example 63: A display system described in any of Examples 60, 61, or 62, wherein the processing electronics are configured to fit the surface to the locus of points and acquire the surface.

[0080] Example 64: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0081] Example 65: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0082] Example 66: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; an eye tracking camera configured to image the eye of a user; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving an image of a user's eye captured by an eye tracking camera, where a flash of light reflection of a different light emitter is observable in the image of the eye captured by the eye tracking camera; estimating a location of a center of rotation of the user's eyes based on a location of a flash of light in the image produced by the eye tracking camera and based on a location of the eye tracking camera and a location of an emitter that produced the flash of light for a plurality of eye poses; processing electronics configured to A display system comprising:

[0083] Example 67: To obtain an estimate of the center of rotation of the eye, the processing electronics: determining a plurality of estimates of a center of corneal curvature of the user's eye based on a plurality of flash reflexes for a plurality of eye postures; determining an estimate of a center of rotation of the user's eye based on a plurality of estimates of centers of corneal curvature of the user's eye for the plurality of eye postures; The system described in Example 66, configured as follows.

[0084] Example 68: To determine the plurality of estimates of corneal curvature of the user's eye, the processing electronics: determining a first direction toward a center of corneal curvature based on at least individual locations of a portion of the plurality of emitters and a location of the eye tracking camera; determining a second direction toward a center of corneal curvature based on at least individual locations of at least a portion of the plurality of emitters and a location of the eye tracking camera; determining an estimate of a center of corneal curvature of the user's eye based on the first and second orientations; The system described in Example 67, configured as follows.

[0085] Example 69: The processing electronics comprises: defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 69. The display system of claim 68, configured to determine a first direction by:

[0086] Example 70: The processing electronics comprises: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 70. The display system of embodiment 69, configured to determine the second direction by:

[0087] Example 71: A system described in any of Examples 68-70, wherein to determine the multiple estimates of the corneal curvature of the user's eye, the processing electronics are configured to determine an area of ​​convergence between a first direction and a second direction and determine an estimate of the center of the corneal curvature of the user's eye.

[0088] Example 72: To obtain an estimate of the center of rotation of the eye, the processing electronics: generating a three-dimensional surface associated with the plurality of estimates of the center of corneal curvature; determining an estimate of a center of rotation of the user's eyes based on the three-dimensional surface; The system described in any one of Examples 19-71, configured as follows.

[0089] Example 73: The system described in Example 72, wherein the processing electronics is configured to fit a surface to multiple estimates of the center of corneal curvature to generate a three-dimensional surface associated with multiple estimates of the center of corneal curvature.

[0090] Example 74: The system described in Example 73, wherein the processing electronics are configured to fit a sphere to multiple estimates of the center of corneal curvature to generate a three-dimensional surface associated with multiple estimates of the center of corneal curvature.

[0091] Example 75: To determine an estimate of the center of rotation of the user's eyes, the processing electronics: Determining two or more normals to a three-dimensional surface; Determine the area of ​​convergence of two or more normals, A system described in any of Examples 72-74, configured so that the area of ​​convergence comprises an estimate of the center of rotation of the user's eye.

[0092] Example 76: A system described in any of Examples 69-75, wherein the one or more images of the user's eye comprise one or more images associated with different gaze vectors of the user's eye.

[0093] Example 77: A system described in any of Examples 69-76, wherein the processing electronics is configured to map the cornea of ​​the user's eye using a gaze target.

[0094] Example 78: A display system described in any of Examples 66-77, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0095] Example 79: A display system described in any of Examples 66-78, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0096] Example 80: A method of determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, the eye having a cornea, the cornea having a center of curvature, capturing a plurality of images of the user's eye with an eye tracking camera configured to image the user's eye and a plurality of light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising a plurality of flashes of light; and obtaining an estimate of a center of rotation of the eye based on a plurality of flashes of light, the step of obtaining an estimate of a center of rotation of the eye comprising: determining a plurality of estimates of a center of corneal curvature of the user's eye based on the plurality of flashes; generating a three-dimensional surface from the multiple estimates of the center of corneal curvature; Using the three-dimensional surface to determine an estimate of the center of rotation of the user's eyes. A method comprising:

[0097] Example 81: The step of determining a plurality of estimates of corneal curvature of an eye of a user includes: determining a first vector based on locations of at least a portion of the plurality of light emitters and a location of the eye tracking camera, the first vector being directed toward a center of corneal curvature; determining a second vector based on locations of at least some of the plurality of light emitters and a location of the eye tracking camera, the second vector being directed toward a center of corneal curvature; determining a region of convergence between the first vector and the second vector to determine an estimate of the center of corneal curvature of the user's eye; The method of Example 80, comprising:

[0098] Example 82: The first direction is defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; The method of Example 81, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,

[0099] Example 83: The second direction is defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; The method of Example 82, wherein the .alpha.-amino acid sequence is determined by the method of Example 82.

[0100] Example 84: A method described in any of Examples 81-83, wherein the step of generating a three-dimensional surface from multiple estimates of the center of corneal curvature includes a step of fitting a surface to the multiple estimates of the center of corneal curvature.

[0101] Example 85: A method described in any of Examples 81-83, wherein the step of generating a three-dimensional surface from multiple estimates of the center of corneal curvature includes a step of fitting a sphere to the multiple estimates of the center of corneal curvature.

[0102] Example 86: The step of determining an estimate of a center of rotation of the user's eye includes: determining two or more vectors normal to a three-dimensional surface; determining a region of convergence of two or more vectors normal to a three-dimensional surface, the region of convergence comprising an estimate of a center of rotation of the user's eyes; The method of any of Examples 81-85, comprising:

[0103] Example 87: A method as described in any of Examples 81-86, wherein the multiple images of the user's eye are provided with images associated with different gaze directions of the user's eye.

[0104] Example 88: The method described in any of Examples 81-87, further comprising a step of mapping the cornea of ​​the user's eye using a gaze target.

[0105] Example 89: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content; an eye tracking camera configured to image the eye of a user; processing electronics in communication with the display and the eye tracking camera, receiving captured images of a plurality of pairs of the user's eyes from an eye tracking camera; obtaining, for each pair of images received from the eye tracking camera, an estimate of a center of corneal curvature of the user's eye based, at least in part, on the respective pair of captured images; determining a three-dimensional surface based on an estimated center of corneal curvature of the user's eye, the center of curvature being obtained based on the captured images of a plurality of pairs of the user's eyes received from the eye tracking camera; Identify the center of curvature of the 3D surface and obtain an estimate of the center of rotation of the user's eye, processing electronics configured to A display system comprising:

[0106] Example 90: A display system as described in Example 89, wherein the processing electronics are configured to fit a three-dimensional surface to an estimated center of corneal curvature of the user's eye obtained based on multiple pairs of captured images of the user's eyes received from an eye tracking camera.

[0107] Example 91: To obtain an estimate of the center of corneal curvature of the user's eye based, at least in part, on the individual pairs of captured images, the processing electronics: determining a first vector along which a center of corneal curvature of the user's eye is estimated to be located based on a first image received from the eye tracking camera location; determining a second vector along which a center of corneal curvature of the user's eye is estimated to be located based on a second image received from the eye tracking camera location, the first and second images corresponding to one of the pair of images; identifying a region of convergence between paths extending in the direction of the first vector and the second vector to obtain an estimate of a center of corneal curvature of the user's eye; The display system of Example 89 or 90, configured as follows:

[0108] Example 92: A method for detecting a glimmer of light using a light source, comprising: To determine a first vector based on a first image of the pair of captured images, the processing electronics defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; identifying a region of convergence of the first plane and the second plane, the region of convergence extending along a direction of the first vector; The display system of Example 91, configured as follows:

[0109] Example 93: To determine a second vector based on a second image in each pair of captured images, the processing electronics: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along a direction of the second vector; The display system of Example 92, configured as follows:

[0110] Example 94: A display system described in any of Examples 89-93, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0111] Example 95: A display system described in any of Examples 89-94, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0112] Example 96: A display system described in any of the above examples, wherein at least a portion of the display is transparent and positioned in front of the user's eyes when the user wears the head-mounted display so as to transmit light from a portion of the environment in front of the user and the head-mounted display to the user's eyes and provide a view of that portion of the environment in front of the user and the head-mounted display.

[0113] Example 97: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; at least one eye tracking camera configured to image the user's eye; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving images of the user's eye captured by at least one eye tracking camera at a first and second location, where flashing reflections of different light emitters are observable in the images of the eye captured by the eye tracking camera; estimating a location of the center of corneal curvature of the user's eye based on a location of the glint reflection in the image produced by the at least one eye tracking camera and based on a location of the at least one eye tracking camera and a location of an emitter that produced the individual glint reflection; processing electronics configured to A display system comprising:

[0114] Example 98: The processing electronics comprises: determining a first direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the at least one eye tracking camera and based on a first location of the at least one eye tracking camera and a location of an emitter that produced the flash of light; determining a second direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the at least one eye tracking camera and based on a second location of the at least one eye tracking camera and a location of an emitter that produced the flash of light; The display system of Example 97, configured as follows:

[0115] Example 99: The processing electronics comprises: defining a first plane including a first location of at least one eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a first location of the at least one eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 99. A display system as described in Example 98, configured to determine a first direction by:

[0116] Example 100: The processing electronics comprises: defining a third plane that includes a second location of the at least one eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane including a second location of the at least one eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; A display system as described in Example 99, configured to determine a second direction by:

[0117] Example 101: A display system described in any of the above examples, wherein the processing electronics is configured to estimate the location of the center of the corneal curvature of the user's eye based on the first and second directions toward the center of the corneal curvature of the user's eye.

[0118] Example 102: The processing electronics comprises: determining a first direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from a first location of the at least one eye tracking camera; determining a second direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from a second location of the at least one eye tracking camera, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0119] Example 103: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the center of corneal curvature of the user's eye based on the convergence of the first and second directions.

[0120] Example 104: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate the location of the center of the corneal curvature of the user's eye by identifying an area of ​​convergence of the first and second directions toward the center of the corneal curvature of the user's eye.

[0121] Example 105: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for different eye postures.

[0122] Example 106: A display system described in any of the above examples, wherein the processing electronics are configured to determine a locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye postures.

[0123] Example 107: A display system as described in Example 106, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye based on the locus of points corresponding to an estimate of the center of corneal curvature of the user's eye for different eye postures.

[0124] Example 108: A display system as described in Example 106 or 107, wherein the processing electronics are configured to determine a surface and obtain an estimate of the center of rotation of the user's eyes based on the locus of points.

[0125] Example 109: A display system as described in Example 106 or 107, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eye by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0126] Example 110: A display system as described in Example 106 or 107, wherein the processing electronics are configured to obtain an estimate of the center of rotation of the user's eyes by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0127] Example 111: A display system described in any of Examples 108, 109, or 110, wherein the processing electronics are configured to fit the surface to the locus of points and acquire the surface.

[0128] Example 112: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation.

[0129] Example 113: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0130] Example 115: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view so that the displayed virtual image content appears to originate from different depths.

[0131] Example 116: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to originate from different depths, and to display virtual image content in the user's field of view.

[0132] Example 117: A display system described in any of the above examples, wherein the display is configured to project diverging light into the user's eye and collimated light to display virtual image content in the user's field of view that appears to originate from different depths. Part III

[0133] Example 1: A display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, comprising: a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eyes; A plurality of light emitters; processing electronics in communication with the display and the first and second eye tracking cameras, receiving images of the user's eyes captured by the first and second eye tracking cameras, where flashing reflections of different light emitters are observable in the images of the eyes captured by the first and second eye tracking cameras; estimating eye parameters based on locations of flash reflections in the images produced by both the first and second eye tracking cameras and based on locations of both the first and second eye tracking cameras and a location of an emitter that produced the individual flash reflections; processing electronics configured to A display system comprising:

[0134] Example 2: The processing electronics comprises: determining a first direction based on a location of the flash of light reflection in one or more images produced by the first eye tracking camera and based on a location of the first eye tracking camera and a location of an emitter that produced the flash of light reflection; determining a second direction based on a location of the flash reflection in one or more images produced by the second eye tracking camera and based on a location of the second eye tracking camera and a location of the emitter that produced the flash reflection; The display system of Example 1, configured to estimate the eye parameter by:

[0135] Example 3: The processing electronics comprises: defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 3. The display system of claim 2, wherein the display system is configured to determine the first direction by:

[0136] Example 4: The processing electronics comprises: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 4. The display system of claim 3, wherein the display system is configured to determine the second direction by:

[0137] Example 5: A display system as described in any of the above examples, wherein the processing electronics is configured to estimate a location of the parameter of the user's eye based on the first and second directions.

[0138] Example 6: The processing electronics comprises: determining the first direction based on at least one first image received from a first eye tracking camera; determining the second direction based on at least one second image received from a second eye tracking camera, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0139] Example 7: A display system according to any of the preceding examples, wherein the processing electronics is configured to obtain an estimate of the parameter based on convergence of the first and second directions.

[0140] Example 8: A display system as described in any of the preceding examples, wherein the processing electronics is configured to estimate the parameter by identifying an area of ​​convergence of the first and second directions.

[0141] Example 9: A display system as described in any of the above examples, wherein the processing electronics is configured to obtain an estimate of an additional parameter of the user's eye based on multiple determinations of other parameters of the user's eye for different eye postures.

[0142] Example 10: A display system according to any of the above examples, wherein the processing electronics is configured to determine a locus of points corresponding to estimates of the user's eye parameters for different eye postures.

[0143] Example 11: A display system as described in Example 10, wherein the processing electronics are configured to obtain estimates of additional parameters of the user's eye based on the locus of points corresponding to estimates of other parameters of the user's eye for different eye postures.

[0144] Example 12: A display system as described in Example 10 or 11, wherein the processing electronics is configured to determine a surface and obtain an estimate of additional parameters of the user's eye based on the locus of points.

[0145] Example 13: A display system as described in Example 10 or 11, wherein the processing electronics are configured to obtain an estimate of an additional parameter of the user's eye by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0146] Example 14: A display system as described in Example 10 or 11, wherein the processing electronics are configured to obtain an estimate of an additional parameter of the user's eye by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0147] Example 15: A display system as described in any of Examples 12, 13, or 14, wherein the processing electronics is configured to fit the surface to the locus of points to obtain the surface.

[0148] Example 16: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the additional parameter.

[0149] Example 17: A display system as described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0150] Example 18: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eyes; A plurality of light emitters; processing electronics in communication with the display and the first and second eye tracking cameras, receiving images of the user's eyes captured by the first and second eye tracking cameras, where flashing reflections of different light emitters are observable in the images of the eyes captured by the first and second eye tracking cameras; estimating a location of a first parameter of the user's eye based on a location of the flash of light in the images produced by both the first and second eye tracking cameras and based on a location of both the first and second eye tracking cameras and a location of an emitter that produced the flash of light for a plurality of eye poses; processing electronics configured to A display system comprising:

[0151] Example 19: To obtain an estimate of a first parameter of the eye, the processing electronics determining a plurality of estimates of a second parameter of the user's eye based on the plurality of flash reflexes for the plurality of eye postures; determining an estimate of a first parameter of the user's eye based on a plurality of estimates of a second parameter of the user's eye for the plurality of eye poses; The display system according to Example 18, configured as follows:

[0152] Example 20: To determine a plurality of estimates of a second parameter of the user's eye, the processing electronics: determining a first direction based on respective locations of at least a portion of the plurality of emitters and a first one of the eye tracking cameras; determining a second direction based on respective locations of at least a portion of the plurality of emitters and a second one of the eye tracking cameras; determining an estimate of a second parameter of the user's eye based on the first and second orientations; The display system of Example 19, configured as follows:

[0153] Example 21: The processing electronics comprises: defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 21. The display system of claim 20, configured to determine the first direction by:

[0154] Example 22: The processing electronics comprises: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 22. The display system of claim 21, configured to determine the second direction by:

[0155] Example 23: A display system described in any of Examples 20-22, wherein to determine multiple estimates of a second parameter of the user's eye, the processing electronics are configured to determine a region of convergence between a first direction and a second direction and determine an estimate of a second parameter of the user's eye.

[0156] Example 24: To obtain the estimate of the first parameter of the eye, the processing electronics comprises: generating a three-dimensional surface associated with the plurality of estimates of the second parameter of the eye; determining an estimate of a first parameter of the user's eye based on the three-dimensional surface; A display system according to any one of Examples 19-23, configured as follows:

[0157] Example 25: A display system as described in Example 24, wherein the processing electronics are configured to fit a surface to the multiple estimates of the second parameter to generate a three-dimensional surface associated with the multiple estimates of the second parameter of the eye.

[0158] Example 26: A display system as described in Example 24, wherein the processing electronics are configured to fit a spherical surface to the multiple estimates of the second parameter to generate a three-dimensional surface associated with the multiple estimates of the second parameter.

[0159] Example 27: To determine an estimate of a first parameter of the user's eye, the processing electronics: Determining two or more normals to a three-dimensional surface; Determine the area of ​​convergence of two or more normals, A display system as described in any of Examples 24-26, configured so that the area of ​​convergence comprises an estimate of a first parameter of the user's eye.

[0160] Example 28: A display system described in any of Examples 21-27, wherein the one or more images of the user's eye comprise one or more images associated with different gaze vectors of the user's eye.

[0161] Example 29: A display system described in any of Examples 21-28, wherein the processing electronics is configured to use a gaze target.

[0162] Example 30: A display system described in any of Examples 18-29, wherein the processing electronics are configured to render a virtual image to be presented to a user's eye using a rendering camera, the rendering camera having a position determined by the first parameter of the eye.

[0163] Example 31: A display system described in any of Examples 18-30, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0164] Example 32: A method of determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, the eye having a cornea, the cornea having a center of curvature, capturing a plurality of images of the user's eye with a plurality of eye tracking cameras configured to image the user's eye and a plurality of light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising a plurality of flashes of light; acquiring a first parameter of the eye based on a plurality of flashes of light; and obtaining a first parameter of the eye includes: determining a plurality of estimates of a second parameter of the user's eye based on the plurality of flashes; generating a three-dimensional surface from the plurality of estimates of the second parameter; determining an estimate of a first parameter of the user's eye using the three-dimensional surface; A method comprising:

[0165] Example 33: The step of determining a plurality of estimates of a second parameter of the user's eye includes: determining a first vector to be pointed based on locations of at least some of the plurality of light emitters and a location of a first camera of the plurality of eye tracking cameras; determining a second vector to be pointed based on locations of at least some of the plurality of light emitters and a location of a second camera of the plurality of eye tracking cameras; determining a region of convergence between the first vector and the second vector to determine an estimate of a second parameter of the user's eye; The method of Example 32, comprising:

[0166] Example 34: The first direction is defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; The method of Example 33, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 29, 32, 33, 34, 35, 36, 37, 38, 39, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,

[0167] Example 35: The second direction is defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; The method of Example 33, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 29, 32, 33, 34, 35, 36, 37, 38, 39, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58,

[0168] Example 36: A method described in any of Examples 32-35, wherein the step of generating a three-dimensional surface from multiple estimates of the second parameter includes a step of fitting a surface to the multiple estimates of the second parameter.

[0169] Example 37: A method described in any of Examples 32-35, wherein the step of generating a three-dimensional surface from multiple estimates of the second parameter includes a step of fitting a sphere to the multiple estimates of the second parameter.

[0170] Example 38: The step of determining an estimate of a first parameter of the user's eye includes: determining two or more vectors normal to a three-dimensional surface; determining a region of convergence of two or more vectors normal to a three-dimensional surface, the region of convergence comprising an estimate of a first parameter of the user's eye; The method of any of Examples 32-37, comprising:

[0171] Example 39: The method of any of Examples 32-38, wherein the multiple images of the user's eye are provided with images associated with different gaze directions of the user's eye.

[0172] Example 40: The method of any of Examples 32-39, further comprising using a gaze target.

[0173] Example 41: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content; first and second eye tracking cameras configured to image the user's eyes; processing electronics in communication with the display and the first and second eye tracking cameras, receiving captured images of a plurality of pairs of the user's eyes from the first and second eye tracking cameras; obtaining an estimate of a second parameter of the user's eye based at least in part on the respective pairs of captured images for the pair of images received from the first and second eye tracking cameras, respectively; determining a three-dimensional surface based on estimated second parameters of the user's eyes obtained based on the captured images of the plurality of pairs of the user's eyes received from the respective first and second eye tracking cameras; Identifying a center of curvature of the 3D surface and obtaining an estimate of a first parameter of the user's eye; processing electronics configured to A display system comprising:

[0174] Example 42: A display system as described in Example 41, wherein the processing electronics are configured to adapt the three-dimensional surface to estimated second parameters of the user's eyes obtained based on multiple pairs of captured images of the user's eyes received from separate first and second eye tracking cameras.

[0175] Example 43: To obtain an estimate of a second parameter of the user's eye based, at least in part, on the individual pairs of captured images, the processing electronics: determining a first vector based on a first image received from the first eye tracking camera; determining a second vector based on a second image received from a second eye tracking camera, the first and second images corresponding to one of the pair of images; identifying a region of convergence between paths extending in the direction of the first vector and the second vector to obtain an estimate of a second parameter of the user's eye; The display system according to Example 41 or 42, configured as follows:

[0176] Example 44: A method for detecting a glare from a user's eye, comprising: To determine a first vector based on a first image of the pair of captured images, the processing electronics defining a first plane that includes a first eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, the location of the second flash of light reflection, and the location of the light emitter corresponding to the second flash of light reflection; identifying a region of convergence of the first plane and the second plane, the region of convergence extending along a direction of the first vector; The display system of Example 43, configured as follows:

[0177] Example 45: To determine a second vector based on a second image in each pair of captured images, the processing electronics: defining a third plane that includes the second eye tracking camera, the location of the third flash of light reflection, and the location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, the location of the fourth flash of light reflection, and the location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along a direction of the second vector; The display system of Example 44, configured as follows:

[0178] Example 46: A display system described in any of Examples 41-45, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the first parameter.

[0179] Example 47: A display system described in any of Examples 41-46, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to originate from different depths and display the virtual image content in the user's field of view, or the display is configured to project light into the user's eye that diverges and is collimated to display virtual image content in the user's field of view that appears to originate from different depths.

[0180] Example 48: A display system described in any of the above examples, wherein at least a portion of the display is transparent and positioned in front of the user's eyes when the user wears the head mounted display so as to transmit light from a portion of the environment in front of the user and the head mounted display to the user's eyes and provide a view of that portion of the environment in front of the user and the head mounted display.

[0181] Example 49: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; an eye tracking camera configured to image the eye of a user; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving an image of a user's eye captured by an eye tracking camera, where a flash of light reflection of a different light emitter is observable in the image of the eye captured by the eye tracking camera; estimating a location of a parameter of the user's eye based on a location of the flash of light in the image produced by the eye tracking camera and based on a location of the eye tracking camera and a location of an emitter that produced the individual flash of light. processing electronics configured to A display system comprising:

[0182] Example 50: The processing electronics comprises: determining a first direction based on a location of the flash of light in one or more images produced by the eye tracking camera and based on a location of the eye tracking camera and a location of an emitter that produced the flash of light; determining a second direction based on a location of the flash of light in one or more images produced by the eye tracking camera and based on a location of the eye tracking camera and a location of the emitter that produced the flash of light; The display system of Example 49, configured as follows:

[0183] Example 51: The processing electronics comprises: defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; A display system as described in Example 50, configured to determine a first direction by:

[0184] Example 52: The processing electronics comprises: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 52. A display system as described in Example 51, configured to determine a second direction by:

[0185] Example 53: A display system described in any of the above examples, wherein the processing electronics is configured to estimate a location of the parameter of the user's eye based on the first and second directions.

[0186] Example 54: The processing electronics comprises: determining the first direction based on at least one first image received from an eye tracking camera location; determining the second direction based on at least one second image received from an eye tracking camera location, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0187] Example 55: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the parameter of the user's eye based on the convergence of the first and second directions.

[0188] Example 56: A display system described in any of the above examples, wherein the processing electronics is configured to estimate the parameter of the user's eye by identifying an area of ​​convergence of the first and second directions.

[0189] Example 57: A display system described in any of the above examples, wherein the processing electronics is configured to obtain estimates of additional parameters based on multiple determinations of the user's eye parameters for different eye postures.

[0190] Example 58: A display system described in any of the above examples, wherein the processing electronics are configured to determine a locus of points corresponding to estimates of the user's eye parameters for different eye postures.

[0191] Example 59: A display system as described in Example 58, wherein the processing electronics are configured to obtain estimates of additional parameters of the user's eye based on the locus of points corresponding to estimates of other parameters of the user's eye for different eye postures.

[0192] Example 60: A display system as described in Example 58 or 59, wherein the processing electronics is configured to determine a surface and obtain an estimate of additional parameters of the user's eye based on the locus of points.

[0193] Example 61: A display system as described in Example 58 or 59, wherein the processing electronics are configured to obtain an estimate of an additional parameter of the user's eye by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0194] Example 62: A display system as described in Example 58 or 59, wherein the processing electronics are configured to obtain an estimate of an additional parameter of the user's eye by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0195] Example 63: A display system described in any of Examples 60, 61, or 62, wherein the processing electronics are configured to fit the surface to the locus of points and acquire the surface.

[0196] Example 64: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the additional parameter.

[0197] Example 65: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to arise from different depths and display virtual image content in the user's field of view, or the display is configured to project light into the user's eye that diverges and is collimated to display virtual image content in the user's field of view that appears to arise from different depths.

[0198] Example 66: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; an eye tracking camera configured to image the eye of a user; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving an image of a user's eye captured by an eye tracking camera, where a flash of light reflection of a different light emitter is observable in the image of the eye captured by the eye tracking camera; estimating a location of a first parameter of the user's eye based on a location of the flash of light in the image produced by the eye tracking camera and based on a location of the eye tracking camera and a location of an emitter that produced the flash of light for a plurality of eye poses; processing electronics configured to A display system comprising:

[0199] Example 67: To obtain an estimate of a first parameter of the eye, the processing electronics comprises: determining a plurality of estimates of a second parameter of the user's eye based on the plurality of flash reflexes for the plurality of eye postures; determining an estimate of a first parameter of the user's eye based on a plurality of estimates of a second parameter of the user's eye for the plurality of eye poses; The system described in Example 66, configured as follows.

[0200] Example 68: To determine a plurality of estimates of a second parameter of the user's eye, the processing electronics: determining a first direction based on individual locations of at least a portion of the plurality of emitters and a location of an eye tracking camera; determining a second direction toward a center of corneal curvature based on at least individual locations of at least a portion of the plurality of emitters and a location of the eye tracking camera; determining an estimate of a center of corneal curvature of the user's eye based on the first and second orientations; The system described in Example 67, configured as follows.

[0201] Example 69: The processing electronics comprises: defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 69. The display system of claim 68, configured to determine a first direction by:

[0202] Example 70: The processing electronics comprises: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 70. The display system of embodiment 69, configured to determine the second direction by:

[0203] Example 71: A system described in any of Examples 68-70, wherein to determine multiple estimates of a second parameter of the user's eye, the processing electronics are configured to determine a region of convergence between a first direction and a second direction and determine an estimate of a second parameter of the user's eye.

[0204] Example 72: To obtain an estimate of a first parameter of the eye, the processing electronics comprises: generating a three-dimensional surface associated with the plurality of estimates of the second parameter; determining an estimate of a first parameter of the user's eye based on the three-dimensional surface; The system described in any one of Examples 19-71, configured as follows.

[0205] Example 73: A system as described in Example 72, wherein the processing electronics are configured to fit a surface to the multiple estimates of the first parameter to generate a three-dimensional surface associated with the multiple estimates of the second parameter.

[0206] Example 74: A system as described in Example 73, wherein the processing electronics are configured to fit a sphere to the multiple estimates of the second parameter to generate a three-dimensional surface associated with the multiple estimates of the second parameter.

[0207] Example 75: To determine an estimate of a first parameter of a user's eye, the processing electronics: Determining two or more normals to a three-dimensional surface; Determine the area of ​​convergence of two or more normals, A system described in any of Examples 72-74, configured so that the area of ​​convergence comprises an estimate of a first parameter of the user's eye.

[0208] Example 76: A system described in any of Examples 69-75, wherein the one or more images of the user's eye comprise one or more images associated with different gaze vectors of the user's eye.

[0209] Example 77: A system described in any of Examples 69-76, wherein the processing electronics is configured to use line of sight targeting.

[0210] Example 78: A display system described in any of Examples 66-77, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the first parameter.

[0211] Example 79: A display system described in any of Examples 66-78, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to arise from different depths and display virtual image content in the user's field of view, or the display is configured to project light into the user's eye that diverges and is collimated to display virtual image content in the user's field of view that appears to arise from different depths.

[0212] Example 80: A method of determining one or more parameters associated with an eye for rendering virtual image content in a display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, the eye having a cornea, the cornea having a center of curvature, capturing a plurality of images of the user's eye with an eye tracking camera configured to image the user's eye and a plurality of light emitters positioned relative to the eye and forming flashes of light thereon, the images comprising a plurality of flashes of light; obtaining an estimate of a first parameter of the eye based on the plurality of flashes of light; and obtaining an estimate of a first parameter of the eye comprises: determining a plurality of estimates of a second parameter of the user's eye based on the plurality of flashes; generating a three-dimensional surface from the plurality of estimates of the second parameter; determining an estimate of a first parameter of the user's eye using the three-dimensional surface; A method comprising:

[0213] Example 81: The step of determining a plurality of estimates of a second parameter of the user's eye includes: determining a first vector based on locations of at least a portion of the plurality of light emitters and a location of the eye tracking camera; determining a second vector based on locations of at least a portion of the plurality of light emitters and a location of the eye tracking camera; determining a region of convergence between the first vector and the second vector to determine an estimate of a second parameter of the user's eye; The method of Example 80, comprising:

[0214] Example 82: The first direction is defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; The method of Example 81, wherein the .alpha.-amino acid is selected from the group consisting of 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28,

[0215] Example 83: The second direction is defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; The method of Example 82, wherein the .alpha.-amino acid sequence is determined by the method of Example 82.

[0216] Example 84: A method described in any of Examples 81-83, wherein the step of generating a three-dimensional surface from multiple estimates of the second parameter includes a step of fitting a surface to multiple estimates of the first parameter.

[0217] Example 85: A method described in any of Examples 81-83, wherein the step of generating a three-dimensional surface from multiple estimates of the second parameter includes a step of fitting a sphere to the multiple estimates of the second parameter.

[0218] Example 86: The step of determining an estimate of a first parameter of the user's eye includes: determining two or more vectors normal to a three-dimensional surface; determining a region of convergence of two or more vectors normal to a three-dimensional surface, the region of convergence comprising an estimate of a first parameter of the user's eye; The method of any of Examples 81-85, comprising:

[0219] Example 87: A method as described in any of Examples 81-86, wherein the multiple images of the user's eye are provided with images associated with different gaze directions of the user's eye.

[0220] Example 88: The method of any of Examples 81-87, further using a gaze target.

[0221] Example 89: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content; an eye tracking camera configured to image the eye of a user; processing electronics in communication with the display and the eye tracking camera, receiving captured images of a plurality of pairs of the user's eyes from an eye tracking camera; obtaining estimates of the user's eye parameters based at least in part on the respective pairs of captured images for each pair of images received from the eye tracking camera; determining a three-dimensional surface based on estimated parameters of the user's eyes obtained based on the captured images of a plurality of pairs of the user's eyes received from the eye tracking camera; Identifying the center of curvature of the 3D surface and obtaining estimates of additional parameters of the user's eye; processing electronics configured to A display system comprising:

[0222] Example 90: A display system as described in Example 89, wherein the processing electronics are configured to adapt a three-dimensional surface to estimated parameters of a user's eyes obtained based on multiple pairs of captured images of the user's eyes received from an eye tracking camera.

[0223] Example 91: To obtain an estimate of an ocular parameter of a user based, at least in part, on the individual pairs of captured images, the processing electronics: determining a first vector based on a first image received from the eye tracking camera; determining a second vector based on a second image received from the eye tracking camera, the first and second images corresponding to one of the pair of images; identifying a region of convergence between paths extending in the direction of the first vector and the second vector to obtain an estimate of an eye parameter of the user; The display system of Example 89 or 90, configured as follows:

[0224] Example 92: A method for detecting a glimmer of light using a light source, comprising: To determine a first vector based on a first image of the pair of captured images, the processing electronics defining a first plane that includes a location of the eye tracking camera, a location of the first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a location of the eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; identifying a region of convergence of the first plane and the second plane, the region of convergence extending along a direction of the first vector; The display system of Example 91, configured as follows:

[0225] Example 93: To determine a second vector based on a second image in each pair of captured images, the processing electronics: defining a third plane that includes a location of the eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes a location of the eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along a direction of the second vector; The display system of Example 92, configured as follows:

[0226] Example 94: A display system described in any of Examples 89-93, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the additional parameter.

[0227] Example 95: A display system described in any of Examples 89-94, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to originate from different depths and display the virtual image content in the user's field of view, or the display is configured to project light into the user's eye that diverges and is collimated to display virtual image content in the user's field of view that appears to originate from different depths.

[0228] Example 96: A display system described in any of the above examples, wherein at least a portion of the display is transparent and positioned in front of the user's eyes when the user wears the head-mounted display so as to transmit light from a portion of the environment in front of the user and the head-mounted display to the user's eyes and provide a view of that portion of the environment in front of the user and the head-mounted display.

[0229] Example 97: A display system configured to project light to an eye of a user to display virtual image content within a field of view of the user, a frame configured to be supported on a user's head; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; at least one eye tracking camera configured to image the user's eye; A plurality of light emitters; processing electronics in communication with the display and the eye tracking camera, receiving images of the user's eye captured by at least one eye tracking camera at a first and second location, where flashing reflections of different light emitters are observable in the images of the eye captured by the eye tracking camera; estimating a location of a parameter of the user's eye based on a location of the flash of light in the image produced by the at least one eye tracking camera and based on a location of the at least one eye tracking camera and a location of an emitter that produced the individual flash of light; processing electronics configured to A display system comprising:

[0230] Example 98: The processing electronics comprises: determining a first direction based on a location of the flash of light reflection in one or more images produced by the at least one eye tracking camera and based on a first location of the at least one eye tracking camera and a location of an emitter that produced the flash of light reflection; determining a second direction based on a location of the flash reflection in one or more images produced by the at least one eye tracking camera and based on a second location of the at least one eye tracking camera and a location of an emitter that produced the flash reflection; The display system of Example 97, configured as follows:

[0231] Example 99: The processing electronics comprises: defining a first plane including a first location of at least one eye tracking camera, a location of a first flash of light reflection, and a location of a light emitter corresponding to the first flash of light reflection; defining a second plane that includes a first location of the at least one eye tracking camera, a location of the second flash of light reflection, and a location of a light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along a first direction; 99. A display system as described in Example 98, configured to determine a first direction by:

[0232] Example 100: The processing electronics comprises: defining a third plane that includes a second location of the at least one eye tracking camera, a location of the third flash of light reflection, and a location of a light emitter corresponding to the third flash of light reflection; defining a fourth plane including a second location of the at least one eye tracking camera, a location of the fourth flash of light reflection, and a location of a light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; A display system as described in Example 99, configured to determine a second direction by:

[0233] Example 101: A display system described in any of the above examples, wherein the processing electronics is configured to estimate a location of the parameter of the user's eye based on the first and second directions.

[0234] Example 102: The processing electronics comprises: determining the first direction based on at least one first image received from a first location of at least one eye tracking camera; determining the second direction based on at least one second image received from a second location of the at least one eye tracking camera, the first and second directions converging towards a region; 4. A display system according to any of the preceding embodiments, configured as follows:

[0235] Example 103: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of the parameter based on the convergence of the first and second directions.

[0236] Example 104: A display system described in any of the above examples, wherein the processing electronics is configured to estimate the parameter of the user's eye by identifying an area of ​​convergence of the first and second directions.

[0237] Example 105: A display system described in any of the above examples, wherein the processing electronics are configured to obtain an estimate of additional parameters of the user's eye based on multiple determinations of other parameters of the user's eye for different eye postures.

[0238] Example 106: A display system described in any of the above examples, wherein the processing electronics are configured to determine a locus of points corresponding to estimates of the user's eye parameters for different eye postures.

[0239] Example 107: A display system as described in Example 106, wherein the processing electronics are configured to obtain estimates of additional parameters of the user's eye based on the locus of points corresponding to estimates of other parameters of the user's eye for different eye postures.

[0240] Example 108: A display system as described in Example 106 or 107, wherein the processing electronics are configured to determine a surface and obtain an estimate of the parameter of the user's eye based on the locus of the points.

[0241] Example 109: A display system as described in Example 106 or 107, wherein the processing electronics are configured to obtain an estimate of an additional parameter of the user's eye by determining a surface and estimating the center of curvature of the surface based on the locus of points.

[0242] Example 110: A display system as described in Example 106 or 107, wherein the processing electronics are configured to obtain an estimate of the parameter of the user's eye by determining a surface based on the locus of points and determining a region where multiple normals to the surface converge.

[0243] Example 111: A display system described in any of Examples 108, 109, or 110, wherein the processing electronics are configured to fit the surface to the locus of points and acquire the surface.

[0244] Example 112: A display system described in any of the above examples, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the additional parameter.

[0245] Example 113: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view at at least one different amount of divergence and collimation, such that the displayed virtual image content appears to originate from different depths.

[0246] Example 115: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view so that the displayed virtual image content appears to originate from different depths.

[0247] Example 116: A display system described in any of the above examples, wherein the display is configured to project light into the user's eye at different amounts of divergence so that the displayed virtual image content appears to originate from different depths, and to display virtual image content in the user's field of view.

[0248] Example 117: A display system described in any of the above examples, wherein the display is configured to project diverging light into the user's eye and collimated light to display virtual image content in the user's field of view that appears to originate from different depths.

[0249] Example 118: A display system described in any of the above examples, wherein at least a portion of the display is transparent and positioned in front of the user's eyes when the user wears the head-mounted display so as to transmit light from a portion of the environment in front of the user and the head-mounted display to the user's eyes and provide a view of that portion of the environment in front of the user and the head-mounted display.

[0250] Example 119: A display system described in any of the above examples, wherein the first parameter includes a center of rotation of the eye.

[0251] Example 120: A display system according to any of the above examples, wherein the second parameter comprises a center of corneal curvature.

[0252] Example 121: A display system according to any of the above examples, wherein the parameter comprises a center of corneal curvature.

[0253] Example 122: A display system described in any of the above examples, wherein the additional parameters include a center of rotation of the eye.

[0254] Any of the above embodiments can be combined. Additionally, any of the above embodiments can be integrated with a head mounted display. Additionally, any of the above embodiments can be implemented with one or more depth planes, such as a single depth plane and / or one or more variable depth planes (e.g., one or more elements with variable focusing power that provide accommodation cues that vary over time).

[0255] Additionally, disclosed herein are devices and methods for determining various values, parameters, such as, but not limited to, anatomical, optical, and geometric features, locations, and orientations. Examples of such parameters include, but are not limited to, for example, the center of rotation of the eye, the center of corneal curvature, the pupil center, the border of the pupil, the center of the iris, the border of the iris, the border of the limbus, the optical axis of the eye, the visual axis of the eye, and the center of gaze. In addition, in some implementations, the center of corneal curvature or the center of cornea refers to the center of curvature of a portion of the cornea or the center of curvature of a spherical surface that coincides with a portion of the surface of the cornea. For example, in some implementations, the center of corneal curvature or the center of cornea refers to the center of curvature of the corneal apex or the center of curvature of a spherical surface that coincides with a portion of the surface of the cornea. In addition, the determination of such values, parameters, etc. as enumerated herein includes estimates thereof and does not necessarily have to precisely match the actual values. For example, determinations of the center of rotation of the eye, center of corneal curvature, center or boundary of the pupil or iris, boundary of the limbus, optical axis of the eye, visual axis of the eye, center of gaze, etc. may be estimates, approximations, or close values ​​that are not identical to actual (e.g., anatomical, optical, or geometric) values ​​or parameters. In some cases, for example, root-mean-square estimation techniques are used to obtain estimates of such values. As an example, certain techniques described herein relate to identifying locations or points where rays or vectors intersect. However, such rays or vectors may not intersect. In this example, the locations or points may be estimated. For example, the locations or points may be determined based on root-mean-square or other estimation techniques (e.g., the locations or points may be estimated to be close or nearest to the rays or vectors). Other processes may also be used to estimate, approximate, or otherwise provide values ​​that may not match actual values. Thus, the terms "determine" and "estimate" or "determined" and "estimated" are used interchangeably herein. References to such determined values ​​may therefore include estimates, approximations, or values ​​that are close to the actual values.Thus, references to determinations of parameters or values ​​above or anywhere herein should not be limited to precise actual values, but may include estimates, approximations, or values ​​close to those values.

[0256] Details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages will become apparent from the description, drawings, and claims. Neither this summary nor the following detailed description purports to define or limit the scope of the inventive subject matter. The present invention provides, for example, the following items. (Item 1) 1. A display system configured to project light to an eye of a user for displaying virtual image content within a field of view of the user, the display system comprising: a frame configured to be supported on a head of the user; and a head mounted display disposed on the frame, the display configured to project light into the user's eye and display virtual image content in the user's field of view; first and second eye tracking cameras configured to image the user's eyes; A plurality of light emitters; processing electronics in communication with the display and the first and second eye tracking cameras, the processing electronics comprising: receiving images of the user's eye captured by the first and second eye tracking cameras, wherein flashing reflections of different light emitters are observable in the eye images captured by the first and second eye tracking cameras; estimating a location of a center of corneal curvature of the user's eye based on a location of the flash of light in the images produced by both the first and second eye tracking cameras, and based on the location of both the first and second eye tracking cameras and the location of the emitter that produced the respective flash of light; and processing electronics configured to: A display system comprising: (Item 2) The processing electronics include: determining a first direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the first eye tracking camera and based on a location of the first eye tracking camera and a location of the emitter that produced the flash of light; determining a second direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the second eye tracking camera and based on a location of the second eye tracking camera and a location of the emitter that produced the flash of light; 2. The display system of item 1, configured to: (Item 3) The processing electronics include: defining a first plane that includes the first eye tracking camera, a location of a first flash of light reflection, and a location of the light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera, a location of a second flash of light reflection, and a location of the light emitter corresponding to the second flash of light reflection; determining a region of convergence of the first plane and the second plane, the region of convergence extending along the first direction; 3. The display system of claim 2, configured to determine the first direction by: (Item 4) The processing electronics include: defining a third plane that includes the second eye tracking camera, a location of a third flash of light reflection, and a location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera, a location of a fourth flash of light reflection, and a location of the light emitter corresponding to the fourth flash of light reflection; determining a region of convergence of the third plane and the fourth plane, the region of convergence extending along the second direction; 4. The display system of claim 3, configured to determine the second direction by: (Item 5) The display system of any of the preceding items, wherein the processing electronics is configured to estimate a location of the center of corneal curvature of the user's eye based on the first and second directions toward the center of corneal curvature of the user's eye. (Item 6) The processing electronics include: determining the first direction along which a center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; determining, based on at least one second image received from the second eye tracking camera, a second direction along which a center of corneal curvature of the user's eye is estimated to be located, the first and second directions converging towards a region; 2. The display system of claim 1, configured to: (Item 7) 2. The display system of claim 1, wherein the processing electronics is configured to obtain an estimate of a center of corneal curvature of the user's eye based on the convergence of the first and second directions. (Item 8) The display system of any of the preceding items, wherein the processing electronics is configured to estimate the location of the center of corneal curvature of the user's eye by identifying an area of ​​convergence of the first and second directions toward the center of corneal curvature of the user's eye. (Item 9) 2. The display system of claim 1, wherein the processing electronics is configured to obtain an estimate of a center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for different eye postures. (Item 10) 2. The display system of claim 1, wherein the processing electronics is configured to determine a locus of points corresponding to estimates of centers of corneal curvature of the user's eye for different eye postures. (Item 11) Item 11. The display system of item 10, wherein the processing electronics is configured to obtain an estimate of a center of rotation of the user's eye based on the locus of points corresponding to estimates of the center of corneal curvature of the user's eye for different eye postures. (Item 12) Item 12. The display system of item 10 or 11, wherein the processing electronics is configured to determine a surface and obtain an estimate of a center of rotation of the user's eyes based on the locus of points. (Item 13) Item 12. The display system of item 10 or 11, wherein the processing electronics is configured to obtain an estimate of a center of rotation of the user's eyes by determining a surface and estimating a center of curvature of the surface based on the locus of points. (Item 14) Item 12. The display system of item 10 or 11, wherein the processing electronics is configured to determine a surface based on the locus of points and obtain an estimate of a center of rotation of the user's eyes by determining a region where multiple normals to the surface converge. (Item 15) Item 15. The display system of any of items 12, 13, or 14, wherein the processing electronics is configured to fit the surface to the locus of points to obtain the surface. (Item 16) A display system as described in any of the preceding items, wherein the processing electronics are configured to render a virtual image to be presented to the user's eye using a rendering camera, the rendering camera having a position determined by the center of rotation. (Item 17) A display system as described in any of the preceding items, wherein the display is configured to project light into the user's eye and display virtual image content in the user's field of view so that the displayed virtual image content appears to originate from different depths. (Item 18) A display system as described in any of the above items, wherein the display is configured to project light into the user's eye at at least one different amount of divergence so that the displayed virtual image content appears to originate from different depths, and to display virtual image content in the user's field of view. (Item 19) A display system as described in any of the preceding items, wherein the display is configured to project diverging light into the user's eye and collimated light into the user's eye to display virtual image content in the user's field of view that appears to originate from different depths. [Brief description of the drawings]

[0257] [Figure 1]FIG. 1 depicts an illustration of a mixed reality scenario involving a virtual reality object and a physical object viewed by a person.

[0258] [Diagram 2] FIG. 2 illustrates diagrammatically an embodiment of a wearable system.

[0259] [Diagram 3] FIG. 3 diagrammatically illustrates example components of a wearable system.

[0260] [Figure 4] FIG. 4 diagrammatically illustrates an example of a waveguide stack of a wearable device for outputting image information to a user.

[0261] [Diagram 5] FIG. 5 illustrates diagrammatically an embodiment of an eye.

[0262] [Figure 5A] FIG. 5A diagrammatically illustrates an exemplary coordinate system for determining eye posture.

[0263] [Figure 6] FIG. 6 is a schematic diagram of a wearable system including an eye tracking system.

[0264] [Figure 7A] FIG. 7A is a block diagram of a wearable system that may include an eye tracking system.

[0265] [Figure 7B] FIG. 7B is a block diagram of a rendering controller in a wearable system.

[0266] [Figure 8] FIG. 8 illustrates an example eye, including the optical and visual axes of the eye and the center of rotation of the eye.

[0267] [Figure 9A] 9A-E illustrate diagrams of example configurations of a wearable system for capturing eye image data for use by an eye tracking module. [Figure 9B] 9A-E illustrate diagrams of example configurations of a wearable system for capturing eye image data for use by an eye tracking module. [Figure 9C] 9A-E illustrate diagrams of example configurations of a wearable system for capturing eye image data for use by an eye tracking module. [Figure 9D] 9A-E illustrate diagrams of example configurations of a wearable system for capturing eye image data for use by an eye tracking module. [Figure 9E] 9A-E illustrate diagrams of example configurations of a wearable system for capturing eye image data for use by an eye tracking module.

[0268] [Figure 10] FIG. 10 shows a graphical illustration of an exemplary center of rotation (CoR) determination process that may be performed by the eye tracking module.

[0269] [Figure 11] FIG. 11 shows an example image of a phosphene on any eye that is used by the eye tracking module to determine an estimated center of rotation.

[0270] [Figure 12A] 12A-D illustrate steps in an exemplary determination of a first plane that includes the location of the first flash of light, a camera that captures an image of the flash of light, and an illumination source that produces the first flash of light. [Figure 12B] 12A-D illustrate steps in an exemplary determination of a first plane that includes the location of the first flash of light, a camera that captures an image of the flash of light, and an illumination source that produces the first flash of light. [Figure 12C]12A-D illustrate steps in an exemplary determination of a first plane that includes the location of the first flash of light, a camera that captures an image of the flash of light, and an illumination source that produces the first flash of light. [Figure 12D] 12A-D illustrate steps in an exemplary determination of a first plane that includes the location of the first flash of light, a camera that captures an image of the flash of light, and an illumination source that produces the first flash of light.

[0271] [Figure 13A] 13A-D illustrate steps in an exemplary determination of a second plane that includes the location of the second flash, the camera, and the illumination source that produces the second flash. [Figure 13B] 13A-D illustrate steps in an exemplary determination of a second plane that includes the location of the second flash, the camera, and the illumination source that produces the second flash. [Figure 13C] 13A-D illustrate steps in an exemplary determination of a second plane that includes the location of the second flash, the camera, and the illumination source that produces the second flash. [Figure 13D] 13A-D illustrate steps in an exemplary determination of a second plane that includes the location of the second flash, the camera, and the illumination source that produces the second flash.

[0272] [Figure 14A] Figures 14A-C illustrate the intersection between the first plane of Figures 12A-D and the second plane of Figures 13A-D, which corresponds to a vector along which the corneal center may lie. [Figure 14B] Figures 14A-C illustrate the intersection between the first plane of Figures 12A-D and the second plane of Figures 13A-D, which corresponds to a vector along which the corneal center may lie. [Figure 14C] Figures 14A-C illustrate the intersection between the first plane of Figures 12A-D and the second plane of Figures 13A-D, which corresponds to a vector along which the corneal center may lie.

[0273] [Figure 15A] 15A-15B illustrate multiple vectors along which the corneal center may lie, acquired using multiple cameras, that may converge or intersect at a location corresponding to or close to the corneal center. [Figure 15B] 15A-15B illustrate multiple vectors along which the corneal center may lie, acquired using multiple cameras, that may converge or intersect at a location corresponding to or close to the corneal center.

[0274] [Figure 16A] 16A-16C illustrate example steps in an example determination of a vector along which the corneal center may lie using a shared illumination source among multiple cameras. [Figure 16B] 16A-16C illustrate example steps in an example determination of a vector along which the corneal center may lie using a shared illumination source among multiple cameras. [Figure 16C] 16A-16C illustrate example steps in an example determination of a vector along which the corneal center may lie using a shared illumination source among multiple cameras.

[0275] [Figure 17A] 17A-17B show an estimation of the 3D surface based on the calculated corneal center. [Figure 17B] 17A-17B show an estimation of the 3D surface based on the calculated corneal center.

[0276] [Figure 18A] 18A and 18B show an example estimate of the center of rotation at the convergence of multiple surface normal vectors normal to a 3D surface, calculated based on the corneal center. [Figure 18B] 18A and 18B show an example estimate of the center of rotation at the convergence of multiple surface normal vectors normal to a 3D surface, calculated based on the corneal center.

[0277] [Figure 19-1] 19A-1 and 19A-2 illustrate an example surface fit to a selected estimated corneal center.

[0278] [Figure 19-2] 19B-1 and 19B-2 show example surface normal vectors that may be normal to a surface that is fitted to an estimated corneal center.

[0279] [Figure 19-3] 19C-1 and 19C-2 illustrate the CoR region estimated based on the points of intersection of surface normal vectors.

[0280] [Figure 19-4] 19D-1 and 19D-2 illustrate exemplary fitting surfaces to different selections of the center of the cornea.

[0281] [Figure 20] FIG. 20 illustrates an exemplary rotation center extraction process that may be implemented by the eye tracking module.

[0282] [Figure 21] FIG. 21 illustrates an example eye tracking process that may use the process of FIG. 20 to determine an estimated center of rotation using the center of corneal curvature. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0283] Throughout the drawings, reference numbers may be reused to indicate correspondence between referenced elements. The drawings are provided to illustrate example embodiments described herein and are not intended to limit the scope of the present disclosure. (Detailed Description)

[0284] Reference is now made to the drawings in which like reference numbers refer to like parts throughout, and in which, unless otherwise indicated, the drawings are schematic and are not necessarily drawn to scale. A. Example of 3D display for wearable system

[0285] A wearable system (also referred to herein as an augmented reality (AR) system) can be configured to present 2D or 3D virtual images to a user. The images may be still images, frames of video, or videos, in combination or the like. At least a portion of the wearable system can be implemented on a wearable device, which may present a VR, AR, or MR environment, alone or in combination, for user interaction. A wearable device can be used synonymously with an AR device (ARD). Additionally, for purposes of this disclosure, the term "AR" is used synonymously with the term "MR."

[0286] Figure 1 depicts an illustration of a mixed reality scenario with certain virtual reality objects and certain physical objects viewed by a person. In Figure 1, an MR scene 100 is depicted in which a user of the MR technology sees a real-world park-like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120. In addition to these items, the user of the MR technology also perceives that they "see" a robotic figure 130 standing on the real-world platform 120, and a flying cartoon-like avatar character 140 that appears to be an anthropomorphic bumblebee, although these elements do not exist in the real world.

[0287] In order for a 3D display to produce a true depth sensation, and more specifically, a simulated sensation of surface depth, it may be desirable to generate, for each point in the display's field of view, an accommodation response that corresponds to that point's virtual depth. If the accommodation response to a display point does not correspond to that point's virtual depth as determined by the binocular depth cues of convergence and stereopsis, the human eye may experience accommodation conflicts, resulting in unstable imaging, deleterious eye strain, headaches, and, in the absence of accommodative information, a near-complete lack of surface depth.

[0288] VR, AR, and MR experiences can be provided by a display system having a display in which images corresponding to multiple depth planes are provided to a viewer. The images may be different for each depth plane (e.g., providing slightly different presentations of a scene or object) and may be focused separately by the viewer's eyes, thereby serving to provide depth cues to the user based on the ocular accommodation required to focus on different image features of a scene located on different depth planes, or based on observing different image features on different depth planes that are out of focus. As discussed elsewhere herein, such depth cues provide a believable perception of depth.

[0289] FIG. 2 illustrates an example of a wearable system 200, which can be configured to provide an AR / VR / MR scene. The wearable system 200 can also be referred to as an AR system 200. The wearable system 200 includes a display 220 and various mechanical and electronic modules and systems to support the functionality of the display 220. The display 220 can be coupled to a frame 230, which is wearable by a user, wearer, or viewer 210. The display 220 can be positioned in front of the eyes of the user 210. The display 220 can present AR / VR / MR content to the user. The display 220 can comprise a head-mounted display (HMD) worn on the user's head.

[0290] In some embodiments, a speaker 240 is coupled to the frame 230 and positioned adjacent the user's ear canal (in some embodiments, another speaker, not shown, is positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). The display 220 can include an audio sensor (e.g., a microphone) 232 to detect audio streams from the environment and capture ambient sounds. In some embodiments, one or more other audio sensors, not shown, are positioned to provide stereo sound reception. Stereo sound reception can be used to determine the location of a sound source. The wearable system 200 can perform voice or speech recognition on the audio stream.

[0291] The wearable system 200 may include an outwardly facing imaging system 464 (shown in FIG. 4 ) that observes the world in the user's surrounding environment. The wearable system 200 may also include an inwardly facing imaging system 462 (shown in FIG. 4 ) that may track the user's eye movements. The inwardly facing imaging system may track either one eye's movements or both eyes' movements. The inwardly facing imaging system 462 may be mounted to the frame 230 and may be in electrical communication with the processing module 260 or 270 that may process image information obtained by the inwardly facing imaging system and determine, for example, the pupil diameter or orientation of the user's 210 eyes, eye movements, or eye posture. The inwardly facing imaging system 462 may include one or more cameras. For example, at least one camera may be used to image each eye. Images obtained by the cameras may be used to determine pupil size or eye posture for each eye separately, thereby allowing the presentation of image information to each eye to be dynamically adjusted for that eye.

[0292] As an example, the wearable system 200 can obtain an image of the user's posture using an outwardly facing imaging system 464 or an inwardly facing imaging system 462. The image may be a still image, a frame of a video, or a video.

[0293] The display 220 can be operably coupled (250) to a local data processing module 260, which can be mounted in a variety of configurations, such as fixedly attached to the frame 230, by wired leads or a wireless connection, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user 210 (e.g., in a backpack configuration, in a belt-attached configuration).

[0294] The local processing and data module 260 may comprise a hardware processor and digital memory such as non-volatile memory (e.g., flash memory), both of which may be utilized to aid in processing, caching, and storing data. The data may include (a) data captured from sensors (e.g., that may be operatively coupled to the frame 230 or otherwise attached to the user 210), such as image capture devices (e.g., cameras in the inward-facing and / or outward-facing imaging systems), audio sensors (e.g., microphones), inertial measurement units (IMUs), accelerometers, compasses, global positioning system (GPS) units, wireless devices, or gyroscopes, or (b) data obtained or processed using the remote processing module 270 or remote data repository 280, possibly for passing to the display 220 after processing or retrieval. The local processing and data module 260 may be operatively coupled to a remote processing module 270 or a remote data repository 280 by a communications link 262 or 264, such as via a wired or wireless communication link, such that these remote modules are available as resources to the local processing and data module 260. In addition, the remote processing module 280 and the remote data repository 280 may be operatively coupled to each other.

[0295] In some embodiments, the remote processing module 270 may comprise one or more processors configured to analyze and process the data or image information. In some embodiments, the remote data repository 280 may comprise a digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, allowing for fully autonomous use from the remote module. B. Exemplary Components of a Wearable System

[0296] FIG. 3 diagrammatically illustrates example components of a wearable system. FIG. 3 illustrates a wearable system 200, which may include a display 220 and a frame 230. A blowup 202 diagrammatically illustrates various components of the wearable system 200. In some implementations, one or more of the components illustrated in FIG. 3 may be part of the display 220. The various components, alone or in combination, may collect various data associated with a user of the wearable system 200 or the user's environment (e.g., auditory or visual data, etc.). It should be understood that other embodiments may have additional or fewer components depending on the application for which the wearable system is used. However, FIG. 3 provides a basic idea of ​​some of the various components and the types of data that may be collected, analyzed, and stored through the wearable system.

[0297] FIG. 3 illustrates an exemplary wearable system 200, which may include a display 220. The display 220 may include a display lens 226, which may be mounted to a housing or frame 230 that corresponds to the user's head, or a frame 230. The display lens 226 may include one or more transparent mirrors positioned in front of the user's eyes 302, 304 by the housing 230, and may be configured to bounce the projected light 338 into the eyes 302, 304 and facilitate beam shaping while also allowing transmission of at least some light from the local environment. The wavefront of the projected light beam 338 may be bent or focused to match a desired focal length of the projected light. As illustrated, two wide-field machine vision cameras 316 (also referred to as world cameras) are coupled to the housing 230 and may image the environment around the user. These cameras 316 may be dual capture visible / non-visible (e.g., infrared) light cameras. The camera 316 may be part of an outward-facing imaging system 464 shown in Figure 4. Images acquired by the world camera 316 may be processed by a pose processor 336. For example, the pose processor 336 may implement one or more object recognizers 708 to identify a pose of the user or another person in the user's environment, or to identify physical objects in the user's environment.

[0298] Continuing with reference to FIG. 3, a pair of scanning laser shaped wavefront (e.g., for depth) light projection modules with display mirrors and optics configured to project light 338 into the eyes 302, 304 are shown. The depicted diagram also shows two miniature infrared cameras 324 paired with infrared light sources 326 (such as light emitting diodes "LEDs") configured to track the user's eyes 302, 304 and support rendering and user input. The cameras 324 may be part of an inwardly facing imaging system 462 shown in FIG. 4. The wearable system 200 may further feature a sensor assembly 339, which may include X, Y, and Z axis accelerometer capabilities and magnetic compass and X, Y, and Z axis gyroscope capabilities, and may preferably provide data at a relatively high frequency, such as 200 Hz. The sensor assembly 339 may be part of an IMU, as described with reference to FIG. 2. The depicted system 200 may also include a head pose processor 336, such as an ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or ARM processor (Advanced Reduced Instruction Set Machine), which may be configured to calculate real-time or near real-time user head pose from the wide FOV image information output from the capture device 316. The head pose processor 336 may be a hardware processor and may be implemented as part of the local processing and data module 260 shown in FIG.

[0299] The wearable system may also include one or more depth sensors 234. The depth sensors 234 may be configured to measure a distance between an object in the environment and the wearable device. The depth sensors 234 may include a laser scanner (e.g., LIDAR), an ultrasonic depth sensor, or a depth-sensing camera. In some implementations where the camera 316 has depth-sensing capabilities, the camera 316 may also be considered a depth sensor 234.

[0300] Also shown is a processor 332 configured to perform digital or analog processing and derive attitude from gyroscope, compass, or accelerometer data from the sensor assembly 339. The processor 332 may be part of a local processing and data module 260, shown in FIG. 2. The wearable system 200 may also include a positioning system, such as, for example, a GPS 337 (Global Positioning System), as shown in FIG. 3, to aid in the attitude and positioning analysis. In addition, the GPS may further provide remote-based (e.g., cloud-based) information about the user's environment. This information may be used to recognize objects or information within the user's environment.

[0301] The wearable system may combine data obtained by the GPS 337 and a remote computing system (e.g., the remote processing module 270, another user's ARD, etc.), which may provide more information about the user's environment. As an example, the wearable system may determine the user's location based on the GPS data and retrieve a world map (e.g., by communicating with the remote processing module 270) that includes virtual objects associated with the user's location. As another example, the wearable system 200 may monitor the environment using a world camera 316 (which may be part of the outward-facing imaging system 464 shown in FIG. 4). Based on the images obtained by the world camera 316, the wearable system 200 may detect objects in the environment (e.g., by using one or more object recognizers). The wearable system may further interpret characters using data obtained by the GPS 337.

[0302] The wearable system 200 may also include a rendering engine 334, which may be configured to provide local rendering information to the user for the user's view of the world and facilitate operation of the scanner and imaging into the user's eye. The rendering engine 334 may be implemented by a hardware processor (e.g., a central processing unit or a graphics processing unit, etc.). In some embodiments, the rendering engine is part of the local processing and data module 260. The rendering engine 334 may be communicatively coupled (e.g., via wired or wireless links) to other components of the wearable system 200. For example, the rendering engine 334 may be coupled to the eye camera 324 via communication link 274 and to the projection subsystem 318 (which may project light into the user's eyes 302, 304 via a scanning laser array in a manner similar to a retinal scanning display) via communication link 272. The rendering engine 34 may also communicate with other processing units, such as, for example, the sensor pose processor 32 and the image pose processor 336, via links 276 and 294, respectively.

[0303] A camera 324 (e.g., a small infrared camera) may be utilized to track eye pose and support rendering and user input. Some example eye poses may include where the user is looking or the depth at which they are focused (which may be estimated using eye convergence and divergence). A GPS 337, gyroscope, compass, and accelerometer 339 may be utilized to provide a rough or fast pose estimate. One or more of the cameras 316 may obtain images and poses that, together with data from associated cloud computing resources, may be utilized to map the local environment and share the user view with others.

[0304] The exemplary components depicted in FIG. 3 are for illustrative purposes only. Multiple sensors and other functional modules are shown together for ease of illustration and description. Some embodiments may include only one or a subset of these sensors or modules. Furthermore, the locations of these components are not limited to the locations depicted in FIG. 3. Some components may be mounted or stored within other components, such as belt-mounted components, handheld components, or helmet components. As an example, the image pose processor 336, the sensor pose processor 332, and the rendering engine 334 may be located within a beltpack and configured to communicate with other components of the wearable system via wireless communication, such as ultra-wideband, Wi-Fi, Bluetooth, or via wired communication. The depicted housing 230 is preferably head-mountable and wearable by a user. However, some components of the wearable system 200 may be worn on other parts of the user's body. For example, the speaker 240 may be inserted into the user's ear to provide sound to the user.

[0305] With regard to projecting light 338 into the user's eyes 302, 304, in some embodiments, the camera 324 may be utilized to measure where the center of the user's eyes is geometrically converged, which generally coincides with the location of the eye's focus or "depth of focus." The three-dimensional surface of all points at which the eyes converge may be referred to as the "monocular locus." The focal length may take on a finite number of depths or may vary infinitely. Light projected from the vergence distance appears focused on the subject's eyes 302, 304, while light in front of or behind the vergence distance is blurred. Examples of wearable systems and other display systems of the present disclosure are also described in U.S. Patent Publication No. 2016 / 0270656, which is incorporated herein by reference in its entirety.

[0306] The human visual system is complex and providing a realistic perception of depth is difficult. A viewer of an object may perceive the object as three-dimensional due to a combination of vergence and accommodation. Vergence movements of the two eyes relative to one another (e.g., rotational movement of the pupils toward or away from one another to converge the gaze of the eyes and fixate on an object) are closely coupled to the focusing of the eye's lenses (or "accommodation"). Under normal conditions, a change in the focus of the eye's lenses or accommodation of the eye to change focus from one object to another at a different distance will automatically produce a corresponding change in vergence to the same distance, a relationship known as the "accommodation-vergence reflex." Similarly, a change in vergence will induce a corresponding change in accommodation under normal conditions. A display system that provides a better match between accommodation and vergence may produce a more realistic and comfortable simulation of a three-dimensional image.

[0307] Furthermore, spatially coherent light with a beam diameter of less than about 0.7 millimeters can be correctly resolved by the human eye regardless of where the eye is focused. Thus, to create the illusion of proper depth of focus, eye vergence-divergence movements may be tracked using the camera 324, and the rendering engine 334 and projection subsystem 318 may be utilized to render all objects on or near the monocular locus in focus and all other objects variable degrees out of focus (e.g., using intentionally created blur). Preferably, the system 220 renders to the user at a frame rate of about 60 frames per second or greater. As explained above, preferably the camera 324 may be utilized for eye tracking, and software may be configured to take up not only vergence-divergence geometry, but also focus location cues to serve as user input. Preferably, such a display system is configured with a suitable brightness and contrast for daytime or nighttime use.

[0308] In some embodiments, the display system preferably has a latency of less than about 20 milliseconds for visual object alignment, an angular alignment of less than about 0.1 degrees, and a resolution of about 1 arc minute, which is believed to be approximately the limit of the human eye, without being limited by theory. The display system 220 may be integrated with a localization system, which may involve a GPS element, optical tracking, a compass, an accelerometer, or other data sources to aid in position and attitude determination. The localization information may be utilized to facilitate accurate rendering within the user's view of the relevant world (e.g., such information would facilitate the glasses knowing their location relative to the real world).

[0309] In some embodiments, the wearable system 200 is configured to display one or more virtual images based on the accommodation of the user's eyes. Unlike traditional 3D display approaches that force the user to focus where the image is projected, in some embodiments, the wearable system is configured to automatically vary the focus of the projected virtual content to allow for more comfortable viewing of one or more images presented to the user. For example, if the user's eyes have a current focus of 1m, the image may be projected to match the user's focus. If the user shifts the focus to 3m, the image is projected to match the new focus. Thus, rather than forcing a predetermined focus on the user, the wearable system 200 of some embodiments allows the user's eyes to function in a more natural manner.

[0310] Such a wearable system 200 may eliminate or reduce the incidence of eye strain, headaches, and other physiological symptoms typically observed with virtual reality devices. To accomplish this, various embodiments of the wearable system 200 are configured to project virtual images at variable focal distances through one or more variable focus elements (VFEs). In one or more embodiments, 3D perception may be achieved through a multi-plane focus system that projects images onto a fixed focal plane from the user. Other embodiments employ a variable planar focus, where the focal plane is moved back and forth in the z-direction to match the user's current state of focus.

[0311] In both the multi-plane focus system and the variable plane focus system, the wearable system 200 may employ eye tracking to determine the vergence-divergence movement of the user's eyes, determine the user's current focus, and project the virtual image at the determined focus. In other embodiments, the wearable system 200 includes a light modulator that variably projects a variably focused light beam in a raster pattern across the retina through a fiber scanner or other light generating source. Thus, the wearable system 200 display's ability to project images at variable focal distances not only facilitates accommodation for the user to view objects in 3D, but may also be used to compensate for the user's ocular anomalies, as further described in U.S. Patent Publication No. 2016 / 0270656, which is incorporated herein by reference in its entirety. In some other embodiments, the spatial light modulator may project the image to the user through various optical components. For example, as further described below, the spatial light modulator may project the image onto one or more waveguides, which then transmit the image to the user. C. Waveguide Stack Assembly

[0312] FIG. 4 illustrates an example of a waveguide stack for outputting image information to a user. The wearable system 400 includes a stack of waveguides or a stacked waveguide assembly 480 that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides 432b, 434b, 436b, 438b, 4400b. In some embodiments, the wearable system 400 can correspond to the wearable system 200 of FIG. 2, of which FIG. 4A diagrammatically illustrates some portions of the wearable system 200 in more detail. For example, in some embodiments, the waveguide assembly 480 can be integrated into the display 220 of FIG. 2.

[0313] 4, the waveguide assembly 480 may also include a number of features 458, 456, 454, 452 between the waveguides. In some embodiments, the features 458, 456, 454, 452 may be lenses. In other embodiments, the features 458, 456, 454, 452 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers or structures to form air gaps).

[0314] The waveguides 432b, 434b, 436b, 438b, 440b or multiple lenses 458, 456, 454, 452 may be configured to transmit image information to the eye with various levels of wavefront curvature or light beam divergence. Each waveguide level may be associated with a particular depth plane and configured to output image information corresponding to that depth plane. Image injection devices 420, 422, 424, 426, 428 may be utilized to inject image information into the waveguides 440b, 438b, 436b, 434b, 432b, respectively, which may be configured to disperse incident light across each individual waveguide for output toward the eye 410. Light exits output surfaces of image launch devices 420, 422, 424, 426, 428 and is launched into corresponding input edges of waveguides 440b, 438b, 436b, 434b, 432b. In some embodiments, a single beam of light (e.g., a collimated beam) may be launched into each waveguide to output an entire field of cloned collimated beams that are directed towards the eye 410 at a particular angle (and divergence) that corresponds to the depth plane associated with the particular waveguide.

[0315] In some embodiments, the image input devices 420, 422, 424, 426, 428 are discrete displays that each generate image information for input into a corresponding waveguide 440b, 438b, 436b, 434b, 432b, respectively. In some other embodiments, the image input devices 420, 422, 424, 426, 428 are the output of a single multiplexed display that may send image information to each of the image input devices 420, 422, 424, 426, 428, for example, via one or more optical conduits (such as fiber optic cables).

[0316] A controller 460 controls the operation of the stacked waveguide assembly 480 and the image input devices 420, 422, 424, 426, 428. The controller 460 includes programming (e.g., instructions in a non-transitory computer-readable medium) that coordinates the timing and provision of image information to the waveguides 440b, 438b, 436b, 434b, 432b. In some embodiments, the controller 460 may be a single integrated device or a distributed system connected by a wired or wireless communication channel. The controller 460 may be part of the processing module 260 or 270 (illustrated in FIG. 2) in some embodiments.

[0317] The waveguides 440b, 438b, 436b, 434b, 432b may be configured to propagate light within each individual waveguide by total internal reflection (TIR). The waveguides 440b, 438b, 436b, 434b, 432b may each be planar or have another shape (e.g., curved) with major top and bottom surfaces and edges extending between the major top and bottom surfaces. In the illustrated configuration, the waveguides 440b, 438b, 436b, 434b, 432b may each include a light extraction optical element 440a, 438a, 436a, 434a, 432a configured to extract light from the waveguide by redirecting light propagating within each individual waveguide out of the waveguide and outputting image information to the eye 410. The extracted light may also be referred to as out-coupled light, and the light extraction optical element may also be referred to as out-coupling optical element. The extracted light beam is output by the waveguide where the light propagating in the waveguide strikes the light redirecting element. The light extraction optical element (440a, 438a, 436a, 434a, 432a) may be, for example, a reflective or diffractive optical feature. Although shown disposed on the bottom major surface of the waveguides 440b, 438b, 436b, 434b, 432b for ease of explanation and clarity of drawing, in some embodiments the light extraction optical element 440a, 438a, 436a, 434a, 432a may be disposed on the top or bottom major surface, or may be disposed directly within the volume of the waveguides 440b, 438b, 436b, 434b, 432b. In some embodiments, the light extraction optical elements 440a, 438a, 436a, 434a, 432a may be formed in a layer of material that is attached to a transparent substrate and forms the waveguides 440b, 438b, 436b, 434b, 432b. In some other embodiments, the waveguides 440b, 438b, 436b, 434b, 432b may be a monolithic piece of material and the light extraction optical elements 440a, 438a, 436a, 434a, 432a may be formed on or within that piece of material.

[0318] Continuing with reference to FIG. 4, as discussed herein, each waveguide 440b, 438b, 436b, 434b, 432b is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 432b closest to the eye may be configured to deliver collimated light to the eye 410 as it is launched into such waveguide 432b. The collimated light may represent an optical infinity focal plane. The next upper waveguide 434b may be configured to send collimated light that passes through a first lens 452 (e.g., a negative lens) before it can reach the eye 410. The first lens 452 may be configured to generate a slight convex wavefront curvature so that the eye / brain interprets the light originating from the next upper waveguide 434b as originating from a first focal plane closer inward from optical infinity toward the eye 410. Similarly, the third upper waveguide 436b passes its output light through both a first lens 452 and a second lens 454 before reaching the eye 410. The combined refractive powers of the first and second lenses 452 and 454 may be configured to produce another incremental amount of wavefront curvature such that the eye / brain interprets the light emerging from the third waveguide 436b as originating from a second focal plane that is closer inward from optical infinity towards the person than was the light from the next upper waveguide 434b.

[0319] The other waveguide layers (e.g., waveguides 438b, 440b) and lenses (e.g., lenses 456, 458) are similarly configured, with the highest waveguide 440b in the stack being used to send its output through all of the lenses between it and the eye for an aggregate focal power representing the focal plane closest to the person. To compensate for the stack of lenses 458, 456, 454, 452 when viewing / interpreting light originating from the world 470 on the other side of the stacked waveguide assembly 480, a compensating lens layer 430 may be placed on top of the stack to compensate for the aggregate power of the lens stacks 458, 456, 454, 452 below. (The compensation lens layer 430 and stacked waveguide assembly 480 may be configured collectively such that light originating from the world 470 is transmitted to the eye 410 with substantially the same level of divergence (or collimation) that the light had when originally received by the stacked waveguide assembly 480.) Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairs. Both the light extraction optical elements of the waveguide and the focusing sides of the lenses may be static (e.g., not dynamic or electro-active). In some alternative embodiments, one or both may be dynamic using electro-active features.

[0320] Continuing with reference to FIG. 4, the light extraction optical elements 440a, 438a, 436a, 434a, 432a may be configured to redirect light out of their respective waveguides and output the light with an appropriate amount of divergence or collimation for a particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have different configurations of light extraction optical elements that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the light extraction optical elements 440a, 438a, 436a, 434a, 432a may be solid or surface features that may be configured to output light at a specific angle. For example, the light extraction optical elements 440a, 438a, 436a, 434a, 432a may be solid holograms, surface holograms, and / or diffraction gratings. Light extraction optical elements such as diffraction gratings are described in U.S. Patent Publication No. 2015 / 0178939, published June 25, 2015, which is incorporated by reference in its entirety.

[0321] In some embodiments, the light extraction optical elements 440a, 438a, 436a, 434a, 432a are diffractive features or "diffractive optical elements" (also referred to herein as "DOEs") that form a diffraction pattern. Preferably, the DOEs have a relatively low diffraction efficiency so that only a portion of the light in the beam is deflected toward the eye 410 at each intersection of the DOE, while the remainder continues traveling through the waveguide via total internal reflection. The light carrying the image information is thus split into several related exit beams that exit the waveguide at multiple locations, which can result in a very uniform pattern of exit emission toward the eye 304 for this particular collimated beam bouncing within the waveguide.

[0322] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which the droplets comprise a diffractive pattern within a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0323] In some embodiments, the number and distribution of depth planes or depths of field may be dynamically varied based on the pupil size or orientation of the viewer's eye. The depth of field may vary inversely with the pupil size of the viewer. As a result, as the size of the pupil of the viewer's eye decreases, the depth of field increases so that one plane that is indistinguishable because its location exceeds the focal depth of the eye may become distinguishable and appear more in focus with a reduction in pupil size and a corresponding increase in depth of field. Similarly, the number of spaced depth planes used to present different images to the viewer may be reduced with a reduced pupil size. For example, the viewer may not be able to clearly perceive the details of both the first and second depth planes at one pupil size without adjusting the accommodation of the eye from one depth plane to the other. However, these two depth planes may be simultaneously well-focused for the user at another pupil size without changing accommodation.

[0324] In some embodiments, the display system may vary the number of waveguides that receive image information based on a determination of a pupil size or orientation, or in response to receiving an electrical signal indicative of a particular pupil size or orientation. For example, if the user's eye is unable to distinguish between two depth planes associated with two waveguides, the controller 460 (which may be an embodiment of the local processing and data module 260) may be configured or programmed to stop providing image information to one of these waveguides. Advantageously, this may reduce the processing burden on the system, thereby increasing the responsiveness of the system. In embodiments in which the DOE for a waveguide is switchable between an on and off state, the DOE may be switched to the off state when the waveguide receives image information.

[0325] In some embodiments, it may be desirable to have the exit beam meet the condition of having a diameter less than the diameter of the viewer's eye. However, meeting this condition may be difficult in light of the variability in the size of the viewer's pupil. In some embodiments, this condition is met over a wide range of pupil sizes by varying the size of the exit beam in response to a determination of the size of the viewer's pupil. For example, as the pupil size decreases, the size of the exit beam may also decrease. In some embodiments, the exit beam size may be varied using a variable aperture.

[0326] The wearable system 400 may include an outwardly facing imaging system 464 (e.g., a digital camera) that images a portion of the world 470. This portion of the world 470 may be referred to as the field of view (FOV) of the world camera, and the imaging system 464 is sometimes also referred to as the FOV camera. The world camera FOV may or may not be the same as the FOV of the viewer 210, which encompasses the portion of the world 470 that the viewer 210 perceives at a given time. For example, in some circumstances the world camera FOV may be larger than the viewer 210 of the wearable system 400. The entire area available for viewing or imaging by the viewer may be referred to as the ocular field of view (FOR). The FOR may include a solid angle of 4π steradians surrounding the wearable system 400, since the wearer may move their body, head, or eyes and perceive virtually any direction in space. In other contexts, the wearer's movements may be more constrained and the wearer's FOR may correspondingly subtend a smaller solid angle. Images obtained from the outward-facing imaging system 464 can be used to track gestures (e.g., hand or finger gestures) made by the user, detect objects in the world 470 in front of the user, etc.

[0327] The wearable system 400 can include an audio sensor 232, e.g., a microphone, to capture ambient sound. As described above, in some embodiments, one or more other audio sensors can be positioned to provide stereo sound reception useful in determining the location of the speech source. The audio sensor 232 can comprise, as another example, a directional microphone, which can also provide such useful directional information regarding where the audio source is located. The wearable system 400 can use information from both the outwardly facing imaging system 464 and the audio sensor 230 in locating the speech source or to determine the active speaker at a particular moment, etc. For example, the wearable system 400 can use voice recognition, alone or in combination with a reflected image of the speaker (e.g., as seen in a mirror), to determine the identity of the speaker. As another example, the wearable system 400 can determine the location of the speaker in the environment based on sounds obtained from a directional microphone. The wearable system 400 can use speech recognition algorithms to analyze sounds originating from the speaker's location, determine the content of the speech, and use voice recognition techniques to determine the speaker's identity (e.g., name or other demographic information).

[0328] The wearable system 400 may also include an inwardly facing imaging system 466 (e.g., a digital camera) that observes user movements, such as eye and face movements. The inwardly facing imaging system 466 may be used to capture images of the eye 410 and determine the size and / or orientation of the pupil of the eye 304. The inwardly facing imaging system 466 may be used to obtain images for use in determining the direction the user is looking (e.g., eye pose) or for biometric identification of the user (e.g., via iris identification). In some embodiments, at least one camera may be utilized for each eye independently to separately determine the pupil size or eye pose of each eye, thereby allowing the presentation of image information to each eye to be dynamically adjusted for that eye. In some other embodiments, the pupil diameter or orientation of only a single eye 410 (e.g., using only a single camera per pair of eyes) is determined and assumed to be similar for both eyes of the user. Images obtained by inward-facing imaging system 466 may be analyzed to determine the user's eye posture or mood, which may be used by wearable system 400 to determine audio or visual content to be presented to the user. Wearable system 400 may also determine head pose (e.g., head position or head orientation) using sensors such as an IMU, accelerometer, gyroscope, etc.

[0329] The wearable system 400 may include a user input device 466 through which a user may input commands into the controller 460 and interact with the wearable system 400. For example, the user input device 466 may include a trackpad, a touch screen, a joystick, a multi-degree-of-freedom (DOF) controller, a capacitive sensing device, a game controller, a keyboard, a mouse, a directional pad (D-pad), a wand, a tactile device, a totem (e.g., serving as a virtual user input device), and the like. A multi-DOF controller may sense user input in possible translation (e.g., left / right, forward / backward, or up / down) or rotation (e.g., yaw, pitch, or roll) of some or all of the controller. A multi-DOF controller that supports translation may be referred to as 3DOF, while a multi-DOF controller that supports translation and rotation may be referred to as 6DOF. In some cases, a user may use a finger (e.g., thumb) to press or swipe on a touch-sensitive input device to provide input to the wearable system 400 (e.g., to provide user input to a user interface provided by the wearable system 400). The user input device 466 may be held by the user's hand during use of the wearable system 400. The user input device 466 may communicate with the wearable system 400 in wired or wireless communication. D. Other Components of the Wearable System

[0330] In many implementations, the wearable system may include other components in addition to or as an alternative to the components of the wearable system described above. The wearable system may include, for example, one or more tactile devices or components. The tactile devices or components may be operable to provide a haptic sensation to the user. For example, the tactile device or component may provide a haptic sensation of pressure or texture upon touching the virtual content (e.g., a virtual object, virtual tool, other virtual structure). The haptics may replicate the sensation of a physical object that the virtual object represents, or may replicate the sensation of an imaginary object or character (e.g., a dragon) that the virtual content represents. In some implementations, the tactile device or component may be worn by the user (e.g., a user-wearable glove). In some implementations, the tactile device or component may be held by the user.

[0331] A wearable system may include one or more physical objects that can be manipulated by, for example, a user to allow input to or interaction with the wearable system. These physical objects may be referred to herein as totems. Some totems may take the form of inanimate objects, e.g., pieces of metal or plastic, walls, the surface of a table. In some implementations, a totem may not actually have any physical input structures (e.g., keys, triggers, joysticks, trackballs, rocker switches). Instead, a totem may simply provide a physical surface, and the wearable system may render a user interface to appear to the user as being on one or more surfaces of the totem. For example, the wearable system may render an image of a computer keyboard and trackpad to appear to reside on one or more surfaces of the totem. For example, the wearable system may render a virtual computer keyboard and virtual trackpad to appear on the surface of a thin rectangular plate of aluminum that serves as the totem. The rectangular plate itself does not have any physical keys or trackpads or sensors. However, the wearable system may detect user manipulation or interaction or touch with the rectangular plate as a selection or input made via a virtual keyboard or virtual trackpad. User input device 466 (shown in FIG. 4) may be an embodiment of the totem, which may include a trackpad, touchpad, trigger, joystick, trackball, rocker or virtual switch, mouse, keyboard, multi-degree-of-freedom controller, or another physical input device. A user may use the totem alone or in combination with posture to interact with the wearable system or other users.

[0332] Examples of tactile devices and totems usable with the wearable devices, HMDs, and display systems of the present disclosure are described in U.S. Patent Publication No. 2015 / 0016777, which is incorporated by reference in its entirety. E. Example of Eye Image

[0333] 5 illustrates an image of an eye 500 with eyelids 504, sclera 508 ("white of the eye"), iris 512, and pupil 516. Curve 516a indicates the pupillary boundary between the pupil 516 and iris 512, and curve 512a indicates the limbal boundary between the iris 512 and sclera 508. Eyelids 504 include upper eyelid 504a and lower eyelid 504b. Eye 500 is illustrated in a natural resting position (e.g., oriented such that both the user's face and gaze would be directed toward a far-distance object directly in front of the user). The natural resting position of the eye 500 may be indicated by a natural resting direction 520, which is a direction perpendicular to the surface of the eye 500 when in the natural resting position (e.g., directly out from the plane for the eye 500 shown in FIG. 5), and which, in this embodiment, is centered within the pupil 516.

[0334] As the eye 500 moves to look towards different objects, the eye pose will change with respect to the natural resting direction 520. The current eye pose can be determined with reference to an eye pose direction 524, which is a direction perpendicular to the surface of the eye (and centered within the pupil 516), but oriented towards the object to which the eye is currently pointed. With reference to the exemplary coordinate system shown in FIG. 5A, the pose of the eye 500 can be represented as two angular parameters indicating the azimuth and zenith deflections of the eye's eye pose direction 524, both relative to the eye's natural resting direction 520. For illustrative purposes, these angular parameters can be represented as θ (azimuth deflection, determined from the origin azimuth angle) and φ (zenith deflection, sometimes also referred to as polar deflection). In some implementations, the angular roll of the eye about the eye pose direction 524 can be included in the determination of the eye pose, and the angular roll can be included in the following analysis. In other implementations, other techniques for determining eye pose can be used, for example pitch, yaw, and optionally roll systems.

[0335] The eye images can be obtained from the video using any suitable process, for example, using video processing algorithms that can extract images from one or more sequential frames. The eye pose can be determined from the eye images using various eye tracking techniques. For example, the eye pose can be determined by considering the lens effect of the cornea on the provided light source. Any suitable eye tracking technique can be used to determine the eye pose. F. Eye Tracking System Implementation Example

[0336] FIG. 6 illustrates a schematic diagram of a wearable system 600 including an eye tracking system. The wearable system 600 may, in at least some embodiments, include components located in a head mounted unit 602 and components located in a non-head mounted unit 604. The non-head mounted unit 604 may be, by way of example, a belt mounted component, a handheld component, a component in a backpack, a remote component, etc. Incorporating some of the components of the wearable system 600 into the non-head mounted unit 604 may help reduce the size, weight, complexity, and cost of the head mounted unit 602. In some implementations, some or all of the functionality described as being implemented by one or more components of the head mounted unit 602 and / or the non-head mounted unit 604 may be provided using one or more components included anywhere in the wearable system 600. For example, some or all of the functionality described below in connection with CPU 612 of head-mounted unit 602 may be provided using CPU 616 of non-head-mounted unit 604, or vice versa. In some embodiments, some or all of such functionality may be provided using peripheral devices of wearable system 600. Further, in some implementations, some or all of such functionality may be provided using one or more cloud computing devices or other remotely located computing devices, in a manner similar to that described above with reference to FIG.

[0337] As shown in FIG. 6, the wearable system 600 can include an eye tracking system including a camera 324 that captures an image of the user's eye 610. If desired, the eye tracking system can also include light sources 326a and 326b (such as light emitting diodes "LEDs"). The light sources 326a and 326b can generate a flash of light (e.g., a reflection from the user's eye that appears in an image of the eye captured by the camera 324). The position of the light sources 326a and 326b relative to the camera 324 can be known, such that the position of the flash of light in the image captured by the camera 324 can be used in tracking the user's eye (as will be discussed in more detail below in connection with FIG. 7). In at least one embodiment, there can be one light source 326 and one camera 324 associated with one of the user's eyes 610. In another embodiment, there can be one light source 326 and one camera 324 associated with each of the user's eyes 610. In still other embodiments, there may be one or more cameras 324 and one or more light sources 326 associated with one or each of the user's eyes 610. As a specific example, there may be two light sources 326a and 326b and one or more cameras 324 associated with each of the user's eyes 610. As another example, there may be three or more light sources, such as light sources 326a and 326b, and one or more cameras 324 associated with each of the user's eyes 610. In some implementations described herein, two or more cameras may be employed to image a given eye.

[0338] The eye tracking module 614 may receive images from the eye tracking camera 324 and may analyze the images and extract various information. As examples, the eye tracking module 614 may detect the user's eye posture, the three-dimensional position of the user's eyes relative to the eye tracking camera 324 (and the head mounted unit 602), the direction of one or both of the user's eyes 610 that are focused on, the user's convergence-divergence depth (e.g., the depth from the user that the user is focused on), the position of the user's pupils, the position of the user's cornea and / or corneal sphere, the center of rotation of one or each of the user's eyes, and the center of gaze of one or each of the user's eyes, or any combination thereof. The eye tracking module 614 may extract such information using techniques described below in connection with Figures 7-11 and / or 12-21. As shown in Figure 6, the eye tracking module 614 may be a software module implemented using the CPU 612 in the head mounted unit 602.

[0339] Although one camera 324 is shown in FIG. 6 as imaging the eye, in some implementations, multiple cameras may image the eye and be used for measurements such as corneal center and / or center of rotation measurements, or otherwise used for eye tracking or other purposes, as discussed herein.

[0340] Data from the eye tracking module 614 may be provided to other components within the wearable system. As an example, such data may be transmitted to components within the non-head mounted unit 604, such as the CPU 616, including software modules for a light field rendering controller 618 and an alignment observer 620.

[0341] The rendering controller 618 may use information from the eye tracking module 614 to adjust the image displayed to the user by the rendering engine 622 (e.g., the rendering engine, which may be a software module within the GPU 620 and may provide images to the display 220). As an example, the rendering controller 618 may adjust the image displayed to the user based on the user's center of rotation or center of viewpoint. In particular, the rendering controller 618 may use information about the user's center of viewpoint to simulate a rendering camera (e.g., to simulate the collection of images from the user's viewpoint) and adjust the image displayed to the user based on the simulated rendering camera.

[0342] A "rendering camera", sometimes referred to as a "pinhole perspective camera" (or simply, a "perspective camera") or a "virtual pinhole camera" (or simply, a "virtual camera"), is a simulated camera for use in rendering virtual image content, possibly from a database of objects in a virtual world. The objects may have a location and orientation relative to a user or wearer, and possibly relative to real objects in the environment surrounding the user or wearer. In other words, a rendering camera may represent a viewpoint in a rendering space from which the user or wearer should view the 3D virtual content (e.g., virtual objects) of the rendering space. The rendering camera may render a virtual image based on a database of virtual objects to be presented to the eye, managed by a rendering engine. The virtual image may be rendered as if it were taken from the user's or wearer's viewpoint. For example, a virtual image may be rendered as if it were captured by a pinhole camera (corresponding to a "rendering camera") with a specific set of intrinsic parameters (e.g., focal length, camera pixel size, principal point coordinates, distortion / warping parameters, etc.) and a specific set of extrinsic parameters (e.g., translation and rotation components relative to the virtual world). The virtual image is taken from the viewpoint of such a camera with the rendering camera's position and orientation (e.g., the rendering camera's extrinsic parameters). It follows that the system may define and / or adjust the intrinsic and extrinsic rendering camera parameters. For example, the system may define a specific set of extrinsic rendering camera parameters such that the virtual image is rendered as if it were captured from the viewpoint of a camera having a specific location relative to the user's or wearer's eyes to provide an image that appears as if it were from the user's or wearer's viewpoint. The system may later dynamically adjust the extrinsic rendering camera parameters on the fly to maintain alignment with the specific location. Similarly, intrinsic rendering camera parameters may also be defined and dynamically adjusted over time.In some implementations, the image is rendered as if it were captured from the viewpoint of a camera that has an aperture (e.g., a pinhole) at a specific location relative to the user's or wearer's eye (such as the center of view or center of rotation or other location).

[0343] In some embodiments, the system may create or dynamically reposition and / or reorient one rendering camera for the user's left eye and another for the user's right eye as the user's eyes are physically separated from one another and therefore consistently positioned in different locations. It follows that in at least some implementations, virtual content rendered from the perspective of a rendering camera associated with the viewer's left eye may be presented to the user through a left eyepiece of a head mounted display (e.g., head mounted unit 602), and virtual content rendered from the perspective of a rendering camera associated with the user's right eye may be presented to the user through a right eyepiece of such head mounted display. Further details discussing the creation, adjustment, and use of rendering cameras in the rendering process are provided in U.S. Patent Application No. 15 / 274,823, entitled "METHODS AND SYSTEMS FOR DETECTING AND COMBINING STRUCTURAL FEATURES IN 3D RECONSTRUCTION," which is expressly incorporated herein by reference in its entirety for all purposes.

[0344] In some examples, one or more modules (or components) of system 600 (e.g., light field rendering controller 618, rendering engine 620, etc.) may determine the position and orientation of a rendering camera in a rendering space based on the user's head and eye position and orientation (e.g., as determined based on head pose and eye tracking data, respectively). That is, system 600 may in effect map the user's head and eye position and orientation to a particular location and angular position in the 3D virtual environment, place and orient the rendering camera to a particular location and angular position in the 3D virtual environment, and render the virtual content for the user as it would be captured by the rendering camera. Further details discussing the real-world / virtual-world mapping process are provided in U.S. Patent Application No. 15 / 296,869, entitled "SELECTING VIRTUAL OBJECTS IN A THREE-DIMENSIONAL SPACE," which is expressly incorporated herein by reference in its entirety for all purposes. As an example, the rendering controller 618 may adjust the depth at which an image is displayed by selecting the depth plane (or depth planes) that will be utilized at any given time to display the image. In some implementations, such depth plane switching may be performed through adjustment of one or more intrinsic rendering camera parameters. For example, the light field rendering controller 618 may adjust the focal length of the rendering camera when performing a depth plane switching or adjustment. As described in more detail below, depth planes may be switched based on a determined convergence-divergence movement or fixation depth of the user.

[0345] The alignment observer 620 may use information from the eye tracking module 614 to identify whether the head mounted unit 602 is properly positioned on the user's head. As an example, the eye tracking module 614 may provide eye location information, such as the location of the center of rotation of the user's eyes, which indicates the three-dimensional position of the user's eyes relative to the camera 324, and the head mounted unit 602 and eye tracking module 614 may use the location information to determine whether the display 220 is properly aligned within the user's field of view, or whether the head mounted unit 602 (or headset) has slipped or is otherwise misaligned with the user's eyes. As examples, the alignment observer 620 may be able to determine whether the head mounted unit 602 has slipped off the bridge of the user's nose, thus moving the display 220 away from the user's eyes and downward therefrom (which may be undesirable), whether the head mounted unit 602 has moved above the bridge of the user's nose, thus moving the display 220 closer to the user's eyes and upward therefrom, whether the head mounted unit 602 has been shifted left or right relative to the bridge of the user's nose, whether the head mounted unit 602 has been lifted above the bridge of the user's nose, or whether the head mounted unit 602 has been moved away from a desired position or range of positions in these or other ways. In general, the alignment observer 620 may be able to determine whether the head mounted unit 602, in general, and the display 220 in particular, are properly positioned in front of the user's eyes. In other words, alignment observer 620 may determine whether the left display in display system 220 is properly aligned with the user's left eye, and the right display in display system 220 is properly aligned with the user's right eye. Alignment observer 620 may determine whether head mounted unit 602 is properly positioned by determining whether head mounted unit 602 is positioned and oriented within a desired range of positions and / or orientations relative to the user's eyes.

[0346] In at least some embodiments, the alignment observer 620 may generate user feedback in the form of alerts, messages, or other content. Such feedback may be provided to the user to inform the user of any misalignment of the head mounted unit 602, along with optional feedback on how to correct the misalignment (such as suggestions to adjust the head mounted unit 602 in a particular manner).

[0347] Exemplary alignment observation and feedback techniques that may be utilized by alignment observer 620 are described in U.S. patent application Ser. No. 15 / 717,747, filed Sep. 27, 2017 (Attorney Docket No. MLEAP.052A2), and U.S. Provisional Patent Application Ser. No. 62 / 644,321, filed Mar. 16, 2018 (Attorney Docket No. MLEAP.195PR), both of which are incorporated by reference in their entireties herein. G. Eye Tracking Module Example

[0348] A detailed block diagram of an exemplary eye tracking module 614 is shown in Figure 7A. As shown in Figure 7A, the eye tracking module 614 may include a variety of different sub-modules, may provide a variety of different outputs, and may utilize a variety of available data in tracking the user's eyes. As an example, the eye tracking module 614 may utilize available data including extrinsic and intrinsic properties of eye tracking such as the geometry of the eye tracking camera 324 relative to the light source 326 and the head mounted unit 602, assumed eye dimensions 704 such as a typical distance of about 4.7 mm between the center of the user's corneal curvature and the average center of rotation of the user's eye or a typical distance between the user's center of rotation and the center of gaze, and per-user calibration data 706 such as the interpupillary distance of a particular user. Additional examples of extrinsic properties, intrinsic properties, and other information that may be employed by the eye tracking module 614 are described in U.S. patent application Ser. No. 15 / 497,726, filed April 26, 2017 (Attorney Docket No. MLEAP.023A7), which is incorporated by reference in its entirety into this specification.

[0349] The image pre-processing module 710 may receive images from an eye camera, such as eye camera 324, and may perform one or more pre-processing (e.g., adjustment) operations on the received images. As examples, the image pre-processing module 710 may apply Gaussian blurring to the images, down-sample the images to a lower resolution, apply an unsharp mask, apply an edge sharpening algorithm, or apply other suitable filters that aid in the later detection, location, and labeling of phosphenes, pupils, or other features in images from the eye camera 324. The image pre-processing module 710 may apply a low-pass filter or a morphological filter, such as an open filter, that may remove high frequency noise from the pupil boundary 516a (see FIG. 5), etc., thereby removing noise that may interfere with pupil and phosphene determination. The image pre-processing module 710 may output the pre-processed images to the pupil identification module 712 and the phosphene detection and labeling module 714.

[0350] The pupil identification module 712 may receive preprocessed images from the image preprocessing module 710 and may identify regions of those images that contain the user's pupil. The pupil identification module 712, in some embodiments, may determine the coordinates of the location of the user's pupil in the eye tracking images from the camera 324, i.e., the coordinates of the center or centroid. In at least some embodiments, the pupil identification module 712 may identify a contour (e.g., a contour of the pupil-iris boundary) in the eye tracking images, identify contour moments (e.g., a center of mass), apply a starburst pupil detection and / or a Canny edge detection algorithm, filter out outliers based on intensity values, identify sub-pixel boundary points, correct for eye camera distortion (e.g., distortion in the images captured by the eye camera 324), apply a random sample consensus (RANSAC) iterative algorithm, fit an ellipse to the boundaries in the eye tracking images, apply a tracking filter to the images, and identify the sub-pixel image coordinates of the user's pupil centroid. The pupil identification module 712 may output pupil identification data (which may indicate areas of the preprocessed image module 712 that have been identified as indicative of the user's pupil) to the flash detection and labeling module 714. The pupil identification module 712 may provide 2D coordinates of the user's pupil in each eye tracking image (e.g., 2D coordinates of the centroid of the user's pupil) to the flash detection module 714. In at least some embodiments, the pupil identification module 712 may also provide the same type of pupil identification data to the coordinate system normalization module 718.

[0351] Pupil detection techniques that may be utilized by pupil identification module 712 are described in U.S. Patent Publication No. 2017 / 0053165, published February 23, 2017, and U.S. Patent Publication No. 2017 / 0053166, published February 23, 2017 (each of which is incorporated by reference in its entirety into this specification).

[0352] The flash detection and labeling module 714 may receive the preprocessed image from module 710 and the pupil identification data from module 712. The flash detection module 714 may use this data to detect and / or identify flashes (i.e., reflections of light from the light source 326 off the user's eye) in areas of the preprocessed image that indicate the user's pupil. As an example, the flash detection module 714 may search for bright areas, sometimes referred to herein as "blobs" or local intensity maxima, in the vicinity of the user's pupil in the eye tracking image. In at least some embodiments, the flash detection module 714 may rescale (e.g., expand) the pupil ellipse to include the additional flashes. The flash detection module 714 may filter the flashes by size and / or intensity. The flash detection module 714 may also determine the 2D location of each of the flashes in the eye tracking image. In at least some examples, the flash detection module 714 may determine the 2D location of the flash relative to the user's pupil, which may also be referred to as the pupil-flash vector. The flash detection and labeling module 714 may label the flashes and output a preprocessed image with the labeled flashes to the 3D corneal center estimation module 716. The flash detection and labeling module 714 may also pass on data such as the preprocessed image from module 710 and the pupil identification data from module 712. In some implementations, the flash detection and labeling module 714 may determine the light source (e.g., among the system's multiple light sources, including infrared light sources 326a and 326b) that produced each identified flash. In these examples, the flash detection and labeling module 714 may label the flashes with information identifying the associated light source and output a preprocessed image with the labeled flashes to the 3D corneal center estimation module 716.

[0353] Pupil and flash detection as implemented by modules such as modules 712 and 714 may use any suitable technique. As an example, edge detection may be applied to the eye image to identify flashes and pupils. Edge detection may be applied by various edge detectors, edge detection algorithms, or filters. For example, a Canny edge detector may be applied to the image to detect edges, such as lines, of the image. Edges may include points located along the lines that correspond to local maximum derivatives. For example, pupil boundary 516a (see FIG. 5) may be located using a Canny edge detector. Once the location of the pupil is determined, various image processing techniques may be used to detect the "pose" of the pupil 116. Determining the eye pose of the eye image may also be referred to as detecting the eye pose of the eye image. Pose may also be referred to as gaze, facing direction, or eye orientation. For example, the pupil may be looking left towards an object, and the pupil pose may be classified as a left-looking pose. Other methods may also be used to detect the location of the pupil or flash. For example, concentric rings may be located in the eye image using a Canny edge detector. As another example, an integro-differential operator may be used to find the limbal boundary of the pupil or iris. For example, the Daugman integro-differential operator, Hough transform, or other iris segmentation techniques can be used to return a curve that estimates the boundary of the pupil or iris.

[0354] The 3D corneal center estimation module 716 may receive the pre-processed images, including the detected phosphene data and pupil identification data, from modules 710, 712, 714. The 3D corneal center estimation module 716 may use these data to estimate the 3D position of the user's cornea. In some embodiments, the 3D corneal center estimation module 716 may estimate the 3D position of the center of the corneal curvature of the eye or the corneal sphere of the user, e.g., the center of an imaginary sphere having a surface portion generally coextensive with the user's cornea. The 3D corneal center estimation module 716 may provide data indicative of the estimated 3D coordinates of the corneal sphere and / or the user's cornea to the coordinate system normalization module 718, the optical axis determination module 722, and / or the light field rendering controller 618. Further details of the operation of the 3D corneal center estimation module 716 are provided herein in connection with FIGS. 11A-16C. Exemplary techniques for estimating the position of ocular features, such as the cornea or corneal sphere, that may be utilized by the 3D corneal center estimation module 716 and other modules in the wearable system of the present disclosure are discussed in U.S. patent application Ser. No. 15 / 497,726, filed April 26, 2017 (Attorney Docket No. MLEAP.023A7), which is incorporated by reference in its entirety into this specification.

[0355] Coordinate system normalization module 718 may optionally be included within eye tracking module 614 (as indicated by its dashed outline). Coordinate system normalization module 718 may receive data indicating the estimated 3D coordinates of the center of the user's cornea (and / or the center of the user's corneal sphere) from 3D corneal center estimation module 716 and may also receive data from other modules. Coordinate system normalization module 718 may normalize the eye camera coordinate system, which may help to compensate for slippage of the wearable device (e.g., slippage of a head-mounted component from its normal resting position on the user's head, which may be identified by alignment observer 620). The coordinate system normalization module 718 may rotate the coordinate system to align the z-axis (e.g., vergence-divergence depth axis) of the coordinate system with the corneal center (e.g., as indicated by the 3D corneal center estimation module 716) and may translate the camera center (e.g., origin of the coordinate system) to a predetermined distance away from the corneal center, such as 30 mm (e.g., module 718 may zoom in or zoom out the eye tracking image depending on whether the eye camera 324 is determined to be closer or farther away than the predetermined distance). Using this normalization process, the eye tracking module 614 may be able to establish consistent orientations and distances in the eye tracking data relatively independent of variations in the headset positioned on the user's head. The coordinate system normalization module 718 may provide the 3D coordinates of the center of the cornea (and / or corneal sphere), the pupil identification data, and the preprocessed eye tracking images to the 3D pupil center locator module 720.

[0356] The 3D pupil center locator module 720 may receive data including the 3D coordinates of the center of the user's cornea (and / or corneal sphere) in a normalized or non-normalized coordinate system, pupil location data, and pre-processed eye tracking images. The 3D pupil center locator module 720 may analyze such data to determine the 3D coordinates of the user's pupil center in a normalized or non-normalized eye camera coordinate system. The 3D pupil center locator module 720 may determine the location of the user's pupil in three dimensions based on the 2D location of the pupil centroid (as determined by module 712), the 3D location of the corneal center (as determined by module 716), assumed eye dimensions 704 such as the size of a typical user's corneal sphere and the typical distance from the corneal center to the pupil center, and optical properties of the eye such as the refractive index of the cornea (relative to the refractive index of air), or any combination thereof. Techniques for estimating the position of eye features, such as the pupil, that may be utilized by the 3D pupil center locator module 720 and other modules in the wearable system of the present disclosure are discussed in U.S. patent application Ser. No. 15 / 497,726, filed April 26, 2017 (Attorney Docket No. MLEAP.023A7), which is incorporated by reference in its entirety herein.

[0357] The optical axis determination module 722 may receive data from modules 716 and 720 indicating the 3D coordinates of the user's cornea and the user's pupil center. Based on such data, the optical axis determination module 722 may identify a vector from the location of the corneal center (e.g., from the center of the corneal sphere) to the user's pupil center, which may define the optical axis of the user's eye. The optical axis determination module 722 may provide outputs to modules 724, 728, 730, and 732 that define the user's optical axis, as an example.

[0358] The center of rotation (CoR) estimation module 724 may receive data from module 722 including parameters of the optical axis of the user's eye (e.g., data indicating the direction of the optical axis in a coordinate system with a known relationship to the head mounted unit 602). For example, the CoR estimation module 724 may estimate the center of rotation of the user's eye. The center of rotation may indicate a point around which the user's eye rotates as the user's eye rotates left, right, up, and / or down. Even if the eye cannot rotate perfectly around a single point, it is assumed that a single point may be sufficient. In at least some embodiments, the CoR estimation module 724 may estimate the center of rotation of the eye by moving the pupil center (as identified by module 720) or the center of corneal curvature (as identified by module 716) a specific distance along the optical axis (as identified by module 722) toward the retina. This specific distance may be the assumed eye dimension 704. As an example, the particular distance between the center of corneal curvature and the CoR may be about 4.7 mm. This distance may be varied for a particular user based on any relevant data, including the user's age, gender, vision prescription, other relevant characteristics, etc. As discussed above, in some implementations, the center of corneal curvature or corneal center refers to the center of curvature of a portion of the cornea or the center of curvature of a spherical surface that coincides with a portion of the surface of the cornea. For example, in some implementations, the center of corneal curvature or corneal center refers to the center of curvature of the corneal apex or the center of curvature of a spherical surface that coincides with a portion of the surface of the corneal apex.

[0359] In at least some embodiments, the CoR estimation module 724 may refine its estimate of the center of rotation of each of the user's eyes over time. As an example, over time, the user will eventually rotate their eyes (to look elsewhere, closer, something further away, or to the left, right, up, or down at one time), causing a shift in the optical axis of each of their eyes. The CoR estimation module 724 may then analyze the two (or more) optical axes identified by module 722 and locate the 3D point of intersection of those optical axes. The CoR estimation module 724 may then determine a center of rotation that is at the 3D point of intersection. Such techniques may provide estimates of the center of rotation with improving accuracy over time.

[0360] Various techniques may be employed to increase the accuracy of the CoR estimation module 724 and the determined CoR positions of the left and right eyes. As an example, the CoR estimation module 724 may estimate the CoR by finding the average point of intersection of the optical axes determined over time for a variety of different eye postures. As an additional example, the module 724 may filter or average the estimated CoR positions over time, may calculate a moving average of the estimated CoR positions over time, and / or may apply a Kalman filter and known dynamics of the eye and eye tracking system to estimate the CoR positions over time. In some implementations, a least squares approach may be taken to determine one or more points of intersection of the optical axes. In such implementations, the system may identify locations where, at a given time, the sum of the squared distances to a given set of optical axes is reduced or minimized as points of optical axis intersection. As a specific example, module 724 may calculate a weighted average of the determined point of optical axis intersection and the assumed CoR location (e.g., 4.7 mm behind the center of the eye's corneal curvature) such that the determined CoR may slowly shift over time from the assumed CoR location (e.g., 4.7 mm behind the center of the eye's corneal curvature) to a slightly different location within the user's eye as eye tracking data is acquired for the user, thereby allowing per-user refinement of the CoR location.

[0361] Under ideal conditions, the 3D position of the true CoR of the user's eye relative to the HMD should change over time by a negligible or minimal amount as the user moves his / her eye (e.g., as the user's eye rotates around its center of rotation). In other words, for a given set of eye movements, the 3D position of the true CoR of the user's eye (relative to the HMD) should hypothetically not vary over time as much as any other point along the optical axis of the user's eye. It therefore follows that the further a point along the optical axis is from the true CoR of the user's eye, the more variation or variance its 3D position will exhibit over time as the user moves his / her eye. In some embodiments, the CoR estimation module 724 and / or other sub-modules of the eye tracking module 614 may utilize this statistical relationship to improve CoR estimation accuracy. In such an embodiment, the CoR estimation module 724 and / or other sub-modules of the eye tracking module 614 may refine its estimate of the CoR 3D position over time by identifying variations in its CoR estimates that have a low amount of variation (e.g., a low amount of variance or standard deviation).

[0362] As a first example, in an embodiment where the CoR estimation module 724 estimates the CoR based on the intersection of multiple different optical axes (each associated with a user looking in a different direction), the CoR estimation module 724 may take advantage of this statistical relationship (the true CoR should have low variance) by introducing a common offset into each of the directions of the optical axes (e.g., shifting each axis by some uniform amount) and determining whether the offset optical axes intersect each other at an intersection point that has low variation, e.g., low variance or standard deviation. This can help to correct for slight systematic errors in the calculation of the directions of the optical axes and refine the estimated location of the CoR to be closer to the true CoR.

[0363] As a second example, in an embodiment in which the CoR estimation module 724 estimates the CoR by moving a particular distance (e.g., the distance between the center of corneal curvature and the CoR) along the optical axis (or other axis), the system may vary, optimize, tune, or otherwise adjust the particular distance between the center of corneal curvature and the CoR over time (e.g., for a large group of images of the eye captured at different times) in such a manner as to reduce or minimize the amount of variation, e.g., variance and / or standard deviation, of the estimated CoR position. For example, if the CoR estimation module 724 initially uses a particular distance value of 4.7 mm (from the center of corneal curvature, along the optical axis) to obtain CoR position estimates, but the true CoR of a given user's eye may be located 4.9 mm behind the center of corneal curvature of the eye (along the optical axis), the initial set of CoR position estimates obtained by the CoR estimation module 724 may exhibit a relatively high amount of variation, e.g., variance or standard deviation. In response to detecting such a relatively high amount of variation (e.g., variance or standard deviation), the CoR estimation module 724 may search for and identify one or more points along the optical axis having a lower amount of variation (e.g., variance or standard deviation), and may identify the 4.9 mm distance as having the lowest amount of variation (e.g., variance or standard deviation), and may accordingly adjust the particular distance value utilized to 4.9 mm.

[0364] The CoR estimation module 724 may search for alternative CoR estimates having lower amounts of variation (e.g., variance and / or standard deviation) in response to detecting that the current CoR estimate has a relatively high amount of variation (e.g., variance or standard deviation), or may search for alternative CoR estimates having lower amounts of variation (e.g., variance and / or standard deviation) as a matter of course after obtaining an initial CoR estimate. In some embodiments, such optimization / adjustment may occur gradually over time, while in other embodiments, such optimization / adjustment may be performed during an initial user calibration session. In embodiments where such a procedure is performed during a calibration procedure, the CoR estimation module 724 may not initially agree / adhere to any assumed particular distance, but rather may collect a set of eye tracking data over time, perform a statistical analysis on the set of eye tracking data, and determine a particular distance value that results in a CoR position estimate with the least possible amount of variation (e.g., variance or standard deviation) based on the statistical analysis.

[0365] The interpupillary distance (IPD) estimation module 726 may receive data from the CoR estimation module 724 indicating the estimated 3D positions of the centers of rotation of the user's left and right eyes. The IPD estimation module 726 may then estimate the user's IPD by measuring the 3D distance between the centers of rotation of the user's left and right eyes. In general, the distance between the estimated CoR of the user's left eye and the estimated CoR of the user's right eye may be approximately equal to the distance between the user's pupil centers when the user is looking at optical infinity (e.g., the optical axes of the user's eyes are approximately parallel to each other), which is a typical definition of interpupillary distance (IPD). The user's IPD may be used by various components and modules within the wearable system. As an example, the user's IPD may be provided to the alignment observer 620 and used in assessing the degree to which the wearable device is aligned with the user's eyes (e.g., whether the left and right display lenses are properly spaced according to the user's IPD). As another example, the user's IPD may be provided to the convergence-divergence depth estimation module 728 and used in determining the user's convergence-divergence depth. The module 726 may employ various techniques, such as those discussed in connection with the CoR estimation module 724, to increase the accuracy of the estimated IPD. As an example, the IPD estimation module 724 may apply filtering, averaging over time, weighted averaging including assumed IPD distances, Kalman filters, etc., as part of estimating the user's IPD in an accurate manner.

[0366] The vergence-divergence movement depth estimation module 728 may receive data from various modules and sub-modules (as illustrated in connection with FIG. 7A ) within the eye tracking module 614. In particular, the vergence-divergence movement depth estimation module 728 may employ data indicative of an estimated 3D position of a pupil center (e.g., as provided by module 720 described above), one or more determined parameters of an optical axis (e.g., as provided by module 722 described above), an estimated 3D position of a center of rotation (e.g., as provided by module 724 described above), an estimated IPD (e.g., a Euclidean distance between estimated 3D positions of centers of rotation) (e.g., as provided by module 726 described above), and / or one or more determined parameters of an optical axis and / or visual axis (e.g., as provided by modules 722 and / or 730 described below). The vergence depth estimation module 728 may detect or otherwise obtain a measurement of the user's vergence depth, which may be the distance from the user at which the user's eyes are focused. As an example, when the user is looking at an object three feet in front of them, the user's left and right eyes have a vergence depth of three feet, while when the user is looking at a distant scene (e.g., the optical axes of the user's eyes are approximately parallel to one another such that the distance between the user's pupil centers may be approximately equal to the distance between the rotation centers of the user's left and right eyes), the user's left and right eyes have a vergence depth of infinity. In some implementations, the vergence depth estimation module 728 may utilize data indicative of the estimated centers of the user's pupils (e.g., as provided by module 720) and determine the 3D distance between the estimated centers of the user's pupils. The vergence-divergence depth estimation module 728 may obtain a measurement of vergence-divergence depth by comparing such determined 3D distance between the pupil centers with an estimated IPD (e.g., the Euclidean distance between the estimated 3D positions of the centers of rotation) (e.g., as shown by module 726 described above).In addition to the 3D distance between pupil centers and the estimated IPD, the vergence movement depth estimation module 728 may utilize known, assumed, estimated, and / or determined geometric shapes to calculate the vergence movement depth. As an example, the module 728 may combine the 3D distance between pupil centers, the estimated IPD, and the 3D CoR position in a trigonometric calculation to estimate (e.g., determine) the user's vergence movement depth. In fact, an evaluation of such determined 3D distance between pupil centers relative to the estimated IPD may serve to indicate a measurement of the user's current vergence movement depth relative to optical infinity. In some examples, the vergence movement depth estimation module 728 may simply receive or have access to data indicative of an estimated 3D distance between estimated centers of the user's pupils for purposes of obtaining such a measurement of the vergence movement depth. In some embodiments, the vergence motion depth estimation module 728 may estimate the vergence motion depth by comparing the left and right optical axes of the user. In particular, the vergence motion depth estimation module 728 may estimate the vergence motion depth by locating the distance from the user where the user's left and right optical axes intersect (or where the projections of the user's left and right optical axes on a plane, such as a horizontal plane, intersect). The module 728 may utilize the user's IPD in this calculation by setting zero depth to be the depth where the user's left and right optical axes are separated by the user's IPD. In at least some embodiments, the vergence motion depth estimation module 728 may determine the vergence motion depth by triangulating eye tracking data with known or derived spatial relationships.

[0367] In some embodiments, the vergence depth estimation module 728 may estimate the user's vergence depth based on the intersection of the user's visual axis (instead of its optical axis), which may provide a more accurate indication of the distance the user is focused on. In at least some embodiments, the eye tracking module 614 may include an optical axis / visual axis mapping module 730. As discussed in more detail in connection with FIG. 10, the user's optical axis and visual axis are generally not aligned. The visual axis is the axis along which a person is looking, while the optical axis is defined by the person's lens and pupil center and may run through the center of the person's retina. In particular, the user's visual axis is generally defined by the location of the user's fovea, which may be offset from the center of the user's retina, thereby resulting in different optical and visual axes. In at least some of these embodiments, the eye tracking module 614 may include an optical axis / visual axis mapping module 730. The optical axis / visual axis mapping module 730 may correct for differences between the user's optical axis and visual axis and provide information about the user's visual axis to other components in the wearable system, such as the vergence-divergence depth estimation module 728 and the light field rendering controller 618. In some embodiments, the module 730 may use the assumed eye dimensions 704, including a typical offset between the optical axis and the visual axis of about 5.2° inward (nasally, toward the user's nose). In other words, the module 730 may shift the user's left optical axis 5.2° nasally (toward the nose) to the right and the user's right optical axis 5.2° nasally (toward the nose) to estimate the direction of the user's left and right optical axes. In other embodiments, the module 730 may utilize per-user calibration data 706 in mapping the optical axis (e.g., as shown by module 722 described above) to the visual axis. As additional examples, module 730 may shift the user's optical axis nasally by 4.0°-6.5°, 4.5°-6.0°, 5.0°-5.4°, etc., or any range formed by any of these values.In some arrangements, module 730 may apply the shift based, at least in part, on characteristics of a particular user, such as their age, gender, vision prescription, or other relevant characteristics, and / or may apply the shift based, at least in part, on a calibration process for the particular user (e.g., to determine the optical axis-visual axis offset for the particular user). In at least some embodiments, module 730 may also shift the origin of the left and right optical axes to correspond to the user's CoP (as determined by module 732) instead of the user's CoR.

[0368] An optional center of perspective (CoP) estimation module 732, when provided, may estimate the location of the user's left and right centers of perspective (CoP). The CoP is a useful location for a wearable system and, in at least some embodiments, may be a location directly in front of the pupil. In at least some embodiments, the CoP estimation module 732 may estimate the location of the user's left and right centers of perspective based on the 3D location of the user's pupil center, the 3D location of the center of the user's corneal curvature, or any such suitable data, or any combination thereof. As an example, the user's CoP may be approximately 5.01 mm in front of the center of the corneal curvature (e.g., 5.01 mm from the center of the corneal sphere, toward the cornea of ​​the eye, in a direction along the optical axis) and approximately 2.97 mm behind the outer surface of the user's cornea along the optical or visual axis. The user's center of perspective may be directly in front of its pupil center. As examples, the user's CoP may be less than about 2.0 mm from the user's pupil, less than about 1.0 mm from the user's pupil, or less than about 0.5 mm from the user's pupil, or any range between any of these values. As another example, the center of gaze may correspond to a location within the anterior chamber of the eye. As other examples, the CoP may be between 1.0 mm and 2.0 mm, about 1.0 mm, 0.25 mm to 1.0 mm, 0.5 mm to 1.0 mm, or 0.25 mm to 0.5 mm from the user's pupil.

[0369] The viewpoint centers described herein (as potentially desirable locations of the rendering camera pinhole and anatomical locations within the user's eye) may be locations that serve to reduce and / or eliminate undesirable parallax shifts. In particular, the optical system of the user's eye roughly parallels a theoretical system formed by a pinhole in front of a lens projecting onto a screen, with the pinhole, lens, and screen roughly corresponding to the user's pupil / iris, lens, and retina, respectively. Furthermore, it may be desirable for there to be little or no parallax shift when two point sources (or objects) at different distances from the user's eye are strictly rotated around the pinhole opening (e.g., rotated along a radius of curvature equal to their respective distances from the pinhole opening). Thus, one would expect the CoP to be located at the eye's pupil center (and such a CoP may be used in some embodiments). However, in addition to the lens and pupil pinhole, the human eye includes a cornea, which imparts additional refractive power to light propagating toward the retina. Thus, the anatomical equivalent of a pinhole in the theoretical system described in this paragraph may be an area of ​​the user's eye that is located between the outer surface of the cornea of ​​the user's eye and the center of the pupil or iris of the user's eye. For example, the anatomical equivalent of a pinhole may correspond to an area within the anterior chamber of the user's eye. For various reasons discussed herein, it may be desirable to set the CoP at such a location within the anterior chamber of the user's eye.

[0370] As discussed above, the eye tracking module 614 may provide data such as estimated 3D positions of the left and right eye centers of rotation (CoR), vergence-divergence movement depth, left and right eye optical axes, 3D positions of the user's eyes, 3D positions of the user's left and right centers of corneal curvature, 3D positions of the user's left and right pupil centers, 3D positions of the user's left and right gaze centers, and the user's IPD to other components in the wearable system, such as the light field rendering controller 618 and the alignment observer 620. The eye tracking module 614 may also include other sub-modules that detect and generate data associated with other aspects of the user's eyes. As an example, the eye tracking module 614 may include an eye blink detection module that provides a flag or other alert whenever the user blinks, and a saccade detection module that provides a flag or other alert whenever the user's eyes saccade (e.g., rapidly shift focus to another point).

[0371] Other methods of determining eye tracking and center of rotation are also possible. Thus, the eye tracking module 614 may vary. In various implementations of the eye tracking module described below, for example, an estimate of the center of rotation is determined based on a plurality of corneal curvature center values. In some implementations, as discussed with reference to, for example, FIGS. 17A-19D, the eye tracking module 614 may estimate the eye's center of rotation by determining the convergence or intersection between surface normal vectors of a surface that is possibly fitted to a plurality of corneal curvature centers for different eye poses. It is noted that one or more features from the eye tracking module 614 described above or elsewhere herein may be included in other implementations of the eye tracking module. H. Rendering Controller Example

[0372] A detailed block diagram of an example light field rendering controller 618 is shown in FIG. 7B. As shown in FIGS. 6 and 7B, the rendering controller 618 may receive eye tracking information from the eye tracking module 614 and provide output to the rendering engine 622, which may generate an image to be displayed for viewing by a user of the wearable system. As an example, the rendering controller 618 may receive vergence-divergence depth, left and right eye rotation centers (and / or gaze centers), and other eye data such as eye blink data, saccade data, etc.

[0373] The depth plane selection module 750 may receive convergence-divergence depth information and other ocular data and may cause the rendering engine 622 to convey content to the user with a particular depth plane (e.g., a particular accommodation or focal length) based on such data. As discussed in connection with FIG. 4, the wearable system may include multiple discrete depth planes formed by multiple waveguides, each conveying image information with a variable level of wavefront curvature. In some embodiments, the wearable system may include one or more variable depth planes, such as optical elements, conveying image information with a time-varying level of wavefront curvature. In these and other embodiments, the depth plane selection module 750 may cause the rendering engine 622 to convey content to the user at a selected depth (e.g., cause the rendering engine 622 to instruct the display 220 to switch depth planes) based in part on the user's convergence-divergence depth. In at least some embodiments, depth plane selection module 750 and rendering engine 622 may render content at different depths and may also generate and / or provide depth plane selection data to display hardware, such as display 220. Display hardware, such as display 220, may perform electronic depth plane switching in response to depth plane selection data (which may be control signals) generated and / or provided by modules, such as depth plane selection module 750 and rendering engine 622.

[0374] In general, it may be desirable for the depth plane selection module 750 to select a depth plane that matches the user's current convergence-divergence depth so that the user is provided with accurate accommodation cues. However, it may also be desirable to switch depth planes in a discreet and unobtrusive manner. As an example, it may be desirable to avoid excessive switching between depth planes and / or to switch depth planes at times when the user is unlikely to notice the switch, such as during an eye blink or eye saccade.

[0375] The hysteresis band crossing detection module 752 may help to avoid excessive switching between depth planes, especially when the user's vergence-divergence motion depth fluctuates at the midpoint or transition point between two depth planes. In particular, the module 752 may cause the depth plane selection module 750 to exhibit hysteresis in its selection of depth planes. As an example, the module 752 may cause the depth plane selection module 750 to switch from a first, more distant depth plane to a second, closer depth plane only after the user's vergence-divergence motion depth passes a first threshold. Similarly, the module 752 may cause the depth plane selection module 750 (and thus may indicate to a display, such as the display 220) to switch to a first, more distant depth plane only after the user's vergence-divergence motion depth passes a second threshold that is further from the user than the first threshold. In the overlap region between the first and second thresholds, module 750 may cause depth plane selection module 750 to maintain whichever depth plane is currently selected as the selected depth plane, thus avoiding excessive switching between depth planes.

[0376] The eye event detection module 750 may receive other eye data from the eye tracking module 614 of FIG. 7A and may cause the depth plane selection module 750 to delay some depth plane switches until an eye event occurs. As an example, the eye event detection module 750 may cause the depth plane selection module 750 to delay a planned depth plane switch until a user blink is detected, or may receive data from an eye blink detection component in the eye tracking module 614 indicating that the user is currently blinking, and in response, cause the depth plane selection module 750 to perform a planned depth plane switch during the blink event (such as by having the module 750 instruct the display 220 to perform a depth plane switch during the blink event). In at least some embodiments, the wearable system may be able to shift content onto a new depth plane during the blink event such that the user is unlikely to perceive the shift. As another example, the eye event detection module 750 may delay a planned depth plane switch until an eye saccade is detected. As discussed in relation to eye blinking, such an arrangement can facilitate discrete shifts in the depth plane.

[0377] If desired, the depth plane selection module 750 may delay a planned depth plane switch only for a limited period of time before performing a depth plane switch, even in the absence of an ocular event. Similarly, the depth plane selection module 750 may perform a depth plane switch when the user's convergence-divergence depth is substantially outside the currently selected depth plane (e.g., when the user's convergence-divergence depth exceeds a predetermined threshold that exceeds the normal threshold for a depth plane switch), even in the absence of an ocular event. These arrangements may help ensure that the eye event detection module 754 does not delay a depth plane switch indefinitely and does not delay a delayed depth plane switch when large accommodation errors are present.

[0378] The rendering camera controller 758 may provide information to the rendering engine 622 indicating the location of the user's left and right eyes. The rendering engine 622 may then generate content by simulating cameras at the user's left and right eye positions and generating content based on the viewpoints of the simulated cameras. As discussed above, the rendering camera is a simulated camera for use in rendering virtual image content, possibly from a database of objects in the virtual world. The objects may have a location and orientation relative to the user or wearer, and possibly relative to real objects in the environment surrounding the user or wearer. The rendering camera may render a virtual image based on a database of virtual objects contained within the rendering engine to be presented to the eyes. The virtual image may be rendered as if it were taken from the viewpoint of the user or wearer. For example, the virtual image may be rendered as if it were captured by a camera (corresponding to a "rendering camera") having an aperture, lens, and detector that views objects in the virtual world. The virtual image is taken from the viewpoint of such a camera, which has the position of the "rendering camera." For example, the virtual image may be rendered as if it had been captured from a camera viewpoint having a specific location relative to the user's or wearer's eyes to provide an image that appears to be from the user's or wearer's point of view. In some implementations, the image is rendered as if it had been captured from a camera viewpoint having an aperture at a specific location relative to the user's or wearer's eyes (such as a viewpoint center or rotation center or other location as discussed herein).

[0379] The rendering camera controller 758 may determine the positions of the left and right cameras based on the left and right eye centers of rotation (CoR) determined by the CoR estimation module 724 and / or based on the left and right eye centers of perspective (CoP) determined by the CoP estimation module 732. In some embodiments, the rendering camera controller 758 may switch between the CoR and CoP locations based on various factors. As examples, the rendering camera controller 758 may, in various modes, align the rendering camera to the CoR location all the time, align the rendering camera to the CoP location all the time, toggle or discretely switch between aligning the rendering camera to the CoR location and aligning the rendering camera to the CoP location over time based on various factors, or dynamically align the rendering camera to any of a range of different positions along the optical (or visual) axis between the CoR and CoP locations over time based on various factors. The CoR and CoP positions may optionally be passed through a smoothing filter 756 (in any of the previously described modes for rendering camera positioning), which may average the CoR and CoP locations over time to reduce noise in these positions and prevent jitter when rendering the simulated rendering camera.

[0380] In at least some embodiments, the rendering camera may be simulated as a pinhole camera with the pinhole located at the location of the estimated CoR or CoP identified by the eye tracking module 614. Because the CoP is offset from the CoR, whenever the rendering camera's position is based on the user's CoP, the location of both the rendering camera and its pinhole shifts as the user's eyes rotate. In contrast, whenever the rendering camera's position is based on the user's CoR, the location of the rendering camera's pinhole does not move with eye rotation, although the rendering camera (behind the pinhole) may move with eye rotation in some embodiments. In other embodiments where the rendering camera's position is based on the user's CoR, the rendering camera may not move (i.e., rotate) with the user's eyes. I. Examples of the Difference Between the Optical Axis and the Visual Axis

[0381] As discussed in connection with the optical axis / visual axis mapping module 730 of FIG. 7A, a user's optical axis and visual axis are generally not aligned, in part due to the user's visual axis being defined by its fovea, which is generally not at the center of the person's retina. Thus, when a person wishes to focus on a particular object, the person aligns its visual axis with the object to ensure that light from the object falls on its fovea, while its optical axis (defined by its pupil center and its corneal curvature center) is actually offset slightly from the object. FIG. 8 is an example of an eye 900 illustrating the eye's optical axis 902, the eye's visual axis 904, and the offset between these axes. In addition, FIG. 8 illustrates the eye's pupil center 906, the eye's corneal curvature center 908, and the eye's mean center of rotation (CoR) 910. In at least some populations, the eye's center of corneal curvature 908 may be approximately 4.7 mm in front of the eye's mean center of rotation (CoR) 910, as indicated by dimension 912. Additionally, the eye's center of perspective 914 may be approximately 5.01 mm in front of the eye's center of corneal curvature 908, approximately 2.97 mm behind the outer surface 916 of the user's cornea, and / or directly in front of the user's pupil center 906 (e.g., corresponding to a location within the anterior chamber of the eye 900). As additional examples, the dimension 912 may be 3.0 mm to 7.0 mm, 4.0 to 6.0 mm, 4.5 to 5.0 mm, or 4.6 to 4.8 mm, or any range between any value within any of these ranges. The eye's center of perspective (CoP) 914 may be a useful location for wearable systems, since, in at least some embodiments, aligning a rendering camera to the CoP may help reduce or eliminate parallax artifacts.

[0382] FIG. 8 also illustrates such locations within the human eye 900 with which the rendering camera pinhole may be aligned. As shown in FIG. 8, the rendering camera pinhole may be aligned with a location 914 along the optical axis 902 or visual axis 904 of the human eye 900 that is closer to the outer surface of the cornea than both (a) the center of the pupil or iris 906 and (b) the center of corneal curvature 908 of the human eye 900. For example, as shown in FIG. 8, the rendering camera pinhole may be aligned with a location 914 along the optical axis 902 of the human eye 900 that is approximately 2.97 millimeters posterior to the outer surface of the cornea 916 and approximately 5.01 millimeters anterior to the center of corneal curvature 908. The rendering camera pinhole location 914 and / or the anatomical region of the human eye 900 to which the location 914 corresponds may be considered to represent the viewpoint center of the human eye 900. 8 represents the shortest line passing through the center of corneal curvature 908 and the center of the pupil or iris 906. The visual axis 904 of the human eye 900 differs from the optical axis 902 because it represents a line extending from the fovea of ​​the human eye 900 to the center of the pupil or iris 906. J. Exemplary Wearable Device Configurations

[0383] 9A-E illustrate an example configuration of components of an example wearable device 2000 for capturing eye image data for use by the eye tracking module 614. For example, as illustrated in FIG. 9A, the wearable device 2000 may be part of a wearable system as described above with reference to FIGS. 2-4. The wearable device 2000 may include a left eyepiece 2010A and a right eyepiece 2010B. The left eyepiece 2010A may be capable of imaging a user's left eye, and the right eyepiece 2010B may be capable of imaging a user's right eye.

[0384] As illustrated in FIG. 9B, the left eyepiece 2010A may include one or more illumination sources 2022. Similarly, the right eyepiece 2010B may include one or more illumination sources 2024. For example, there may be four illumination sources 2022 and four illumination sources 2024. The illumination sources 2022 may be positioned within the left eyepiece 2010A and emit light toward the user's left eye 2012A. The illumination sources 2022 may be positioned so as not to obstruct the user's view through the left eyepiece 2010A. For example, the illumination sources 2022 may be positioned around the edge of the display within the left eyepiece 2010A so as not to obstruct the user's view through the display. Similarly, the illumination sources 2024 may be positioned within the right eyepiece 2010B and emit light toward the user's right eye 2012B. The illumination source 2024 may be positioned so as not to obstruct the user's view through the right eyepiece 2010B. For example, the illumination source 20204 may be positioned around the edge of the display in the right eyepiece 2010B so as not to obstruct the user's view through the display. The illumination sources 2022, 2024 may emit light in the visible or non-visible light. For example, the illumination sources 2022, 2024 may be infrared (IR) LEDs. The illumination sources may also be differently located or configured.

[0385] As illustrated in FIG. 9B, the left eyepiece 2010A may include a left eye imaging system. The left eye imaging system may include one or more inwardly facing cameras (2014, 2016). For example, the left eye imaging system may include a left eye tracking camera 2014 for the left eyepiece 2010A and a right eye tracking camera 2016 for the left eyepiece 2010A. The left eye tracking camera 2014 and the right eye tracking camera 2016 may be located to the left and right of each other, respectively, possibly to the left and right of the center of the left eyepiece, respectively. Similarly, the right eyepiece 2010B may include a right eye imaging system. The right eye imaging system may include one or more inwardly facing cameras (2018, 2020). For example, the right eye imaging system may include a left eye tracking camera 2018 for the right eyepiece 2010B and a right eye tracking camera 2020 for the right eyepiece 2010B. The left eye tracking camera 2018 and the right eye tracking camera 2020 may be located to the left and right of each other, respectively, possibly to the left and right of the center of the right eyepiece, respectively. One or more cameras in the left eye tracking system 2010A and one or more cameras in the right eye tracking system 2010B may be positioned in the wearable device 2000 so as to not interfere with capturing images of the user's eyes. Other configurations are possible.

[0386] The field of view of the imaging system for the left eyepiece 2010A may be capable of imaging all or a useful portion of the user's left eye 2012A (not necessarily imaging the right eye or a portion thereof useful for eye tracking) in many different eye pose positions. Similarly, the field of view of the imaging system for the right eyepiece 2010B may be capable of imaging all or a useful portion of the user's right eye 2012B (not necessarily imaging the left eye or a portion thereof useful for eye tracking) in many different eye pose positions. For example, a user may be able to move their eyes up to 50 degrees from the central line of sight in any direction during normal movement. The imaging systems may be collectively positioned to image substantially all of the full range of motion (e.g., 50 degrees) of the user's eyes during their normal movement.

[0387] Figure 9C illustrates an example field of view 2030 of the right eye tracking camera 2016 of the left eyepiece 2010A and an example field of view 2032 of the right eye tracking camera 2020 of the right eyepiece 2010B. Figure 9D illustrates an example field of view 2040 of the right eye tracking camera 2014 of the left eyepiece 2010A and an example field of view 2042 of the right eye tracking camera 2018 of the right eyepiece 2010B. Figure 9E illustrates how the fields of view 2030 and 2040 from the left eye tracking camera 2014 and right eye tracking camera 2016 of the left eyepiece 2010A, respectively, may overlap to image substantially all of the user's left eye 2012A. In addition, Figure 9E illustrates how the fields of view 2040 and 2042 from the left eye tracking camera 2018 and the right eye tracking camera 2020 of the right eyepiece 2010B, respectively, may overlap to image substantially all of the image user's right eye 2012B. Variations are possible. For example, the number and locations of the cameras may be different. Other types of imaging systems may also be used. K. Example of Locating Center of Rotation Using an Eye Tracking System

[0388] To simplify the eye tracking system (or the process within the eye tracking module 614), it may be desirable to reduce the number of variables required to determine the center of rotation (CoR) of the human eye. Advantageously, reducing the number of variables used to determine the CoR may also improve eye tracking accuracy. For example, because the CoR may be used to determine a gaze vector for use in eye tracking, increased errors in the CoR may result in less accurate eye tracking. Errors in the CoR may result from errors introduced during the determination of variables used to calculate the CoR. For example, CoR calculation may involve extracting the pupil circle center and modeling the corneal sphere. Both of these processes may introduce errors and contribute to inaccuracies. Therefore, it may be advantageous to extract the CoR using a limited number of variables.

[0389] Described herein are systems and methods for extracting CoR primarily or entirely from corneal data. Advantageously, due to reasons similar to those discussed above, the system can improve the accuracy of eye tracking systems. For example, the system may require fewer assumptions and therefore reduce the potential for introducing errors. In addition to or as an alternative to improved accuracy, the system can improve other aspects of eye tracking systems. For example, the system may rely on shorter eye exposure to an illumination source. Shorter eye exposure can reduce risks associated with prolonged eye exposure to an illumination source, reduce illumination power consumption, and provide high ambient light rejection. In another embodiment, the system may not require a wide field of view. The reduced field of view requirement can allow more flexibility in the hardware design of the wearable system.

[0390] In some examples, the center of rotation (CoR) of the human eye can be extracted from the corneal data. FIG. 10 shows a graphical view of an example CoR extraction system 1000 that may be implemented by the eye tracking module 614. For example, the wearable system may use an illumination system comprising one or more illumination sources to generate two or more flashes of light on the cornea 1020 of the user's eye 1010. In various implementations, the illumination system comprises multiple distinct regions from which light is output. These regions may correspond to separate light emitters or light sources. The flash detection and labeling module 714 may extract the flash location on the cornea 1020 of the eye 1010. As described below, the 3D corneal center estimation module 716 may determine an approximated corneal curvature 1018 based on the flash location and calculate an estimated center 1012 of the approximated corneal curvature 1018. Different eye postures may provide different approximated corneal curvatures 1018 and associated estimated centers of corneal curvature 1012. The CoR estimation module 724 may determine an estimated CoR based on multiple estimated centers of corneal curvature 1012 by fitting a surface to the estimated centers 1012 and determining a region 1016 of convergence or intersection of a set of surface normal vectors 1014 normal to the surface. The estimated CoR may be obtained from this region 1016, e.g., the estimated CoR may be at or within this region 1016.

[0391] Optionally, the CoR estimate may be further checked using the eye tracking module 614. For example, as described in more detail below, if the CoR has moved relative to the wearable device during use of the device, a new measurement of the corneal center 1012 may be tested by measuring the distance between the newly calculated corneal center 1012 and a surface fitted to the set of calculated corneal centers 1012. If the distance is too large, the eye tracking module 614 may pause eye tracking or switch to a different method of determining the CoR or a different eye tracking method. In some embodiments, the switch may be temporary until enough data has been collected to reduce the overall error.

[0392] Advantageously, the CoR extraction 1000 may employ one or more assumptions. For example, the CoR extraction 1000 may assume that the phosphene extraction is accurate, that the geometry of the eye 1010 is known, that the radius of the cornea (or two radii in the case of corneal astigmatism) is known, or that the data was collected during normal or random movements of the user's gaze. L. Exemplary Eye Tracking Environment

[0393] As discussed above, the CoR may be determined from multiple estimated centers of corneal curvature 1012. For example, a surface may be fitted to the estimated center of corneal curvature 1012 and multiple surface normal vectors 1014 normal to this surface may be obtained. A region 1016 of convergence of the set of these surface normal vectors 1014 may be identified. An estimated CoR may be obtained from this region of convergence 1016, e.g., the estimated CoR may be at or within this region 1016.

[0394] To obtain multiple estimated centers of corneal curvature 1012, phosphenes may be produced on the eye using an illumination source and imaged by a camera as described above. FIG. 11 shows an example image of any on-eye phosphenes used by an eye tracking module to determine an estimated center of rotation. For example, as discussed above, the wearable system may include an imaging system. The imaging system may image the user's eye 1110 and produce an eye image 1101. The wearable system may include one or more illumination sources 1102 with spatially distinct regions that output light. Thus, light from the illumination source 1102 may produce one or more phosphenes 1104 on the user's eye 1110 that are reflections of these spatially distinct regions that illuminate with light.

[0395] The imaging system of the wearable system may be part of an eye tracking assembly (e.g., as shown in FIGS. 9A-E). The imaging system may include one or more cameras. For example, the imaging system may include a single camera at a location 1106 associated with the user's eye 1110. In another embodiment, the imaging system may include multiple cameras, which may be located at different locations associated with the user's eye 1110.

[0396] The illumination source 1102 can include one or more light sources, such as light emitting diodes (LEDs). The illumination source may emit light in the visible or non-visible range (e.g., infrared (IR) light). For example, the illumination source 1102 can be an infrared (IR) LED. The illumination source 1102 can be part of an eye tracking assembly (e.g., as illustrated in FIGS. 9A-E).

[0397] The illumination source 1102 may produce one or more specular reflections 1104 on the cornea of ​​the user's eye 1110. The specular reflections 1104 may also be referred to as flashes. For example, there may be two illumination sources (1102A, 1102B). The illumination sources may be configured to produce two or more discrete flashes (1104A, 1104B) on the user's eye 1110. FIG. 11 shows an image of the user's eye with the flashes thereon. FIG. 11 also shows a view of the camera 1106 (represented by the origin of the coordinate system) relative to the eye 1110 and the location of the flashes 1104A, 1104B thereon and to the illumination sources 1102A, 1102B in their relative locations. M. Exemplary Extraction of a Vector Along Which the Corneal Center Lies Using a Single Camera

[0398] As discussed above, a camera at location 1106 may image the flashes of light 1104A, 1104B produced by illumination sources 1102A, 1102B on the user's eye 1110. In FIG. 12A-D, a first plane 1220 may be determined that includes the flash of light 1104A, the camera capturing an image of the flash of light at location 1106, and the location of the illumination source 1102A that produces the flash of light. In particular, the module may determine the first plane 1220 that includes the first illumination source 1102A and the first flash of light 1104A. Similarly, as illustrated in FIG. 13A-D, a second plane 1320 may be determined that includes the flash of light 1104B, the camera capturing an image of the flash of light at location 1106, and the location of the illumination source 1102B that produces the flash of light. In particular, the module 716 may determine a second plane 1320 based on the second illumination source 1102B and the second flash of light 1104B. As illustrated in Figures 14A-C, the module 716 may determine an intersection between the first plane 1220 and the second plane 1320. The intersection between the first plane 1220 and the second plane 1320 may define a vector 1410 that is oriented along where the corneal center is located. As shown in Figures 14A-C, this vector 1410 may also extend along a direction that includes the camera location 1106.

[0399] In some implementations, the module 716 may determine the first plane 1220 by determining a set of lines 1210, 1212, 1214 between the first illumination source 1102A, the first flash of light 1104A, and the camera location 1106. As illustrated in FIG. 12A, the module 716 may determine a first line 1210 extending between the camera location 1106 and a location in the image plane 1101A of the first flash of light 1104A that may be produced by the first illumination source 1102A. As illustrated in FIG. 12B, the module 716 may determine a second line 1212 extending between the camera location 1106 and a location of the illumination source 1102A that produced the first flash of light 1104A. 12C, module 716 may determine a third line 1214 that is projected between a location in image plane 1101A of illumination source 1102A and first flash 1104A. As illustrated in FIG. 12D, any two of these lines 1210, 1210, and 1214 may define a plane 1220 within which the corneal center may reside.

[0400] Similarly, in some implementations, the module 716 may determine the second plane 1320 by determining a set of lines 1310, 1312, 1314 between the second illumination source 1102B, the second flash of light 1104B, and the camera location 1106. As illustrated in FIG. 13A, the module 716 may determine a first line 1310 extending between the camera location 1106 and a location in the image plane 1101A of the second flash of light 1104B that may be produced by the second illumination source 1102B. As illustrated in FIG. 13B, the module 716 may determine a second line 1313 extending between the camera location 1106 and a location of the second illumination source 1102B that produced the second flash of light 1104A. 13C, module 716 may determine a third line 1314 extending between a location in image plane 1101A of second flash of light 1104B and second illumination source 1102B. As shown in FIG. 13D, lines 1310, 1310, and 1314 may define a plane 1320 within which the corneal center may reside.

[0401] However, in some implementations, the first plane 1220 can be determined directly from the locations of the first illumination source 1102A and the first flash 1104A and the camera location 1106, without necessarily separately defining the lines 1210, 1210, and 1214. Similarly, the second plane 1320 can be determined directly from the locations of the second illumination source 1102B and the second flash 1104B and the camera location 1106, without necessarily separately defining the lines 1310, 1310, and 1314.

[0402] Module 716 may identify an intersection between first and second planes 1220 and 1320. As illustrated in Figures 14A and 14B, the intersection of first plane 1220 and second plane 1320 may define a vector 1410 that extends along a direction that may involve or otherwise include the origin of camera location 1106. As shown in Figure 14C, vector 1410 may point toward the corneal center location.

[0403] The module 716 may repeat the estimation process multiple times to generate one or more corneal vectors 1410. For example, the module 716 may determine a first plane 1220 and use it to define a vector based on the first illumination source 1102A and the first flash of light 1104A with multiple different camera locations 1106. The camera locations 1106 can be varied in relation to the user's eye 1110 (e.g., in relation to the distance to the user's eye 1110, or the horizontal or vertical position relative to the eye, or any combination thereof) or in relation to the location of the illumination source (1102A, 1102B). The module 716 may determine a vector 1410 for one or more of the camera locations 1106. The module 716 may then determine the corneal center from the intersection of the two or more vectors, as described above. If the two or more vectors do not intersect, the corneal center may be interpolated or otherwise extrapolated from the vector data. Additionally or alternatively, the eye tracking module 614 may collect and analyze more data to determine the corneal center.

[0404] The module 716 may repeat the estimation process while varying one or more parameters associated with the eye tracking environment 1100. For example, the module 716 may repeat the process with different camera locations or for different gaze directions of the user's eyes. The eye tracking module 614 may utilize gaze targets to ensure that the user maintains their eye posture while the parameters are varied. For example, the eye tracking module 614 may estimate one or more vectors 1410 while the user directs their gaze to the gaze target while varying a parameter such as the camera location 1106 or the location of the illumination source 1102. Additionally or alternatively, the eye tracking module 614 may estimate one or more vectors 1410 while the user naturally moves their gaze while using the wearable device. For example, the eye tracking module 614 may capture data associated with different parameters during the natural movement of the user's eyes.

[0405] The iterative estimation process may result in multiple vectors 1410 pointing toward the corneal center associated with a particular eye pose. Module 716 may determine an area of ​​intersection or convergence of the multiple vectors 1410 to generate an estimated center of corneal curvature. N. Exemplary extraction of vectors along which the corneal center is located using multiple cameras

[0406] In various implementations, multiple cameras may be employed to image the eye, and images from the multiple cameras may be used to determine the center of corneal curvature of the eye. In particular, module 716 may determine a vector (1510, 1530) along which the corneal center may be located. Figures 15A-16C illustrate steps in an exemplary process for determining such a vector using multiple cameras. For example, as illustrated in Figure 15A, a first camera at a first location 1506 may image flashes 1504A, 1504B produced by illumination sources 1502A, 1502B on the user's eye 1501, and a second camera at a location 1526 may image flashes 1524A, 1524B produced by illumination sources 1522A, 1522B on the user's eye 1501. The module 716 may determine a first vector 1510 based on data associated with a first camera and illumination source 1502A, 1502B at a location 1506 and may determine a second vector 1530 associated with a second camera and illumination source 1522A, 1522B at a location 1526. As illustrated in FIG. 15B , the module 716 may estimate the corneal center 1520 by determining the convergence or intersection between the first vector 1510 and the second vector 1530.

[0407] To obtain the first vector 1510, the module 716 may identify a first plane 1512 by determining a set of lines (not shown) between the first illumination source 1502A, the first flash location 1504A in the image plane 1503A, and the first camera at the first location 1506. The module 716 may determine a second plane 1514 by determining a set of lines (not shown) between the second illumination source 1502B, the second flash location 1504B in the image plane 1503A, and the second camera location 1506. The module 716 may determine the vector 1510 by determining the intersection between these first and second planes 1512 and 1514. The intersection of these planes 1512 and 1514 may define a vector 1510 with an origin at the camera location 1506 that points toward the central location of the corneal curvature.

[0408] However, in some implementations, the first plane 1512 can be determined directly from the locations of the first illumination source 1502A, the first flash 1504A, and the first camera 1106, without necessarily separately defining one or more lines. Similarly, the second plane 1514 can be determined directly from the locations of the second illumination source 1502B, the second flash 1504B, and the first camera 1506, without necessarily separately defining one or more lines.

[0409] The module 716 may similarly determine a first plane 1532 by determining a set of lines (not shown) between the first illumination source 1522A, the first flash location 1524A in the image plane 1503B, and the first camera at location 1526. The module 716 may determine a second plane 1534 by determining a set of lines (not shown) between the second illumination source 1522B, the second flash location 1524B in the image plane 1503B, and the camera location 1526. The module 716 may determine a second vector 1530 by determining an intersection between these first and second planes 1532 and 1534. The intersection of the planes 1532 and 1534 may define a vector 1530 with an origin at the camera location 1526, which may point toward the central location of the corneal curvature. However, in some implementations, the first plane 1532 can be determined directly from the locations of the first illumination source 1522A, the first flash 1524A, and the second camera 1526, without necessarily separately defining one or more lines. Similarly, the second plane 1534 can be determined directly from the locations of the second illumination source 1522B, the second flash 1524B, and the second camera 1526, without necessarily separately defining one or more lines.

[0410] As illustrated in FIG. 15B, the module 716 may determine a center of corneal curvature location based on these first and second vectors 1510 and 1530. For example, the module 716 may determine a convergence or intersection 1520 of these vectors 1510 and 1530. The convergence or intersection 1520 may correspond to an approximate corneal center location. If the vectors 1510 and 1530 do not intersect, the center of corneal curvature may be interpolated or otherwise extrapolated from the vector data. Additionally or alternatively, the eye tracking module 614 may collect and analyze more data to determine the center of corneal curvature 1520.

[0411] 16A-16C illustrate another exemplary process for determining the center of corneal curvature using multiple cameras. As illustrated in FIG. 16A, the wearable system may have a set of shared illumination sources 1602A, 1602B that may be used in conjunction with multiple eye cameras. The shared illumination sources 1602A, 1602B may additionally or alternatively be a set of separate illumination sources associated with one or more cameras. The set of shared illumination sources 1602A, 1602B may produce flashes of light 1604A, 1604B, 1604C, 1604D on the user's eye.

[0412] 16B, the module 716 may determine a set of planes using the shared illumination sources 1602A, 1602B. For example, the module 716 may determine a first plane 1630 by determining a set of lines (not shown) between the first illumination source 1602A, a first flash location 1604A in the image plane 1503A, and a first camera at location 1506. The module 716 may determine a second plane 1632 by determining a set of lines (not shown) between the second illumination source 1602B, a second flash location 1604B in the first image plane 1503A, and the first camera location 1506.

[0413] However, in some implementations, the first plane 1630 can be determined directly from the locations of the first illumination source 1602A, the first flash 1604A in the first image plane 1503A, and the first camera 1506, without necessarily separately defining one or more lines. Similarly, the second plane 1632 can be determined directly from the locations of the second illumination source 1602B, the second flash 1604B, and the first camera 1506, without necessarily separately defining one or more lines.

[0414] The module 716 may determine the first distinct plane 1634 by determining a set of lines (not shown) between the first illumination source 1602A, the first flash location 1604C in the image plane 1503B, and the second camera at location 1526. The module 716 may determine the separate distinct plane 1636 by determining a set of lines (not shown) between the second illumination source 1602B, the second flash location 1604D in the second image plane 1503B, and the second camera location 1526.

[0415] However, in some implementations, the distinct first plane 1634 can be determined directly from the locations of the first illumination source 1602A, the first flash 1604C in the image plane 1503B, and the second camera 1526, without necessarily separately defining one or more lines. Similarly, the distinct second plane 1636 can be determined directly from the locations of the second illumination source 1602B, the second flash 1604D, and the second camera 1526, without necessarily separately defining one or more lines.

[0416] 16C , module 614 may determine the intersection between planes 1630 and 1632 to determine vector 1610. The intersection of planes 1630 and 1632 may define vector 1610, with an origin at camera location 1506, which may point toward the corneal center location. Similarly, module 614 may determine the intersection between planes 1634 and 1636 to determine vector 1630. The intersection of planes 1634 and 1636 may define vector 1630, with an origin at camera location 1526, which may point toward the corneal center location.

[0417] 16C , the module 716 may determine a location of the center of corneal curvature based on the vectors 1610 and 1630. For example, the module 716 may determine a convergence or intersection 1620 of the first and second vectors 1610 and 1630. The convergence or intersection 1620 may correspond to an approximate location of the center of corneal curvature. If the first and second vectors 1610 and 1630 do not intersect, the center of corneal curvature may be interpolated or otherwise extrapolated from the vector data. Additionally or alternatively, the eye tracking module 614 may collect and analyze more data to determine the center of corneal curvature.

[0418] The module 716 may repeat the estimation process for multiple gaze directions of the user's eyes. For example, the wearable system may display one or more gaze targets to which the user may direct his / her gaze. The eye tracking module 614 may estimate one or more vectors 1410 while the user directs his / her gaze to the gaze targets. Additionally or alternatively, the eye tracking module 614 may estimate one or more vectors 1410 while the user naturally moves his / her gaze while using the wearable device. For example, the eye tracking module 614 may capture data associated with different parameters during natural movement of the user's eyes. As described below, data captured at different eye postures or gaze vectors of the user's eyes may be used to calculate multiple corneal centers, which may be used by the CoR estimation module 724 to estimate the CoR. O. Estimation of the center of rotation

[0419] The center of rotation (CoR) estimation module 724 may determine an estimated center of rotation based on an estimated center of corneal curvature 1012. For example, the CoR estimation module 724 may fit a surface to one or more estimated corneal centers of curvature and determine a set of surface normal vectors normal to the fitted surface. The surface normal vectors may converge or intersect at a point or region that may correspond to the estimated CoR.

[0420] To determine the surface, module 614 may analyze multiple eye images. For example, the wearable system may image the user's eye 1501 (e.g., with an inward-facing imaging system 462) while the user's eye 1501 is in one or more eye postures. In some implementations, module 614 may prompt one or more eye postures or gaze directions through the display of gaze targets on a display of the wearable device. Additionally or alternatively, module 614 may collect data associated with one or more eye postures that naturally occur during use of the wearable device.

[0421] 17A and 17B, module 614 may determine multiple corneal centers of curvature 1712 based on data collected by the wearable system while the user's eye is in one or more eye postures. For example, module 614 may perform a corneal curvature center estimation process using one or more cameras as part of module 716, as described above, multiple times (e.g., for different gaze directions or eye postures of the user's eye 1501). The output of the corneal center estimation process of module 716 may include multiple estimated corneal centers of curvature 1712.

[0422] The corneal centers of curvature 1712 may be located within a region 1710 in three-dimensional (3D) space. The region 1710 may fall within the corneal sphere 1022. Without subscribing to any particular scientific theory, the corneal centers of curvature 1712 may be approximately aligned within the region 1710 according to the shape of the corneal curvature 1018. For example, the corneal centers of curvature 1712 may be aligned within the region 1710 to outline a shape that is approximately parallel to or substantially identical to the shape of the cornea 1020. If the cornea is approximately spherical, the corneal centers 1712 may approximately follow the corneal curvature 1018 at a distance that approximately corresponds to a radius of the cornea. In the case of astigmatism (or if the cornea is not approximately spherical), the corneal centers 1712 may approximately follow the corneal curvature 1018 at a distance that approximately corresponds to one or more radii of the corneal geometry.

[0423] In various implementations, the module 614 may determine whether the multiple corneal centers 1712 fall within a determined tolerance of an expected distance from the surface 1022 of the cornea to the center 1022 of the corneal sphere. For example, the corneal sphere 1022 may be spherical or astigmatic (e.g., having a geometry other than a spherical shape). The expected distance may correspond to a distance to the center of the geometry of the corneal sphere 1022. For example, if the corneal geometry is spherical, the expected distance may be the radius of the corneal sphere 1022. If the corneal centers 1712 fall outside the determined tolerance, the module 614 may reduce the contribution of outliers in further analysis. For example, the module 614 may exclude the outlier data points from further analysis. Additionally or alternatively, if a threshold number of corneal centers 1712 fall outside the determined tolerance, module 614 may stop analysis until further data is obtained or switch to a different method of determining the center of rotation.

[0424] As shown in FIG. 17B, the module 724 may fit a 3D surface 1714 to the multiple corneal centers 1712. The module 724 may fit the 3D surface using, for example, regression analysis. The module 724 may determine the fit utilizing a suitable surface or curve fitting technique. The module 724 may fit the corneal centers 1712 to a low order polynomial 3D surface 1714 using, for example, polynomial regression. In another example, the module 724 may apply a geometric fit to the corneal centers 1712 (e.g., a total least squares fit). In some examples, the surface 1714 may have a similar curvature to the corneal curvature 1018. In other examples, the surface 1714 may have a different shape than the corneal curvature 1018.

[0425] Module 724 may determine a set of surface normal vectors that are normal to surface 1714. FIG. 18A illustrates an example calculation 1800 of the CoR (or eyeball center "EBC") using surface normal vectors 1814. For example, module 716 may determine a set of estimated corneal centers 1812. Module 724 may fit surface 1714 (as shown in FIG. 17B) to the estimated corneal centers 1812. Module 724 may then determine one or more surface normal vectors 1814 that are normal to surface 1714. The surface normal vectors 1814 may result from an estimated center of corneal curvature 1812. For example, module 724 may determine a surface normal vector 1814 for each estimated center of corneal curvature 1812 used to determine surface 1714. Fewer surface normals may be used in some implementations. Additionally or alternatively, the surface normal vector 1814 may originate from other points on the surface 1714 .

[0426] The module 724 may determine a region 1802 of convergence of the surface normal vectors 1814. For example, as illustrated in the inset 1801 of FIG. 18A, some or all of the surface normal vectors 1814 may converge or intersect within the region 1802 of 3D space. The region 1802 of 3D space may be a point of intersection or a volume of 3D space (e.g., volume 1920 in FIGS. 19C and 19D) where the normal vectors intersect and / or converge. The volume of 3D space may be centered around the central point of intersection or convergence of the surface normal vectors 1814. The volume of 3D space may be large enough to encompass most of the points of intersection.

[0427] The region of convergence 1802 may include different areas of convergence or intersection corresponding to different gaze directions or eye postures. For example, the region of convergence 1802 may include a sub-region 1820 corresponding to a first gaze direction (e.g., downward gaze) and a sub-region 1822 corresponding to a second gaze direction (e.g., upward gaze). In some examples, the sub-regions 1820, 1822 may correspond to approximated CoRs associated with regions of a display of the wearable device. For example, the first sub-region 1820 may correspond to an upper region of the display and the second sub-region 1822 may correspond to a lower region of the display.

[0428] The module 724 may determine the CoR by analyzing the region of convergence 1802. For example, the module 724 may determine the CoR by determining the mode or median of the convergence or intersection of the vectors 1814. Additionally or alternatively, the module 724 may determine the CoR by first determining a line-of-sight based convergence or intersection, such as the mode or median of the convergence or intersection of the vectors 1814 in the sub-regions 1820, 1822, and then determining the mode or median based on those line-of-sight based convergence or intersection. Additionally or alternatively, the module 724 may perform a different analysis of the convergence or intersection to determine the CoR. For example, the module 724 may utilize a machine learning algorithm to determine the CoR.

[0429] In some examples, variations in the calculated corneal center of curvature may result in a wider region of convergence 1824 as opposed to a single point of intersection. FIG. 18B illustrates an example CoR calculation 1803 using region 1824. For example, the calculated corneal center of curvature 1832 may be noisy relative to the fitted 3D surface 1830. The noisy corneal center 1832 may result in region 1824 in which the CoR or eyeball center (EBC) is likely based on the intersection of a vector (not shown) with an origin at the corneal center 1832. In some implementations, module 614 may use region 1824 in calculating gaze direction. For example, module 614 may determine the CoR as the center of region 1824 or some other location within or on region 1824 or based otherwise thereon.

[0430] In various implementations, the module 724 may select a portion of the estimated corneal center 1910 and determine the CoR. FIGS. 19A-1 and 19A-2 illustrate an example surface 1912 that is fitted to a portion of the estimated corneal center 1910 that may be selected using a data reduction process. FIGS. 19B-1 and 19B-2 show an example vector 1916 that may be normal to the surface 1912. The vector 1916 may occur at the selected estimated corneal center 1910. FIGS. 19C-1 and 19C-2 illustrate an estimated CoR region 1920 based on the points of area convergence or intersection of the vectors 1916. As shown in FIGS. 19D-1 and 19D-2, where the module 724 does not select the corneal center 1910 for fitting to the surface 1912, many of the vectors 1922 may not converge or intersect within the region 1920.

[0431] In various implementations, the module 724 may select the estimated corneal centers 1910 based on the determined area of ​​convergence of the normal vectors 1916. For example, the module 724 may determine a large area in which the normal vectors 1922 intersect. In some implementations, if the large area has a volume that exceeds a threshold volume, the module 724 may determine a smaller set of corneal centers 1910 and use it to determine the CoR. In some implementations, the threshold volume may include a suitable volume for determining the CoR that is associated with a threshold accuracy of the eye tracking based on its CoR. For example, a volume of 30 percent of the user's eye volume may be associated with an 80% decrease in accuracy in eye tracking. If the determined volume exceeds the threshold volume, the module 724 may select a smaller set of corneal centers 1910 based on any number of suitable data selection criteria, as described below.

[0432] Additionally or alternatively, the module 724 may select an estimated corneal center 1910 for analysis using any number of data reduction processes, such as machine learning algorithms or filtering processes. For example, the module 724 may filter the data to eliminate outliers. The filter may include determining a confidence score associated with a given corneal center 1910 and selecting the corneal center 1910 based on the confidence score. In some examples, the confidence score may be determined based on the deviation of the corneal center of the curvature 1910 from a secondary calculation or the determination of the corneal center of the curvature 1910 or surface 1912. In some examples, the confidence score may be based on the location of the corneal center of the curvature 1910 relative to the fitted surface 1912 (e.g., the deviation of the corneal center 1910 from the fitted surface 1912). In some examples, the confidence score may be determined based on the error calculated within the scintillation extraction utilized to determine the corneal center of the curvature 1910. For example, glint extraction may have high error if there is error in the eye image that is analyzed to extract the glint (e.g., due to blur, obstructions in the image, distortion, or other noise sources). P. Exemplary Uses of Rotational Extraction of the Center of Corneal Curvature

[0433] 20 illustrates an example center of rotation extraction process 2100 that may be implemented by the eye tracking module 614. For example, the center of rotation extraction process 2100 may include one or more corneal center estimation processes 2108, an application block 2116, a vector determination block 2118, a convergence or intersection determination block 2120, and a center of rotation determination block 2122.

[0434] The module 614 can implement several blocks as part of one or more corneal center estimation processes 2108. For example, the corneal center estimation process 2108 can include an image receiving block 2110, a phosphene determination block 2112, and a corneal center determination block 2114.

[0435] In the image receiving block 2110, the module 614 can receive one or more images of the user's eye. The images can be obtained from an imaging system associated with a wearable device worn by the user. For example, the wearable device can be a head mounted display including a left eyepiece 2010A and a right eyepiece 2...

Claims

1. A display system configured to display virtual image content within a field of view of a user by projecting light onto the user's eye, the display system comprising: a frame configured to be supported on a head of the user; and a head mounted display disposed on the frame, the head mounted display configured to display virtual image content in the user's field of view by projecting light into the user's eye; a first eye tracking camera and a second eye tracking camera configured to image the user's eye; A plurality of light emitters; processing electronics in communication with the head mounted display and the first and second eye tracking cameras; Equipped with The processing electronics include: receiving images of the user's eye captured by the first eye tracking camera and the second eye tracking camera, wherein flashing reflexes of a plurality of different light emitters are observable in the images of the eye captured by the first eye tracking camera and the second eye tracking camera; estimating a location of a center of corneal curvature of the user's eye based on a location of the flash of light in the images generated by both the first eye tracking camera and the second eye tracking camera, and based on the locations of both the first eye tracking camera and the second eye tracking camera and the location of the light emitter that generated each flash of light; determining a locus of points corresponding to estimates of centers of corneal curvature of the user's eye for a plurality of different eye postures; determining a surface based on the locus of points; obtaining an estimate of a center of rotation of the user's eye by estimating a center of curvature of the surface; A display system configured to:

2. The processing electronics include: determining a first direction toward a center of corneal curvature of the user's eye based on a location of a flash of light in one or more images produced by the first eye tracking camera and based on a location of the first eye tracking camera and a location of the light emitter that produced the flash of light; determining a second direction toward a center of corneal curvature of the user's eye based on a location of the flash of light in one or more images produced by the second eye tracking camera and based on a location of the second eye tracking camera and a location of the light emitter that produced the flash of light; The display system of claim 1 configured to:

3. The processing electronics include: defining a first plane that includes the first eye tracking camera and a location of a first flash of light reflection and a location of the light emitter corresponding to the first flash of light reflection; defining a second plane that includes the first eye tracking camera and a location of a second flash of light reflection and a location of the light emitter corresponding to the second flash of light reflection; determining a first intersection of the first plane and the second plane, the first intersection defining a first vector, the first vector extending along the first direction; 3. The display system of claim 2, configured to determine the first orientation by:

4. The processing electronics include: defining a third plane that includes the second eye tracking camera and a location of a third flash of light reflection and a location of the light emitter corresponding to the third flash of light reflection; defining a fourth plane that includes the second eye tracking camera and a location of a fourth flash of light reflection and a location of the light emitter corresponding to the fourth flash of light reflection; determining a second intersection of the third plane and the fourth plane, the second intersection defining a second vector, the second vector extending along the second direction; 4. The display system of claim 3, configured to determine the second orientation by:

5. The display system of any one of claims 2 to 4, wherein the processing electronics is configured to estimate a location of a center of corneal curvature of the user's eye based on the first direction and the second direction toward a center of corneal curvature of the user's eye.

6. The processing electronics include: determining the first direction in which a center of corneal curvature of the user's eye is estimated to be located based on at least one first image received from the first eye tracking camera; determining the second direction in which a center of corneal curvature of the user's eye is estimated to be located based on at least one second image received from the second eye tracking camera; The device is configured to: The display system of any one of claims 2 to 5, wherein the first direction and the second direction converge towards a certain area.

7. 7. The display system of claim 2, wherein the processing electronics is configured to derive an estimate of a center of corneal curvature of the user's eye based on where the first direction and the second direction converge.

8. 8. The display system of claim 2, wherein the processing electronics is configured to estimate a location of a center of corneal curvature of the user's eye by identifying a region where the first direction and the second direction converge toward a center of corneal curvature of the user's eye.

9. 9. The display system of claim 1, wherein the processing electronics is configured to derive an estimate of a center of rotation of the user's eye based on multiple determinations of the center of corneal curvature of the user's eye for multiple different eye postures.

10. 2. The display system of claim 1, wherein the processing electronics is configured to derive an estimate of a center of rotation of the user's eye based on the locus of points corresponding to estimates of a center of corneal curvature of the user's eye for a plurality of different eye postures.

11. A display system configured to display virtual image content within a field of view of a user by projecting light onto the user's eye, the display system comprising: a frame configured to be supported on a head of the user; and a head mounted display disposed on the frame, the head mounted display configured to display virtual image content in the user's field of view by projecting light into the user's eye; a first eye tracking camera and a second eye tracking camera configured to image the user's eye; A plurality of light emitters; processing electronics in communication with the head mounted display and the first and second eye tracking cameras; Equipped with The processing electronics include: receiving images of the user's eye captured by the first eye tracking camera and the second eye tracking camera, wherein flashing reflexes of a plurality of different light emitters are observable in the images of the eye captured by the first eye tracking camera and the second eye tracking camera; estimating a location of a center of corneal curvature of the user's eye based on a location of the flash of light in the images generated by both the first eye tracking camera and the second eye tracking camera, and based on the locations of both the first eye tracking camera and the second eye tracking camera and the location of the light emitter that generated each flash of light; determining a locus of points corresponding to estimates of centers of corneal curvature of the user's eye for a plurality of different eye postures; determining a surface based on the locus of points and obtaining an estimate of a center of rotation of the user's eyes by determining a region where a plurality of normals to the surface converge; A display system configured to:

12. 12. A display system as claimed in claim 1 or claim 11, wherein the processing electronics is configured to obtain the surface by fitting the surface to the locus of points.

13. A display system according to any preceding claim, wherein the processing electronics is configured to render a virtual image to be presented to the user's eye by using a rendering camera, the rendering camera having a position determined by the centre of rotation.

14. A display system as described in any one of claims 1 to 13, wherein the head mounted display is configured to display virtual image content in the user's field of view by projecting light into the user's eye such that the displayed virtual image content appears to originate from a plurality of different depths.

15. 15. A display system as described in any one of claims 1 to 14, wherein the head mounted display is configured to display virtual image content in the user's field of view by projecting light into the user's eye at at least one different amount of divergence such that the displayed virtual image content appears to originate from a plurality of different depths.

16. The display system of any one of claims 1 to 15, wherein the head-mounted display is configured to project diverging light into the user's eye and to project collimated light into the user's eye to display virtual image content in the user's field of view that appears to originate from a plurality of different depths.

Citation Information

Patent Citations

  • Visual axis measuring apparatus

    JP2009297323A

  • Method and apparatus for measuring visual axis

    JP2012029940A

  • See-through display brightness control

    US20130114043A1

  • Dynamic display calibration based on eye-tracking

    US20170124928A1

  • Head pose and distraction estimation

    US20190213402A1