Angularly segmented hot mirrors for eye tracking

Abstract

Providing angle-segmented hot mirrors for eye tracking. [Solution] An embodiment of an imaging system for use with a head-mounted display (HMD) is disclosed. The imaging system may include a forward-facing imaging camera, and the surface of the HMD's display may include an off-axis diffractive optical element (DOE) or hot mirror configured to reflect light to the imaging camera. The DOE or hot mirror may be segmented, for example, using different segments having different angles or different refractive powers. The imaging system can be used for eye tracking, biometric identification, multi-view reconstruction of the three-dimensional shape of the eye, etc. A method for manufacturing angularly segmented optical elements is also provided. This method may include injection molding.

Description

[Technical Field]

[0001] (Cross-reference of related applications) This application claims the benefit of priority of U.S. Provisional Application No. 62 / 880,499, filed on 30 July 2019, titled "Angularly Segmented Hot Mirror for Eye Tracking," which is incorporated herein by reference in its entirety by reference under 35 U.S.C § 119(e) (U.S. Patent Act).

[0002] This disclosure relates to virtual reality and augmented reality imaging and visualization systems, imaging systems for acquiring images of the eye, and methods for manufacturing optical elements for these imaging systems. [Background technology]

[0003] Modern computing and display technologies are driving the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or parts thereof are presented to the user in a manner that appears, or can be perceived, as real. Virtual reality, or "VR," scenarios typically involve the presentation of digital or virtual image information without transparency to other real-world visual inputs; augmented reality, or "AR," scenarios typically involve the presentation of digital or virtual image information as an extension of the visualization of the real world around the user; and mixed reality, or "MR," relates to the fusion of the real and virtual worlds to create a new environment in which physical and virtual objects coexist and interact in real time. In conclusion, the human visual perception system is highly complex, making it challenging to produce VR, AR, or MR technologies that facilitate a comfortable, natural, and rich presentation of virtual image elements among other virtual or real-world image elements. The systems and methods disclosed herein address various challenges related to VR, AR, and MR technologies. [Overview of the project] [Means for solving the problem]

[0004] An embodiment of a head-mounted display (HMD) configured to be worn on the user's head is disclosed. The HMD comprises a frame having a pair of temples; a pair of optical elements supported by the frame such that each of the pair of optical elements can be positioned in front of the user's eyes; a forward-facing imager mounted on one of the temples; and a reflective element positioned within or on one of the pair of optical elements, configured to reflect infrared light toward the forward-facing imager, which is configured to receive infrared light reflected by the reflective element. Each of the pair of optical elements may be transparent to visible light. The reflective element may include multiple segments having the same or different optical properties. The imager may be configured to obtain an image of the wearer's eyes. The HMD may include a processor that analyzes the image obtained by the imager for eye tracking, biometric identification, multi-view reconstruction of eye shape, estimation of the eye's accommodation state, or imaging of the eye's retina. Reflective elements can be segmented using different segments with different angles or different refractive powers.

[0005] Embodiments of imaging systems for use with head-mounted displays (HMDs) are disclosed. The imaging system may include a forward-facing imaging camera, and the surface of the HMD's display may include off-axis diffractive optical elements (DOEs) or hot mirrors configured to reflect light to the imaging camera. The DOEs or hot mirrors may be segmented, for example, using different segments having different angles or different refractive powers. The imaging system can be used for eye tracking, biometric identification, multi-view reconstruction of the three-dimensional shape of the eye, and the like. Methods for manufacturing angularly segmented optical elements are also provided. These methods may include injection molding. Several embodiments are provided below.

[0006] Example 1: A method for manufacturing a segmented hot mirror, comprising the steps of: providing a first mold having a first cavity, the first cavity comprising a first surface having a first portion that is at a non-zero angle with respect to a second portion; positioning a hot mirror film adjacent to at least a first portion and a second portion of the first surface of the first cavity; introducing a first polymer material into the first cavity of the first mold to form a first molded component; removing the first molded component from the first mold, the first molded component comprising at least a portion of the hot mirror film; positioning the first molded component in a second mold having a second cavity; introducing a second polymer material into the second cavity to form a second molded component, the second polymer material covering at least a portion of the hot mirror film; and removing the second molded component from the second mold.

[0007] Example 2: The method according to Example 1, wherein the non-zero angle is in the range of 2 to 25 degrees.

[0008] Example 3: The method according to Example 1, wherein the non-zero angle is in the range of 5 to 20 degrees.

[0009] Example 4: The method according to any one of Examples 1-3, wherein the hot mirror film is substantially transparent to visible light and substantially reflective to infrared light.

[0010] Example 5: The method according to any one of Examples 1-4, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of about 800 nm to 900 nm.

[0011] Example 6: The method according to any one of Examples 1-5, wherein the first polymer is the same as the second polymer.

[0012] Example 7: The method according to any one of Examples 1-6, wherein the first polymer or the second polymer is substantially transparent to visible light and infrared light.

[0013] Example 8: The method according to any one of Examples 1-7, wherein the first polymer or the second polymer comprises a thermoplastic polymer.

[0014] Example 9: The method according to any one of Examples 1-8, wherein the first polymer or the second polymer comprises polycarbonate or polymethyl methacrylate (PMMA).

[0015] Example 10: The method according to any one of Examples 1-9, further comprising removing a portion of the hot film that extends outside the first molded component or the second molded component.

[0016] Example 11: The step of disposing the first molded component within a second mold having a second cavity comprises orienting the first molded component such that the hot mirror film is disposed toward the central region of the second cavity. The method according to any one of Examples 1-10.

[0017] Example 12: The method according to any one of Examples 1-11, wherein the first mold has a vent between the first portion and the second portion.

[0018] Example 13: The method according to any one of Examples 1-12, further comprising disposing at least one infrared light source within the second cavity of the second mold.

[0019] Example 14: The method according to Example 13, wherein at least one infrared light source is disposed on the polymer film, and the method comprises disposing the polymer film within the second cavity of the second mold.

[0020] Example 15: The polymer film is the method described in Example 14, comprising polyethylene terephthalate (PET).

[0021] Example 16: The method according to any one of Examples 1-15, wherein the first surface of the first mold comprises a third portion adjacent to a second portion, the third portion being at a second non-zero angle with respect to the second portion.

[0022] Example 17: The method according to any one of Examples 1-16, wherein a first or second portion of the first surface comprises a curved region.

[0023] Example 18: The method according to any one of Examples 1-17, further comprising the step of attaching a second molded component to a display for an augmented, composite, or virtual reality device.

[0024] Example 19: A method for forming an optical element, comprising the steps of: positioning an optical film adjacent to a segmented surface of a first mold, the segmented surface comprising a first portion and a second portion at a non-zero angle with respect to the first portion, and the optical film being substantially transparent in a first wavelength range and substantially reflective in a second wavelength range different from the first wavelength range; introducing a first polymer into a first cavity of the first mold to form a first optical element, the first polymer being substantially transparent in a first and second wavelength range, and the first optical element comprising at least a portion of an optical film; positioning the first optical element in a second mold; introducing a second polymer into a second mold to form a second optical element, the second polymer covering at least a portion of the optical film of the first optical element; and removing the second optical element from the second mold.

[0025] Example 20: The method according to Example 19, wherein the first wavelength range includes at least a portion of the visible wavelength range, and the second wavelength range includes at least a portion of the infrared wavelength range.

[0026] Example 21: The method according to Example 19 or Example 20, wherein the first mold is provided with a vent between the first part and the second part.

[0027] Example 22: The method according to any one of Examples 19-21, wherein the first or second portion of the segmented surface is substantially flat.

[0028] Example 23: The method according to any one of Examples 19-22, further comprising the step of terminating the optical film at the first edge of the first optical element or the second edge of the second optical element.

[0029] Example 24: The method according to any one of Examples 19-23, further comprising the step of placing a light source in a second mold.

[0030] Example 25: The method according to any one of Examples 19-24, wherein the non-zero angle is in the range of 2 to 25 degrees.

[0031] Example 26: The method according to any one of Examples 19-25, wherein the optical film comprises a diffractive element or a holographic element.

[0032] Example 27: A method for forming an optical element, comprising the steps of: applying an optical film to a first surface of a first optical element, the first surface comprising a first section and a second section, the second section being at a non-zero angle with respect to the first section; and applying the second optical element to the first optical element such that the optical film is positioned between the first optical element and the second optical element, thereby forming an optical element.

[0033] Example 28: The method according to Example 27, wherein the first optical element, the second optical element, and the optical film are optically transparent in the visible range.

[0034] Example 29: The method according to Example 28, wherein the optical film is optically reflective in infrared light, and the first and second optical elements are optically transparent in infrared light.

[0035] Example 30: The method according to any one of Examples 27-29, wherein the step of applying the optical film includes the step of adhering the optical film onto the first surface.

[0036] Example 31: The method according to any one of Examples 27-29, wherein the step of applying the optical film includes the step of depositing the optical film onto a first surface.

[0037] Example 32: The method according to any one of Examples 27-31, further comprising the step of injection molding a first optical element.

[0038] Example 33: The method according to any one of Examples 27-32, wherein the step of applying the second optical element includes the step of injection molding.

[0039] Example 34: A segmented hot mirror formed according to any one of the methods described in Examples 1-33.

[0040] Example 35: An optical element formed according to any one of the methods of Examples 1-34.

[0041] Example 36: A display comprising the segmented hot mirror described in Example 34 or the optical element described in Example 35.

[0042] Example 37: An augmented, virtual, or mixed reality display device comprising the display described in Example 36.

[0043] Example 38: A method for manufacturing a hot mirror configured for coupling with a display element, comprising the steps of: positioning a hot mirror layer along first and second surfaces of a first mold, wherein the first surface forms a rising angle with the plane of the second surface, the first and second surfaces of the first mold align with the first surface of a layer of reflective material, and the hot mirror layer is configured to transmit visible light through it and reflect infrared light; forming an intermediate structure by introducing a first transparent material into the interior of a mold and aligning it with the first surface of a layer of reflective material; and forming an optical element by introducing a second transparent material into the interior of a second mold and aligning it with the second surface of a layer of reflective material, wherein the second mold houses the intermediate structure, and the first surface of the layer of reflective material faces the second surface of the layer of reflective material.

[0044] Example 39: The method of Example 38, further comprising the step of coupling an optical element to a display element, which is configured for insertion into a frame configured to be fitted by a user.

[0045] Example 40: The method according to any one of Examples 38-39, wherein the first mold has a shape different from that of the second mold.

[0046] Example 41: The method according to any one of Examples 38-40, wherein the first mold has a shape different from that of the second mold.

[0047] Example 42: The method according to any one of Examples 38-41, wherein the first and second transparent materials are identical.

[0048] Example 43: The method according to any one of Examples 38-42, wherein at least one of the first and second transparent materials is glass or plastic.

[0049] Example 44: The rise angle is approximately 3 o ~35 oThe method described in any one of Examples 38-43.

[0050] Example 45: The method according to any one of Examples 38-44, wherein the outer surface of the intermediate structure comprises a second surface of a layer of reflective material.

[0051] Example 46: The method according to any one of Examples 38-45, wherein the step of introducing a second transparent material into the interior of a second mold includes the step of forming a layer of the second transparent material on the second surface of the reflective material.

[0052] Example 47: The method according to any one of Examples 38-46, further comprising the step of inserting an optical element into a frame configured to be mounted on the user's head.

[0053] Example 48: The method according to Example 47, further comprising the step of mounting a camera to a frame, wherein the camera is configured to image the user's eyes.

[0054] Example 49: A hot mirror manufactured according to the method described in any one of Examples 38-48.

[0055] Example 50: A head-mounted display system comprising: a frame configured to be supported on the head of a user; an optical element configured to display an image to the user, configured to transmit light from the environment to the user's eye and to provide the user with a view of a portion of the environment; a forward-facing imager configured to receive light from the optical element; and a reflective element, at least partially located within the optical element, the reflective element comprising first and second segments, the first segment angled at a non-zero angle with respect to the second segment, the first segment configured to produce a first image of the eye, which is captured by the forward-facing imager, and the second segment configured to produce a second image of the eye, which is captured by the forward-facing imager.

[0056] Example 51: The head-mounted display system according to Example 50, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0057] Example 52: A head-mounted display system according to any of Examples 50-51, wherein the first segment has a different refractive power from the second segment.

[0058] Example 53: A head-mounted display system according to any of Examples 50-52, wherein the non-zero angle is in the range of 2 to 25 degrees.

[0059] Example 54: A head-mounted display system according to any of Examples 50-53, wherein the reflective element comprises a hot mirror film that is substantially transparent to visible light and substantially reflective to infrared light.

[0060] Example 55: The head-mounted display system according to Example 54, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of approximately 800 nm to 900 nm.

[0061] Example 56: A head-mounted display system according to any of Examples 50-55, wherein the optical element comprises at least one infrared light source disposed on or at least partially within it.

[0062] Example 57: A head-mounted display system according to any one of Examples 50-56, wherein the optical element comprises polyethylene terephthalate (PET).

[0063] Example 58: A head-mounted display system according to any of Examples 50-57, further comprising: a non-transient memory configured to store images of the user's eyes acquired by a forward-facing imager; and a hardware processor communicating with the non-transient memory, which is programmed to access the images of the eyes and perform one or more of the following: tracking the user's eyes, extracting biometric information associated with the user's eyes, reconstructing the shape of a part of the user's eyes, estimating the accommodative state of the user's eyes, or imaging the retina, iris, or other elements of the user's eyes.

[0064] Example 59: The head-mounted display system according to Example 58, wherein the hardware processor is programmed to estimate eye orientation using the shape of a portion of the user's eye.

[0065] Example 60: A head-mounted display system according to any of Examples 50-59, wherein the optical element is positioned in front of the user's first eye.

[0066] Example 61: A head-mounted display system according to any of Examples 50-60, wherein the frame supports a second reflective element having multiple reflective segments, and the second optical element is positioned in front of the user's second eye.

[0067] Example 62: A head-mounted display system according to any one of Examples 50-61, wherein at least one of the first or second segments is configured to generate a separate virtual camera that images an eye at infinity.

[0068] Example 63: A head-mounted display system according to Example 62, wherein, in order to form an image of the eye, the imaging device uses a first segment when the user is looking upwards and a second segment when the user is looking downwards.

[0069] Example 64: A head-mounted display system according to any of Examples 62-63, wherein, in order to image the eye, the display system selects a segment of a first or second segment that has less occlusion due to the user's eyelashes or eyelids.

[0070] Example 65: A head-mounted display system comprising: a frame configured to be supported on the head of a user; a display disposed on the frame; and an eyepiece configured to receive light from the display and project the light into the user's eye to display virtual image content in the user's field of view, wherein the eyepiece includes a transparent portion, which is arranged to transmit light from a portion of the environment in front of the user and the head-mounted display to the user's eye and to provide a view of the portion of the environment in front of the user and the head-mounted display, the eyepiece comprising at least one layer; and at least one light source, which is at least partially embedded in the at least one layer and directs light towards the user's eye.

[0071] Example 66: The head-mounted display system according to Example 65, further comprising an imaging device configured to form an image of the user's eyes.

[0072] Example 67: The head-mounted display system according to Example 65, further comprising a forward-facing imaging unit configured to form an image of the user's eyes.

[0073] Example 68: A head-mounted display system according to Example 67, wherein the reflective element comprises first and second segments, the first segment being angled to the second segment at a non-zero angle, the first segment being configured to produce a first image of the eye to be captured by a forward-facing imager, and the second segment being configured to produce a second image of the eye to be captured by a forward-facing imager.

[0074] Example 69: The head-mounted display system according to Example 68, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0075] Example 70: An optical eyepiece according to any of Examples 68-69, wherein the first segment has a different refractive power from the second segment.

[0076] Example 71: A head-mounted display system according to any of Examples 68-70, wherein the non-zero angle is in the range of 2 to 25 degrees.

[0077] Example 72: A head-mounted display system according to any one of Examples 68-71, wherein the reflective element comprises a hot mirror film that is substantially transparent to visible light and substantially reflective to infrared light.

[0078] Example 73: The head-mounted display system according to Example 72, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of approximately 800 nm to 900 nm.

[0079] Example 74: The head-mounted display system according to any one of Examples 65-73, wherein the eyepiece comprises at least one waveguide.

[0080] Example 75: The eyepiece is a head-mounted display system according to any one of Examples 65-74, comprising a stack of layers.

[0081] Example 76: The head-mounted display system according to Example 75, wherein the stack of layers comprises at least one waveguide.

[0082] Example 77: A head-mounted display system according to Example 75, wherein the stack of layers comprises multiple waveguides.

[0083] Example 78: A head-mounted display system according to any of Examples 75-77, wherein a reflective element comprising first and second segments is included in a stack of layers, the first segment being angled at a non-zero angle with respect to the second segment, the first segment being configured to produce a first image of the eye to be captured by a forward-facing imager, and the second segment being configured to produce a second image of the eye to be captured by a forward-facing imager.

[0084] Example 79: The head-mounted display system according to Example 78, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0085] Example 80: A head-mounted display system according to any one of Examples 65-79, wherein the at least one light source comprises an infrared light source.

[0086] Example 81: A head-mounted display system according to any one of Examples 65-80, wherein the at least one light source is configured to form a flash of light over the user's eyes.

[0087] Example 82: The head-mounted display system according to any one of claims 65-81, wherein the at least one light source comprises a solid-state emitter.

[0088] Example 83: A head-mounted display system according to any of Examples 65-82, wherein the at least one light source embedded in the at least one layer comprises at least one LED.

[0089] Example 84: A head-mounted display system according to any one of Examples 65-83, further comprising a conductive material for supplying power to the at least one light source, wherein the conductive material is transparent to visible light.

[0090] Example 85: The head-mounted display system according to Example 84, wherein the conductive material comprises indium tin oxide.

[0091] Example 86: The head-mounted display system according to any one of Examples 65-85, wherein the at least one layer comprises a transparent layer.

[0092] Example 87: A head-mounted display system according to any one of Examples 65-86, wherein the at least one layer comprises a polymer.

[0093] Example 88: A method for forming an optical element, comprising the steps of: applying an optical film to a first surface of a first transparent body, the first surface comprising a first section and a second section, the second section being at a non-zero angle with respect to the first section; applying a second transparent body to a first transparent body such that the optical film is positioned between the first transparent body and the second transparent body; and placing a layer comprising at least one light source on at least one of the first or second transparent bodies.

[0094] Example 89: The optical film is optically transparent within the visible light range, as described in Example 88.

[0095] Example 90: The method according to Example 89, wherein the optical film is optically reflective in infrared light, and the first and second optical elements are optically transparent in infrared light.

[0096] Example 91: The method according to any one of Examples 88-90, wherein the step of applying the optical film includes the step of adhering the optical film to the first surface.

[0097] Example 92: The method according to any one of Examples 88-91, wherein the step of applying the optical film includes the step of depositing the optical film onto a first surface.

[0098] Example 93: The method according to any one of Examples 88-92, further comprising the step of injection molding a first transparent body.

[0099] Example 94: The method according to any one of Examples 88-93, wherein the step of applying the second optical element includes the step of injection molding.

[0100] Example 95: The method according to any one of Examples 88-94, wherein the layer comprises at least one light source and is placed on the first transparent body.

[0101] Example 96: The method according to any one of Examples 88-95, wherein the layer comprises at least one light source and is disposed on the second transparent body.

[0102] Example 97: The method according to any one of Examples 88-96, further comprising the step of placing at least one light source inside a first transparent body.

[0103] Example 98: The method according to any one of Examples 88-97, further comprising the step of placing at least one light source inside a second transparent body.

[0104] Example 99: The method of any of Examples 88-98, further comprising the step of embedding, at least partially, the at least one light source within the first transparent body.

[0105] Example 100: The method according to any one of Examples 88-99, further comprising the step of embedding, at least partially, the at least one light source within the second transparent body.

[0106] Example 101: The method according to any one of Examples 88-100, comprising at least one light source, wherein the layer comprises a transparent layer.

[0107] Example 102: The method according to any one of Examples 88-101, wherein the at least one light source comprises a solid-state emitter.

[0108] Example 103: The method according to any one of Examples 88-102, wherein the at least one light source comprises an LED.

[0109] Example 104: A segmented hot mirror formed according to any one of the methods described in Examples 88-103.

[0110] Example 105: An optical element formed according to any one of the methods of Examples 88-103.

[0111] Example 106: A display comprising the segmented hot mirror described in Example 104 or the optical element described in Example 105.

[0112] Example 107: An augmented, virtual, or mixed reality display device comprising the display described in Example 106.

[0113] Details of one or more implementations of the subject matter described herein are shown in the accompanying drawings and the description below. Other features, aspects, and advantages will be evident from the description, drawings, and claims. Neither this summary nor the following detailed description claims to define or limit the scope of the subject matter of the invention. The present invention provides, for example, the following: (Item 1) A method for manufacturing segmented hot mirrors, wherein the method is A step of providing a first mold having a first cavity, wherein the first cavity comprises a first surface having a portion that is at a non-zero angle with respect to a second portion, The steps include: positioning the hot mirror film adjacent to at least the first and second portions of the first surface of the first cavity; The steps include: pouring a first polymer material into a first cavity of the first mold to form a first molded component; A step of removing the first molded component from the first mold, wherein the first molded component includes at least a portion of the hot mirror film, The steps include placing the first molded component into a second mold having a second cavity, A step of introducing a second polymer material into the second cavity to form a second molded component, wherein the second polymer material covers at least a portion of the hot mirror film, The steps include removing the second molded component from the second mold and Methods that include... (Item 2) The method according to item 1, wherein the non-zero angle is in the range of 2 to 25 degrees. (Item 3) The method according to item 1, wherein the non-zero angle is in the range of 5 to 20 degrees. (Item 4) The method according to any one of items 1-3, wherein the hot mirror film is substantially transparent to visible light and substantially reflective to infrared light. (Item 5) The hot mirror film according to any one of items 1-4, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of about 800 nm to 900 nm. (Item 6) The method according to any one of items 1-5, wherein the first polymer is identical to the second polymer. (Item 7) The method according to any one of items 1-6, wherein the first polymer or the second polymer is substantially transparent to visible light and infrared light. (Item 8) The method according to any one of items 1-7, wherein the first polymer or the second polymer includes a thermoplastic polymer. (Item 9) The method according to any one of items 1-8, wherein the first polymer or the second polymer comprises polycarbonate or polymethyl methacrylate (PMMA). (Item 10) The method according to any one of items 1-9, further comprising the step of removing a portion of the hot film that extends outside the first molded component or the second molded component. (Item 11) The method according to any one of items 1-10, wherein the step of placing the first molded component in a second mold having a second cavity includes the step of oriented the first molded component such that the hot mirror film is positioned toward the central region of the second cavity. (Item 12) The method according to any one of items 1-11, wherein the first mold comprises a vent between the first part and the second part. (Item 13) The method according to any one of items 1-12, further comprising the step of placing at least one infrared light source in a second cavity of the second mold. (Item 14) The method according to item 13, wherein the at least one infrared light source is placed on a polymer film, and the method includes the step of placing the polymer film in a second cavity of the second mold. (Item 15) The polymer film is the method described in item 14, comprising polyethylene terephthalate (PET). (Item 16) The method according to any one of items 1-15, wherein the first surface of the first mold comprises a third portion adjacent to the second portion, the third portion being at a second non-zero angle with respect to the second portion. (Item 17) The method according to any one of items 1-16, wherein the first or second portion of the first surface comprises a curved region. (Item 18) The method according to any one of items 1-17, further comprising the step of attaching the second molded component to a display for an augmented, composite, or virtual reality device. (Item 19) A method for forming an optical element, wherein the method is A step of placing an optical film adjacent to a segmented surface of a first mold, wherein the segmented surface comprises a first portion and a second portion at a non-zero angle with respect to the first portion, and the optical film is substantially transparent in a first wavelength range and substantially reflective in a second wavelength range different from the first wavelength range. Steps include: introducing a first polymer into a first cavity of a first mold to form a first optical element, wherein the first polymer is substantially transparent in a first wavelength range and a second wavelength range, and the first optical element comprises at least a portion of the optical film; The steps include placing the first optical element inside the second mold, The steps include: pouring a second polymer into the second mold to form a second optical element, wherein the second polymer covers at least a portion of the optical film of the first optical element; The steps include removing the second optical element from the second mold and Methods that include... (Item 20) The method according to item 19, wherein the first wavelength range includes at least a portion of the visible wavelength range, and the second wavelength range includes at least a portion of the infrared wavelength range. (Item 21) The method according to item 19 or item 20, wherein the first mold comprises a vent between the first part and the second part. (Item 22) The method according to any one of items 19-21, wherein the first or second portion of the segmented surface is substantially flat. (Item 23) The method according to any one of items 19-22, further comprising the step of terminating the optical film at the first edge of the first optical element or the second edge of the second optical element. (Item 24) The method according to any one of items 19-23, further comprising the step of placing a light source in the second mold. (Item 25) The non-zero angle is in the range of 2 to 25 degrees, as described in any one of items 19-24. (Item 26) The optical film comprising a diffractive element or a holographic element, as described in any one of items 19-25. (Item 27) A method for forming an optical element, wherein the method is A step of applying an optical film to a first surface of a first optical element, wherein the first surface comprises a first section and a second section, the second section being at a non-zero angle with respect to the first section. The steps include applying the second optical element to the first optical element and forming the optical element so that the optical film is positioned between the first optical element and the second optical element, and Methods that include... (Item 28) The method according to item 27, wherein the first optical element, the second optical element, and the optical film are optically transparent in the visible light. (Item 29) The method according to item 28, wherein the optical film is optically reflective in infrared light, and the first optical element and the second optical element are optically transparent in infrared light. (Item 30) The method according to any one of items 27-29, wherein the step of applying the optical film includes the step of adhering the optical film onto the first surface. (Item 31) The method according to any one of items 27-29, wherein the step of applying the optical film includes the step of depositing the optical film onto the first surface. (Item 32) The method according to any one of items 27-31, further comprising the step of injection molding the first optical element. (Item 33) The method according to any one of items 27-32, wherein the step of applying the second optical element includes the step of injection molding. (Item 34) A segmented hot mirror formed according to any one of the methods described in items 1-18. (Item 35) An optical element formed according to any one of the methods described in items 19-33. (Item 36) A display comprising a segmented hot mirror as described in item 34 or an optical element as described in item 35. (Item 37) An augmented, virtual, or mixed reality display device having the display described in item 36. (Item 38) A method for manufacturing a hot mirror configured to be coupled with a display element, the method is A step of arranging a hot mirror layer along first and second surfaces of a first mold, wherein the first surface forms a rising angle with the plane of the second surface, the first and second surfaces of the first mold align with the first surface of the reflective material layer, and the hot mirror layer is configured to transmit visible light through it and reflect infrared light. The process involves the steps of: introducing a first transparent material into the mold and aligning it with the first surface of the reflective material layer to form an intermediate structure; A step of forming an optical element by introducing a second transparent material into the interior of a second mold and aligning it with a second surface of the layer of reflective material, wherein the second mold houses the intermediate structure, and the first surface of the layer of reflective material faces the second surface of the layer of reflective material. Methods that include... (Item 39) The method of item 38, further comprising the step of coupling the optical element to a display element configured for insertion into a frame configured to be worn by a user. (Item 40) The method according to any one of items 38-39, wherein the first mold has a shape different from the shape of the second mold. (Item 41) The method according to any one of items 38-40, wherein the first mold has a shape different from the shape of the second mold. (Item 42) The first and second transparent materials are identical, as described in any one of items 38-41. (Item 43) The method according to any one of items 38-42, wherein at least one of the first and second transparent materials includes glass or plastic. (Item 44) The aforementioned rising angle is approximately 3 o ~35 o The method described in any one of items 38-43. (Item 45) The method according to any one of items 38-44, wherein the outer surface of the intermediate structure comprises a second surface of the layer of the reflective material. (Item 46) The method according to any one of items 38-45, wherein the step of introducing a second transparent material into the interior of a second mold includes the step of forming a layer of the second transparent material on the second surface of the reflective material. (Item 47) The method according to any one of items 38-46, further comprising the step of inserting the optical element into a frame configured to be worn on the user's head. (Item 48) The method according to item 47, further comprising the step of mounting a camera to the frame, wherein the camera is configured to image the user's eyes. (Item 49) A hot mirror manufactured in accordance with the method described in any one of items 38-48. (Item 50) A head-mounted display system, A frame configured to be supported above the user's head, An optical element configured to display an image to a user, wherein the optical element is configured to transmit light from the environment to the user's eyes and to provide the user with a view of a portion of the environment. A forward-facing imaging device configured to receive light from the aforementioned optical element, A reflective element, at least partially disposed within the optical element, comprising a first and a second segment, the first segment being angled to the second segment at a non-zero angle, the first segment being configured to produce a first image of the eye configured to be captured by the forward-facing imager, and the second segment being configured to produce a second image of the eye configured to be captured by the forward-facing imager. A head-mounted display system equipped with the following features. (Item 51) The head-mounted display system according to item 50, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE). (Item 52) A head-mounted display system according to any one of items 50-51, wherein the first segment has a different refractive power from the second segment. (Item 53) The head-mounted display system according to any one of items 50-52, wherein the non-zero angle is in the range of 2 to 25 degrees. (Item 54) The head-mounted display system according to any one of items 50-53, wherein the reflective element comprises a hot mirror film that is substantially transparent to visible light and substantially reflective to infrared light. (Item 55) The head-mounted display system according to item 54, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of about 800 nm to 900 nm. (Item 56) The head-mounted display system according to any one of items 50-55, wherein the optical element comprises at least one infrared light source disposed on or at least partially therein. (Item 57) The head-mounted display system according to any one of items 50-56, wherein the optical element includes polyethylene terephthalate (PET). (Item 58) A non-transient memory configured to store the image of the user's eye acquired by the forward-facing imaging device, A hardware processor that communicates with the non-transient memory, wherein the hardware processor Accessing the aforementioned eye image, The following is a summary, that is, Tracking the user's eyes, Extracting biometric information associated with the user's eyes, Reconstructing the shape of a part of the user's eye, Estimating the user's eye's near and far focusing state, or To form an image on the retina, iris, or other elements of the user's eye. Implement one or more of the following: A hardware processor and A head-mounted display system as described in any of items 50 or 57, further comprising the features described herein. (Item 59) The head-mounted display system according to item 58, wherein the hardware processor is programmed to estimate the orientation of the eye using the shape of a part of the user's eye. (Item 60) The optical element is positioned in front of the user's first eye in the head-mounted display system according to any one of items 50-59. (Item 61) The head-mounted display system according to any one of items 50-60, wherein the frame supports a second reflective element having a plurality of reflective segments, and the second optical element is positioned in front of the user's second eye. (Item 62) A head-mounted display system according to any one of items 50-61, wherein at least one of the first or second segments is configured to generate a separate virtual camera that images the eye at infinity. (Item 63) The head-mounted display system according to item 62, wherein, in order to form an image of the eye, the imaging device uses the first segment when the user is looking upward, and the second segment when the user is looking downward. (Item 64) A head-mounted display system according to any one of items 62-63, wherein, in order to form an image of the eye, the display system selects a segment of the first or second segment having less occlusion by the user's eyelashes or eyelids. (Item 65) A head-mounted display system, A frame configured to be supported above the user's head, A display arranged on the frame, An eyepiece, wherein the eyepiece is configured to receive light from the display, project the light into the user's eye, and display virtual image content in the user's field of view, and the eyepiece includes a transparent portion arranged to allow light from a portion of the environment in front of the user and the head-mounted display to pass through to the user's eye, and to provide a view of the portion of the environment in front of the user and the head-mounted display, and the eyepiece comprises at least one layer, At least one light source, the at least one light source being at least partially embedded in the at least one layer and directing light toward the user's eyes A head-mounted display system equipped with the following features. (Item 66) The head-mounted display system according to item 65, further comprising an imaging device configured to form an image of the user's eyes. (Item 67) The head-mounted display system according to item 65, further comprising a forward-facing imaging device configured to form an image of the user's eyes. (Item 68) A head-mounted display system according to item 67, wherein the reflective element comprises first and second segments, the first segment being angled with respect to the second segment at a non-zero angle, the first segment being configured to produce a first image of the eye to be captured by the forward-facing imager, and the second segment being configured to produce a second image of the eye to be captured by the forward-facing imager. (Item 69) The head-mounted display system according to item 68, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE). (Item 70) The first segment has a different refractive power from the second segment, as described in any of items 68-69. (Item 71) The head-mounted display system according to any one of items 68-70, wherein the non-zero angle is in the range of 2 to 25 degrees. (Item 72) The head-mounted display system according to any one of items 68-71, wherein the reflective element comprises a hot mirror film that is substantially transparent to visible light and substantially reflective to infrared light. (Item 73) The head-mounted display system according to item 72, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of about 800 nm to 900 nm. (Item 74) The head-mounted display system according to any one of items 65-73, wherein the eyepiece comprises at least one waveguide. (Item 75) The eyepiece is a head-mounted display system according to any one of items 65-74, comprising a stack of layers. (Item 76) The stack of the aforementioned layers comprises at least one waveguide, the head-mounted display system according to item 75. (Item 77) The stack of the aforementioned layers comprises a plurality of waveguides, the head-mounted display system as described in item 75. (Item 78) A head-mounted display system according to any one of items 75-77, wherein a reflective element comprising first and second segments is included in a stack of the layers, the first segment being angled with respect to the second segment at a non-zero angle, the first segment being configured to produce a first image of the eye to be captured by the forward-facing imager, and the second segment being configured to produce a second image of the eye to be captured by the forward-facing imager. (Item 79) The head-mounted display system according to item 78, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE). (Item 80) The head-mounted display system according to any one of items 65-79, wherein the at least one light source comprises an infrared light source. (Item 81) A head-mounted display system according to any one of items 65-80, wherein the at least one light source is configured to form a flash of light over the user's eyes. (Item 82) The head-mounted display system according to any one of items 65-81, wherein the at least one light source comprises a solid-state emitter. (Item 83) A head-mounted display system according to any one of items 65-82, wherein the at least one light source embedded in the at least one layer comprises at least one LED. (Item 84) A head-mounted display system according to any one of items 65-83, further comprising a conductive material for supplying power to the at least one light source, wherein the conductive material is transparent to visible light. (Item 85) The head-mounted display system according to item 84, wherein the conductive material comprises indium tin oxide. (Item 86) The head-mounted display system according to any one of items 65-85, wherein at least one of the layers comprises a transparent layer. (Item 87) The head-mounted display system according to any one of items 65-86, wherein the at least one layer comprises a polymer. (Item 88) A method for forming an optical element, wherein the method is A step of applying an optical film to a first surface of a first transparent body, wherein the first surface comprises a first section and a second section, the second section being at a non-zero angle with respect to the first section, The steps include applying the second transparent body to the first transparent body so that the optical film is positioned between the first transparent body and the second transparent body, The steps include: placing a layer having at least one light source on at least one of the first or second transparent bodies; Methods that include... (Item 89) The optical film is optically transparent within the visible light range, as described in item 88. (Item 90) The method according to item 89, wherein the optical film is optically reflective in infrared light, and the first optical element and the second optical element are optically transparent in infrared light. (Item 91) The method according to any one of items 88-90, wherein the step of applying the optical film includes the step of adhering the optical film onto the first surface. (Item 92) The method according to any one of items 88-91, wherein the step of applying the optical film includes the step of depositing the optical film onto the first surface. (Item 93) The method according to any one of items 88-92, further comprising the step of injection molding the first transparent body. (Item 94) The method according to any one of items 88-93, wherein the step of applying the second optical element includes the step of injection molding. (Item 95) The layer comprising the at least one light source is disposed on the first transparent body, as described in any of items 88-94. (Item 96) The layer comprising the at least one light source is disposed on the second transparent body, as described in any of items 88-95. (Item 97) The method according to any one of items 88-96, further comprising the step of arranging the at least one light source inside the first transparent body. (Item 98) The method according to any one of items 88-97, further comprising the step of arranging the at least one light source inside the second transparent body. (Item 99) The method according to any one of items 88-98, further comprising the step of embedding, at least in part, the at least one light source within the first transparent body. (Item 100) The method according to any one of items 88-99, further comprising the step of embedding, at least in part, the at least one light source within the second transparent body. (Item 101) The method according to any one of items 88-100, wherein the layer comprising at least one light source comprises a transparent layer. (Item 102) The method according to any one of items 88-101, wherein the at least one light source comprises a solid-state emitter. (Item 103) The method according to any one of items 88-102, wherein the at least one light source comprises an LED. (Item 104) A segmented hot mirror formed according to any one of the methods described in items 88-103. (Item 105) An optical element formed according to any one of the methods described in items 88-103. (Item 106) A display comprising a segmented hot mirror as described in item 104 or an optical element as described in item 105. (Item 107) An augmented, virtual, or mixed reality display device having the display described in item 106. [Brief explanation of the drawing]

[0114] [Figure 1] Figure 1 illustrates an example of an augmented reality scenario involving a virtual reality object and a real-world object that are visible to a person.

[0115] [Figure 2] Figure 2 schematically illustrates an embodiment of the wearable display system.

[0116] [Figure 3] Figure 3 schematically illustrates aspects of an approach to simulating a 3D image using multiple depth planes.

[0117] [Figure 4] Figure 4 schematically illustrates an example of a waveguide stack for outputting image information to the user.

[0118] [Figure 5] Figure 5 shows an exemplary output beam that can be produced by a waveguide.

[0119] [Figure 6] Figure 6 is a schematic diagram showing a display system that includes a waveguide device, an optical coupler subsystem for optically coupling light to or from the waveguide device, and a control subsystem used in the generation of a multifocal stereoscopic display, image, or light field.

[0120] [Figure 7A] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye. [Figure 7B] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye. [Figure 7C] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye. [Figure 7D] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye. [Figure 7E] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye. [Figure 7F] Figures 7A-7F schematically illustrate an embodiment of an imaging system that includes a forward-facing camera that uses a reflective off-axis diffractive optical element (DOE) to image the wearer's eye.

[0121] [Figure 7G] Figures 7G and 7H schematically show embodiments of a DOE having multiple segments that may have different optical properties (e.g., reflection angle, refractive power, etc.). [Figure 7H] Figures 7G and 7H schematically show embodiments of a DOE having multiple segments that may have different optical properties (e.g., reflection angle, refractive power, etc.).

[0122] [Figure 8] Figure 8 shows another embodiment of an optical system for eye tracking.

[0123] [Figure 9] Figure 9 shows a series of plots illustrating examples of gaze sensitivity (in pixels per degree) over horizontal gaze angle (in degrees) for various configurations of a hot mirror, related to different differences in interpupillary distance (IPD) and / or different eyebox offsets along the axial direction (z-axis).

[0124] [Figure 10] Figure 10 shows an exemplary optical system with segmented reflective elements.

[0125] [Figure 11] Figure 11 shows an exemplary eye image reflected from an angularly segmented hot mirror embodiment.

[0126] [Figure 12] Figures 12A–12E illustrate various stages of an exemplary manufacturing process for an optical element (e.g., one comprising an angularly segmented hot mirror).

[0127] [Figure 13] Figures 13A-13C show optional steps for manufacturing an optical element that includes a light source (e.g., an infrared LED).

[0128] [Figure 14] Figure 14 shows an exemplary method for manufacturing optical elements such as angle-segmented hot mirrors.

[0129] Throughout the drawings, reference numbers may be reused to indicate correspondences between the referenced elements. The drawings are provided to illustrate exemplary embodiments described herein and are not intended to limit the scope of this disclosure. [Modes for carrying out the invention]

[0130] Detailed explanation overview The wearer's eyes in a head-mounted display (HMD) can be imaged using a reflective off-axis diffractive optical element (DOE). In some implementations, the DOE may be a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE). The wearer's eyes can also be imaged, or alternatively, using a hot mirror (e.g., one that is transparent to visible light and reflective in infrared light). The resulting image can be used for purposes such as tracking one or both eyes, imaging the retina, reconstructing the eye shape in three dimensions, and extracting biometric information (e.g., iris recognition) from the eye.

[0131] Head-mounted displays (HMDs) can utilize information about the wearer's eye condition for various purposes. For example, this information can be used to estimate the wearer's gaze direction or for biometric identification. However, imaging the wearer's eyes with an HMD can be difficult. The distance between the HMD and the wearer's eyes is short. Furthermore, gaze tracking requires a larger field of view, while biometric identification requires a relatively high number of pixels on the target on the iris. Regarding imaging systems that attempt to accomplish both of these purposes, the requirements of the two tasks are significantly in conflict. Moreover, both problems can be further complicated by obstruction from the eyelids and eyelashes.

[0132] Embodiments of imaging systems described herein address some or all of these problems. For example, an imaging system may include an imaging unit configured to view the wearer's eyes. The imaging system may be mounted close to the wearer's temple (e.g., on the frame of a wearable display system, e.g., on the earpiece). In some embodiments, a second imaging unit may be used for the wearer's other eye so that each eye is imaged separately. The imaging unit may include an infrared digital camera that is sensitive to infrared radiation. The imaging unit may be mounted facing forward (towards the wearer's field of vision) and directed towards the eye, rather than facing backward. By positioning the imaging unit closer to the wearer's ears, the weight of the imaging unit may also be closer to the ears, and the HMD may be easier to wear compared to an HMD in which the imaging unit is facing backward and positioned closer to the front of the HMD. In addition, by positioning a forward-facing imager near the wearer's temples, the distance from the wearer's eyes to the imager becomes approximately twice as long as that of a rearward-facing imager positioned near the front of the HMD. Since the depth of field of the image is roughly proportional to this distance, the depth of field for a forward-facing imager is approximately twice that of a rearward-facing imager. A greater depth of field for the imager can be advantageous for imaging the eye area of ​​a wearer who has a large or protruding nose, brow ridge, etc.

[0133] The imaging device can be positioned to view the inner surface of an otherwise transparent optical element. The optical element can be part of the display of an HMD (or lenses in a pair of glasses). The optical element can have a surface that reflects a first range of wavelengths but is substantially transparent to a second range of wavelengths (different from the first range of wavelengths). The first range of wavelengths can be in the infrared spectrum, and the second range of wavelengths can be in the visible spectrum. For example, the optical element can have a hot mirror that reflects infrared light but transmits visible light. Visible light from the outside world can be transmitted through the optical element and perceived by the wearer. In fact, the imaging system acts like a virtual imaging device directed backward toward the wearer's eye. The virtual imaging device can image virtual infrared light propagating from the wearer's eye through the optical element. The hot mirror (or other DOEs described herein) can be positioned on the inner surface of the optical element, on the outer surface of the optical element, or within the optical element (e.g., a stereoscopic HOE).

[0134] In some embodiments, the optical element comprises multiple segments with different optical properties, such as angle or refractive power. The different segments of the optical element have the advantage of being able to reflect light to the imager when the wearer is looking in different directions.

[0135] An example of a manufacturing process for producing segmented optical elements is provided. The manufacturing process may include injection molding. The injection mold may include a segmented surface with different segments having different angles (or refractive forces).

[0136] Infrared radiation can include radiation with wavelengths in the range of 700 nm to 10 μm. Infrared radiation can also include near-infrared radiation with wavelengths in the range of 700 nm to 1.5 μm. In many implementations, eye imaging occurs in the near-infrared range with wavelengths of 700 nm to 900 nm. 3D display

[0137] Figure 1 illustrates an example of an augmented reality scenario involving a virtual reality object and a real-world object that is visible to a person. Figure 1 depicts an augmented reality scene 100, in which a user of AR technology sees a real-world park-like setting 110 featuring people, trees, buildings in the background, and a concrete platform 120. In addition to these items, the user of AR technology also perceives "seeing" a robotic figure 130 standing on the real-world platform 120 and a flying cartoonish avatar character 140 that appears to be a personification of a bumblebee, although these elements do not exist in the real world.

[0138] It is desirable for a three-dimensional (3D) display to generate a true sense of depth, more specifically, a simulated sense of surface depth, by generating a distance-accommodative response for each point within the display's field of view that corresponds to its virtual depth. If the distance-accommodative response for a display point does not correspond to the virtual depth of that point, as determined by the binocular depth cues for convergence and stereopsis, the human eye may experience distance-accommodative collision, which can result in unstable image formation, harmful eye strain, headaches, and, in the absence of distance-accommodative information, a near-complete loss of surface depth.

[0139] VR, AR, and MR experiences can be provided by a display system having a display that provides the viewer with images corresponding to multiple depth planes. The images may differ for each depth plane (e.g., providing slightly different presentations of a scene or object) and may be individually focused by the viewer's eyes, thereby helping to provide the user with depth cues based on the eye's accommodation required to focus on different image features relating to a scene located on different depth planes, and / or based on observing different image features on different depth planes that are out of focus. As discussed in any part of this specification, such depth cues provide a credible perception of depth.

[0140] Figure 2 illustrates an embodiment of a wearable display system 200 that may be used to present a VR, AR, or MR experience to a display system wearer or viewer 204. The display system 200 includes a display 208 and various mechanical and electronic modules and systems to support the functionality of the display 208. The display 208 may be coupled to a frame 212, which is wearable by the display system user, wearer, or viewer 204 and is configured to position the display 208 in front of the wearer's eyes. The display 208 may be a light field display. In some embodiments, a speaker 216 is coupled to the frame 212 and positioned adjacent to the user's ear canal (in some embodiments, another speaker, not shown, is positioned adjacent to the user's other ear canal to provide stereo / shapeable sound control). The display 208 is operably coupled to the local data processing module 224 by wired or wireless connections, etc., and may be mounted in various configurations, such as being fixedly attached to the frame 212, fixed to a helmet or hat worn by the user, embedded in headphones, or otherwise detachably attached to the user 204 (for example, in a backpack configuration, in a belt-connected configuration).

[0141] The local processing and data module 224 may include a hardware processor and non-transient digital memory such as non-volatile memory (e.g., flash memory), both of which may be used to assist in data processing, caching, and storage. The data may include (a) data captured from sensors (e.g., operably coupled to frame 212 or otherwise attached to user 204) such as image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, and / or gyroscopes, and / or (b) possibly data obtained and / or processed using the remote processing module 228 and / or remote data repository 232 for transmission to display 208 after such processing or reading. The local processing and data module 224 may be operably coupled to the remote processing module 228 and the remote data repository 232 by communication links 236 and / or 240, such as via wired or wireless communication links, so that these remote modules 228 and 232 are available as resources to the local processing and data module 224. In addition, the remote processing module 228 and the remote data repository 232 may be operably coupled to each other.

[0142] In some embodiments, the remote processing module 228 may comprise one or more processors configured to analyze and process data and / or image information, such as video information, captured by the image capture device. The video data may be stored locally in the local processing and data module 224 and / or the remote data repository 232. In some embodiments, the remote data repository 232 may comprise a digital data storage facility, which may be available through the internet or other networking configurations in a “cloud” resource configuration. In some embodiments, all data is stored and all calculations are performed in the local processing and data module 224, enabling fully autonomous use from the remote module.

[0143] The human visual system is complex, and providing a realistic perception of depth is challenging. While not limited by theory, it is believed that a viewer of an object may perceive it as three-dimensional due to a combination of convergence-divergence movements and accommodation. The convergence-divergence movements of two eyes relative to each other (i.e., rotational movements of the pupils toward or away from each other to converge the lines of sight and fixate on an object) are closely related to the focusing (or "accommodation") of the eye's lens. Under normal conditions, a change in the focal point of the eye's lens or the eye's accommodation to change focus from one object to another at a different distance will automatically produce a corresponding change in convergence-divergence movements toward the same distance, under a relationship known as the "accommodation-convergence-divergence reflex." Similarly, a change in convergence-divergence movements will, under normal conditions, induce a corresponding change in accommodation. A display system that provides a better match between accommodation and convergence-divergence movements can form a more realistic or comfortable simulation of three-dimensional images.

[0144] Figure 3 illustrates aspects of an approach to simulating a three-dimensional image using multiple depth planes. Referring to Figure 3, objects at various distances from eyes 302 and 304 on the z-axis are accommodated by eyes 302 and 304 so that those objects are in focus. Eyes 302 and 304 take on specific accommodated states, focusing objects at different distances along the z-axis. As a result, a specific accommodated state can be said to be associated with one of the specific depth planes 306, having an associated focal length such that an object or part of an object in a particular depth plane is in focus when the eye is accommodated to that depth plane. In some embodiments, the three-dimensional image may be simulated by providing a different presentation of the image for each of eyes 302 and 304, and by providing a different presentation of the image corresponding to each of the depth planes. Although shown as separate for the sake of clarity in the illustration, it should be understood that the fields of view of eyes 302 and 304 may overlap, for example, as the distance along the z-axis increases. Furthermore, although shown as flat for the sake of illustration, it should be understood that the contour of the depth plane may be curved in physical space so that all features within the depth plane are in focus with the eye in a particular distance-accommodated state. While not limited by theory, it is thought that the human eye can typically interpret a finite number of depth planes to provide depth perception. Consequently, a highly realistic simulation of perceived depth can be achieved by providing the eye with different representations of images corresponding to each of these limited number of depth planes. Waveguide stack assembly

[0145] Figure 4 illustrates an embodiment of a waveguide stack for outputting image information to a user. The display system 400 includes a waveguide stack or stacked waveguide assembly 405, which may be used to provide three-dimensional perception to the eyes 410 or the brain using a plurality of waveguides 420, 422, 424, 426, and 428. In some embodiments, the display system 400 may correspond to the system 200 in Figure 2, and Figure 4 shows some parts of that system 200 in more detail. For example, in some embodiments, the waveguide assembly 405 may be integrated into the display 208 in Figure 2.

[0146] Continuing with Figure 4, the waveguide assembly 405 may also include several features 430, 432, 434, and 436 between the waveguides. In some embodiments, features 430, 432, 434, and 436 may be lenses. In some embodiments, features 430, 432, 434, and 436 may not be lenses; rather, they may be spacers (e.g., cladding layers and / or structures for forming air gaps).

[0147] Waveguides 420, 422, 424, 426, 428 and / or multiple lenses 430, 432, 434, 436 may be configured to transmit image information to the eye using varying levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a specific depth plane and configured to output image information corresponding to that depth plane. Image input devices 440, 442, 444, 446, 448 may be used to input image information into waveguides 420, 422, 424, 426, 428, each of which may be configured to disperse incident light across each individual waveguide for output toward the eye 410. The light exits from the output surface of the image input devices 440, 442, 444, 446, 448 and is input into the corresponding input edges of waveguides 420, 422, 424, 426, 428. In some embodiments, a single beam of light (e.g., a collimated beam) may be introduced into each waveguide and output the entire field of cloned collimated beams, which is directed toward the eye 410 at a specific angle (and divergence) corresponding to a depth plane associated with a particular waveguide.

[0148] In some embodiments, the image input devices 440, 442, 444, 446, and 442 are discrete displays, each generating image information to be input into the corresponding waveguides 420, 422, 424, 426, and 428, respectively. In some other embodiments, the image input devices 440, 442, 444, 446, and 448 are output terminals of a single multiplexed display, which can send image information to each of the image input devices 440, 442, 444, 446, and 448, for example, via one or more optical conduits (such as fiber optic cables).

[0149] The controller 450 controls the operation of the stacked waveguide assembly 405 and the image input devices 440, 442, 444, 446, and 448. In some embodiments, the controller 450 includes programming (e.g., instructions in a non-transient computer-readable medium) to coordinate the timing and delivery of image information to the waveguides 420, 422, 424, 426, and 428. In some embodiments, the controller 450 may be a single integrated device or a distributed system connected by wired or wireless communication channels. In some embodiments, the controller 450 may be part of a processing module 224 or 228 (illustrated in Figure 2). In some embodiments, the controller may communicate with an inward-facing imaging system 452 (e.g., a digital camera), an outward-facing imaging system 454 (e.g., a digital camera), and / or a user input device 466. An inward-facing imaging system 452 (e.g., a digital camera) can be used to capture an image of the eye 410 and, for example, to determine the size and / or orientation of the pupil of the eye 410. An outward-facing imaging system 454 can be used to image a portion of the world 456. The user can interact with the display system 400 by inputting commands to the controller 450 via a user input device 466.

[0150] Waveguides 420, 422, 424, 426, and 428 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Waveguides 420, 422, 424, 426, and 428 may each be planar or have another shape (e.g., curved), with a main upper and lower surface and a rim extending between their main upper and lower surfaces. In the illustrated configuration, waveguides 420, 422, 424, 426, and 428 may each include light extraction optical elements 460, 462, 464, 466, and 468, respectively, configured to extract light from the waveguide by redirecting the light propagating within each individual waveguide outwards from the waveguide and outputting image information to the eye 410. The extracted light may also be referred to as externally coupled light, and the light extraction optical elements may also be referred to as externally coupled optical elements. The extracted beam of light is output by a waveguide to the location where the light propagating within the waveguide strikes the light redirection element. The light extraction optical elements (460, 462, 464, 466, 468) may be, for example, reflective and / or diffracting optical features. For the sake of clarity and to facilitate the explanation, they are shown positioned on the bottom main surface of the waveguides 420, 422, 424, 426, 428, but in some embodiments, the light extraction optical elements 460, 462, 464, 466, 468 may be positioned on the top and / or bottom main surface, and / or directly within the volume of the waveguides 420, 422, 424, 426, 428. In some embodiments, the light extraction optical elements 460, 462, 464, 466, 468 may be mounted on a transparent substrate and formed within a layer of material forming the waveguides 420, 422, 424, 426, 428. In some other embodiments, the waveguides 420, 422, 424, 426, and 428 may be monolithic pieces of material, and the light extraction optical elements 460, 462, 464, 466, and 468 may be formed on and / or inside the material piece.

[0151] Continuing with reference to Figure 4, as discussed herein, each waveguide 420, 422, 424, 426, 428 is configured to emit light and form an image corresponding to a particular depth plane. For example, the waveguide 420 closest to the eye may be configured to deliver collimated light to the eye 410 as it is introduced into such waveguide 420. The collimated light may represent the optical infinity focal plane. The next waveguide 422 may be configured to emit collimated light that passes through a first lens 430 (e.g., a negative lens) before it can reach the eye 410. The first lens 430 may generate a slight convex wavefront curvature so that the eye / brain interprets the light emanating from the next waveguide 422 as emanating from a first focal plane that is closer inward from optical infinity toward the eye 410. Similarly, the third waveguide 424 passes its output light through both the first lens 430 and the second lens 432 before reaching the eye 410. The combined refractive power of the first and second lenses 430 and 432 may be configured to generate another increment of wavefront curvature so that the eye / brain interprets the light emanating from the third waveguide 424 as originating from a second focal plane that is even closer inward toward the person from optical infinity than the light from the next waveguide 422.

[0152] Other waveguide layers (e.g., waveguides 426, 428) and lenses (e.g., lenses 434, 436) are configured similarly, with the highest waveguide 428 in the stack emitting its output through all the lenses between it and the eye for a concentrated focusing power representing the focal plane closest to the person. When viewing / interpreting light originating from the other side world 456 of the stacked waveguide assembly 405, a compensating lens layer 438 may be positioned on top of the stack to compensate for the stack of lenses 430, 432, 434, 436, and to compensate for the concentrated power of the lower lens stacks 430, 432, 434, 436. Such a configuration provides the same number of perceived focal planes as there are available waveguide / lens pairs. The focusing surfaces of the light extraction optical elements 460, 462, 464, 466, 468 of the waveguides 420, 422, 424, 426, 428 and the lenses 430, 432, 434, 436 may both be static (e.g., not dynamic or electrically active). In some alternative embodiments, either or both may be dynamic using electrically active features.

[0153] Continuing with Figure 4, the light extraction optical elements 460, 462, 464, 466, and 468 may be configured to redirect light from their respective waveguides for specific depth planes associated with the waveguides, and to output the light with an appropriate amount of divergence or collimation. As a result, waveguides having different associated depth planes may have different configurations of light extraction optical elements that output light with different amounts of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the light extraction optical elements 460, 462, 464, 466, and 468 may be three-dimensional or surface features that can be configured to output light at specific angles. For example, the light extraction optical elements 460, 462, 464, 466, and 468 may be three-dimensional holograms, surface holograms, and / or diffraction gratings. Optical elements for light extraction, such as diffraction gratings, are described in U.S. Patent Publication No. 2015 / 0178939, published on June 25, 2015 (which is incorporated herein by reference as a whole). In some embodiments, features 430, 432, 434, and 436 may not be lenses; rather, they may simply be spacers (e.g., cladding layers and / or structures for forming air gaps).

[0154] In some embodiments, the light extraction optical elements 460, 462, 464, 466, and 468 are diffraction features, i.e., “diffractive optical elements” (also referred to herein as “DOEs”), that form a diffraction pattern. Preferably, the DOEs have relatively low diffraction efficiency such that only a portion of the beam light is deflected toward the eye 410 at each intersection of the DOEs, while the remainder continues to travel through the waveguide via total internal reflection. The light carrying the image information is therefore split into several associated emission beams that exit the waveguide at multiple locations, resulting in a very uniform pattern of emission toward the eye 410 with respect to this particular collimated beam bouncing within the waveguide.

[0155] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer-dispersed liquid crystals in which microdroplets have a diffraction pattern in a host medium, and the refractive index of the microdroplets can be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the refractive index of the microdroplets can be switched to one that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0156] In some embodiments, the number and distribution of depth planes and / or depth of field may vary dynamically based on the pupil size and / or orientation of the viewer's eye. In some embodiments, an inward-facing imaging system 452 (e.g., a digital camera) may be used to capture an image of the eye 410 and determine the pupil size and / or orientation of the eye 410. In some embodiments, the inward-facing imaging system 452 may be mounted on a frame 212 (as shown in Figure 2) and may communicate with processing modules 224 and / or 228 that process image information from the inward-facing imaging system 452 and can determine, for example, the pupil diameter and / or orientation of the user's eye 204.

[0157] In some embodiments, an inward-facing imaging system 452 (e.g., a digital camera) can observe user movements, such as eye and face movements. The inward-facing imaging system 452 may be used to capture an image of the eye 410 and to determine the pupil size and / or orientation of the eye 410. The inward-facing imaging system 452 may be used to determine the direction the user is looking (e.g., eye posture) or to obtain an image for the user's biometric identification (e.g., via iris recognition). The image obtained by the inward-facing imaging system 452 may be analyzed to determine the user's eye posture and / or mood, which may be used by the display system 400 to determine the audio or visual content to be presented to the user. The display system 400 may also use sensors such as an inertial measurement unit (IMU), accelerometer, and gyroscope to determine head posture (e.g., head position or head orientation). Head posture may be used alone or in combination with eye posture to interact with support tracking and / or to present audio content.

[0158] In some embodiments, one camera may be used for each eye to independently determine the pupil size and / or orientation of each eye, thereby allowing the presentation of image information to each eye to be dynamically adjusted to that eye. In some embodiments, at least one camera may be used for each eye to independently determine the pupil size and / or eye orientation of each eye, thereby allowing the presentation of image information to each eye to be dynamically adjusted to that eye. In some other embodiments, the pupil diameter and / or orientation of only one eye 410 is determined (for example, using only one camera per pair of eyes) and is assumed to be similar to that of both eyes of the viewer 204.

[0159] For example, depth of field may change inversely with the viewer's pupil size. As a result, as the pupil size of the viewer's eye decreases, the depth of field increases so that one plane that is indistinguishable because its location is beyond the eye's depth of focus becomes discernible and appears more in focus with the decrease in pupil size and the corresponding increase in depth of field. Similarly, the number of spaced depth planes used to present different images to the viewer may decrease with a decreased pupil size. For example, it may not be possible for a viewer to clearly perceive the details of both a first and a second depth plane at one pupil size without adjusting the eye's accommodation from one depth plane to the other. However, these two depth planes can simultaneously be sufficiently in focus for the user at a different pupil size without changing accommodation.

[0160] In some embodiments, the display system may vary the number of waveguides receiving image information based on the determination of pupil size and / or orientation, or in response to the reception of an electrical signal indicating a particular pupil size and / or orientation. For example, if the user's eye is unable to distinguish between two depth planes associated with two waveguides, the controller 450 may be configured or programmed to stop providing image information to one of these waveguides. Advantageously, this can reduce the processing load on the system and thereby increase the system's responsiveness. In embodiments where the DOE for a waveguide is switchable between on and off states, the DOE may be switched off when the waveguide receives image information.

[0161] In some embodiments, it may be desirable to satisfy the condition that the emitted beam has a diameter less than the diameter of the viewer's eye. However, satisfying this condition may be difficult in light of the variability of the viewer's pupil size. In some embodiments, this condition is satisfied over a wide range of pupil sizes by varying the size of the emitted beam in response to the determination of the viewer's pupil size. For example, as the pupil size decreases, the size of the emitted beam may also decrease. In some embodiments, the size of the emitted beam may be varied using a variable aperture.

[0162] The display system 400 may include an outward-facing imaging system 454 (e.g., a digital camera) that images a portion of the world 456. This portion of the world 456 may sometimes be referred to as the field of view (FOV), and the imaging system 454 may also be referred to as an FOV camera. The entire area available for viewing or imaging by the viewer 204 may be referred to as the eye-moving field of view (FOR). FOR may include a solid angle of 4π steradians surrounding the display system 400. In some implementations of the display system 400, FOR may include substantially all of the solid angle of the display system 400 around the user 204 (in front of, behind, above, below, or to the side of the user) because the user 204 may move their head and eyes to view objects surrounding the user. Images obtained from the outward-facing imaging system 454 can be used to track gestures made by the user (e.g., hand or finger gestures) and to detect objects in the world 456 in front of the user.

[0163] The display system 400 may include a user input device 466 that allows the user to input commands to a controller 450 and interact with the display system 400. For example, the user input device 466 may include a trackpad, touchscreen, joystick, multi-degree-of-freedom (DOF) controller, capacitive sensing device, game controller, keyboard, mouse, directional pad (D-pad), wand, tactile device, totem (e.g., functioning as a virtual user input device), etc. In some cases, the user may use a finger (e.g., thumb) to press or swipe over a touch-sensitive input device to provide input to the display system 400 (e.g., to provide user input to a user interface provided by the display system 400). The user input device 466 may be held in the user's hand while using the display system 400. The user input device 466 can be connected to the display system 400 via wired or wireless communication.

[0164] Figure 5 shows an embodiment of an outgoing beam output by a waveguide. Although one waveguide is shown, other waveguides within the waveguide assembly 405 may function similarly, and it should be understood that the waveguide assembly 405 includes multiple waveguides. Light 505 is injected into the waveguide 420 at its input edge 510 and propagates through the waveguide 420 by TIR. At the point where light 505 collides with the DOE 460, a portion of the light exits the waveguide as an outgoing beam 515. The outgoing beams 515 are shown as substantially parallel, but they may also be redirected to propagate towards the eye 410 at a certain angle (e.g., forming a divergent outgoing beam) depending on the depth plane associated with the waveguide 420. It should be understood that a nearly parallel-emitting beam may represent a waveguide with an optical extraction element that externally couples the light and forms an image that appears to be set on the depth plane at long distances from the eye 410 (e.g., optical infinity). Other waveguides or other sets of optical extraction elements may output a more divergent emitted beam pattern that requires the eye 410 to adjust for closer distances and focus on the retina, and which would be interpreted by the brain as light from a distance closer to the eye 410 than optical infinity.

[0165] Figure 6 shows another embodiment of the display system 400, which includes a waveguide apparatus, an optical coupler subsystem for optically coupling light to or from the waveguide apparatus, and a control subsystem. The display system 400 can be used to generate multifocal stereoscopic images or light fields. The display system 400 may include one or more primary planar waveguides 604 (only one is shown in Figure 6) and one or more DOEs 608 associated with at least one of the primary waveguides 604. The planar waveguides 604 can be analogous to waveguides 420, 422, 424, 426, and 428 discussed with reference to Figure 4. The optical system may employ a dispersed waveguide apparatus to relay light along a first axis (vertical or Y-axis in the diagram of Figure 6) and extend the effective exit pupil of light along the first axis (e.g., Y-axis). The dispersed waveguide apparatus may include, for example, a dispersed plane waveguide 612 and at least one DOE 616 (illustrated by a double dashed line) associated with the dispersed plane waveguide 612. The dispersed plane waveguide 612 may be similar to or the same as a primary plane waveguide 604 having a different orientation in at least some respects. Similarly, at least one DOE 616 may be similar to or the same as a DOE 608 in at least some respects. For example, the dispersed plane waveguide 612 and / or DOE 616 may be made of the same material as the primary plane waveguide 604 and / or DOE 608, respectively. The optical system shown in Figure 6 can be integrated into the wearable display system 200 shown in Figure 2.

[0166] The relayed and dilated light is optically coupled from the dispersed waveguide apparatus into one or more primary plane waveguides 604. The primary plane waveguide 662 preferably relays the light along a second axis (e.g., horizontal or X-axis in the diagram of Figure 6) perpendicular to the first axis. It should be noted that the second axis can be a non-orthogonal axis to the first axis. The primary plane waveguide 604 expands the effective exit path of the light along its second axis (e.g., X-axis). For example, a dispersed plane waveguide 612 can relay and expand the light along the vertical or Y-axis and pass its light through a primary plane waveguide 604 that relays and expands the light along the horizontal or X-axis.

[0167] The display system 400 may include one or more colored light sources (e.g., red, green, and blue laser light) 620 that can be optically coupled into the proximal end of a single-mode optical fiber 624. The distal end of the optical fiber 624 may be screwed or received through a hollow tube 628 made of piezoelectric material. The distal end protrudes from the tube 628 as an unfixed, flexible cantilever 632. The piezoelectric tube 628 can be associated with four quadrant electrodes (not shown). The electrodes may be plated, for example, on the outside, outer surface, outer circumference, or diameter of the tube 628. A core electrode (not shown) may also be located in the core, center, inner circumference, or inner diameter of the tube 628.

[0168] For example, a drive electronic device 636, electrically coupled via wire 640, drives a pair of opposing electrodes to independently bend the piezoelectric tube 628 along two axes. The protruding distal tip of the optical fiber 624 has a mechanical resonance mode. The resonance frequency may depend on the diameter, length, and material properties of the optical fiber 624. By vibrating the piezoelectric tube 628 near the first mechanical resonance mode of the fiber cantilever 632, the fiber cantilever 632 can be vibrated and swept through a large deflection.

[0169] By stimulating resonant vibrations in two axes, the tip of the fiber cantilever 632 is scanned in two axes within an area that fills a two-dimensional (2-D) scan. By modulating the intensity of the light source 620 in synchronization with the scanning of the fiber cantilever 632, the light emitted from the fiber cantilever 632 forms an image. A description of such a setup is provided in U.S. Patent Publication 2014 / 0003762, which is incorporated herein by reference in its entirety.

[0170] Component 644 of the optical coupler subsystem collimates light emitted from the scanning fiber cantilever 632. The collimated light is reflected by the mirrored surface 648 into a narrow-dispersion planar waveguide 612 containing at least one diffractive optical element (DOE) 616. The collimated light propagates along the dispersive planar waveguide 612 perpendicular to the diagram in Figure 6 by total internal reflection, thereby repeatedly intersecting with the DOE 616. The DOE 616 preferably has low diffraction efficiency. This causes a portion of the light (e.g., 10%) to be diffracted toward the edge of the larger primary planar waveguide 604 at each point of intersection with the DOE 616, and via TIR, a portion of the light continues along its original trajectory along the length of the dispersive planar waveguide 612.

[0171] At each point of intersection with DOE616, additional light is diffracted toward the entrance of the primary waveguide 612. By splitting the incident light into multiple external coupling sets, the exit pupil of the light is vertically expanded by DOE616 within the dispersed plane waveguide 612. This vertically expanded light, externally coupled from the dispersed plane waveguide 612, is incident on the edge of the primary plane waveguide 604.

[0172] Light entering the primary waveguide 604 propagates horizontally along the primary waveguide 604 (relative to the diagram in Figure 6) via the TIR. As the light intersects the DOE 608 at multiple points, it propagates horizontally along at least a portion of the length of the primary waveguide 604 via the TIR. The DOE 608 may be advantageously designed or configured to have a phase profile, which is the sum of linear and radially symmetric diffraction patterns, and to produce both deflection and focusing of light. The DOE 608 may have a low diffraction efficiency (e.g., 10%) such that only a portion of the beam of light is deflected towards the viewer's eye at each intersection of the DOE 608, while the rest of the light continues to propagate through the waveguide 604 via the TIR.

[0173] At each point of intersection between the propagating light and the DOE608, a portion of the light is diffracted toward the adjacent surface of the primary waveguide 604, allowing the light to escape from the TIR and be emitted from the surface of the primary waveguide 604. In some embodiments, the radially symmetric diffraction pattern of the DOE608 also imparts a certain focal level to the diffracted light, shapes the wavefront of the individual beams (e.g., imparts curvature), and steers the beams to an angle that matches the designed focal level.

[0174] Therefore, these different paths can be used to couple light outside the primary plane waveguide 604 by resulting in different filling patterns in the DOE 608 at different angles, focal levels, and / or in the exit pupil. Different filling patterns in the exit pupil can, for the benefit of being, be used to generate a light field display with multiple depth planes. Each layer in the waveguide assembly or a set of layers in a stack (e.g., three layers) may be employed to generate individual colors (e.g., red, blue, and green). Thus, for example, a first set of three adjacent layers may be employed to generate red, blue, and green light at a first depth of focus, respectively. A second set of three adjacent layers may be employed to generate red, blue, and green light at a second depth of focus, respectively. Multiple sets may be employed to generate a full 3D or 4D color image light field with various depths of focus. Exemplary optical system for eye imaging using an off-axis imaging machine

[0175] The eyes of a wearer of a head-mounted display (HMD) (for example, the wearable display system 200 shown in Figure 2) can be imaged using a reflective off-axis diffractive optical element (DOE) (which may be a holographic optical element (HOE) in some implementations). The resulting image can be used for tracking one or both eyes, imaging the retina, reconstructing the eye shape in three dimensions, extracting biometric information from the eye (e.g., iris recognition), etc.

[0176] There are several reasons why a head-mounted display (HMD) may use information about the wearer's eye condition. For example, this information can be used to estimate the wearer's gaze direction or for biometric identification. However, this problem is difficult due to the short distance between the HMD and the wearer's eyes. This is further complicated by the fact that gaze tracking requires a larger field of view, while biometric identification requires a relatively high number of pixels on the target on the iris. With respect to imaging systems that will attempt to accomplish both of these objectives, the requirements of the two tasks are significantly in conflict. Finally, both problems are further complicated by obstruction by the eyelids and eyelashes. Embodiments of imaging systems described herein address some or all of these problems. Various embodiments of the imaging system 700 described herein with reference to Figures 7A–7H can be used in conjunction with an HMD, including a display device described herein (e.g., the wearable display system 200 shown in Figure 2, and the display system 400 shown in Figures 4 and 6).

[0177] Figure 7A schematically illustrates an embodiment of the imaging system 700, which includes an imager 702b used to view eye 304 and mounted close to the wearer's temple (e.g., on the frame 212 of the wearable display system 200, e.g., on the earpiece). In other embodiments, a second imager is used for the wearer's other eye 302 so that each eye is imaged separately. The imager 702b may include an infrared digital camera that is sensitive to infrared radiation. The imager 702b is mounted facing forward (towards the wearer's field of vision) and directed towards eye 304 (as camera 452 shown in Figure 4), rather than facing backward. By positioning the imaging unit 702b closer to the wearer's ear, the weight of the imaging unit 702b is also closer to the ear, and the HMD may be easier to wear compared to an HMD in which the imaging unit is rear-facing and positioned closer to the front of the HMD (for example, close to the display 208). In addition, by positioning the forward-facing imaging unit 702b near the wearer's temple, the distance from the wearer's eye 304 to the imaging unit is approximately twice that of a rear-facing imaging unit positioned near the front of the HMD (for example, compared to the camera 452 shown in Figure 4). Since the depth of field of the image is approximately proportional to this distance, the depth of field for the forward-facing imaging unit 702b is approximately twice that of a rear-facing imaging unit. The greater depth of field of the imaging unit 702b may be advantageous for imaging the eye area of ​​a wearer who has a large or protruding nose, brow ridge, etc.

[0178] The imaging device 702b is positioned to view the inner surface 704 of an otherwise transparent optical element 706. The optical element 706 may be part of the display 208 of an HMD (or lenses in a pair of glasses). The optical element may be transparent to at least 10%, 20%, 30%, 40%, 50%, or more of the visible light incident on the optical element. In other embodiments, the optical element 706 does not need to be transparent (e.g., in a virtual reality display). The optical element 706 may comprise a reflective element 708. The reflective element 708 may be a surface that reflects a first range of wavelengths but is substantially transparent to a second range of wavelengths (different from the first range of wavelengths). The first range of wavelengths may be in the infrared spectrum, and the second range of wavelengths may be in the visible spectrum. For example, the reflective element 708 may comprise a hot mirror that reflects infrared light but transmits visible light. In such embodiments, infrared light 710a, 712a, and 714a from the wearer propagates to the optical element 706, is reflected from there, and yields reflected infrared light 710b, 712b, and 714b, which can be imaged by the imager 702b. In some embodiments, the imager 702b may be sensitive to or capable of capturing at least a subset (non-empty subset and / or subset less than all) of a first range of wavelengths reflected by the reflecting element 708. For example, the reflecting element 708 may reflect infrared light in the range of 700 nm to 1.5 μm, and the imager 702b may be sensitive to or capable of capturing near-infrared light in the wavelength range of 700 nm to 900 nm. In another embodiment, the reflective element 708 may reflect infrared light in the range of 700 nm to 1.5 μm, and the imaging unit 702b may include a filter that filters out infrared light in the range of 900 nm to 1.5 μm so that the imaging unit 702b can capture near-infrared light in the wavelength range of 700 nm to 900 nm.

[0179] Visible light from the outside world 456 can be transmitted through the optical element 706 and can be perceived by the wearer. In fact, the imaging system 700 shown in FIG. 7A acts like a virtual imaging machine 702c that is retro-directed towards the wearer's eye 304. The virtual imaging machine 702c can image virtual infrared light 710c, 712c, 714c (shown as dotted lines) that propagates from the wearer's eye 304 through the optical element 706. The hot mirror (or other DOE described herein) can be disposed on the inner surface 704 of the optical element 706, but this is not limiting. In other embodiments, the hot mirror or DOE can be disposed on the outer surface of the optical element 706 or within the optical element 706 (e.g., a volumetric HOE).

[0180] FIG. 7B schematically illustrates another example of the imaging system 700. In this embodiment, perspective distortion can be reduced or eliminated by the combined use of a perspective control lens assembly 716b (e.g., a shift lens assembly, a tilt lens assembly, or a tilt-shift lens assembly) and the imaging machine 702b. In some embodiments, the perspective control lens assembly 716b can be part of the lens of the imaging machine 702b. The perspective control lens 716b can be configured such that the normal to the imaging machine 702b is substantially parallel to the normal to the region of the surface 704 that includes the DOE (or HOE) or the hot mirror. In fact, the imaging system 700 shown in FIG. 7B acts like a virtual imaging machine 702c with a virtual perspective control lens assembly 716c that is retro-directed towards the wearer's eye 304.

[0181] In addition or alternatively, as schematically shown in FIG. 7C, the reflective element 708 of the optical element 706 may have an off-axis holographic mirror (OAHM) that is used to reflect light 710a, 712a, 714a on its surface 704 and facilitate viewing of the eye 304 by the camera imaging machine 702b that captures the reflected light 710b, 712b, 714b. The OAHM 708 may also have refractive power and in that case can be an off-axis volumetric diffractive optical element (OAVDOE), as schematically shown in FIG. 7D. In the embodiment shown in FIG. 7D, the effective location of the virtual camera 702c is at infinity (not shown in FIG. 7D).

[0182] In some embodiments, the HOE (e.g., OAHM or OAVDOE) or hot mirror can be divided into a plurality of segments. Each of these segments can have different optical properties or characteristics, including, for example, the reflection angle or refractive power at which the segment reflects the incident (infrared) light. The segments can be configured such that light is reflected from each segment toward the imaging machine 702b. As a result, the image acquired by the imaging machine 702b will also be divided into a corresponding number of segments and will effectively view the eye from different angles. FIG. 7E schematically illustrates an embodiment of the display system 700 having an OAHM or hot mirror with three segments 718a1, 718a2, 718a3 that act as individual virtual cameras 702c1, 702c2, 702c3 that image the eye 304 at different angular locations. Additional embodiments of the display system with segmented optical elements are described with reference to FIGS. 8 and 10.

[0183] Figure 7F schematically illustrates another embodiment of the display system 700, each having a refractive power (e.g., segmented OAVDOE) or hot mirror, with each segment having three segments 718a1, 718a2, and 718a3, each generating a virtual camera at infinity and imaging the eye 304 at different angular locations. Three segments are schematically illustrated in Figures 7E and 7F, but this is illustrative and not limiting. In other embodiments, two, four, five, six, seven, eight, nine or more segments can be utilized. None of these segments of the HOE or hot mirror have refractive power, or some or all of them may have refractive power.

[0184] In Figures 7E and 7F, the three segments 718a1, 718a2, and 718a3 are shown spaced apart horizontally across the optical element 706. In other embodiments, the segments can be spaced vertically on the optical element 706. For example, Figure 7G schematically shows a DOE or hot mirror 718 having two vertically spaced segments 718a1 and 718a2, where segment 718a1 is configured to backreflect light toward the imager 702b (which may be in the same substantially horizontal plane as segment 718a1), and segment 718a2 is configured to reflect light upward toward the imager 702b. Similar to a bifocal lens, the arrangement shown in Figure 7G may be advantageous in allowing the imaging system 700 to use a reflected image acquired by the imaging unit 702b from the upper segment 718a1 (illustrated via a solid arrow) when the wearer is looking straight ahead through the upper portion of the HMD, and to use a reflected image from the lower segment 718a2 (illustrated via a dashed arrow) when the wearer is looking downward through the lower portion of the HMD.

[0185] As illustrated with reference to Figure 8-10, another arrangement can enable the imaging system 700 to use a reflected image from the outer segment obtained by the imaging machine 702b when the wearer is looking forward or away from the nose, and to use a reflected image from the inner segment when the wearer is looking toward the nose.

[0186] A mixture of horizontally and vertically spaced segments can also be used in other embodiments. For example, Figure 7H shows another embodiment of the HOE or hot mirror 718 with a 3x3 array of segments. The imaging machine 702b can obtain reflection data from each of these nine segments representing rays originating from different areas and angular directions of the eye region. Two exemplary rays that propagate from the eye region to the HOE or hot mirror 718 and are reflected back by the imaging machine 702b are shown as solid and dashed lines. The imaging system 700 (or processing module 224 or 228) can analyze the reflection data from multiple segments to multifocally calculate the three-dimensional shape of the eye or the direction of the eye's line of sight (e.g., eye orientation).

[0187] Embodiments of the optical system 700 utilizing segments may have several advantages. For example, segments can be used individually by selecting specific segments best suited to a particular task, or they can be used collectively to multifocally estimate the three-dimensional shape or orientation of the eye. In the former case, this selectivity can be used, for example, to select an image of the wearer's iris that is least obstructed by the eyelids or eyelashes. In the latter case, the three-dimensional reconstruction of the eye can be used to estimate orientation (e.g., by estimating the location of corneal bulges) or accommodative state (e.g., by estimating lens-induced distortion on the apparent location of the pupil). Angle segmentation

[0188] In some implementations, it may be advantageous to provide a wider range of angles over which the user's eyes can be tracked using the embodiments described herein. For example, it may be advantageous to increase the line-of-sight sensitivity of any eye-tracking imaging device (e.g., a camera) along at least a portion of the eye's line-of-sight orientation.

[0189] Figure 8 shows another embodiment of the optical system 700 capable of performing eye tracking. The optical system in Figure 8 shares many common features with those of the optical system 700 shown in Figure 7A. Figure 8 shows an imaging device 702b configured to image the user's eye 304. As light is reflected from the user's eye 304 or a part thereof (e.g., cornea, retina, iris, sclera, etc.), the light may be reflected from at least the reflective surface of a partially reflective element 708. The reflective element 708 may be located on a substrate 804, which may provide stability to the reflective surface (e.g., part of a thin film or coating). The substrate 804 may contain a polymer material such as plastic. Opposite the substrate 804 (relative to the user's eye 304) may be an optical element 706. The optical element 706 may be a variable focus element (VFE) or light field display or other display element configured to project virtual content onto the user's eye. For example, the optical element 706 may include a waveguide assembly 405 in Figure 4 or a waveguide device described with reference to Figure 4-6. In some embodiments, the substrate 804 and the optical element 706 may be a single element.

[0190] The reflective element 708 may be configured to substantially reflect light in a range of wavelengths and / or substantially transmit light in a second range of wavelengths. The first and second ranges of wavelengths may be different from each other. The first range of wavelengths may substantially include infrared wavelengths or a specific subrange thereof (e.g., near-infrared). For example, the first range may include wavelengths corresponding to about 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm, 1000 nm, any value in between, or any endpoint within that range. The second range of wavelengths may substantially include visible wavelengths or a specific subrange thereof. For example, the second range may include wavelengths corresponding to about 390 nm, 450 nm, 500 nm, 550 nm, 600 nm, 650 nm, 700 nm, any value in between, or any endpoint within that range. The reflective element 708 may include a hot mirror. The reflective element 708 may include a reflective material, a coating, a diffractive optical element (DOE), and / or a holographic optical element (e.g., HOE718, OAHM as described above). In some embodiments, the first and second wavelength ranges may have at least partial overlap with each other.

[0191] As illustrated with reference to Figures 7A-7G, light from the eye is reflected by the reflective element 708 and imaged by a physical camera 702b (for example, near the user's temple). This optical array functions as if a virtual camera 702c, positioned in front of the user's eye, were to image the light 709c as if it were originating directly from the eye.

[0192] The line of sight angle 812 of the user's eye 304 can be defined between the optical axis 810 and the direction of the user's eye 304's line of sight. The optical axis 810 may be in the direction of the eye's natural resting position (e.g., facing forward). The line of sight angle 812 is zero when the user's line of sight is in the direction of the optical axis 810, negative when the user's line of sight is directed toward the user's nose 705 (e.g., towards the nose), and positive when the user's line of sight is directed toward the user's nearest temple (e.g., toward the temple, away from the nose 705, toward the corresponding imaging device 702b). As shown, the eyebox, which is imaged by the physical imaging device 702b, is represented by the prism 710b, and the eyebox of the virtual imaging device 702c is represented by the prism 710c.

[0193] Figure 9 shows a series of plots 904a–904e representing line-of-sight sensitivity (in pixels per degree) over horizontal line-of-sight angle (in degrees) for various configurations of a hot mirror, related to different differences in interpupillary distance (IPD) or different eyebox offsets along the axial direction (z-axis). In these embodiments, the hot mirror was not segmented and generally resembled the arrangement shown in Figure 8. The eyebox may define the area of ​​focus, including horizontal and vertical dimensions, which may be limited by the distance of the eye to the optical element 706. The eyebox may have dimensions, for example, about 30 mm × 30 mm × 13 mm. The interpupillary distance (IPD) may be within a range of up to about 9.5 mm between the eyes. The line-of-sight angle 812 may be within a range of about ±22 degrees (horizontal) to ±55 degrees (vertical). Other dimensions are also possible.

[0194] The line-of-sight sensitivity can be defined as the ratio between the measured movement of an eye feature (such as a flash or a pupil) imaged by an imaging machine (in pixel units) and the movement of the eye feature (in degrees). As can be seen from the embodiment in FIG. 9, the line-of-sight sensitivity can be lower at negative angles (such as a line of sight directed towards the nose) than at positive angles (such as a line of sight directed towards the temple away from the nose). The lower sensitivity from the book regarding the nose line-of-sight direction (negative line-of-sight angle) is likely due to the larger angle between the line-of-sight direction and the virtual camera 702c (which can approach 90 degrees when the user is looking directly towards the nose). In contrast, when the user's line of sight is on the more temple side (positive line-of-sight angle), the user is looking more directly at the virtual position of the virtual camera 702c.

[0195] Therefore, it may be beneficial to increase the line-of-sight sensitivity of the imaging device for negative line-of-sight angles (such as a line of sight oriented towards the nose side). As described herein, improving the nose line-of-sight sensitivity can be accomplished by orienting the first segment of the reflective element 708 at a non-zero angle with respect to the second segment so that the virtual imaging machine 702c can image the user's eye better when looking at the nose side.

[0196] FIG. 10 shows an exemplary optical system 700 having an angularly segmented reflective element 708. The reflective element 708 may include a first segment 708a and a second segment 708b. The second segment 708b may be angled with respect to the first segment 708a. The rising angle 728 may be defined between the second segment 708b and a plane or surface including the first segment 708a. The angle 728 may be steep as shown. For example, the angle 728 may be within the range of about 2° to about 35°, within the range of about 5° to about 20°, or within some other range. The angle 728 is about 3 o 、5 o 、7 o 、10 o 、12 o 、15 o 、18 o 、20 o 、25o , 30 o ,33 o , 35 o , 40 o , 50 o , 60 o , 70 o , 80 o , or may fall within a range having any value in between, or any endpoint within that range. Two segments 708a, 708b are shown in Figure 10, but this is for illustrative purposes only and is not limiting. Any preferred number or arrangement of segments can be used in other embodiments, for example, as described with reference to Figures 7E-7H. Furthermore, the first segment 708a is 0 o It may be positioned with a rising angle (for example, substantially parallel to the main plane of the substrate 804 as shown in Figure 10). However, in some embodiments, the first segment 708a may be tilted with respect to the main plane of the substrate 804 and / or the optical element 706.

[0197] As shown in Figure 10, the angled properties of the two segments 708a and 708b of the reflective element 708 result in corresponding virtual imaging devices 702c1 and 702c2, respectively. The first virtual imaging device 702c1 represents imaging from the first segment 708a, and the second virtual imaging device 702c2 represents imaging from the second segment 708b. When the user's line of sight is more nasal (e.g., towards the nose 705 shown in Figure 10), the line-of-sight sensitivity of the eye-tracking imaging system is improved because the virtual camera 702c2 is positioned more toward the nasal region (compared to the location of the first virtual camera 702c1). The angle between the nasal line of sight direction and the virtual imaging device 702c2 is reduced compared to the corresponding angle for the optical system 700 shown in Figure 8, for example. Therefore, the use of angularly segmented reflective elements 708 enables the eye-tracking system to provide improved gaze sensitivity for gaze angles located on the nasal (e.g., using virtual imager 702c2) and temple (e.g., using virtual imager 702c1) sides.

[0198] A larger rise angle 728 may require a thicker substrate 804. Therefore, various embodiments can be used that strike an effective balance between providing relatively high line-of-sight sensitivity at the nose angle and achieving a relatively thin substrate 804. In some such embodiments, the angle 728 is about 15° and the thickness of the substrate 804 is about 2 mm.

[0199] While angled segments of discrete straight lines are described for illustrative purposes, the angled portions of the reflective element 708 may be curved, at least partially, such as a portion of the surface of a quadratic surface (e.g., a sphere, ellipsoid, paraboloid, or hyperboloid). One or more of the segments may have refractive power. In addition, or alternatively, multiple segments may be angled horizontally (e.g., as shown in Figure 10) or vertically (e.g., in and out of the plane in Figure 10). Many other alternatives are also possible.

[0200] Figure 11 shows an exemplary image 1100 of an eye from an angularly segmented reflective element. In this experiment, the reflective element is a hot mirror (HM) including a flat segment and a segment angled at 15°. Image 1100 includes a first image portion 1108a and a second image portion 1108b, separated by a boundary line 1112. The image of the boundary line 1112 may be an artifact of the way the reflective element 708 (e.g., shown in Figure 10) was formed for this experiment. Image 1100 shows the eye and flashes 1116a and 1116b of a light source reflected from the cornea of ​​the eye. Note that the flashes 1116a and 1116b may be seen within image portions 1108a and 1108b, respectively. Exemplary method for manufacturing angularly segmented reflective elements

[0201] Figures 12A–12E illustrate various stages of an exemplary manufacturing process for an optical element 1224 (e.g., an angularly segmented hot mirror). The manufacturing process may comprise injection molding and may include two injection molding stages (e.g., a first injection molding stage described below with reference to Figure 12A and a second injection molding stage described below with reference to Figure 12D).

[0202] Figure 12A shows the first injection molding step in which the first transparent material 1208 is introduced into the first mold 1212. The first mold 1212 may include a plurality of parts, such as a first part 1212a and a second part 1212b, which are joined together to form a cavity 1270 into which the first transparent material 1208 is introduced, as shown. To form an angled segment of the optical element 1224, the second surface 1214b may be angled with respect to the first surface 1214a by a rise angle 1228. The rise angle 1228 may be selected to provide a rise angle 728 of the second segment 708b, as described with reference to Figure 10. The rise angle 1228 may be defined between the second surface 1214b and a plane or surface that includes at least a portion of the first surface 1214a. The angle may be steep, as shown. The rising angle 1228 may be in the range of approximately 2° to approximately 35°, approximately 5° to approximately 20°, or any other range. For example, angle 1228 is approximately 3 o , 5 o , 7 o , 10 o , 12 o , 15 o , 18 o , 20 o ,twenty five o , 30 o ,33 o , 35 o , 40 o , 50 o , 60 o , 70 o , 80 o It may also fall within any value in that range, or within any range that has any endpoints within that range.

[0203] The reflective material 1204 can be disposed on or adjacent to the first surface 1214a of the first portion 1212a and the second surface 1214b of the second portion 1212b. The reflective material 1204 can include a hot mirror film. For example, the reflective material 1204 can be substantially transmissive within the visible portion of the electromagnetic spectrum and substantially reflective within the infrared portion of the electromagnetic spectrum. For example, the reflective material 1204 may be transmissive to at least 50%, 60%, 70%, 80%, 90%, or more of the visible light incident thereon. The reflective material 1204 may be reflective to at least 30%, 40%, 50%, 60%, 70%, 80%, 90%, or more of the infrared light incident thereon. As an example, the hot mirror film may include the 3M HM-825nm film available from 3M Corporation.

[0204] The reflective material 1204 may be configured to substantially reflect light in a range of wavelengths or substantially transmit light in a second range of wavelengths. The first and second ranges of wavelengths may be different from each other. The first range of wavelengths may include infrared wavelengths or a specific subrange of infrared wavelengths. For example, the first range may include wavelengths corresponding to approximately 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm, 1000 nm, any value in between, or any endpoint within that range. The second range of wavelengths may substantially include visible wavelengths or a specific subrange of visible wavelengths. For example, the second range may include wavelengths corresponding to approximately 390 nm, 450 nm, 500 nm, 550 nm, 600 nm, 650 nm, 700 nm, any value in between, or any endpoint within that range. The reflective material 1204 may include reflective materials, coatings, and / or holographic or diffractive optical elements (e.g., HOE718, OAHM as described above). In some embodiments, the first and second wavelength ranges may have some overlap with each other. The reflective material 1204 may have a thickness of less than 2 mm. For example, the thickness may fall within a range having about 0.1 mm, 0.2 mm, 0.3 mm, 0.4 mm, 0.5 mm, 0.6 mm, 0.7 mm, 0.8 mm, 0.9 mm, 1 mm, 1.2 mm, 1.5 mm, 1.7 mm, any value in between, or any endpoint within that range.

[0205] One or more fluid (e.g., air or gas) vents may be located along the first surface 1214a, the second surface 1214b, or at the confluence point 1222 between surfaces 1214a and 1214b. The use of such vents may be advantageous in allowing the reflective material 1204 to be pressed against surfaces 1214a and 1214b by the pressure of the transparent material 1208 when the material 1208 is introduced into the first mold 1212. For example, allowing a gas (e.g., air) initially located within the cavity 1270 to vent as the material 1208 is introduced into the cavity 1270 may allow the reflective material 1204 (which may be in the form of a thin film) to form a sharp angle at the confluence point 1222 between the two segments.

[0206] The first transparent material 1208 may be transparent in the visible and infrared spectral regions to allow visible and infrared light to pass through the material 1208 and reach the reflective element 1204. As described above, the reflective element 1204 may then reflect the infrared component of the incident light. The first transparent material 1208 may contain polymers or plastics. For example, the first transparent material 1208 may contain elastomers, thermoplastics, thermosetting resins, or other polymers. Exemplary materials include polyamides, polypropylene, high-density polyethylene, acrylonitrile butadiene styrene, polycarbonate, polymethyl methacrylate (PMMA), or any combination thereof.

[0207] The injection molding step described with reference to Figure 12A results in the formation of an intermediate structure 1216a, which can be removed from the first mold 1212. The intermediate structure 1216a can be referred to as the first molded component, as it will be used in a second injection molding step described with reference to Figure 12D. Figure 12B shows the intermediate structure 1216a outside the first mold 1212. In some ways, pieces of the reflective material 1204 may extend beyond the body 1217 of the intermediate structure 1216a, and these pieces may be removed (e.g., by cutting, trimming, polishing, etc.) at one or more endpoints 1218a, 1218b to form the intermediate structure 1216.

[0208] The intermediate structure 1216 may be formed as described above, wherein the reflective material 1208 is contained within the first mold 1212, and in other embodiments, the body 1217 of the intermediate structure 1216 may be formed first (e.g., via injection molding), and then the reflective material 1204 may be bonded to or attached to the body 1217, or coated or deposited thereon.

[0209] The manufacturing method may optionally include a second injection molding step in which a second body 1219 is formed on an intermediate structure to provide an optical element 1224. Figure 12D shows an embodiment of this second injection molding step. The intermediate structure 1216 may be placed in a second mold 1220. The second mold 1220 may include two or more parts, such as a first part 1220a and a second part 1220b, as shown, with a cavity 1280 formed between them. The first and second parts 1220a, 1220b of the second mold 1220 may be molded to form two main surfaces that are substantially flat. The two main surfaces may be substantially parallel to each other so that the terminated optical element has a substantially flat outer surface. Other relationships are also possible; for example, the inner surfaces of the first or second parts 1220a, 1220b can be curved, which provides refractive power to the optical element or allows for a better fit when attached to the display element 706.

[0210] The second transparent material 1230 may be placed into the second cavity 1280 of the second mold 1220. The second transparent material 1230 may be placed such that the reflective material 1208 is positioned between the first body 1217 and the cavity 1280. The second transparent material 1230 may be substantially identical to the first transparent material 1208. Other materials may also be considered, for example, the first and second materials may have different refractive indices or different visible or infrared transmittances.

[0211] The second injection molding stage forms the optical element 1224, which is shown in Figure 12E and removed from the second mold 1220. As can be seen therefrom, the reflective element 1208 is positioned between the first body 1217 (formed during the first injection molding stage) and the second body 1219 (formed during the second injection molding stage). The two transparent bodies 1217, 1219 thereby protect the reflective element 1204 from exposure to environmental conditions (e.g., dust, humidity, etc.) or from touch by the user of the wearable system 200. The optical element 1224 can be used as an angle-segmented reflective element (e.g., reflective element 708 and substrate 804), as described with reference to Figure 10. For example, the optical element 1224 can be bonded or mounted to the optical display element 706.

[0212] The optical element 1224 may be formed as described above, but in other embodiments, the second body 1219 may be formed separately (e.g., via injection molding) and then bonded or attached to the intermediate structure 1216.

[0213] The manufacturing process for the optical element may include additional, optional, or different steps. For example, Figures 13A-13B show an embodiment in which the optical element 1224 is formed to include one or more light sources.

[0214] Figure 13A shows an embodiment of a polymer layer 1232 comprising two light sources 1236a and 1236b. An electronic network may be included on or within the polymer layer 1232 to provide power to the light sources 1236a and 1236b. The network may be formed from a conductive material that is transparent to visible light, such as indium tin oxide (ITO). One or both of the light sources 1236a and 1236b may be configured to emit infrared light with wavelengths such as approximately 700 nm, 750 nm, 800 nm, 850 nm, 900 nm, 950 nm, 1000 nm, any value in between, or any endpoint within that range. In some embodiments, the light sources 1236a and 1236b comprise SFH4055 infrared light-emitting diodes (LEDs) (available from Osram Opto Semiconductors). Two light sources are shown in Figure 13A, but light sources 1, 3, 4, 5, 6, or more may be used in other embodiments.

[0215] The polymer layer 1232 may comprise any polymer, such as plastic. For example, the polymer layer 1232 may contain polyethylene terephthalate (PET). The polymer layer 1232 may be a rigid material. The polymer layer 1232 may have a thickness of less than 1 mm. For example, the thickness may be within a range of 0.05 mm, 0.1 mm, 0.15 mm, 0.2 mm, 0.3 mm, 0.4 mm, 0.5 mm, 0.6 mm, 0.7 mm, 0.8 mm, 0.9 mm, any value in between, or any endpoint within that range.

[0216] Figure 13B shows an exemplary process for including a polymer layer 1232 during a second injection molding step, as described with reference to Figure 12D. As shown, the polymer layer 1232 may be placed in the second mold 1220 prior to the introduction of the second transparent material 1230. The light sources 1236a, 1236b may be oriented to face the reflective material 1204 such that the light sources 1236a, 1236b are positioned inside the second body 1219. The portion of the polymer layer 1232 extending outside the body 1217, 1219 may be removed. Figure 13C shows the optical element 1224 after demolding. The light sources 1236a, 1236b may be arranged so that the emitted light passes through the optical element 1224 without first being incident on the reflective material 1204 (which may be reflective to the wavelength of light emitted by the source). Light from light sources 1236a and 1236b can be used to provide corneal flashes, which are used for eye tracking. Exemplary manufacturing method

[0217] Figure 14 is a flowchart for an exemplary method 1400 for manufacturing optical elements such as angularly segmented hot mirrors. The segmented hot mirrors may be injection molded as described above with reference to Figures 12A-13C.

[0218] In block 1404, a first mold having a first cavity may be provided. The first cavity may include a first surface having a first portion that is at a non-zero angle with respect to a second portion. The angle may be any angle described above, for example, a rise angle of 728 or 1228. For example, the angle may range from 2 to 25 degrees or 5 to 20 degrees.

[0219] In block 1408, method 1400 may include the step of positioning a hot mirror film adjacent to at least a first portion and a second portion of the first surface of the first cavity. The hot mirror film may be substantially transparent to visible light and substantially reflective to infrared light, as described above. The first mold may include a vent between the first portion and the second portion, which may allow air to be vented from the mold so that the hot mirror film can form an acute angle 1232.

[0220] In block 1412, the first polymer material may be introduced into the first cavity of the first mold to form a first molded component (for example, an intermediate structure 1216a or 1216, as described with reference to Figures 12B and 12C). In block 1416, the first molded component may be removed from the first mold. The first molded component may include at least a portion of a hot mirror film, for example, as shown in Figures 12B and 12C.

[0221] In block 1420, method 1400 may include the step of placing a first molded component in a second mold having a second cavity. The first molded component can be oriented such that the hot mirror film is positioned toward the central region of the second cavity (see, for example, Figure 12D). In block 1424, method 1400 may include the step of introducing a second polymer material into the second cavity such that the second polymer material covers at least a portion of the hot mirror film, thereby forming a second molded component (see, for example, Figure 12D). The first polymer material may be identical to the second polymer material. One or both of the polymers may be substantially transparent to visible and infrared light. One or both of the first or second polymers may include a thermoplastic polymer. Exemplary polymers may include polycarbonate, polymethyl methacrylate (PMMA), and / or any other material described herein in relation to the insertable material. In block 1428, the second molded component can be removed from the second mold.

[0222] Method 1400 may optionally include the step of removing a portion of the hot film that extends outside the first molded component or the second molded component (see, for example, Figures 12B and 12C). In addition, or alternatively, Method 1400 may include the step of placing at least one infrared light source in the second cavity of the second mold (see, for example, Figures 13A and 13B). For example, at least one infrared light source may be placed on a polymer film. The polymer film may contain polyethylene terephthalate (PET) and / or any other polymer. Method 1400 may also include the step of placing the polymer film in the second cavity of the second mold.

[0223] A reflective element may include two portions angled relative to each other, but in other embodiments, additional angled portions may be formed. For example, the first surface of the first mold may include a third portion adjacent to a second portion, such that the third portion is at a second non-zero angle with respect to the second portion. The first and / or second portions of the first surface may include curvature such as one or more curved regions. In some embodiments, one or more curved regions may include different curvatures and / or orientations relative to each other. As described herein, method 1400 may include the step of mounting the second molded component to a display for an augmented, composite, or virtual reality device.

[0224] In one embodiment, a method for manufacturing an optical element may include the step of applying an optical film to a first surface of a first optical element, the first surface comprising a first section and a second section. The second section may be positioned at a non-zero angle with respect to the first section. The method may further include the step of applying the second optical element to the first optical element such that the optical film is positioned between the first optical element and the second optical element, thereby forming the optical element. The optical film may include features of one or more of the hot mirror film, reflective element 708, and / or reflective material 1204 described herein. Additional aspects

[0225] In the first aspect, a head-mounted display (HMD) configured to be worn on the head of a user is disclosed. The HMD comprises a frame having a pair of ear-hooks; a pair of optical elements supported by the frame such that each of the pair of optical elements can be positioned in front of the user's eyes; a forward-facing imager mounted on one of the pair of ear-hooks; a reflective element positioned within or on one of the pair of optical elements; and a reflective element configured to reflect infrared light toward the forward-facing imager, which is configured to receive infrared light reflected by the reflective element.

[0226] In the second aspect, the pair of optical elements are each transparent to visible light, as described in aspect 1 of the HMD.

[0227] In a third aspect, the HMD described in side 1 or side 2 is configured such that each pair of optical elements is configured to display an image to the user.

[0228] On the fourth side, the pair of optical elements each include a light field display, as shown in side 3 of the HMD.

[0229] In the fifth aspect, the HMD described in aspect 4 comprises a waveguide stack configured to output an image to the user as a light field display.

[0230] In the sixth aspect, the reflective element is a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE), as described in any one of aspects 1-5.

[0231] In the seventh aspect, the reflective element is transmittance to visible light, as described in any one of aspects 1-6 of the HMD.

[0232] In the eighth aspect, the HMD according to any one of aspects 1-7, wherein the reflective element comprises a plurality of segments, and each segment within the plurality of segments has optical properties different from the optical properties of at least one other segment within the plurality of segments.

[0233] In the ninth aspect, the optical properties include the angle of reflection or refractive power of the HMD as described in aspect 8.

[0234] The tenth side is the HMD described in side 8 or side 9, wherein the multiple segments comprise two, three, four, five, six, seven, eight, or nine segments.

[0235] In the eleventh side, the forward-facing imaging device is mounted on the temple portion of one of the pair of earpieces, as described in any one of sides 1-10 of the HMD.

[0236] In the twelfth aspect, the imaging device comprises an eye-line control lens assembly, as described in any one of aspects 1-11 of the HMD.

[0237] In the 13th aspect, the eye-line control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in the HMD on the 12th aspect.

[0238] In a fourteenth aspect, a display system is disclosed. The display system comprises an optical element configured to display an image to a user, the optical element being positioned in front of the user's eyes; a forward-facing imager; and a reflective element located within or on the optical element, the reflective element being configured to reflect infrared light received from the user's eyes toward the forward-facing imager.

[0239] In the 15th aspect, the optical element is a display system as described in aspect 14, comprising a light field display.

[0240] In the sixteenth aspect, the display system according to aspect 14 or 15, wherein the reflective element includes a hot mirror, an off-axis diffractive optical element (DOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0241] In the 17th aspect, the display system according to any one of aspects 14-16, wherein the reflective element comprises multiple segments having different refractive powers or different reflection angles.

[0242] The 18th aspect further comprises a non-transient memory configured to store images of a user's eyes acquired by a forward-facing imager, and a hardware processor communicating with the non-transient memory, which is programmed to access images of the eyes and perform one or more of the following: tracking the user's eyes, extracting biometric information associated with the user's eyes, reconstructing the shape of parts of the user's eyes, estimating the accommodative state of the user's eyes, or imaging the retina, iris, or other elements of the user's eyes, as described in any one of aspects 14-17.

[0243] In the 19th aspect, a head-mounted display system is disclosed. The HDM comprises a frame configured to support the display system described in any one of aspects 14-18 such that the optical elements are positioned in front of the user's first eye.

[0244] In the 20th aspect, the frame supports the second display system described in any one of aspects 14-18, the head-mounted display system as described in aspect 19, such that the optical elements of the second display system are positioned in front of the user's second eye.

[0245] In the 21st aspect, an imaging system is disclosed. The imaging system comprises a reflective element that reflects light in a first wavelength range, and an imaging machine that is sensitive to light in a non-empty subset less than all of the first wavelength range, and is configured to be oriented to capture the light reflected by the reflective element.

[0246] In the 22nd aspect, the imaging system according to aspect 21, wherein the reflective element includes a hot mirror, a holographic optical element (HOE), an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0247] In the 23rd aspect, the imaging system according to any one of aspects 21-22, wherein the first wavelength range includes the infrared wavelength range.

[0248] The imaging system according to any one of aspects 21-23, wherein the imaging system comprises an optical element, the optical element comprises a reflective element, and the optical element is transparent to at least 50% of the visible light incident on the optical element.

[0249] In the 25th aspect, the imaging system according to any one of aspects 21-24 comprises a reflective element with multiple segments.

[0250] The imaging system according to side 25, wherein in side 26, the first segment in the plurality of segments has optical properties different from those of the second segment in the plurality of segments.

[0251] In aspect 27, the imaging system according to aspect 26, wherein the optical properties of a first segment in a plurality of segments or the optical properties of a second segment in a plurality of segments include the angle of reflection or the refractive power.

[0252] In aspect 28, the imaging system according to any one of aspects 25-27, wherein the plurality of segments comprises at least two segments.

[0253] In the 29th aspect, two of the multiple segments are arranged horizontally, as described in any one of aspects 25-28 of the imaging system.

[0254] In the 30th aspect, two of the multiple segments are arranged vertically, as described in any one of aspects 25-29 of the imaging system.

[0255] In the 31st aspect, the imaging system described in any one of aspects 25-30, wherein some of the multiple segments are arranged within a grid.

[0256] In the 32nd aspect, the imaging system according to any one of aspects 21-31 further comprises an eye-line control lens assembly.

[0257] In the 33rd aspect, the line-of-sight control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in the imaging system of the 32nd aspect.

[0258] In the 34th aspect, an imaging system for indirectly capturing an image of a user's eye is disclosed. The imaging system comprises a reflective element that reflects light in a first wavelength range, the reflective element including an off-axis holographic mirror (OAHM) or an off-axis stereodiffractive optical element (OAVDOE), the reflective element being oriented to reflect light propagating from the user's eye when the imaging system is placed in front of the user's eye, and an imaging machine that is sensitive to light in a non-empty subset less than all of the first wavelength range, and is oriented to form an image of the user's eye by capturing the light propagating from the user's eye reflected by the reflective element.

[0259] The imaging system according to side 34, wherein, in side 35, the image of the user's eye formed by the imaging machine and the image of the user's eye formed by a camera positioned in front of the user's eye are indistinguishable.

[0260] In the 36th aspect, the imaging system according to aspect 35, wherein the image of the user's eye, which is imaged by the imaging machine, is effectively imaged by a camera positioned directly in front of the user's eye.

[0261] In aspect 37, the imaging system according to any one of aspects 35-36, wherein the effective location of the camera positioned in front of the user's eye is at infinity.

[0262] In aspect 38, the imaging system according to any one of aspects 35-37, wherein the first wavelength range includes the infrared wavelength range.

[0263] The imaging system according to any one of the 39th aspect, comprising an optical element, wherein the optical element comprises a reflective element, and the optical element is transparent to at least 50% of the visible light incident on the optical element, as described in any one of aspects 35-38.

[0264] In the 40th aspect, the imaging system according to any one of aspects 35-39 comprises a reflective element with multiple segments.

[0265] The imaging system according to side 40, wherein in side 41, the first segment in the plurality of segments has optical properties different from those of the second segment in the plurality of segments.

[0266] In the 42nd aspect, the imaging system according to aspect 41, wherein the optical properties of a first segment in a plurality of segments or the optical properties of a second segment in a plurality of segments include the angle of reflection or the refractive power.

[0267] In the 43rd aspect, the imaging system according to any one of aspects 40-42, wherein the multiple segments comprise at least two segments.

[0268] In the 44th aspect, two of the multiple segments are arranged horizontally, as described in any one of aspects 40-43 of the imaging system.

[0269] In the 45th aspect, two of the multiple segments are arranged vertically, as described in any one of aspects 40-44 of the imaging system.

[0270] In the 46th aspect, the imaging system described in any one of the aspects 40-45, wherein some of the multiple segments are arranged in a grid.

[0271] In the 47th aspect, the imaging system according to any one of aspects 34-46 further comprises an eye-line control lens assembly.

[0272] In the 48th aspect, the line-of-sight control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in the imaging system of the 47th aspect.

[0273] In the 49th aspect, an imaging system is disclosed. The imaging system comprises a display having reflective elements that reflect light in a first wavelength range, including a hot mirror, an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE); and an imaging machine that is sensitive to light in the first wavelength range and is configured to be oriented to capture light reflected by at least the reflective elements.

[0274] In the 50th aspect, the imaging system described in aspect 49, wherein the first wavelength range includes the infrared wavelength range.

[0275] In the 51st aspect, the imaging system according to aspect 49 or aspect 50, wherein the display is substantially transparent to visible light.

[0276] In aspect 52, the imaging system according to any one of aspects 49-51, wherein the reflective element comprises a plurality of segments, and each segment within the plurality of segments has optical properties different from the optical properties of at least one other segment within the plurality of segments.

[0277] In aspect 53, the imaging system described in aspect 52, wherein the optical properties include the angle of reflection or the refractive power.

[0278] The imaging system according to side 52 or side 53, wherein the multiple segments comprise two, three, four, five, six, seven, eight, or nine segments.

[0279] In aspect 55, the imaging system according to any one of aspects 49-54 further comprises an eye-line control lens assembly.

[0280] In the 56th aspect, the line-of-sight control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in the imaging system of the 55th aspect.

[0281] Aspect 57 further comprises a non-transient data storage device configured to store images obtained by an imaging machine, and a hardware processor communicating with the non-transient data storage device, which is programmed with executable instructions to analyze an image and perform one or more of the following: eye tracking, biometric identification, multi-view reconstruction of the shape of the eye, estimation of the eye's accommodation state, or imaging of the eye's retina, iris, or other distinguishing pattern. This is the imaging system according to any one of aspects 21-56.

[0282] In aspect 58, a head-mounted display (HMD) is disclosed. The HMD comprises an imaging system as described in any one of aspects 21-57.

[0283] The HMD according to side 58, in side 59, comprises a frame having a portion configured to be worn near the ear, and an imaging device positioned near the portion.

[0284] In the 60th aspect, the imaging system is configured to image the wearer's first eye, and the HMD comprises a second imaging system as described in any one of aspects 21-57, wherein the second imaging system is configured to image the wearer's second eye, as described in aspect 58 or 59.

[0285] In Aspect 61, HMD is an Augmented Reality Device (ARD), as defined in any one of Aspects 58–60.

[0286] In Aspect 62, a method for creating a virtual camera is disclosed. The method includes the steps of providing an imaging system in front of an object to be imaged, thereby creating a virtual camera in front of the object, the imaging system comprising: a reflective element that reflects light in a first wavelength range, the reflective element including an off-axis holographic mirror (OAHM) or an off-axis stereodiffractive optical element (OAVDOE), the reflective element being oriented to reflect light propagating from the object when the imaging system is placed in front of the object; and an imaging machine that is sensitive to light in a non-empty subset less than all of the first wavelength range, the imaging machine being oriented to image an image of the object by capturing light propagating from the object reflected by the reflective element, the image of the object imaged by the imaging machine and the image of the object imaged by a camera in front of the object are indistinguishable.

[0287] In aspect 63, the method according to aspect 62, wherein the first wavelength range includes the infrared wavelength range.

[0288] The method according to any one of the aspects 62-63, wherein the imaging system comprises an optical element, the optical element comprises a reflective element, and the optical element is transparent to at least 50% of the visible light incident on the optical element.

[0289] In aspect 65, the reflective element comprises multiple segments, as described in any one of aspects 62-64.

[0290] The method according to aspect 65, wherein in aspect 66, the first segment in the plurality of segments has optical properties different from those of the second segment in the plurality of segments.

[0291] In aspect 67, the method according to aspect 66, wherein the optical properties of a first segment in a plurality of segments or the optical properties of a second segment in a plurality of segments include the angle of reflection or the refractive power.

[0292] Aspect 68 is the method according to any one of Aspects 65-67, wherein the plurality of segments comprises at least two segments.

[0293] The method according to any one of the 69th side, wherein two of the multiple segments are arranged horizontally, as described in any one of the 65-68 sides.

[0294] In the 70th aspect, two of the multiple segments are arranged vertically, as described in any one of the 65-69 aspects.

[0295] Aspect 71 is the method described in any one of Aspects 65-70, wherein some of the multiple segments are arranged within a grid.

[0296] In the 72nd aspect, the imaging machine further comprises an eye-line control lens assembly, according to the method of any one of aspects 62-71.

[0297] In the 73rd aspect, the eye-line control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in the 72nd aspect.

[0298] In Aspect 74, a method for imaging an object using a virtual camera is disclosed. The method includes the steps of: providing an imaging system in front of an object to be imaged and creating a virtual camera in front of the object, wherein the imaging system comprises: a reflective element that reflects light in a first wavelength range, the reflective element including an off-axis holographic mirror (OAHM) or an off-axis stereodiffractive optical element (OAVDOE), the reflective element being oriented to reflect light propagating from the object when the imaging system is placed in front of the object; and an imaging machine that is sensitive to light in a non-empty subset less than all of the first wavelength range, and is oriented to image an image of the object by capturing light propagating from the object reflected by the reflective element; and imaging an image of the object by capturing light propagating from the object reflected by the reflective element, wherein the image of the object imaged by the imaging machine and the image of the object imaged by the camera in front of the object are indistinguishable.

[0299] In aspect 75, the method as described in aspect 74, wherein the first wavelength range includes the infrared wavelength range.

[0300] The method according to any one of the aspects 74-75, wherein the imaging system comprises an optical element, the optical element comprises a reflective element, and the optical element is transparent to at least 50% of the visible light incident on the optical element.

[0301] In aspect 77, the reflective element comprises multiple segments, as described in any one of aspects 74-76.

[0302] The method according to aspect 77, wherein in aspect 78, the first segment in the plurality of segments has optical properties different from those of the second segment in the plurality of segments.

[0303] In aspect 79, the method according to aspect 78, wherein the optical properties of a first segment in a plurality of segments or the optical properties of a second segment in a plurality of segments include the angle of reflection or the refractive power.

[0304] In aspect 80, the method according to any one of aspects 77-79, wherein the multiple segments comprise at least two segments.

[0305] In the 81st aspect, two of the multiple segments are arranged horizontally, as described in any one of the 77-80 aspects.

[0306] In the 82nd aspect, two of the multiple segments are arranged vertically, as described in any one of the 77-81 aspects.

[0307] Aspect 83 is the method described in any one of Aspects 77-82, wherein some of the multiple segments are arranged within a grid.

[0308] In aspect 84, the imaging device further comprises an eye-line control lens assembly, as described in any one of aspects 74-83.

[0309] In aspect 85, the eye-line control lens assembly comprises a deflection lens, a tilt lens, or a deflection-tilt lens, as described in aspect 84.

[0310] In aspect 86, an imaging assembly is disclosed. The imaging assembly comprises a see-through element (e.g., a display), a viewing camera positioned to view the display, a lens associated with the camera, and a reflective element on the display that makes the display reflective to all or part of the wavelengths to which the display is sensitive.

[0311] In aspect 87, the assembly described in aspect 86 includes a hot mirror, an off-axis holographic mirror (OAHM), or an off-axis stereodiffractive optical element (OAVDOE).

[0312] In aspect 88, the assembly is integrated into a wearable structure such as a pair of eyeglasses or a helmet, as described in any one of aspects 86-87.

[0313] In the 89th aspect, the reflective element is segmented, as described in any of the 86-88 assemblies.

[0314] In aspect 90, the assembly is configured for the use of segmented OAHMs to select the best possible viewing angles for a specific task (e.g., eye tracking or biometric identification), as described in aspect 89.

[0315] In Aspect 91, the assembly is configured for the use of a number of segment subimages for stereoscopic or multi-view three-dimensional reconstruction of the shape of the eye, as described in any one of Aspects 89-90.

[0316] In the 92nd aspect, a three-dimensional reconstruction of the shape of the eye is used to estimate the accommodative state of the eye, as described in the 91st aspect.

[0317] The assembly according to side 92, wherein the step of estimating the accommodative state of the eye includes comparing the apparent location and shape of the pupil and iris of the eye across multiple images of the same wearer of the assembly.

[0318] In Aspect 94, the step of estimating the accommodative state of the eye is the assembly described in any one of Aspects 92-93, which is used to determine the magnification state of the lens.

[0319] In Aspect 95, the assembly is configured for the use of image segments as input to an information fusion algorithm, as described in any one of Aspects 86-94.

[0320] In aspect 96, an information fusion algorithm is used to improve the apparent resolution of the eye or the quality of information extraction therefrom, as described in aspect 95 of the assembly.

[0321] In Aspect 97, the information fusion algorithm is an assembly described in any one of Aspects 95-96, including an image super-resolution technique.

[0322] In Aspect 98, an information fusion algorithm is used to improve the image of the iris of the eye, as described in any one of Aspects 95-97.

[0323] In Aspect 99, the information fusion algorithm is an assembly according to any one of Aspects 95–98, comprising iris code extraction (e.g., John Daugman, et al. 2006) and subsequent fusion of the resulting iris codes to form a single estimate of the wearer's iris codes.

[0324] In aspect 100, the assembly is configured for the use of image segments to improve eye pose estimation or tracking, as described in any of aspects 86-99.

[0325] In aspect 101, the assembly described in aspect 100 is used in conjunction with an image segment to improve the coverage of the eye in pose estimation, with a three-dimensional reconstruction of the eye, iris, pupil, and cornea (or any subset thereof).

[0326] In Aspect 102, the reflective element is an assembly as described in any one of Aspects 86-101, including an OAVDOE that includes a refractive force for adding or reducing beam divergence.

[0327] In aspect 103, the reflective element is an assembly as described in any one of aspects 86-102, comprising any number of segments (e.g., two, three, six, or nine segments).

[0328] In aspect 104, the reflective element is configured to reflect infrared light, and the viewing camera is sensitive to infrared light, as described in any one of aspects 86-103 of the assembly.

[0329] On side 105, the assembly described on side 104 includes a hot mirror, the reflective element configured to reflect in infrared light but otherwise be transparent to visible light.

[0330] The assembly according to any one of the sides 86-105, further comprising, in side 106, an offset lens (for example, as in tilt-shift photography) having a normal to a viewing camera parallel to the normal of a surface having a reflective element.

[0331] In aspect 107, a head-mounted display (HMD) is disclosed. The HMD comprises a pair of displays, each display comprising an imaging assembly as described in any one of aspects 86-106, with one assembly of the pair configured for each of the wearer's eyes. conclusion

[0332] The processes, methods, and algorithms described herein and / or depicted in accompanying diagrams may be embodied in code modules executed by one or more physical computing systems, hardware computer processors, application-specific circuits, and / or electronic hardware configured to perform specific computer instructions, thereby being fully or partially automated. For example, a computing system may include a general-purpose computer (e.g., a server) or a dedicated computer, dedicated circuit, etc., programmed with specific computer instructions. Code modules may be installed in a dynamic link library, compiled and linked into an executable program, or written in an interpreted programming language. In some implementations, specific operations and methods may be performed by circuits specific to a given function.

[0333] Furthermore, functional implementations of the present disclosure are sufficiently mathematical, computational, or technically complex that application-specific hardware (utilizing appropriate specialized executable instructions) or one or more physical computing devices may need to implement the functionality, for example, due to the volume or complexity of the computations involved, or to provide results substantially in real time. For example, video may contain many frames, each frame may have millions of pixels, and specifically programmed computer hardware needs to process the video data to provide a desired image processing task or application in a commercially reasonable amount of time.

[0334] Code modules or any type of data may be stored on any type of non-transient computer-readable medium, such as physical computer storage devices, including hard drives, solid-state memory, random-access memory (RAM), read-only memory (ROM), optical discs, volatile or non-volatile storage devices, combinations thereof, and / or equivalents. The method and modules (or data) may also be transmitted as data signals generated on various computer-readable transmission media, including wireless-based and wired / cable-based media (e.g., as part of a carrier wave or other analog or digital propagation signal), and may take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed process or process steps may be stored persistently or otherwise in any type of non-transient tangible computer storage device, or communicated via computer-readable transmission media.

[0335] Any process, block, state, step, or functionality in the flowcharts described herein and / or depicted in the accompanying diagrams should be understood as potentially representing a code module, segment, or portion of code containing one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in the process. Various processes, blocks, states, steps, or functionality can be combined, rearranged, added to, removed from, modified, or otherwise altered from the exemplary embodiments provided herein. In some embodiments, additional or different computing systems or code modules may implement some or all of the functionality described herein. The methods and processes described herein are also not limited to any particular sequence, and the blocks, steps, or states associated therewith may be implemented in other appropriate sequences, for example, sequentially, in parallel, or in some other manner. Tasks or events may be added to or removed from the exemplary embodiments disclosed. Furthermore, the separation of various system components in the implementations described herein is for illustrative purposes only and should not be understood as requiring such separation in all implementations. It should be understood that the program components, methods, and systems described can generally be integrated together in a single computer product or packaged across multiple computer products. Many implementation variations are possible.

[0336] This process, method, and system may be implemented in a network (or distributed) computing environment. Network environments include enterprise-wide computer networks, intranets, local area networks (LANs), wide area networks (WANs), personal area networks (PANs), cloud computing networks, crowdsourced computing networks, the Internet, and the World Wide Web. The network may be a wired or wireless network or any other type of communication network.

[0337] Each system and method of this disclosure has several innovative aspects, none of which alone contribute to or are required for the desirable attributes disclosed herein. The various features and processes described above may be used independently of each other or in various combinations. All possible combinations and secondary combinations are intended to fall within the scope of this disclosure. Various modifications of the implementations described herein may be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations without departing from the spirit or scope of this disclosure. Accordingly, the claims are not intended to be limited to the implementations shown herein, but should be given the broadest scope consistent with the disclosure, principles, and novel features disclosed herein.

[0338] Some features described herein in the context of separate implementations may also be implemented in combinations within a single implementation. Conversely, various features described in the context of a single implementation may also be implemented separately in multiple implementations or in any preferred secondary combination. Furthermore, features described above as acting in a combination and further claimed as such, but one or more features from the claimed combination may, in some cases, be removed from the combination, and the claimed combination may be subject to secondary combinations or variations of secondary combinations. No single feature or group of features is required or essential in any embodiment.

[0339] In particular, conditional statements used herein, such as “can,” “could,” “might,” “may,” “eg,” and equivalents, are generally intended to convey that one embodiment includes certain features, elements, and / or steps, while other embodiments do not, unless otherwise specifically stated or understood in the context in which they are used. Therefore, such conditional statements are generally not intended to imply that features, elements, and / or steps are required in any way for one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or steps are included or should be implemented in any particular embodiment, whether or not the author inputs or prompts them. The terms “comprising,” “including,” “having,” and equivalents are synonyms and are used in a non-restrictive manner to encompass additional elements, features, actions, behaviors, etc. Furthermore, the term "or," when used, for example, to connect a list of elements, is used in its inclusive sense (and not in its exclusive sense) so that the term "or" means one, some, or all of the elements in the list. In addition, the articles "a," "an," and "the," as used in this application and the attached claims, should be interpreted as meaning "one or more" or "at least one" unless otherwise specified.

[0340] As used herein, the phrase "at least one of ~" referring to a list of items refers to any combination of those items that includes a single element. In one embodiment, "at least one of A, B, or C" is intended to encompass A, B, C, A and B, A and C, B and C, and A, B, and C. Connecting phrases such as "at least one of X, Y, and Z" are generally understood differently in contexts such as those used to convey that an item, term, etc., may be at least one of X, Y, or Z, unless otherwise specifically stated. Thus, such connecting phrases are generally not intended to suggest that one embodiment requires the presence of at least one of X, at least one of Y, and at least one of Z, respectively.

[0341] Similarly, while actions may be depicted in a diagram in a specific order, it should be recognized that such actions do not necessarily need to be performed in a specific or sequential order shown, or that not all illustrated actions need to be performed, in order to achieve the desired result. Furthermore, a diagram may graphically depict one or more exemplary processes in the form of a flowchart. However, other actions not depicted may also be incorporated into the graphically illustrated exemplary methods and processes. For example, one or more additional actions may be performed before, after, simultaneously with, or in between any of the illustrated actions. In addition, actions may be rearranged or rearranged in other implementations. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the implementations described above should not be understood as requiring such separation in all implementations, and it should be understood that the described program components and systems may generally be integrated together in a single software product or packaged in multiple software products. In addition, other implementations are also within the scope of the following claims. In some cases, the actions enumerated in the claims may be performed in a different order, and the desired result can still be achieved.

Claims

1. A head-mounted display system (200, 700), wherein the head-mounted display system (200, 700) is A frame (212) configured to be supported on the head of the user (204), An optical element (706) coupled to the frame, wherein the optical element (706) is configured to display virtual image content to the user, and the optical element (706) is at least partially transparent to provide the user with a view of part of the environment, A forward-facing imager (702b) coupled to the frame, wherein the forward-facing imager (702b) is configured to receive infrared light reflected from the user's eyes (302, 304) through the optical elements, A reflective element (708) configured to reflect the infrared light reflected from the eye toward the forward-facing imaging machine, wherein the reflective element (708) is at least partially located within the optical element (706), and the reflective element (708) comprises a plurality of segments including a first segment (708a) and a second segment (708b), the first segment being angled at a non-zero angle with respect to the second segment, the first segment being configured to produce a first image of the eye configured to be captured by the forward-facing imaging machine, and the second segment being configured to produce a second image of the eye configured to be captured by the forward-facing imaging machine, Hardware processors (224, 228), wherein the hardware processors (224, 228) are Accessing multiple images of the eye generated by the forward-facing imaging device based on the infrared light reflected from the eye, The lateral aspect of the eye is determined by analyzing the aforementioned multiple images, and determining the lateral aspect of the eye is When the eye is looking inward toward the user's nose or straight ahead in a first line of sight direction, the first of the multiple images generated using infrared light reflected by the first segment is analyzed. When the eye is looking in a second outward line of sight that is away from the nose, the second of the multiple images generated using infrared light reflected by the second segment is analyzed. The side view of the eye is determined based on the first and second sets of images among the aforementioned set of images. including and The hardware processors (224, 228) are programmed to do this. A head-mounted display system (200, 700) equipped with the following features.

2. The head-mounted display system (200, 700) according to claim 1, wherein the reflective element (708) includes a hot mirror or an off-axis diffractive optical element (DOE) or an off-axis holographic mirror (OAHM) or an off-axis stereodiffractive optical element (OAVDOE).

3. The head-mounted display system (200, 700) according to any one of claims 1 to 2, wherein the first segment has a different refractive power than the second segment.

4. The head-mounted display system (200, 700) according to any one of claims 1 to 3, wherein the non-zero angle is in the range of 2 to 25 degrees.

5. The head-mounted display system (200, 700) according to any one of claims 1 to 4, wherein the reflective element (708) comprises a hot mirror film, the hot mirror film being substantially transparent to visible light and substantially reflective to infrared light.

6. The head-mounted display system (200, 700) according to claim 5, wherein the hot mirror film is substantially transparent to light in a first wavelength range of 400 nm to 700 nm and substantially reflective to light in a second wavelength range of 800 nm to 900 nm.

7. The head-mounted display system (200, 700) according to any one of claims 1 to 6, wherein the optical element (706) comprises at least one infrared light source, and the at least one infrared light source is disposed on or at least partially within the optical element (706).

8. The head-mounted display system (200, 700) according to any one of claims 1 to 7, wherein the optical element (706) comprises polyethylene terephthalate (PET).

9. The head-mounted display system (200, 700) further comprises a non-transient memory configured to store the plurality of images of the user's (204) eyes (302, 304) generated by the forward-facing imaging device (702b), Determining the lateral aspect of the eye is Tracking the eyes of the aforementioned user, To extract biometric information associated with the user's eye, Reconstructing the shape of a part of the user's eye, Estimating the user's eye's near and far accommodation state, or To form an image of the retina, iris, or other elements of the user's eye. A head-mounted display system (200, 700) according to any one of claims 1 to 8, comprising performing one or more of the following.

10. The head-mounted display system (200, 700) according to claim 9, wherein the hardware processor (224, 228) is programmed to estimate the orientation of the eye by utilizing the shape of a part of the user's eye.

11. The head-mounted display system (200, 700) according to any one of claims 1 to 10, wherein the optical element (706) is positioned in front of the user's first eye.

12. The head-mounted display system (200, 700) according to any one of claims 1 to 11, wherein the frame (212) supports a second reflective element having a plurality of reflective segments, and the second optical element is positioned in front of the user's second eye.

13. The head-mounted display system (200, 700) according to any one of claims 1 to 12, wherein at least one of the first segment (708a) or the second segment (708b) is configured to generate individual virtual cameras that image the eye at infinity.

14. The head-mounted display system (200, 700) according to claim 13, wherein the first segment (708a) is oriented at a non-zero angle with respect to the second segment (708b), so that the first segment optimally reflects infrared light to image the eye when the eye is looking inward toward the nose.

15. The head-mounted display system (200, 700) according to any one of claims 13 to 14, wherein the display system selects a segment from among the plurality of segments that has less occlusion by the user's eyelashes or eyelids relative to the other segments in order to form an image of the eye.

16. The head-mounted display system (200, 700) according to any one of claims 1 to 15, wherein determining the side of the eye is based on a selected image from a first plurality of images and a second plurality of images, the selected image exhibiting the least occlusion of the eye by the eyelid or eyelashes among the analyzed images.

17. The reflective element is A first molded component comprising a first polymer material, the first molded component comprising a first component surface and a second component surface facing the first component surface, wherein the first component surface is a segmented surface including a first segment and a second segment, and the second component surface is flat and parallel to the first segment of the first component surface, A hot mirror film disposed on the first segment and the second segment of the segmented surface of the first molded component, A second molded component comprising a second polymer material, wherein the second molded component is positioned such that the hot mirror film is located between the first molded component and the second molded component. A head-mounted display system (200, 700) according to any one of claims 1 to 16, comprising the above.

18. The head-mounted display system (200, 700) according to claim 17, wherein the hot mirror film is substantially transparent to visible light and substantially reflective to infrared light, and the first polymer material and the second polymer material are substantially transparent to visible light and infrared light.

Images