Perspective computer display system

Abstract

The present invention relates to a see-through computer display system. A head-mounted display having an improved high-transmittance see-through view of the surrounding environment, the see-through view having an overlaid high-contrast display image, the head-mounted display comprising: upper optics having a first optical axis and non-polarizing lower optics having a second optical axis, the upper optics comprising: an emissive image source providing image light comprising one or more narrow spectral bands of light; one or more lenses; and a stray light trap, and the non-polarizing lower optics comprising a planar beam splitter and a curved partial reflector angled relative to the first and second optical axes, wherein one or more of the reflective surfaces is shaped to reflect a majority of incident light within the one or more narrow spectral bands and transmit a majority of incident visible light from the surrounding environment.

Description

[0001] This application is a divisional application. The name of the invention of the parent application is “Perspective Computer Display System”, the application date is February 16, 2016, and the application number is 201680002425.3.

[0002] Priority Declaration

[0003] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 884,567 (ODGP-3017-U01), filed on October 15, 2015.

[0004] This application claims the benefit of priority to the following U.S. patent application: U.S. Patent Application No. 14 / 635,390, filed March 2, 2015 (ODGP-2014-U01), which is incorporated herein by reference in its entirety.

[0005] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 670,677, filed on March 27, 2015, entitled “See-Through Computer Display Systems” (ODGP-2015-U01).

[0006] This application claims the benefit of priority to U.S. application No. 14 / 741,943, filed on June 17, 2015 (ODGP-2016-U01).

[0007] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 813,969, filed on July 30, 2015, entitled “SEE-THROUGH COMPUTER DISPLAY SYSTEMS” (ODGP-2017-U01).

[0008] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 851,755, filed on September 11, 2015, entitled “SEE-THROUGH COMPUTER DISPLAY SYSTEMS” (ODGP-2018-U01).

[0009] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 861,496, filed on September 22, 2015, entitled “SEE-THROUGH COMPUTER DISPLAY SYSTEMS” (ODGP-2019-U01).

[0010] This application claims the benefit of priority to U.S. non-provisional application No. 14 / 623,932, filed February 17, 2015 (ODGP-3016-U01).

[0011] All of the above applications are incorporated herein by reference in their entirety. Technical Field

[0012] The present invention relates to see-through computer display systems. Background Art

[0013] Head-mounted displays (HMDs), and in particular HMDs that provide a see-through view of an environment, are valuable devices. Presenting content in a see-through display can be a complex operation while ensuring an optimized user experience. Improved systems and methods for presenting content in a see-through display are needed to improve the user experience. Summary of the Invention

[0014] Aspects of the present invention relate to methods and systems for a see-through computer display system having transition capabilities from augmented reality (i.e., high see-through transmission through a display) to virtual reality (i.e., low see-through or no see-through transmission through a display).

[0015] In one aspect, a head-mounted display may include a display panel and a processor, the display panel being sized and positioned to produce a field of view for presenting digital content to a user's eyes, the processor being adapted to present the digital content to the display panel such that the digital content is presented only in a portion of the field of view, the portion being centered within the field of view such that horizontally opposite edges of the field of view are blank areas. The processor may be further adapted to shift the digital content into one of the blank areas to adjust the convergence distance of the digital content and thereby change the perceived distance from the user to the digital content. The digital content may include augmented reality objects. The perceived distance may be within arm's reach of the user. The convergence distance may be adjusted corresponding to the type of digital content being displayed or the use case associated with the augmented reality object. The convergence may be measured by an eye imaging system of the head-mounted display. The eye imaging system images the frontal perspective of the user's eyes.

[0016] In one aspect, a head-mounted display may include a display panel and a processor, the display panel being sized and positioned to produce a field of view for presenting digital content to a user's eyes, the processor being adapted to present the digital content to the display panel such that the digital content is presented only in a portion of the field of view, the portion being in the middle of the field of view such that horizontally opposite edges of the field of view are blank areas. The processor may be further adapted to shift the digital content into one of the blank areas to adjust the position of the digital content based on a focus distance of the digital content.

[0017] In one aspect, a head-mounted display may include a display panel and a processor, the display panel being sized and positioned to produce a field of view for presenting digital content to a user's eyes, the processor being adapted to present the digital content to the display panel such that the digital content is presented only in a portion of the field of view, the portion being in the middle of the field of view such that horizontally opposite edges of the field of view are blank areas. The processor may be further adapted to shift the digital content into one of the blank areas to adjust the position of the digital content based on an indication that the user is looking toward an edge of the digital content. The indication that the user is looking toward an edge of the digital content may be based on an eye image captured by a camera in the head-mounted display. The indication that the user is looking toward an edge of the digital content may be based on an indication that the user turned their head quickly followed by an indication that the user turned their eyes.

[0018] In one aspect, a head-mounted display may include a display panel and a processor, the display panel being sized and positioned to produce a field of view for presenting digital content to a user's eyes, the processor being adapted to present the digital content to the display panel such that the digital content is presented only in a portion of the field of view, the portion being centered such that horizontally opposite edges of the field of view are blank areas, wherein each blank area comprises approximately 10% or more of the lateral area of ​​the field of view. The processor may be further adapted to shift the digital content into one of the blank areas to adjust the position of the digital content. The total amount of blank area in the field of view, including the combined left and right portions of the field of view, remains constant while the left and right portions are varied to position the digital content within the field of view. The digital content may be positioned to adjust a convergence distance associated with the digital content. The digital content may be positioned to adjust an interpupillary distance associated with the digital content.

[0019] These and other systems, methods, objects, features and advantages of the present invention will be apparent to those skilled in the art from the following detailed description of preferred embodiments and the accompanying drawings.All documents mentioned herein are hereby incorporated by reference in their entirety. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The embodiments are described with reference to the following figures. The same numerals are used throughout to refer to similar features and components shown in the figures.

[0021] Figure 1 A head-mounted computing system in accordance with the principles of the present invention is illustrated.

[0022] Figure 2 A head-mounted computing system with an optical system in accordance with the principles of the present invention is illustrated.

[0023] Figure 3aA large prior art optical arrangement is shown.

[0024] Figure 3b An upper optical module in accordance with the principles of the present invention is shown.

[0025] Figure 4 An upper optical module in accordance with the principles of the present invention is shown.

[0026] Figure 4a An upper optical module in accordance with the principles of the present invention is shown.

[0027] Figure 4b An upper optical module in accordance with the principles of the present invention is shown.

[0028] Figure 5 An upper optical module in accordance with the principles of the present invention is shown.

[0029] Figure 5a An upper optical module in accordance with the principles of the present invention is shown.

[0030] Figure 5b Illustrated is an upper optical module and dark light trap in accordance with the principles of the present invention.

[0031] Figure 5c Illustrated is an upper optical module and dark light trap in accordance with the principles of the present invention.

[0032] Figure 5d Illustrated is an upper optical module and dark light trap in accordance with the principles of the present invention.

[0033] Figure 5e Illustrated is an upper optical module and dark light trap in accordance with the principles of the present invention.

[0034] Figure 6 Illustrated are upper and lower optical modules in accordance with the principles of the present invention.

[0035] Figure 7 The angles of the combiner elements are illustrated in accordance with the principles of the present invention.

[0036] Figure 8 Illustrated are upper and lower optical modules in accordance with the principles of the present invention.

[0037] Figure 8a Illustrated are upper and lower optical modules in accordance with the principles of the present invention.

[0038] Figure 8b Illustrated are upper and lower optical modules in accordance with the principles of the present invention.

[0039] Figure 8c Illustrated are upper and lower optical modules in accordance with the principles of the present invention.

[0040] Figure 9An eye imaging system in accordance with the principles of the present invention is illustrated.

[0041] Figure 10 A light source according to the principles of the present invention is shown.

[0042] Figure 10a A backlighting system according to the principles of the present invention is shown.

[0043] Figure 10b A backlighting system in accordance with the principles of the present invention is illustrated.

[0044] Figures 11a to 11d A light source and a filter according to the principles of the present invention are shown.

[0045] Figures 12a to 12c A light source and quantum dot system according to the principles of the present invention is illustrated.

[0046] Figures 13a to 13c A peripheral lighting system according to the principles of the present invention is shown.

[0047] Figures 14a to 14h A light suppression system according to the principles of the present invention is illustrated.

[0048] Figure 15 Illustrated is an external user interface according to the principles of the present invention.

[0049] Figures 16a to 16c A distance control system according to the principles of the present invention is shown.

[0050] Figures 17a to 17c A force interpretation system according to the principles of the present invention is illustrated.

[0051] Figures 18a to 18c A user interface mode selection system in accordance with the principles of the present invention is illustrated.

[0052] Figure 19 An interactive system according to the principles of the present invention is illustrated.

[0053] Figure 20 Illustrated is an external user interface according to the principles of the present invention.

[0054] Figure 21 FIGURE 2 illustrates a representation of mD trajectories presented in accordance with the principles of the present invention.

[0055] Figure 22 FIGURE 2 illustrates a representation of mD trajectories presented in accordance with the principles of the present invention.

[0056] Figure 23 Illustrated is an environment for mD scanning according to the principles of the present invention.

[0057] Figure 23aFIGURE 2 illustrates a representation of mD trajectories presented in accordance with the principles of the present invention.

[0058] Figure 24 A stray light suppression technique according to the principles of the present invention is illustrated.

[0059] Figure 25 A stray light suppression technique according to the principles of the present invention is illustrated.

[0060] Figure 26 A stray light suppression technique according to the principles of the present invention is illustrated.

[0061] Figure 27 A stray light suppression technique according to the principles of the present invention is illustrated.

[0062] Figures 28a to 28c Illustration of DLP mirror angle.

[0063] Figures 29 to 33 An eye imaging system in accordance with the principles of the present invention is illustrated.

[0064] Figure 34 and 34a A structured eye illumination system according to the principles of the present invention is illustrated.

[0065] Figure 35 Illustrate eye glint in the prediction of eye direction analysis according to the principles of the present invention.

[0066] Figure 36a Illustrated are eye characteristics that can be used for individual identification through systematic analysis in accordance with the principles of the present invention.

[0067] Figure 36b The illustration shows reflections of digital content exiting a wearer's eye that can be analyzed according to the principles of the present invention.

[0068] Figure 37 Illustrated is imaging of the eye along various virtual target lines and various focal planes in accordance with the principles of the present invention.

[0069] Figure 38 Illustrated is content control with respect to eye movement based on eye imaging according to the principles of the present invention.

[0070] Figure 39 Illustrated is an illustration of eye imaging and eye convergence in accordance with the principles of the present invention.

[0071] Figure 40 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0072] Figure 41 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0073] Figure 42 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0074] Figure 43 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0075] Figure 44 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0076] Figure 45 Graphs the various headings over time in the example.

[0077] Figure 46 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0078] Figure 47 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0079] Figure 48 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0080] Figure 49 Illustrated is the position of content depending on sensor feedback in accordance with the principles of the present invention.

[0081] Figure 50 Illustration of light impinging the eye in accordance with the principles of the present invention.

[0082] Figure 51 Illustrated is a view of an eye in accordance with the principles of the present invention.

[0083] Figure 52a and 52b Illustrated is a view of an eye with a structured light pattern in accordance with the principles of the present invention.

[0084] Figure 53 An optics module according to the principles of the present invention is illustrated.

[0085] Figure 54 An optics module according to the principles of the present invention is illustrated.

[0086] Figure 55 Shown are a series of example spectra measured for various controlled substances using a form of infrared spectroscopy.

[0087] Figure 56 The infrared absorption spectrum for glucose is shown.

[0088] Figure 56a 、 56b , 56c and 56d depict examples of eye blinks.

[0089] Figure 56e Graph depicting the measured anterior and posterior spherical radii of the human eye.

[0090] Figure 57 The photo shows a person walking with the HWC mounted on his head.

[0091] Figure 58 Illustrated is a system for receiving, developing and using movement heading, view heading, eye heading and / or persistence information from(s) HWCs.

[0092] Figure 59 A rendering technique according to the principles of the present invention is illustrated.

[0093] Figure 60 A rendering technique according to the principles of the present invention is illustrated.

[0094] Figure 61 A rendering technique according to the principles of the present invention is illustrated.

[0095] Figure 62 A rendering technique according to the principles of the present invention is illustrated.

[0096] Figure 63 A rendering technique according to the principles of the present invention is illustrated.

[0097] Figure 64 A rendering technique according to the principles of the present invention is illustrated.

[0098] Figure 65 A rendering technique according to the principles of the present invention is illustrated.

[0099] Figure 66 A rendering technique according to the principles of the present invention is illustrated.

[0100] Figure 67 An optical configuration according to the principles of the present invention is shown.

[0101] Figure 68 An optical configuration according to the principles of the present invention is shown.

[0102] Figure 69 An optical configuration according to the principles of the present invention is shown.

[0103] Figure 70 An optical configuration according to the principles of the present invention is shown.

[0104] Figure 71 An optical configuration according to the principles of the present invention is shown.

[0105] Figure 72 An optical element according to the principles of the present invention is shown.

[0106] Figure 73 An optical element according to the principles of the present invention is shown.

[0107] Figure 74 An optical element according to the principles of the present invention is shown.

[0108] Figure 75 An optical element according to the principles of the present invention is shown.

[0109] Figure 76 Illustrated is an illustration of optical elements in a see-through computer display in accordance with the principles of the present invention.

[0110] Figure 77 An optical element according to the principles of the present invention is shown.

[0111] Figure 78 An optical element according to the principles of the present invention is shown.

[0112] Figure 79a Illustrated is a schematic diagram of an upper optical device according to the principles of the present invention.

[0113] Figure 79 Illustrated is a schematic diagram of an upper optical device according to the principles of the present invention.

[0114] Figure 80 A stray light control technique according to the principles of the present invention is illustrated.

[0115] Figure 81a and 81b A display with gap and masking techniques according to the principles of the present invention is illustrated.

[0116] Figure 82 Illustrated is an upper module with a trimmed polarizer in accordance with the principles of the present invention.

[0117] Figure 83 An optical system with stacked multiple polarizer films in accordance with the principles of the present invention is illustrated.

[0118] Figure 84a and 84b A partially reflective layer in accordance with the principles of the present invention is shown.

[0119] Figure 84c A multiple polarizer with complex curved stacks according to the principles of the present invention is shown.

[0120] Figure 84d A multiple polarizer with curved stacks in accordance with the principles of the present invention is shown.

[0121] Figure 85 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0122] Figure 86 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0123] Figure 87 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0124] Figure 88 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0125] Figure 89 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0126] Figure 90 An optical system adapted for a head mounted display according to the principles of the present invention is shown.

[0127] Figure 91 An optical system according to the principles of the present invention is shown.

[0128] Figure 92 An optical system according to the principles of the present invention is shown.

[0129] Figure 93 An optical system according to the principles of the present invention is shown.

[0130] Figure 94 An optical system according to the principles of the present invention is shown.

[0131] Figure 95 An optical system according to the principles of the present invention is shown.

[0132] Figure 96 An optical system according to the principles of the present invention is shown.

[0133] Figure 97 An optical system according to the principles of the present invention is shown.

[0134] Figure 98 An optical system according to the principles of the present invention is shown.

[0135] Figure 99 An optical system according to the principles of the present invention is shown.

[0136] Figure 100 An optical system according to the principles of the present invention is shown.

[0137] Figure 101 An optical system according to the principles of the present invention is shown.

[0138] Figure 102 An optical system according to the principles of the present invention is shown.

[0139] Figure 103 、 103a and 103b illustrate an optical system according to the principles of the present invention.

[0140] Figure 104 An optical system according to the principles of the present invention is shown.

[0141] Figure 105 A blocking optic in accordance with the principles of the present invention is illustrated.

[0142] Figure 106a 、 106b and 106c illustrate a blocking optics system in accordance with the principles of the present invention.

[0143] Figure 107 A full color image in accordance with the principles of the present invention is shown.

[0144] 108A and 108B illustrate color breakup management according to the principles of the present invention.

[0145] Figure 109 A time-scale sequence according to the principles of the present invention is illustrated.

[0146] Figure 110 A time-scale sequence according to the principles of the present invention is illustrated.

[0147] Figure 111a and 111b Illustrate images displayed sequentially according to the principles of the present invention.

[0148] Figure 112 A see-through display with a rotating component in accordance with the principles of the present invention is illustrated.

[0149] Figure 113 An optics module having a twisted reflective surface according to the principles of the present invention is shown.

[0150] Figure 114 Illustrated is the PCB and see-through optics module location within a glasses form factor in accordance with the principles of the present invention.

[0151] Figure 115 Illustrated is the PCB and see-through optics module location within a glasses form factor in accordance with the principles of the present invention.

[0152] Figure 116 Illustrated is the PCB and see-through optics module location within a glasses form factor in accordance with the principles of the present invention.

[0153] Figure 117 Illustrated is a user interface in accordance with the principles of the present invention.

[0154] Figure 118 Illustrated is a user interface in accordance with the principles of the present invention.

[0155] Figure 119 A lens arrangement according to the principles of the present invention is shown.

[0156] Figure 120 and 121 An eye imaging system in accordance with the principles of the present invention is illustrated.

[0157] Figure 122 The identification process according to the principles of the present invention is illustrated.

[0158] Figure 123 and 124 A combiner assembly according to the principles of the present invention is illustrated.

[0159] Figure 125 Graph showing the sensitivity of the human eye to brightness.

[0160] Figure 126 is a graph showing lightness (L*) as perceived by the human eye versus measured brightness (illuminance) of hue.

[0161] Figure 127 is an illustration of a see-through view of the surrounding environment with an outline illustrating that the displayed field of view is smaller than a typical see-through field of view.

[0162] Figure 128 is an illustration of a captured image of the surrounding environment, which may be of a much larger field of view than the displayed image, so that a cropped version of the captured image of the environment can be used for the alignment process.

[0163] Figure 129a and 129b Illustrated are first and second target images with invisible markings.

[0164] Figure 130 and 131 Illustrated is a target overlaid onto a perspective view, wherein the target is moved using eye tracking controls, in accordance with the principles of the present invention.

[0165] Figure 132 A diagram showing multiply folded optics including solid prisms for a head-mounted display in accordance with the principles of the present invention.

[0166] Figure 133a 、 133b and 133c show diagrams of steps associated with bonding a reflective plate to a solid prism in accordance with the principles of the present invention.

[0167] Figure 134 A diagram showing multiply folded optics for reflecting an image source in accordance with the principles of the present invention, wherein a backlight assembly is positioned behind a reflective plate.

[0168] Figure 135 A diagram showing a prismatic film bonded to a reflective plate in accordance with the principles of the present invention.

[0169] Figure 135a A diagram illustrating a multiply folded optical device in accordance with the principles of the present invention showing two cones of illumination light provided by a prismatic film.

[0170] Figure 136 、 137 and 138 show diagrams of different embodiments of additional optical elements included in a solid prism for imaging a user's eye in accordance with the principles of the present invention.

[0171] Figure 139 A diagram showing an eye imaging system for multiply folded optics according to the principles of the present invention, where the image source is a self-luminous display.

[0172] Figure 140a and 140b is a diagrammatic representation of an eye imaging system in accordance with the principles of the present invention.

[0173] Figure 141a and 141b is an illustration of a folded optical device including a waveguide having an angled partially reflective surface and a powered reflective surface in accordance with the principles of the present invention.

[0174] Figure 142a and 142b is an illustration of folded optics for a head-mounted display including a waveguide having at least one holographic optical element and an image source in accordance with the principles of the present invention.

[0175] Figure 143 is an illustration of folded optics for a head-mounted display in which illumination light is injected into a waveguide and redirected by a holographic optical element so that the user's eyes are illuminated in accordance with the principles of the present invention.

[0176] Figure 144 A diagram showing folded optics for a head-mounted display in which a series of angled partially reflecting mirrors are included in a waveguide in accordance with the principles of the present invention.

[0177] Figure 145 A diagram showing a beam splitter-based optical module for a head-mounted display in accordance with the principles of the present invention.

[0178] Figure 146 A diagram showing an optical module for a head mounted display in accordance with the principles of the present invention.

[0179] Figure 146aDiagram showing a side view of an optics module including a corrective lens element.

[0180] Figure 147 Diagram showing left and right optics modules connected together in a rack in accordance with the principles of the present invention.

[0181] Figure 148 Shown are left and right images provided at nominal vergence distances within left and right display fields of view in accordance with the principles of the present invention.

[0182] Figure 149 It shows how left and right images can be shifted laterally toward each other within left and right display fields of view in accordance with the principles of the present invention.

[0183] Figure 150a and 150b A mechanism for moving an image source according to the principles of the present invention is shown.

[0184] Figure 151a and 151b Diagram showing upper and lower wedges from the position of an image source in accordance with the principles of the present invention.

[0185] Figure 152 Diagram showing a spring clip applying force to an image source in accordance with the principles of the present invention.

[0186] Figure 153a 、 153b and 154 show diagrams of example display optics including eye imaging according to the principles of the present invention.

[0187] Figure 155a 、 155b , 156a, 156b, 157a, 157b, 158a, 158b, 159a and 159b show diagrams of focus adjustment modules according to the principles of the present invention.

[0188] Figure 160 A diagram showing an example of a multiply folded optic viewed from the eye position in accordance with the principles of the present invention.

[0189] Figure 161 and 162 An optical system according to the principles of the present invention is shown.

[0190] Figure 163A illustrates a sudden change in the appearance of content in the field of view of a see-through display.

[0191] FIG163B illustrates a managed appearance system in which content is visually scaled back as it enters a transition zone near the edge of the field of view.

[0192] Figure 164 A hybrid field of view is illustrated that includes a centered field of view and an extended field of view positioned at or near an edge of the centered field of view or overlapping an edge of the centered field of view.

[0193] Figure 165 Illustrated is a hybrid display system where the main centered field of view is created using optics in the upper module and the extended field of view is created using a display system mounted above the combiner.

[0194] 166A-166D illustrate examples of extended display, or extended image content optical configurations.

[0195] Figure 167 Another optical system is shown that uses a hybrid optical system including a main display optical system and an extended field of view optical system.

[0196] 168A-168E illustrate various embodiments in which a see-through display panel is positioned directly in front of a user's eyes in a head-worn computer to provide an extended and / or overlapping field of view in a hybrid display system.

[0197] Figure 169 A cross-sectional illustration of an example optics assembly for a head-mounted display is shown in accordance with the principles of the present invention.

[0198] Figure 170 Diagram showing a light trap operating in accordance with the principles of the present invention to reduce stray light.

[0199] Figure 171 A diagram showing a simple optical system providing a 60 degree display field of view in accordance with the principles of the present invention.

[0200] Figure 172 Graph showing the acuity of a typical human eye versus angular position in the field of view.

[0201] Figure 173 A graph showing acuity versus eccentricity of a typical human eye in simplified form, highlighting the decrease in acuity with eccentricity and the difference between achromatic and chromatic acuity.

[0202] Figures 174A and 174B show typical graphs of eye movement and head movement given in radians versus time.

[0203] Figure 175 is a graph showing the effective relative colorless acuity provided by a typical human eye within the eye's field of view when including movement of the eye, compared to the acuity of the fovea.

[0204] Figure 176is a graph showing the minimum design MTF versus angular field position required to provide a uniformly sharp viewing image in a wide field of view display image.

[0205] Figure 177 is a graph showing the relative MTF that needs to be provided by the display optics for a wide field of view display to provide a sharpness that matches the acuity of the human eye in the peripheral regions of the display field of view.

[0206] Figure 178 Shown with Figure 171 Figure 1 shows a modeled MTF curve associated with an optical system of FIG. 1 , where the MTF curves are shown for various angular positions within the display field of view.

[0207] Figure 179 is an illustration of a resolution chart where the sharpness of the image has been reduced by blurring the peripheral portions of the image to simulate an image from optics that provides a less sharp peripheral region for a central sharp region of + / - 15 degrees.

[0208] Figure 180 and 181 is a diagram showing how an image is shifted within a display field of view as a user moves their head in accordance with the principles of the present invention.

[0209] Figure 182 Illustrated is a blank portion of a display field of view in which the area from which the image has been displaced is displayed as a dark area to enable the user to see through to the surrounding environment in the blank portion in accordance with the principles of the present invention.

[0210] Figure 183 A diagram showing a wide display field of view, wherein a user may select to display a smaller field of view for a given image or application (eg, a game) to improve a personal viewing experience, in accordance with the principles of the present invention.

[0211] Figure 184 and 185 Shown is the physical arrangement of an optical system according to the principles of the present invention.

[0212] Figure 186 The field of view of a 30:9 format and the field of view of a 22:9 format are shown, wherein the two fields of view have the same vertical field of view and different horizontal fields of view in accordance with the principles of the present invention.

[0213] Figure 187 Depicts the user's eyes looking through the display field of view.

[0214] Figure 188 Depicted are lateral image shifts within the displayed field of view.

[0215] Figure 189Depicted are illustrations of left and right display images as they would be presented within a display field of view.

[0216] Figure 190 Depicted are illustrations of left and right display images as they would be presented within a display field of view.

[0217] Figure 191 Depicted is a diagram showing a user's eyes looking through a display field of view.

[0218] Figure 192 Depicted are illustrations of left and right display images as they would be presented within a display field of view.

[0219] While the invention has been described in connection with certain preferred embodiments, other embodiments will be apparent to those of ordinary skill in the art and are contemplated herein. DETAILED DESCRIPTION

[0220] One aspect of the present invention relates to a head-mounted computing ("HWC") system. In some instances, HWCs involve systems that mimic the appearance of head-mounted glasses or sunglasses. The glasses can be a fully developed computing platform, such as including a computer display that is presented to the user's eyes in each of the lenses of the glasses. In embodiments, the lenses and displays can be configured to allow a person wearing the glasses to see the environment through the lenses while also seeing a digital image, which forms an overlaid image perceived by the person as a digitally enhanced image of the environment or augmented reality ("AR").

[0221] HWC involves more than just placing a computing system on a person's head. The system may need to be designed as a lightweight, compact, and fully functional computer display, such as one that includes a high-resolution digital display that provides a high level of emersion, consisting of displayed digital content and a see-through view of the surrounding environment. Unlike those used in more conventional computers (such as laptops), user interfaces and control systems adapted for HWC devices may be required. To make HWC and associated systems most effective, the glasses may be equipped with sensors to determine environmental conditions, geographic location, relative positioning to other points of interest, objects identified through imaging and movement by the user or other users in a connected group, and so on. In an approach generally referred to as context-aware HWC, the HWC can then change its mode of operation to match the conditions, location, positioning, movement, and so on. The glasses may also need to be connected to other systems, either locally or via a network, wirelessly or otherwise. Control of the glasses can be achieved through the use of external devices, automatically through contextually collected information, through user gestures captured by the glasses' sensors, and so on. Each technology can be further refined based on the software application being used in the glasses. The glasses may further be used to control or coordinate with external devices associated with the glasses.

[0222] refer to Figure 1, presents an overview of an HWC system 100. As shown, the HWC system 100 includes an HWC 102, which in this example is configured as glasses to be worn on the head with sensors that enable the HWC 102 to be aware of objects and conditions in an environment 114. In this example, the HWC 102 also receives and interprets control inputs, such as gestures and movements 116. The HWC 102 can communicate with an external user interface 104. The external user interface 104 can provide a physical user interface for obtaining control instructions from a user of the HWC 102, and the external user interface 104 and HWC 102 can communicate bidirectionally to implement the user's commands and provide feedback to an external device 108. The HWC 102 can also communicate bidirectionally with an externally controlled or coordinated local device 108. For example, the external user interface 104 can be used in conjunction with the HWC 102 to control the externally controlled or coordinated local device 108. The externally controlled or coordinated local device 108 may provide feedback to the HWC 102, and a customized GUI may be presented in the HWC 102 based on the specific identified device 108 or type of device. The HWC 102 may also interact with remote devices and information sources 112 via the network connection 110. Moreover, the external user interface 104 may be used in conjunction with the HWC 102 to control or otherwise interact with any of the remote devices 108 and information sources 112 in a manner similar to when the external user interface 104 is used to control or otherwise interact with an externally controlled or coordinated local device 108. Similarly, HWC 102 may interpret gestures 116 (e.g., captured from forward-facing sensors, downward-facing sensors, upward-facing sensors, rearward-facing sensors (such as camera(s), rangefinders, IR sensors, etc.)) or environmental conditions sensed in the environment 114 to control a local or remote device 108 or 112.

[0223] We will now describe in more detail Figure 1 Each of the main elements described above; however, these descriptions are intended to provide general guidance and should not be interpreted as limiting. Additional descriptions of each element may also be further described in this article.

[0224] HWC 102 is a computing platform designed to be worn on a person's head. HWC 102 can take many different forms to suit many different functional requirements. In some cases, HWC 102 will be designed in the form of conventional glasses. The glasses may or may not have an active computer graphics display. In cases where HWC 102 already has an integrated computer display, the display can be configured as a see-through display to enable digital images to be overlaid on the user's view of the environment 114. There are many see-through optical designs that can be used, including those with reflective displays (e.g., LCoS, DLP), emissive displays (e.g., OLED, LED), holograms, TIR waveguides, and the like. In embodiments, the illumination system used in conjunction with the display optics can be a solid-state illumination system, such as LEDs, OLEDs, quantum dots, quantum dot LEDs, and the like. Furthermore, the optical configuration can be single- or dual-purpose. It can also include vision correction optics. In embodiments, the optics can be packaged as contact lenses. In other embodiments, the HWC 102 may be in the form of a helmet with a see-through shield, sunglasses, safety glasses, goggles, a mask, a fire helmet with a see-through shield, a police helmet with a see-through shield, a military helmet with a see-through shield, a practical form customized for a certain work task (e.g., inventory control, logistics, repair, maintenance, etc.), etc.

[0225] The HWC 102 may also have a number of integrated computing facilities, such as an integrated processor, integrated power management, communication structures (e.g., cell net, WiFi, Bluetooth, local area connectivity, mesh connectivity, remote connectivity (e.g., client server, etc.)), etc. The HWC 102 may also have a number of location-aware sensors, such as GPS, an electronic compass, an altimeter, a tilt sensor, an IMU, etc. It may also have other sensors, such as a camera, a rangefinder, a hyperspectral camera, a Geiger counter, a microphone, a spectral illumination detector, a temperature sensor, a chemical sensor, a biosensor, a humidity sensor, an ultrasonic sensor, etc.

[0226] The HWC 102 may also have integrated control technology. This integrated control technology can be context-based, passive, active, user-controlled, and so on. For example, the HWC 102 may have integrated sensors (e.g., cameras) that capture user hand or body gestures 116, enabling an integrated processing system to interpret these gestures and generate control commands for the HWC 102. In another example, the HWC 102 may have sensors that detect movement (e.g., nodding, shaking the head, etc.), including accelerometers, gyroscopes, and other inertial measurements, where an integrated processor can interpret the movement and generate control commands in response. The HWC 102 may also automatically control itself based on measured or sensed environmental conditions. For example, if the environment is bright, the HWC 102 may increase the brightness or contrast of the displayed image. In embodiments, the integrated control technology may be mounted on the HWC 102 to enable the user to interact directly with it. For example, the HWC 102 may have button(s), a touch-capacitive interface, and so on.

[0227] As described herein, the HWC 102 can communicate with an external user interface 104. The external user interface can take many different forms. For example, a cell phone screen can be suitable for obtaining user input for controlling aspects of the HWC 102. The external user interface can be a dedicated UI such as a keyboard, a touch surface, (a plurality of) buttons, a joystick, etc. In embodiments, the external controller can be integrated into another device (such as a ring, a watch, a bicycle, a car, etc.). In each case, the external user interface 104 can include sensors (e.g., an IMU, an accelerometer, a compass, an altimeter, etc.) to provide additional input for controlling the HWD 104.

[0228] As described herein, the HWC 102 can control or coordinate with other local devices 108. The external devices 108 can be audio devices, visual devices, vehicles, cell phones, computers, etc. For example, the local external device 108 can be another HWC 102, where information can then be exchanged between the separate HWCs 108.

[0229] Similar to how the HWC 102 can control or coordinate with a local device 106, the HWC 102 can control or coordinate with a remote device 112, such as by the HWC 102 communicating with the remote device 112 via the network 110. Furthermore, the remote device 112 can take many forms. Among these forms is another HWC 102. For example, each HWC 102 can transmit its GPS location so that all HWCs 102 know where they are located.

[0230] Figure 2 A HWC 102 is illustrated having an optical system comprising an upper optical module 202 and a lower optical module 204. While the upper optical module 202 and the lower optical module 204 will generally be described as separate modules, it should be understood that this is merely illustrative and that the invention encompasses other physical configurations, such as where the two modules are combined into a single module or where the elements comprising the two modules are configured into more than two modules. In an embodiment, the upper module 202 comprises a computer controlled display (e.g., LCoS, DLP, OLED, etc.) and image light delivery optics. In an embodiment, the lower module comprises eye delivery optics configured to receive image light from the upper module and deliver the image light to the eye of the wearer of the HWC. Figure 2 , it should be noted that while the upper optical module 202 and the lower optical module 204 are illustrated in one side of the HWC to enable image light to be delivered to one eye of the wearer, it is envisioned by the present invention that embodiments will incorporate two image light delivery systems, one for each eye.

[0231] Figure 3b The upper optical module 202 according to the principles of the present invention is illustrated. In this embodiment, the upper optical module 202 includes a DLP (also known as a DMD or digital micromirror device) computer-operated display 304 (which includes pixels composed of rotatable mirrors, such as, for example, the DLP3000 available from Texas Instruments), a polarized light source 302, a ¼-wave retarder film 308, a reflective polarizer 310, and a field lens 312. The polarized light source 302 provides substantially uniformly polarized light that is directed generally toward the reflective polarizer 310. The reflective polarizer reflects light of one polarization state (e.g., S-polarized light) and transmits light of another polarization state (e.g., P-polarized light). The polarized light source 302 and the reflective polarizer 310 are oriented such that the polarized light from the polarized light source 302 is generally reflected toward the DLP 304. The light then passes through the ¼-wave film 308 once before illuminating the pixels of the DLP 304 and then again after being reflected by the pixels of the DLP 304. During the two passes through the ¼ wave film 308, the light is converted from one polarization state to another (e.g., from S-polarized light to P-polarized light). The light then passes through the reflective polarizer 310. With the DLP pixel(s) in the "on" state (i.e., the mirrors are positioned to reflect light toward the field lens 312), the "on" pixels generally reflect light along the optical axis and into the field lens 312. This light, reflected by the "on" pixels and generally directed along the optical axis of the field lens 312, will be referred to as image light 316. The image light 316 then passes through the field lens to be used by the lower optical module 204.

[0232] The light provided by the polarized light source 302 (which is subsequently reflected by the reflective polarizer 310 before it reflects from the DLP 304) will generally be referred to as illumination light. The light reflected by the "off" pixels of the DLP 304 is reflected at a different angle than the light reflected by the "on" pixels, so that the light from the "off" pixels is generally directed away from the optical axis of the field lens 312 and toward the side of the upper optical module 202 as shown in FIG3. The light reflected by the "off" pixels of the DLP 304 will be referred to as dark-state light 314.

[0233] The DLP 304 operates as a computer-controlled display and is generally considered a MEMs device. DLP pixels consist of small mirrors that can be steered. The mirrors typically flip from one angle to another. These two angles are generally referred to as states. When light is used to illuminate the DLP, the mirrors will reflect light in directions depending on the state. In the embodiments herein, the two states are generally referred to as "on" and "off," which are intended to describe the conditions of the display pixels. "On" pixels will appear to a viewer of the display as emitting light because the light is directed along the optical axis and into the field lens and associated rest of the display system. "Off" pixels will appear to a viewer of the display as not emitting light because the light from these pixels is directed to the sides of the optical housing and into a light trap or light dump where it is absorbed. The pattern of "on" and "off" pixels generates image light, which is perceived by the viewer of the display as a computer-generated image. A full-color image is presented to the user by sequentially providing complementary colors (such as red, green, and blue) for the illumination light. wherein the sequence is presented in a recurring cycle that is faster than a user can perceive as separate images, and as a result the user perceives a full-color image composed of the sum of the sequential images. Bright pixels in the image are provided by pixels that remain in an "on" state for the entire time of the cycle, while darker pixels in the image are provided by pixels that switch between the "on" and "off" states within the time of the cycle, or, in the case of a video sequence of images, within the frame time.

[0234] Figure 3a A diagram of a system for a DLP 304 is shown in which an unpolarized light source 350 is pointed directly at the DLP 304. In this case, the angle required for the illumination light is such that the field lens 352 must be positioned significantly away from the DLP 304 to avoid clipping of the illumination light by the field lens 352. The large distance between the field lens 352 and the DLP 304, combined with the straight path of the dark-state light 354, means that the light trap for the dark-state light 354 is also located at a significant distance from the DLP. For these reasons, this configuration is larger in size than the upper optics module 202 of the preferred embodiment.

[0235] exist Figure 3b The configuration illustrated in FIG can be lightweight and compact, allowing it to fit into a small portion of a HWC. For example, the upper module 202 illustrated herein can be physically adapted to be mounted within the upper frame of a HWC, enabling image light to be directed into the lower optical module 204 for presenting digital content to the wearer's eyes. The packaging of the components that combine to generate the image light (i.e., polarized light source 302, DLP 304, reflective polarizer 310, and quarter-wave film 308) is very lightweight and compact. The height of the system, excluding the field lens, can be less than 8 mm. The width (i.e., front to back) can be less than 8 mm. The weight can be less than 2 grams. The compactness of this upper optical module 202 allows for a compact mechanical design of the HWC, and the lightweight nature of these embodiments helps make the HWC lightweight, providing a comfortable HWC for the wearer.

[0236] exist Figure 3b The configuration illustrated in FIG can produce sharp contrast, high brightness, and deep blacks, especially when compared to LCD or LCoS displays used for HWCs. The "on" and "off" states of the DLP provide a strong differentiator in the light reflection path representing "on" pixels and "off" pixels. As will be discussed in more detail below, the dark state light reflected from the "off" pixels can be managed to reduce stray light in the display system to produce an image with high contrast.

[0237] Figure 4 Another embodiment of the upper optical module 202 according to the principles of the present invention is shown. This embodiment includes a light source 404, but in this case, the light source is capable of providing unpolarized illumination light. The illumination light from the light source 404 is directed into the TIR wedge 418 so that the illumination light is incident on the inner surface of the TIR wedge 418 (shown as ) at an angle exceeding the critical angle as defined by Equation 1. Figure 4 on the angled lower surface of the TRI wedge 418 in FIG.

[0238] Critical angle = arc-sin(l / n) Equation 1.

[0239] Where the critical angle is the angle beyond which illumination light reflects from an interior surface when the interior surface comprises an interface from a solid with a higher refractive index (n) to air with a refractive index of 1 (e.g., for an acrylic interface with a refractive index of n = 1.5 to air, the critical angle is 41.8 degrees; for a polycarbonate interface with a refractive index of n = 1.59 to air, the critical angle is 38.9 degrees). Thus, a TIR wedge 418 is associated with the thin air gap 408 along the interior surface to create an interface between the solid with a higher refractive index and air. By selecting the angle of the light source 404 relative to the DLP 402 corresponding to the angle of the interior surface of the TIR wedge 418, the illumination light is turned toward the DLP 402 at an angle suitable for providing image light 414 reflected from "on" pixels. Here, illumination light is provided to the DLP 402 at an angle approximately twice that of the pixel mirrors in the DLP 402 that are in the "on" state, so that after reflecting from the pixel mirrors, image light 414 is generally directed along the optical axis of the field lens. Depending on the state of the DLP pixels, illumination light from the "on" pixels is reflected as image light 414 that is directed toward the field lens and the lower optical module 204, while illumination light reflected from the "off" pixels (generally referred to herein as "dark" state light, "off" pixel light, or "off" state light) 410 is directed in a separate direction that can be captured and not used in the image ultimately presented to the wearer's eye.

[0240] A light trap for dark-state light 410 can be positioned along an optical axis defined by the direction of dark-state light 410 and in the side of the housing, which functions to absorb the dark-state light. To this end, the light trap can consist of an area outside the cone of image light 414 from "on" pixels. The light trap is typically constructed of a light-absorbing material, including a coating of black paint or other light-absorbing material, to prevent light scattering from the dark-state light from degrading the image perceived by the user. Additionally, the light trap can be recessed into the wall of the housing or include a mask or guard to block scattered light and prevent the light trap from being viewed adjacent to the displayed image.

[0241] Figure 4 The embodiment also includes a corrective wedge 420 to correct for the refractive effects of the image light 414 as it exits the TIR wedge 418. By including the corrective wedge 420 and providing a thin air gap 408 (e.g., 25 microns), the image light from the "on" pixels is generally maintained in a direction along the optical axis of the field lens (i.e., the same direction as that defined by the image light 414), so it passes into the field lens and the lower optical module 204. As shown in Figure 4As shown in FIG, image light 414 from an "on" pixel exits corrective wedge 420 generally perpendicular to the surface of corrective wedge 420, while dark-state light exits at an oblique angle. As a result, the direction of image light 414 from an "on" pixel is largely unaffected by refraction as it exits the surface of corrective wedge 420. In contrast, dark-state light 410 is significantly altered in direction by refraction as it exits corrective wedge 420.

[0242] Figure 4 The embodiment shown in FIG has a Figure 3b The advantages are similar to those discussed in connection with the embodiments. Figure 4 The size and weight of the upper module 202 depicted in FIG may be approximately 8×8 mm, with a weight of less than 3 grams. Figure 3b The configuration shown in the figure is Figure 4 The difference in overall performance between the configurations shown in the figure is Figure 4 Embodiments of the invention do not require the use of polarized light as supplied by light source 404. This can be advantageous in some circumstances, as will be discussed in more detail below (e.g., increased see-through transparency of the HWC optics from the user's perspective). Figure 4 The embodiment depicted in uses polarized light. Figure 4 The embodiment and Figure 3b An additional advantage over the embodiment shown in FIG is that the dark-state light (shown as DLP-off light 410) is directed at a steeper angle away from the optical axis of the image light 414 due to the increased refraction encountered when the dark-state light 410 exits the corrective wedge 420. This steeper angle of the dark-state light 410 allows the light trap to be positioned closer to the DLP 402, thereby reducing the overall size of the upper module 202. Since the light trap does not interfere with the field lens, the light trap can also be made larger, thereby increasing the efficiency of the light trap and, as a result, reducing stray light and improving the contrast of the image perceived by the user. Figure 4a Illustration of the combination Figure 4 The described embodiment has an example set of corresponding angles at various surfaces where a beam of light is reflected through the upper optical module 202. In this example, the DLP mirror is provided at 17 degrees to the surface of the DLP device. The angles of the TIR wedges are selected relative to each other to provide TIR reflected illumination light at the correct angle for the DLP mirror while allowing image light and dark state light to pass through the thin air gap, and various angle combinations are possible to achieve this.

[0243] Figure 5 Another embodiment of an upper optical module 202 according to the principles of the present invention is shown. Figure 4As in the embodiment shown in Figure 5 The embodiment shown in does not require the use of polarized light. Polarized light can be used in conjunction with this embodiment, but it is not required. Figure 5 The optical module 202 is depicted in conjunction with Figure 4 The optical modules presented are similar; however, Figure 5 504. As can be seen from the illustration, the off-light redirecting wedge 502 allows the image light 414 to continue generally along the optical axis toward the field lens and into the lower optical module 204 (as shown). However, the off-light 504 is redirected substantially toward the side of the corrective wedge 420 where it passes into the light trap. This configuration can allow for further high compactness in the HWC because the light trap (not shown) intended to absorb the off-light 504 can be positioned laterally adjacent to the upper optical module 202, as opposed to below it. Figure 5 In the embodiment depicted in FIG, there is a thin air gap (with Figure 4 There is also a thin air gap between the corrective wedge 420 and the off light redirecting wedge 502. There can be a HWC mechanical configuration that ensures a light trap for the dark state light is located elsewhere, and Figure 5 The diagram depicted in should be seen as illustrating the concept that off light can be redirected to create compactness of the overall HWC. Figure 5a Illustration of the combination Figure 5 An example of an embodiment is described, with more detail added regarding the relative angles at various surfaces, and the ray traces for image light and dark light are shown as they pass through upper optical module 202. Again, various angle combinations are possible.

[0244] Figure 4b A diagram of a further embodiment is shown in which a solid transparent matched wedge set 456 is provided with a reflective polarizer 450 at the interface between the wedges. Wherein the interfaces between the wedges in the wedge set 456 are provided at a certain angle so that the illumination light 452 from the polarized light source 458 is reflected at an angle suitable for the DLP mirror "on" state (e.g., 34 degrees for a 17 degree DLP mirror) so that the reflected image light 414 is provided along the optical axis of the field lens. The general geometry of the wedges in the wedge set 456 is similar to that in Figure 4 and 4a. A quarter-wave film 454 is provided on the surface of DLP 402 so that illumination light 452 is of one polarization state (e.g., S polarization state), while image light 414 is converted to another polarization state (e.g., P polarization state) upon passing through quarter-wave film 454, reflecting from the DLP mirror, and returning through quarter-wave film 454. The reflective polarizer is oriented so that illumination light 452 of its polarization state is reflected and image light 414 of its other polarization state is transmitted. Since dark-state light from "off" pixels 410 also passes through quarter-wave film 454 twice, it is also of the other polarization state (e.g., P polarization state) so that it is transmitted by reflective polarizer 450.

[0245] The angles of the faces of wedge set 450 correspond to the angles required to provide illumination light 452 at the angles required by the DLP mirrors when in the "on" state so that reflected image light 414 is reflected from the DLP along the optical axis of the field lens. Wedge set 456 provides an internal interface where a reflective polarizer film can be positioned to redirect illumination light 452 toward the mirrors of the DLP 402. The wedge set also provides matching wedges on opposite sides of the reflective polarizer 450 so that image light 414 from the "on" pixels exits wedge set 450 substantially perpendicular to the exit surface, while dark state light from the "off" pixels 410 exits at an oblique angle to the exit surface. As a result, image light 414 is substantially unrefracted upon exiting wedge set 456, while dark state light from the "off" pixels 410 is significantly refracted upon exiting wedge set 456, as shown in FIG. Figure 4b As shown in .

[0246] By providing a solid, transparent, matched wedge set, the flatness of the interface is reduced, as variations in flatness have negligible effects as long as they are within the cone angle of illumination light 452. This can be f# 2.2 with a 26-degree cone angle. In a preferred embodiment, an optical adhesive is used to bond the reflective polarizer between the matched inner surfaces of wedge set 456 to reduce Fresnel reflections at the interface on either side of reflective polarizer 450. The optical adhesive can be index-matched to the material of wedge set 456, and the segments of wedge set 456 can all be made of the same material, such as BK7 glass or cast acrylic. The wedge material can also be selected to have low birefringence to reduce non-uniformity in brightness. Wedge set 456 and quarter-wave film 454 can also be bonded to DLP 402 to further reduce Fresnel reflections at DLP interface losses. Furthermore, because image light 414 is substantially normal to the exit surface of wedge set 456, the flatness of the surface is not critical to maintaining the wavefront of image light 414 so that high image quality can be obtained in the displayed image without requiring very tight tolerance flatness on the exit surface.

[0247] Another embodiment of the present invention which is not shown in the figure is Figure 4b and Figure 5 In this embodiment, the wedge group 456 consists of three wedges, wherein the general geometry of the wedges in the wedge group corresponds to Figure 5 and 5a The geometry shown in . Figure 4b Similar to the case shown in FIG, a reflective polarizer is joined between the first and second wedges, however, unlike Figure 5 The embodiment of is similarly provided with a third wedge. There is a thin air gap at an angle between the second and third wedges so that the dark state light is TIR reflected towards the side of the second wedge where it is absorbed in the light trap. This embodiment (like in Figure 4b ) uses a polarized light source, as previously described. The difference in this embodiment is that the image light is transmitted through a reflective polarizer and through a thin air gap that is angled so that it exits perpendicular to the exit surface of the third wedge.

[0248] Figure 5b The upper optical module 202 is shown with a dark light trap 514a. Figure 4 and 4aAs described, image light can be generated from the DLP when using a TIR and corrective lens configuration. The upper module can be mounted in an HWC housing 510, and the housing 510 can include a dark light trap 514a. The dark light trap 514a is generally positioned / configured / formed in optical alignment with the dark light optical axis 512. As shown, the dark light trap can have a depth that causes it to internally reflect dark light to further absorb it and prevent it from combining with image light passing through the field lens. The dark light trap can have a shape and depth that allows it to absorb dark light. Furthermore, in embodiments, the dark light trap 514b can be made of or coated with a light-absorbing material. In embodiments, the recessed light trap 514a can include a baffle to block the view of dark state light. This can be combined with a black surface or a textured or fibrous surface to help absorb light. The baffle can be part of the light trap associated with the housing, field lens, or the like.

[0249] Figure 5c Another embodiment is shown having a light trap 514b. As can be seen in the illustration, the shape of the trap is configured to enhance internal reflections within the light trap 514b to increase absorption of the dark light 512. Figure 5d Another embodiment is shown having a light trap 514c. As can be seen in the illustration, the shape of the trap 514c is configured to enhance internal reflections to increase absorption of the dark light 512.

[0250] Figure 5e Another embodiment of the upper optical module 202 is shown with a dark light trap 514d. This embodiment of the upper module 202 includes a closed light reflecting wedge 502, such as in combination with Figure 5 and 5a As shown and described in the embodiment of Figure 5e As can be seen in FIG5 , light trap 514d is positioned along the optical path of dark light 512. Dark light trap 514d may be configured as described in other embodiments herein. Figure 5e The embodiment of the light trap 514d illustrated in FIG includes black areas on the sidewalls of the wedge, where the sidewalls are positioned substantially away from the optical axis of the image light 414. Additionally, baffles 5252 may be added to one or more edges of the field lens 312 to block the view of the light trap 514d adjacent to the displayed image seen by the user.

[0251] Figure 6The diagram shows the combination of the upper optical module 202 and the lower optical module 204. In this embodiment, the image light projected from the upper optical module 202 may or may not be polarized. The image light reflects off the flat combiner element 602 so that it is directed toward the user's eyes. The combiner element 602 is a partially reflective mirror that reflects the image light while transmitting a significant portion of the light from the environment, so that the user can see through the combiner element and see the environment around the HWC.

[0252] Combiner 602 can include a holographic pattern to form a holographic mirror. If a monochromatic image is desired, a single wavelength reflective design can be used for the holographic pattern on the surface of combiner 602. If the intention is to have multiple colors reflected from the surface of combiner 602, a multi-wavelength holographic mirror can be included on the combiner surface. For example, in a three-color embodiment, where red, green, and blue pixels are generated in the image light, the holographic mirror is reflective for wavelengths that substantially match the wavelengths of the red, green, and blue light provided by the light source. This configuration can be used as a wavelength-specific mirror, where light of a predetermined wavelength from the image light is reflected toward the user's eye. This configuration can also be configured so that substantially all other wavelengths in the visible light pass through combiner element 602, so that the user has a substantially clear view of their surroundings when looking through combiner element 602. When using a combiner that is a holographic mirror, the transparency between the user's eye and the surroundings can be approximately 80%. A holographic mirror can be constructed using laser light to generate an interference pattern in the holographic material of the combiner, wherein the wavelength of the laser light corresponds to the wavelength of light subsequently reflected by the holographic mirror.

[0253] In another embodiment, the combiner element 602 may include a notch mirror comprised of a multilayer coated substrate, wherein the coatings are designed to substantially reflect the wavelengths of light provided by the light source and substantially transmit the remaining wavelengths in the visible spectrum. For example, in a scenario where red, green, and blue light are provided by the light source to enable a full-color image to be provided to the user, the notch mirror is a tristimulus notch mirror, wherein the multilayer coatings are designed to reflect narrow bands of red, green, and blue light that match the bands provided by the light source, while the remaining visible wavelengths are transmitted through the coatings to enable a view of the environment through the combiner. In another example where a monochromatic image is provided to the user, the notch mirror is designed to reflect a single narrow band of light that matches the wavelength range of the light provided by the light source, while transmitting the remaining visible wavelengths to enable a see-through view of the environment. The combiner 602 with the notch mirror will operate from the user's perspective in a similar manner to a combiner that includes a holographic pattern on the combiner element 602. Due to the match between the color of the image light and the reflected wavelength of the notch mirror, the combiner with the three-color notch mirror will reflect the "on" pixels toward the eye, and the wearer will be able to see the surrounding objects with high clarity. When using the three-color notch mirror, the transparency between the user's eye and the surrounding objects can be approximately 80%. In addition, due to the reduced scattering of the imaging light by the combiner, the image provided by the upper optical module 202 with the notch mirror combiner can provide a higher contrast image than the holographic mirror combiner.

[0254] Light can escape through combiner 602 and, because the light is generally directed downward onto the user's cheek, can create a face glow. When using a holographic mirror combiner or a tristimulus notch mirror combiner, the escaping light can be captured to avoid face glow. In embodiments, if the image light is polarized before the combiner, a linear polarizer can be laminated to or otherwise associated with the combiner, with the polarizer's transmission axis oriented relative to the polarized image light so that any escaping image light is absorbed by the polarizer. In embodiments, the image light will be polarized to provide S-polarized light to the combiner for better reflection. As a result, the linear polarizer on the combiner will be oriented to absorb S-polarized light and transmit P-polarized light. This also provides a preferred orientation for polarized sunglasses.

[0255] If the image light is not polarized, a microlouvered film (such as a privacy filter) can be used to absorb escaping image light while providing the user with a see-through view of the environment. In this case, the absorption or transmission of the microlouvered film depends on the angle of the light, with steep angle light being absorbed and light at smaller angles being transmitted. For this reason, in embodiments, the combiner with the microlouvered film is oriented at an angle greater than 45 degrees to the optical axis of the image light (for example, the combiner can be oriented at 50 degrees) so that the image light from the field lens is incident on the combiner at an oblique angle.

[0256] Figure 7 Illustrated are embodiments of the combiner element 602 at various angles when the combiner element 602 comprises a holographic mirror. Typically, a mirrored surface reflects light at an angle equal to the angle at which the light is incident on the mirrored surface. Typically, this necessitates that the combiner element be at 45 degrees (i.e., 602a) if the light is presented to the combiner perpendicularly so that the light can be reflected horizontally towards the wearer's eyes. In embodiments, the incident light can be presented at angles other than perpendicular so that the mirrored surface can be oriented at other than 45 degrees, but in all cases where a mirrored surface is employed (including the tristimulus notch mirrors described previously), the angle of incidence equals the angle of reflection. As a result, increasing the angle of the combiner 602a requires that the incident image light be presented to the combiner 602a at a different angle, which positions the upper optical module 202 to the left of the combiner, as in Figure 7 In contrast, the holographic mirror combiner included in an embodiment can be configured so that light is reflected at an angle different from the angle at which the light is incident on the holographic mirrored surface. This allows the angle of the combiner element 602b to be freely selected independently of the angle of the incident image light and the angle of the light reflected into the wearer's eye. In an embodiment, the angle of the combiner element 602b is greater than 45 degrees (in Figure 7 ), as this allows for a more laterally compact HWC design. The increased angle of the combiner element 602b reduces the front-to-back width of the lower optical module 204 and can allow for a thinner HWC display (i.e., the element farthest from the wearer's eyes can be closer to the wearer's face).

[0257] Figure 8Another embodiment of the lower optical module 204 is shown. In this embodiment, polarized image light provided by the upper optical module 202 is directed into the lower optical module 204. The image light reflects off a polarizer 804 and is directed to a focusing partially reflective mirror 802, which is adapted to reflect polarized light. An optical element (such as a ¼ wave film) positioned between the polarizer 804 and the partially reflective mirror 802 is used to alter the polarization state of the image light so that light reflected by the partially reflective mirror 802 is transmitted by the polarizer 804 to present the image light to the wearer's eye. The user is also able to see through the polarizer 804 and the partially reflective mirror 802 to view the surrounding environment. As a result, the user perceives a combined image composed of the displayed image light superimposed on a see-through view of the environment.

[0258] While many embodiments of the present invention have been referred to as upper and lower modules comprising certain optical components, it should be understood that the image light and dark light generation and management functions described in conjunction with the upper module can be arranged to direct light in other directions (e.g., upward, to the side, etc.). In embodiments, it may be preferable to mount the upper module 202 above the wearer's eye, in which case the image light will be directed downward. In other embodiments, generating light from the side of the wearer's eye or from below the wearer's eye may be preferable. Furthermore, the lower optical module is generally configured to deliver image light to the wearer's eye and allow the wearer to see through the lower optical module, which can be achieved through various optical components.

[0259] Figure 8a An embodiment of the present invention is shown in which the upper optical module 202 is arranged to direct image light into a TIR waveguide 810. In this embodiment, the upper optical module 202 is positioned above the wearer's eye 812 and the light is directed horizontally into the TIR waveguide 810. The TIR waveguide is designed to internally reflect the image light in a series of downward TIR reflections until it reaches a portion in front of the wearer's eye, where the light exits the TIR waveguide 812 and passes into the wearer's eye. In this embodiment, an external shield 814 is positioned in front of the TIR waveguide 810.

[0260] Figure 8b An embodiment of the present invention is shown in which the upper optical module 202 is arranged to guide image light into the TIR waveguide 818. In this embodiment, the upper optical module 202 is arranged on the side of the TIR waveguide 818. For example, when the HWC is configured as a pair of head-mounted glasses, the upper optical module can be located in or near the arm of the HWC. The TIR waveguide 818 is designed to internally reflect the image light in a series of TIR reflections until it reaches the portion in front of the wearer's eye, where the light exits the TIR waveguide 812 and is passed into the wearer's eye.

[0261] Figure 8c Still further embodiments of the present invention are illustrated in which the upper optical module 202 directs polarized image light into an optical guide 828, where the image light passes through a polarizing reflector 824, changes polarization upon reflection from an optical element 822 (which, for example, includes a ¼ wave film), and is then reflected by the polarizing reflector 824 toward the wearer's eye due to the change in polarization of the image light. The upper optical module 202 can be positioned to direct light toward a reflector 820 to laterally position the upper optical module 202, or in other embodiments, the upper optical module 202 can direct the image light directly toward the polarizing reflector 824. It should be understood that the present invention encompasses other optical arrangements intended to direct image light toward the wearer's eye.

[0262] Another aspect of the present invention relates to eye imaging. In an embodiment, a camera is used in conjunction with the upper optical module 202 so that the wearer's eye can be imaged using the pixels on the DLP in the "off" state. Figure 9 A system is illustrated in which an eye imaging camera 802 is mounted and angled so that the field of view of the eye imaging camera 802 is redirected toward the wearer's eye by the mirror pixels of the DLP 402 in the "off" state. In this manner, the eye imaging camera 802 can be used to image the wearer's eye along the same optical axis as the display image presented to the wearer. The image light presented to the wearer's eye illuminates the wearer's eye so that the eye can be imaged by the eye imaging camera 802. In this process, light reflected from the eye passes back through the optical train of the lower optical module 204 and a portion of the upper optical module to where it is reflected by the "off" pixels of the DLP 402 toward the eye imaging camera 802.

[0263] In one embodiment, the eye imaging camera can image the wearer's eye at a time when there are enough "off" pixels to achieve the desired eye image resolution. In another embodiment, the eye imaging camera collects eye image information from the "off" pixels over time and forms a time-lapse image. In another embodiment, a modified image is presented to the user that includes enough "off" state pixels at the desired resolution and brightness available to the camera for imaging the wearer's eye, and the eye image capture is synchronized with the presentation of the modified image.

[0264] Eye imaging systems can be used for security systems. If the eye is not identified (e.g., by eye characteristics including retina or iris characteristics, etc.), the HWC may not allow access to the HWC or other systems. In some embodiments, the HWC may be used to provide continuous secure access. For example, eye security confirmation can be a continuous, near-continuous, real-time, quasi-real-time, periodic, etc. process so that the wearer is effectively constantly verified as known. In embodiments, the HWC can be worn and tracked for eye security for access to other computer systems.

[0265] Eye imaging systems can be used to control HWCs. For example, blinking, winking, or particularly eye movements can be used as a control mechanism for software applications operating on the HWC or associated devices.

[0266] The eye imaging system can be used in determining how or when the HWC 102 delivers digitally displayed content to the wearer. For example, the eye imaging system can determine that the user is looking in one direction, and the HWC can then change the resolution in an area of ​​the display or provide certain content associated with something in the environment the user may be looking at. Alternatively, the eye imaging system can identify different users and change the enabled features or displayed content provided to the users. Users can be identified from a database of user eye characteristics located on the HWC 102 or remotely located on the network 110 or server 112. Furthermore, the HWC can identify a primary user or group of primary users based on the eye characteristics, where the primary user(s) are provided with an enhanced feature set and all other users are provided with a different feature set. Thus, in this use case, the HWC 102 uses the identified eye characteristics to enable or disable features, and the eye characteristics only need to be analyzed compared to a relatively small database of individual eye characteristics.

[0267] Figure 101 and 4. The diagram illustrates a light source (e.g., a polarized light source (such as polarized light sources 302 and 458) if the light from the solid-state light source is polarized) and a light source 404 that may be used in association with the upper optics module 202. In an embodiment, to provide a uniform surface of light 1008 that is directed directly or indirectly into the upper optics module 202 and toward the DLP of the upper optics module, the solid-state light source 1002 may be projected into a backlighting optical system 1004. The solid-state light source 1002 may be one or more LEDs, laser diodes, or OLEDs. In an embodiment, the backlighting optical system 1004 includes an extended section having a length / distance ratio greater than 3, wherein the light undergoes multiple reflections from the sidewalls to homogenize the light supplied by the solid-state light source 1002. The backlighting optical system 1004 can also include a surface opposite to where the uniform light 1008 emerges from the backlight 1004 to redirect the light toward the DLP 302 and reflective polarizer 310 or the DLP 402 and TIR wedge 418 (e.g., a TIR wedge 418). Figure 10 The backlight optical system 1004 may also include a structure for collimating the uniform light 1008 to provide light to the DLP with a smaller angular distribution or a narrower cone angle. A diffuser or polarizer can be used on the incident surface or the exit surface of the backlight optical system. A diffuser can be used to expand or homogenize the exiting light from the backlight to improve the uniformity of the uniform light 1008 or increase the angular spread of the uniform light 1008. An elliptical diffuser, which diffuses light more in some directions and less in other directions, can be used to improve the uniformity or spread of the uniform light 1008 in a direction perpendicular to the optical axis of the uniform light 1008. A linear polarizer can be used to convert the unpolarized light supplied by the solid-state light source 1002 into polarized light, thereby polarizing the uniform light 1008 in a desired polarization state. A reflective polarizer can be used on the exit surface of the backlight 1004 to polarize the uniform light 1008 to a desired polarization state, while reflecting other polarization states back into the backlight, where it is recycled through multiple reflections within the backlight 1004 and at the solid-state light source 1002. Thus, by including a reflective polarizer at the exit surface of the backlight 1004, the efficiency of the polarized light source is improved.

[0268] Figure 10a and 10bAn illustration of structures in a backlight optical system 1004 that can be used to redirect light provided by a light source to an entrance surface 1045 and then collimate the light in a direction transverse to the optical axis of the exiting uniform light 1008. Structure 1060 comprises an angled sawtooth pattern in a transparent waveguide, wherein the left edge of each sawtooth shears off steep angled light rays, thereby limiting the angle of the redirected light. A steep surface on the right side of each sawtooth (as shown) then redirects the light so that it reflects off the left angled surface of each sawtooth and is redirected towards the exit surface 1040. Figure 10a and 10b The sawtooth surface shown on the lower surface of FIG can be smooth and coated (e.g., with an aluminum coating or a dielectric mirror coating) to provide a high level of reflectivity without scattering. Structure 1050 includes a curved surface on the left side (as shown) to focus rays after they pass through exit surface 1040, thereby providing a mechanism for collimating uniform light 1008. In a further embodiment, a diffuser can be provided between solid-state light source 1002 and incident surface 1045 to homogenize the light provided by solid-state light source 1002. In yet another embodiment, a polarizer can be used between the diffuser and incident surface 1045 of backlight 1004 to provide a polarized light source. Because the sawtooth pattern provides a smooth reflective surface, the polarization state of light can be preserved from incident surface 1045 to exit surface 1040. In this embodiment, light incident on the backlight from solid-state light source 1002 passes through a polarizer so that the light is polarized in the desired polarization state. If the polarizer is an absorbing linear polarizer, light of the desired polarization state is transmitted while light of the other polarization state is absorbed. If the polarizer is a reflective polarizer, light of the desired polarization state is transmitted into the backlight 1004 while light of the other polarization state is reflected back into the solid-state light source 1002, where the light can be recycled as previously described to improve the efficiency of the polarized light source.

[0269] Figure 11a The light source 1100 is shown as being usable in association with the upper optics module 202. In an embodiment, the light source 1100 can provide light to the backlighting optical system 1004, as described above in conjunction with the Figure 10 As described. In an embodiment, the light source 1100 includes a tricolor notch filter 1102. The tricolor notch filter 1102 has narrow bandpass filters for three wavelengths, as in Figure 11c As indicated in the transmission diagram 1108. Figure 11bThe graph shown as 1104 in FIG illustrates the output of three different colored LEDs. One can see that the bandwidth of the emission is narrow, but they have long tails. A three-color notch filter 1102 can be used in conjunction with such LEDs to provide a light source 1100 whose emission is as shown in FIG. Figure 11d 110. The light source 1100 can be used in conjunction with a combiner 602 having a holographic reflector or a tristimulus notch reflector to provide a narrow band of light that is reflected toward the wearer's eyes with less wasted light that is not reflected by the combiner, thereby improving efficiency and reducing escape light that can cause face glow.

[0270] Figure 12a Another light source 1200 is shown that can be used in association with the upper optics module 202. In an embodiment, the light source 1200 can provide light to the backlighting optical system 1004, as described above in conjunction with Figure 10 As described. In an embodiment, the light source 1200 includes a quantum dot cover glass 1202. Wherein, the quantum dots absorb light of shorter wavelengths and emit light of longer wavelengths ( Figure 12b An example is shown where UV spectrum 1202 applied to quantum dots results in the quantum dots emitting narrow bands (shown as PL spectrum 1204), which depend on the size and material composition of the quantum dots. As a result, the quantum dots in the quantum dot cover glass 1202 can be tailored to provide narrow bandwidth light (e.g., red, green, and blue) emission depending on one or more bands of different quantum dots included, as shown in FIG. Figure 12c As illustrated in the diagram shown in FIG, three different quantum dots are used. In embodiments, the LED driver lamp emits UV light, deep blue, or blue light. For sequential illumination of different colors, multiple light sources 1200 are used, each of which includes a quantum dot cover glass 1202 with quantum dots selected to emit one of the desired colors. Light sources 1100 can be used in conjunction with a combiner 602 having a holographic mirror or a tristimulus notch mirror to provide a narrow transmission band of light that is reflected toward the wearer's eyes with less wasted light (not reflected).

[0271] Another aspect of the present invention relates to generating peripheral image lighting effects for a person wearing a HWC. In embodiments, a solid-state lighting system (e.g., LEDs, OLEDs, etc.) or other lighting system may be included within the optics of the lower optical module 204. The solid-state lighting system may be arranged so that lighting effects are rendered outside the field of view (FOV) of the presented digital content, creating an immersive effect for the person wearing the HWC. To this end, the lighting effects may be rendered for any portion of the HWC visible to the wearer. The solid-state lighting system may be digitally controlled by an integrated processor on the HWC. In embodiments, the integrated processor controls the lighting effects in coordination with the digital content presented within the HWC's FOV. For example, a movie, picture, game, or other content may be displayed or played within the HWC's FOV. The content may depict a bomb exploding on the right side of the FOV at the same instant, and the solid-state lighting system within the upper module's optics may flash rapidly in sync with the FOV image effect. This effect may not be immediate; it may be more persistent, for example, to indicate a general glow or color on one side of the user. A solid state lighting system may be color controlled, for example using red, green, and blue LEDs, so that the color control can be coordinated with the content of a digital presentation within the field of view.

[0272] Figure 13a The optical components of the lower optical module 204 are illustrated along with the outer lens 1302 . Figure 13a Also shown is an embodiment including effect LEDs 1308a and 1308b. Figure 13a The image light 1312 is shown being directed into the upper optical module as described elsewhere herein where it will reflect off the combiner element 1304 as described elsewhere herein. The combiner element 1304 in this embodiment is angled toward the wearer's eyes at the top of the module and away from the wearer's eyes at the bottom of the module, as also described in conjunction with FIG. Figure 8 As described and illustrated (e.g., at a 45 degree angle). Figure 13aImage light 1312 provided by the optical system (not shown) reflects off combiner element 1304 away from the wearer's eye toward collimating mirror 1310, as described elsewhere herein. Image light 1312 then reflects and focuses off collimating mirror 1304, passes back through combiner element 1304, and is directed into the wearer's eye. The wearer is also able to view the surrounding environment through the transparency of combiner element 1304, collimating mirror 1310, and external lens 1302 (if included). As described elsewhere herein, various surfaces are polarized to create an optical path for the image light and provide transparency to enable the wearer to view the surrounding environment. The wearer will generally perceive the image light as forming an image within FOV 1305. In embodiments, external lens 1302 may be included. External lens 1302 is an external lens that may or may not be corrective and may be designed to conceal the underlying optical module components in an attempt to make the HWC appear to be similar to standard glasses or sunglasses.

[0273] exist Figure 13a In the embodiment illustrated in FIG, effects LEDs 1308a and 1308b are positioned to the sides of combiner element 1304 and external lens 1302 and / or collimating reflector 1310. In an embodiment, effects LED 1308a is positioned within the confines defined by combiner element 1304 and external lens 1302 and / or collimating reflector. Effects LEDs 1308a and 1308b are also positioned outside of FOV 1305. In this arrangement, effects LEDs 1308a and 1308b can provide lighting effects within the lower optical module outside of FOV 1305. In an embodiment, light emitted from effects LEDs 1308a and 1308b can be polarized so that the light passes through combiner element 1304 and does not pass through external lens 1302 and / or collimating reflector 1310 toward the wearer's eyes. This arrangement provides peripheral lighting effects to the wearer in a more private setting by not transmitting the lighting effects into the surrounding environment through the front of the HWC. However, in other embodiments, the effects LEDs 1308a and 1308b may be unpolarized, thereby making the lighting effects provided purposefully viewable by others in the environment for entertainment, such as to give the wearer an eye glow effect corresponding to image content viewed by the wearer.

[0274] Figure 13b Illustration of the combination Figure 13a13. A cross-section of the depicted embodiment. As illustrated, the effects LEDs 1308a are located in the upper-front region of the interior of the optics of the lower optical module. It should be understood that the placement of the effects LEDs 1308a in the depicted embodiment is merely illustrative and that alternative layouts are encompassed by the present invention. Additionally, in embodiments, one or more effects LEDs 1308a may be present on each of the two sides of the HWC to provide peripheral lighting effects near one or both of the wearer's eyes.

[0275] Figure 13c An embodiment is illustrated in which the combiner element 1304 is angled away from the eye at the top and toward the eye at the bottom (e.g., in accordance with the holographic or notch filter embodiments described herein). In this embodiment, the effect LED 1308a is located on the outer lens 1302 side of the combiner element 1304 to provide a covert appearance of the lighting effect. As with other embodiments, Figure 13c The effect LED 1308a may include a polarizer to enable emitted light to pass through the polarizing element associated with the combiner element 1304 and be blocked by the polarizing element associated with the external lens 1302.

[0276] Another aspect of the present invention relates to mitigating light escaping from the space between the wearer's face and the HWC itself. Another aspect of the present invention relates to maintaining a controlled lighting environment close to the wearer's eyes. In an embodiment, both maintenance of the lighting environment and mitigation of light escape are achieved by including a removable and replaceable flexible shield for the HWC. Wherein, corresponding to the use of a display for each eye, a removable and replaceable shield can be provided for one eye or both eyes. For example, in a night vision application, the display for only one eye can be used for night vision, while the display for the other eye is turned off to provide good see-through when moving between areas where visible light is available and dark areas where night vision enhancement is desired.

[0277] Figure 14aA removable and replaceable flexible eye mask 1402 is shown with an opening 1408 that can be quickly attached to and removed from the HWC 102 using magnets 1404. Other attachment methods can be used, but for the purposes of illustrating the present invention, we will focus on the magnet embodiment. In an embodiment, a magnet can be included in the eye mask 1402, and a magnet of opposite polarity can be included (e.g., embedded) in the frame of the HWC 102. With the opposite polarity configuration, the magnets of the two elements will attract quite strongly. In another embodiment, one of the elements can have a magnet and the other side can have metal for attraction. In an embodiment, the eye mask 1402 is a flexible, elastic shield. In an embodiment, the eye mask 1402 can be an elastic bellows design to accommodate flexibility and more closely align with the wearer's face. Figure 14b A removable and replaceable flexible eye mask 1402 is shown adapted to form a single eye mask. In embodiments, a single eye mask can be used on each side of a HWC to cover both eyes of the wearer. In embodiments, a single eye mask can be used in conjunction with a HWC that includes only one computer display for each eye. These configurations block light generated and generally directed toward the wearer's face by covering the space between the wearer's face and the HWC. An opening 1408 allows the wearer to look through the opening 1408 to view displayed content and the surrounding environment through the front of the HWC. Image light in the lower optical module 204 can be prevented from being emitted from the front of the HWC by, for example, an internal optics polarization scheme as described herein.

[0278] Figure 14c Another embodiment of a light suppression system is shown. In this embodiment, eye shield 1410 can be similar to eye shield 1402, but includes a front light shield 1412. Front light shield 1412 can be opaque to prevent light from escaping the front lens of the HWC. In other embodiments, front light shield 1412 is polarized to prevent light from escaping the front lens. In a polarized arrangement, in embodiments, internal optical elements of the HWC (e.g., of the lower optical module 204) can polarize light transmitted toward the front of the HWC, and front light shield 1412 can be polarized to prevent light from transmitting through front light shield 1412.

[0279] In an embodiment, an opaque front light shield 1412 may be included and the digital content may include an image of the surrounding environment to enable the wearer to visualize the surrounding environment. A night vision environment image may be presented to one eye and the opaque front light shield 1412 may be used to cover the ambient light path for that eye. In other embodiments, this arrangement may be associated with both eyes.

[0280] Another aspect of the present invention relates to automatically configuring the lighting system(s) in the HWC 102. In embodiments, display lighting and / or effect lighting, as described herein, can be controlled in a manner appropriate for when the eye mask 1402 is attached to or removed from the HWC 102. For example, at night, when ambient light is low, the lighting system(s) in the HWC can enter a low-light mode to further control any stray light escaping from the HWC and the area surrounding it. Operating shielded at night while using night vision or standard vision may require a solution that prevents as much light from escaping as possible, so the user clips on the eye mask(s) 1402 and the HWC can then enter low-light mode. In some embodiments, low-light mode may only be entered when the eye mask 1402 is attached, if the HWC identifies low-light conditions in the environment (e.g., detected by an ambient light level sensor). In embodiments, the low-light level can be determined to be midway between full light and low light, depending on ambient conditions.

[0281] Another aspect of the present invention relates to automatically controlling the type of content displayed in the HWC when the eye mask 1402 is attached to or removed from the HWC. In embodiments, when the eye mask(s) 1402 are attached to the HWC, the content displayed can be limited in amount or color. For example, the display(s) can enter a simplified content delivery mode to limit the amount of information displayed. This can be done to reduce the amount of light generated by the display(s). In embodiments, the display(s) can be changed from a color display to a monochrome display to reduce the amount of light generated. In embodiments, the monochrome lighting can be red to limit the impact on the wearer's eyes, thereby maintaining the ability to see better in the dark.

[0282] Another aspect of the present invention relates to a system that is adapted to quickly transition from a see-through system to a non-see-through system or a very low-transmittance see-through system for a more immersive user experience. The transition system may include interchangeable lenses, eye masks, and optics adapted to provide a user experience in both modes. The lens, for example, may be "blacked out" to provide an experience in which the user's full attention is devoted to digital content, and then the lens may be switched to a high-transmittance lens so that the digital content enhances the user's view of the surrounding environment. Another aspect of the present invention relates to a low-transmittance lens that allows the user to see through the lens, but remains dark enough to maintain the majority of the user's attention on the digital content. The slight see-through can provide the user with a visual connection to the surrounding environment, which can reduce or eliminate nausea and other issues associated with completely removing the surrounding view when viewing digital content.

[0283] Figure 14d The diagram illustrates a head-worn computer system 102 having a see-through digital content display 204 adapted to include a removable external lens 1414 and a removable eyecup 1402. The eyecup 1402 can be attached to the head-worn computer 102 using magnets 1404 or other attachment systems (e.g., mechanical attachment, a snug friction fit between the arms of the head-worn computer 102, etc.). The eyecup 1402 can be attached when the user wants to reduce stray light from escaping the confines of the head-worn computer, create a more immersive experience by removing the otherwise visible peripheral view of the surrounding environment, and so on. The removable external lens can be of several types to accommodate various experiences. It can have no transmission or very low transmission to create a dark background for digital content, creating an immersive experience for the digital content. It can also have high transmission so that the user can see through the see-through display and lens to view the surrounding environment, creating a system for a heads-up display, augmented reality display, assisted reality display, and so on. Lens 1414 can be dark in the middle to provide a dark background for the digital content (i.e., a dark background behind the see-through field of view from the user's perspective) and provide higher transmission areas elsewhere. For example, lens 1414 can have a transmission in the range of 2 to 5%, 5 to 10%, 10 to 20% for immersive effects, and a transmission above 10% or 20% for augmented reality effects. Lens 1414 can also have adjustable transmission to facilitate changes in the system effect. For example, lens 1414 can be a lens with electronically adjustable tint (e.g., liquid crystal, or with crossed polarizers with adjustment for the level of crossing).

[0284] In embodiments, the eye mask may have transparent or partially transparent areas to provide some visual connection to the user's surroundings. This may also reduce or eliminate nausea or other sensations associated with completely removing the view of the surrounding environment.

[0285] Figure 14e The head-worn computer 102 is shown assembled with the eye mask 1402 without the lenses in place. In embodiments, the lenses can be held in place using magnets 1418 for easy removal and replacement. In embodiments, the lenses can be held in place using other systems, such as a mechanical system.

[0286] Another aspect of the present invention relates to an effects system that generates out-of-field-of-view effects in a see-through display of a head-worn computer. The effects can be, for example, lighting effects, sound effects, haptic effects (e.g., through vibration), air movement effects, and the like. In an embodiment, the effects generation system is mounted on the eye mask 1402. For example, a lighting system (e.g., LED(s), OLED, etc.) can be mounted on the inner surface 1420 or exposed through the inner surface 1420, such as Figure 14f , so that they can create lighting effects (e.g., bright lights, tinted lights, subtle color effects) in coordination with the content being displayed in the field of view of the see-through display. The content may be, for example, a movie or game, and the explosion may be scripted to occur to the right of the content. To match the content, a bright flash of light may be generated by the effects lighting system to create a stronger effect. As another example, the effects system may include a vibration system mounted on the side, near the temple, or otherwise, and when the same explosion occurs, the vibration system may generate vibrations on the right side to enhance the user experience, indicating that the explosion has real sound waves that produce the vibrations. As yet another example, the effects system may include an air system, where the effect is a puff of air blown into the user's face. This can create a sense of proximity to a fast-moving object in the content. The effects system may also include speakers pointed toward the user's ears, attachments for earbuds, and so on.

[0287] In embodiments, the effects generated by the effects system can be scripted by the author to coordinate with the content. In embodiments, sensors can be placed within the eye mask to monitor content effects that will then cause the effect(s) to be generated (e.g., light sensors to measure bright lighting effects or peripheral lighting effects).

[0288] The effects system in the eye mask can be powered by an internal battery, and in some embodiments, the battery can also provide additional power to the head-worn computer 102 as a backup system. In some embodiments, the effects system is powered by a battery in the head-worn computer. Power can be delivered through an attachment system (e.g., magnets, mechanical system) or a dedicated power system.

[0289] The effects system can receive data and / or commands from the head-mounted computer via a wired or wireless data connection. The data can come, for example, via an attachment system, a separate line, or via Bluetooth or other short-range communication protocols.

[0290] In one embodiment, the eye mask is made of reticulated foam, which is very lightweight and can conform to the contours of the user's face. Due to the open-cell nature of the material, the reticulated foam also allows air circulation, which can reduce user fatigue and increase user comfort. The eye mask can be made of other materials, such as soft, rigid, and flexible, and can have another material around the periphery that contacts the face for comfort. In one embodiment, the eye mask can include a fan to exchange air between the external environment and an interior space, where the interior space is partially defined by the user's face. The fan can operate very slowly and at low power to exchange air to keep the user's face cool. In one embodiment, the fan can have a variable speed controller and / or a temperature sensor can be positioned to measure the temperature in the interior space, thereby controlling the temperature in the interior space to a specified range, temperature, etc. The interior space is generally characterized by the space enclosed in front of the user's eyes and upper cheek (where the eye mask encloses the area).

[0291] Another aspect of the present invention relates to flexibly mounting an audio headset on the head-worn computer 102 and / or eyecup 1402. In an embodiment, the audio headset is mounted using a relatively rigid system with flexible joint(s) (e.g., a swivel joint at the connection to the eyecup, a swivel joint in the middle of a rigid arm, etc.) and extension(s) (e.g., a telescoping arm) to provide adjustability to the user to allow for a comfortable fit on, in, or around the user's ear. In an embodiment, the audio headset is mounted using a flexible system that is more flexible throughout, such as using a wire-based connection.

[0292] Figure 14g The diagram shows a head-worn computer 102 with removable lenses 1414 and mounted eye shield 1402. In embodiments, the head-worn computer includes a see-through display (as disclosed herein). Eye shield 1402 also includes mounted audio headphones 1422. In this embodiment, mounted audio headphones 1422 are mounted to eye shield 1402 and have an audio cable connection (not shown). In embodiments, the audio cable connection can connect to an internal wireless communication system (e.g., Bluetooth, NFC, WiFi) to establish a connection with the processor in the head-worn computer. In embodiments, the audio cable can connect to a magnetic connector, a mechanical connector, or the like to establish a connection.

[0293] Figure 14h The illustrated eye mask 1402 is not mounted with a mounted audio headset 1422. As illustrated, the mechanical design of the eye mask is adapted to fit onto a head-worn computer to provide visual isolation or partial isolation as well as audio headset.

[0294] In an embodiment, eye shields 1402 can be adapted to be removably mounted on a head-worn computer 102 having a see-through computer display. Audio headphones 1422 with adjustable mounting can be connected to the eye shields, where the adjustable mounting can provide extension and rotation to provide a mechanism for the head-worn computer user to align the audio headphones with the user's ears. In an embodiment, the audio headphones include an audio cable connected to a connector on the eye shield, and the eye shield connector can be adapted to removably mate with a connector on the head-worn computer. In an embodiment, the audio headphones can be adapted to receive audio signals from the head-worn computer via a wireless connection (e.g., Bluetooth, WiFi). As described elsewhere herein, the head-worn computer can have a removable and replaceable front lens. The eye shields can include a battery to power systems within the eye shields. The eye shields can have a battery to power systems within the head-worn computer.

[0295] In an embodiment, the eye mask may include a fan adapted to exchange air between an interior space partially defined by the user's face and the external environment, thereby cooling the air in the interior space and on the user's face. In an embodiment, the audio headset may include a vibration system (e.g., a vibration motor, a piezoelectric motor, etc. in the armature and / or in the supra-auricular section) adapted to provide tactile feedback to the user coordinated with digital content presented on a see-through computer display. In an embodiment, the head-worn computer includes a vibration system adapted to provide tactile feedback to the user coordinated with digital content presented on a see-through computer display.

[0296] In an embodiment, the eye mask 1402 is adapted to be removably mounted on a head-mounted computer having a see-through computer display. The eye mask may also include a flexible audio headset mounted to the eye mask, wherein the flexibility provides a mechanism for the user of the head-mounted computer to align the audio headset with the user's ear. In an embodiment, the flexible audio headset is mounted to the eye mask using a magnetic connection. In an embodiment, the flexible audio headset may be mounted to the eye mask using a mechanical connection.

[0297] In an embodiment, the audio earphones may be spring or otherwise loaded so that the earphones are pressed inwardly toward the user's ears for a more secure fit.

[0298] refer to Figure 15Now, we turn to a description of a specific external user interface 104, generally referred to as a pen 1500. The pen 1500 is a specially designed external user interface 104 capable of operating, for example, as a user interface for many different styles of HWC 102. The pen 1500 generally follows the form of a conventional pen, a familiar user-held device that creates an intuitive physical interface for many of the operations performed within the HWC system 100. The pen 1500 can be one of several user interfaces 104 used in conjunction with controlling operations within the HWC system 100. For example, the HWC 102 can monitor gestures 116 and interpret them as control signals, wherein the pen 1500 can also be used as a user interface for the same HWC 102. Similarly, a remote keyboard can be used as an external user interface 104 in conjunction with the pen 1500. The combination of user interfaces or the use of only one control system generally depends on the operation(s) being performed within the HWC system 100.

[0299] While the pen 1500 may follow the general form of a conventional pen, it contains a number of technologies that enable it to function as the external user interface 104 . Figure 15 The technology included in the pen 1500 is illustrated. As can be seen, the pen 1500 may include a camera 1508, which is arranged to view through a lens 1502. The camera can then focus, such as through the lens 1502, to image the surface on which the user is writing or making other movements to interact with the HWC 102. There are situations where the pen 1500 will also have ink, graphite, or other systems to enable the content being written to be visible on the writing surface. There are other situations where the pen 1500 does not have such a physical writing system and therefore has no deposit on the writing surface, in which case the pen will simply transmit data or commands to the HWC 102. The lens configuration is described in more detail herein. The function of the camera is to capture information from the unstructured writing surface so that the pen stroke can be interpreted as intended by the user. To assist in predicting the intended stroke path, the pen 1500 may include a sensor, such as an IMU 1512. Of course, the IMU can be included in the pen 1500 as a separate part of the pen (e.g., gyroscope, accelerometer, etc.), or the IMU can be included as a single unit. In this example, the IMU 1512 is used to measure and predict the movement of the pen 1500. In turn, the integrated microprocessor 1510 will take the IMU information and the camera information as input and process the information to form a prediction of the movement of the pen tip.

[0300] The pen 1500 may also include a pressure monitoring system 1504, such as to measure the pressure applied on the lens 1502. As will be described in greater detail herein, the pressure measurement can be used to predict the user's intent to change line thickness, line type, stroke type, click, double-click, etc. In embodiments, the pressure sensor may be constructed using any force or pressure measurement sensor located behind the lens 1502, including, for example, a resistive sensor, a current sensor, a capacitive sensor, a voltage sensor (such as a piezoelectric sensor), etc.

[0301] The pen 1500 may also include a communication module 1518, such as for bidirectional communication with the HWC 102. In one embodiment, the communication module 1518 may be a short-range communication module (e.g., Bluetooth). The communication module 1518 may be securely paired with the HWC 102. The communication module 1518 is configured to transmit data and commands to and from the microprocessor 1510 of the pen 1500. The microprocessor 1510 may be programmed to interpret data generated from, for example, the camera 1508, the IMU 1512, and the pressure sensor 1504, and then transmit commands to the HWC 102, for example, via the communication module 1518. In another embodiment, data collected by the microprocessor from any of the input sources (e.g., the camera 1508, the IMU 1512, the pressure sensor 1504) may be transmitted to the HWC 102 by the communication module 1518, and the HWC 102 may perform data processing and predict the user's intent when the pen 1500 is in use. In yet another embodiment, the data may be further transferred to a remote device 112 (such as a server) via the network 110 for data processing and prediction. The command may then be transmitted back to the HWC 102 for execution (e.g., display writing in the glasses display, selecting within the UI of the glasses display, controlling the remote external device 112, controlling the local external device 108), etc. The pen may also include a memory 1514 for long-term or short-term use.

[0302] The pen 1500 may also include a number of physical user interfaces, such as a quick launch button 1522, a touch sensor 1520, and the like. The quick launch button 1522 can be adapted to provide the user with a quick way to jump to software applications in the HWC system 100. For example, the user may be a frequent user of a communication software package (e.g., email, text, Twitter, Instagram, Facebook, Google+, etc.) and may program the quick launch button 1522 to command the HWC 102 to launch an application. The pen 1500 may be provided with several quick launch buttons 1522, which may be user-programmable or factory-programmable. The quick launch buttons 1522 can be programmed to perform actions. For example, one of the buttons can be programmed to clear the HWC 102's digital display. This creates a quick way for the user to clear the screen on the HWC 102 for any reason, such as to better view the surroundings. The quick launch button functionality will be discussed in further detail below. The touch sensor 1520 can be used to obtain gesture-style input from the user. For example, the user can take out a single finger and apply it across the touch sensor 1520 to scroll a page.

[0303] The pen 1500 may also include a laser pointer 1524. The laser pointer 1524 may be coordinated with the IMU 1512 to coordinate gestures and laser pointing. For example, a user may use the laser 1524 during a presentation to help guide an audience with a graphical explanation, and the IMU 1512 may interpret the user gestures as commands or data inputs simultaneously or when the laser 1524 is turned off.

[0304] 16A-C illustrate several embodiments of a lens and camera arrangement 1600 for the pen 1500. One aspect involves maintaining a constant distance between the camera and the writing surface so that the writing surface can be kept in focus to better track the movement of the pen 1500 across the writing surface. Another aspect involves maintaining an angled surface that follows the circumference of the writing tip of the pen 1500 so that the pen 1500 can roll or partially roll in the user's hand to create the feel and freedom of a conventional writing instrument.

[0305] FIG16A illustrates an embodiment of the writing lens end of pen 1500. The configuration includes a ball lens 1604, a camera or image capture surface 1602, and a domed cap lens 1608. In this arrangement, the camera views the writing surface through ball lens 1604 and domed cap lens 1608. Ball lens 1604 causes the camera to focus so that the camera views the writing surface when pen 1500 is held in a natural writing position (e.g., with pen 1500 in contact with the writing surface). In embodiments, ball lens 1604 should be separated from the writing surface to achieve the highest resolution of the writing surface at camera 1602. In embodiments, ball lens 1604 is separated by approximately 1 to 3 mm. In this configuration, domed cap lens 1608 provides a surface that maintains a constant distance between ball lens 1604 and the writing surface (e.g., substantially independent of the angle used to write on the writing surface). For example, in embodiments, the field of view of the camera in this arrangement will be approximately 60 degrees.

[0306] The dome cover lens or other lens 1608 used to physically interact with the writing surface will be transparent or transmissive within the effective bandwidth of the camera 1602. In embodiments, the dome cover lens 1608 can be spherical or other shapes and composed of glass, plastic, sapphire, diamond, etc. In other embodiments, where lower resolution imaging of the surface is acceptable, the pen 1500 can omit the dome cover lens 1608 and the spherical lens 1604 can be in direct contact with the surface.

[0307] FIG16B illustrates another configuration in which the construction is somewhat similar to that described in conjunction with FIG16A ; however, rather than using a domed cover lens 1608, this embodiment uses a spacer 1610 to maintain a predictable distance between the ball lens 1604 and the writing surface, wherein the spacer can be spherical, cylindrical, tubular, or other shapes that provide spacing while allowing an image to be obtained by the camera 1602 through the lens 1604. In a preferred embodiment, the spacer 1610 is transparent. Furthermore, while the spacer 1610 is shown as spherical, other shapes such as ellipsoids, toroids, hemispheres, cones, cylinders, or other forms may be used.

[0308] FIG16C illustrates yet another embodiment in which the structure includes a post 1614, such as one extending through the center of the lensed end of pen 1500. Post 1614 can be an ink deposition system (e.g., an ink cartridge), a graphite deposition system (e.g., a graphite holder), or a pseudo-post whose purpose is primarily alignment. The choice of post type depends on the pen's intended use. For example, if the user intends to use pen 1500 as a regular ink deposition pen with a fully functional external user interface 104, an ink system post would be the best choice. If there is no need to make "writing" visible on the writing surface, a pseudo-post would be the best choice. The embodiment of FIG16C includes camera(s) 1602 and associated lenses 1612, where the cameras 1602 and lenses 1612 are positioned to capture the writing surface without substantial interference from post 1614. In embodiments, pen 1500 can include multiple cameras 1602 and lenses 1612 to enable more or all of the circumference of tip 1614 to function as an input system. In an embodiment, the pen 1500 includes a contoured grip that keeps the pen aligned in the user's hand so that the camera 1602 and lens 1612 remain pointed at a surface.

[0309] Another aspect of the pen 1500 involves sensing the force applied by the user to the writing surface using the pen 1500. Force measurements can be used in a variety of ways. For example, force measurements can be tracked as discrete values ​​or discrete events and compared to thresholds in determining the user's intent. For example, a user may desire a force that is interpreted as a "click" when selecting an object. A user may also desire multiple force applications that are interpreted as multiple clicks. There may be times when a user holds the pen 1500 in a certain position or holds a certain part of the pen 1500 (e.g., a button or touchpad) while simultaneously clicking to perform a certain action (e.g., a 'right click'). In embodiments, force measurements can be used to track force and force trends. For example, force trends can be tracked and compared to a threshold limit. There can be one such threshold limit, multiple limits, a group of related limits, and so on. For example, when the force measurement indicates a relatively constant force that generally falls within a range of related thresholds, the microprocessor 1510 can interpret the force trend as an indication that the user desires to maintain a current writing style, writing tip type, line thickness, stroke type, and so on. In the event that the force trend appears to have intentionally moved outside of the set thresholds, the microprocessor may interpret this action as an indication that the user wishes to change the current writing style, writing tip type, line thickness, stroke type, etc. Once the microprocessor has made a determination of the user's intent, the change in the current writing style, writing tip type, line thickness, stroke type, etc. may be performed. In embodiments, the change may be indicated to the user (e.g., in a display of the HWC 102), and the user may be presented with an opportunity to accept the change.

[0310] FIG17A illustrates an embodiment of a force-sensing surface tip 1700 of a pen 1500. The force-sensing surface tip 1700 includes a surface-connected tip 1702 (e.g., a lens as described elsewhere herein) in combination with a force or pressure monitoring system 1504. As a user writes on a surface or simulates writing on a surface using the pen 1500, the force monitoring system 1504 measures the force or pressure applied by the user to the writing surface and transmits the data to a microprocessor 1510 for processing. In this configuration, the microprocessor 1510 receives the force data from the force monitoring system 1504 and processes the data to make predictions about the user's intent to apply the particular force currently being applied. In embodiments, processing may be provided at a location other than the pen (e.g., at a server in the HWC system 100, on the HWC 102). For the sake of brevity, when reference is made herein to processing information at the microprocessor 1510, processing of the information contemplates processing of the information at a location other than the pen. The microprocessor 1510 can be programmed with force threshold(s), force signature(s), a force signature library, and / or other features intended to guide the inference process in determining the user's intent based on the measured force or pressure. The microprocessor 1510 can further be programmed to make inferences based on the force measurements as to whether the user has attempted to initiate a discrete action (e.g., a user interface selection 'click') or is performing a constant action (e.g., writing within a specific writing style). The inference process is important because it enables the pen 1500 to function as an intuitive external user interface 104.

[0311] FIG17B illustrates a trend graph of force 1708 versus time 1710 with a single threshold 1718. Threshold 1718 can be set at a level indicating a discrete force application that indicates a user's desire to cause an action (e.g., select an object in a GUI). For example, event 1712 can be interpreted as a click or select command because the force quickly increases from below threshold 1718 to above threshold 1718. Event 1714 can be interpreted as a double-click because the force quickly increases above threshold 1718, drops below threshold 1718, and then repeats essentially rapidly. A user can also cause the force to exceed threshold 1718 and hold for a period of time, indicating that the user is intending to select an object in a GUI (e.g., a GUI presented in a display of HWC 102) and "hold" for further action (e.g., moving the object).

[0312] While thresholds can be used to assist in interpreting the user's intent, characteristic force event trends can also be used. Thresholds and features can be used in combination or each method can be used alone. For example, a single-click feature can be represented by a force trend feature or a set of features. For example, (multiple) single-click features may require the trend to meet the following criteria: rise time between x and y values, hold time between a and b values, and fall time between c and d values. Features can be stored for various functions (such as click, double-click, right-click, hold, move, etc.). The microprocessor 1510 can compare the real-time force or pressure tracking with features from the feature library to make a decision and issue commands to the software application executed in the GUI.

[0313] FIG17C illustrates a trend chart of force 1708 versus time 1710 with multiple thresholds 1718. As an example, the force trend is plotted on the chart using several pen force or pressure events. As noted, there are both presumably intentional events 1720 and presumably unintentional events 1722. The two thresholds 1718 in FIG17C create three force zones: lower, medium, and higher ranges. The initial portion of the trend indicates that the user is applying force in the lower range. This may mean that the user is writing with a given line thickness and has no intention of changing the thickness of their writing. The trend then shows a significant increase in force 1720 into the intermediate force range. This force change from the trend appears sudden and is sustained thereafter. The microprocessor 1510 can interpret this as an intentional change and, as a result, alter the operation according to pre-set rules (e.g., changing line width, increasing line thickness, etc.). The trend then continues with a second, apparently intentional event 1720 into the higher force range. During performance in the higher force range, the force drops below the upper threshold 1718. This may indicate an unintentional force change and the microprocessor may detect the change in range, yet not implement the change in operation being coordinated by the pen 1500. As indicated above, trend analysis may be accomplished with thresholds and / or signatures.

[0314] Generally, in this disclosure, instrument stroke parameter changes may refer to changes in line type, line thickness, tip type, stroke type, stroke width, stroke pressure, color, and other forms of writing, coloring, painting, and the like.

[0315] Another aspect of the pen 1500 relates to selecting an operating mode for the pen 1500 based on contextual information and / or selection interface(s). The pen 1500 may have several operating modes. For example, the pen 1500 may have a writing mode, in which the pen 1500's user interface(s) (e.g., writing surface end, quick launch button 1522, touch sensor 1520, motion-based gestures, etc.) are optimized or selected for tasks associated with writing. As another example, the pen 1500 may have a wand mode, in which the pen's user interface(s) are optimized or selected for tasks associated with software or device control (e.g., HWC 102, external local device, remote device 112, etc.). As another example, the pen 1500 may have a presentation mode, in which the user interface(s) are optimized or selected to assist the user in presenting a presentation (e.g., pointing with the laser pointer 1524 while controlling the presentation or an application related to the presentation using the button(s) 1522 and / or gestures). For example, the pen may have a mode that is optimized or selected for the particular device the user is trying to control. The pen 1500 may have a variety of other modes and one aspect of the present invention relates to selecting such a mode.

[0316] FIG18A illustrates automatic user interface mode(s) selection based on contextual information. The microprocessor 1510 can be programmed with IMU thresholds 1814 and 1812. Thresholds 1814 and 1812 can serve as upper and lower bounds for the angles 1804 and 1802 of the pen 1500 for certain predicted positions during certain prediction modes. For example, when the microprocessor 1510 determines that the pen 1500 is being held or otherwise positioned within an angle 1802 corresponding to a writing threshold 1814, the microprocessor 1510 can then initiate a writing mode for the pen user interface. Similarly, if the microprocessor 1510 (e.g., via the IMU 1512) determines that the pen is being held at an angle 1804 that falls within predetermined pen recognition thresholds 1812, the microprocessor can initiate a pen recognition mode for the pen user interface. Both of these examples may be referred to as context-based user interface mode selection, as the mode selection is based on contextual information (e.g., location) that is automatically collected and then used through an automatic evaluation process to automatically select the pen's user interface mode(s).

[0317] As with other examples presented herein, the microprocessor 1510 can monitor contextual trends (e.g., the angle of the pen over time) to attempt to decide whether to stay in one mode or change modes. For example, through signatures, thresholds, trend analysis, etc., the microprocessor can determine that the change is unintentional and therefore a user interface mode change is not desired.

[0318] FIG18B illustrates automatic user interface mode selection based on contextual information. In this example, pen 1500 (e.g., via its microprocessor) monitors whether the camera at writing surface end 1508 is imaging a writing surface in close proximity to the writing surface end of pen 1500. If pen 1500 determines that a writing surface is within a predetermined, relatively short distance, pen 1500 may determine that a writing surface is present 1820 and may enter the writing mode user interface mode(s). If pen 1500 does not detect a relatively close writing surface 1822, it may predict that the pen is not currently being used as a writing implement and may enter the non-writing user interface mode(s).

[0319] FIG18C illustrates manual user mode selection. The user interface mode(s) may be selected based on twisting a segment 1824 of the pen 1500 housing, clicking an end button 1828, pressing a quick launch button 1522, interacting with the touch sensor 1520, detecting a predetermined action (e.g., a click) at a pressure monitoring system, detecting a gesture (e.g., by an IMU), etc. Manual mode selection may involve selecting an item in a GUI associated with the pen 1500 (e.g., an image presented on a display of the HWC 102).

[0320] In an embodiment, a confirmation selection may be presented to the user in the event that the mode is to be changed. The presentation may be physical (e.g., a vibration in the pen 1500), through a GUI, through a light indicator, etc.

[0321] Figure 19 The figure shows a pair of pen usage scenarios 1900 and 1901. There are many usage scenarios, and for the reader's further understanding, we have combined Figure 19 A pair of usage scenarios are presented as a way of illustrating the usage scenarios. As such, the usage scenarios should be considered illustrative and non-limiting.

[0322] Usage scenario 1900 is a writing scenario in which pen 1500 is used as a writing instrument. In this example, quick launch button 122A is pressed to launch a note application 1910 in GUI 1908 of HWC 102 display 1904. Once quick launch button 122A is pressed, HWC 102 launches the note application 1910 and puts the pen into writing mode. The user uses pen 1500 to scribe a symbol 1902 on a writing surface, which is recorded and sent to HWC 102, where a symbol representing the scribe is displayed 1912 within the note application 1910.

[0323] Usage scenario 1901 is a gesture scenario in which the pen 1500 is used as a gesture capture and command device. In this example, the quick launch button 122B is activated and the pen 1500 activates the recognition pen mode so that applications launched on the HWC 102 can be controlled. Here, the user sees an application selector 1918 in the (multiple) displays of the HWC 102, in which different software applications can be selected by the user. The user gestures with the pen (e.g., slides, rotates, turns, etc.) to move the application selector 1918 from application to application. Once the correct application is identified (e.g., highlighted) in the selector 1918, the user can gesture or click or otherwise interact with the pen 1500 to select and launch the identified application. Once the application is launched, the recognition pen mode can be used, for example, to scroll, rotate, change applications, select items, initiate processes, etc.

[0324] In an embodiment, the quick launch button 122A may be activated and the HWC 102 may launch an application selector to present a collection of applications to the user. For example, the quick launch button may launch a selector showing all communication programs available for selection (e.g., SMS, Twitter, Instagram, Facebook, email, etc.) so that the user can select the program the user wants and then enter writing mode. As a further example, the launcher may present selections for various other groups that are related or categorized as commonly selected at a given time (e.g., Microsoft Office products, communication products, productivity products, note-taking products, organizational products, etc.).

[0325] Figure 20 Still another embodiment of the present invention is shown. Figure 20 A watchband clip-on controller 2000 is shown. The watchband clip-on controller may be a controller for controlling the HWC 102 or devices in the HWC system 100. The watchband clip-on controller 2000 has a fastener 2018 (e.g., a rotatable clip) that is mechanically adapted to attach to a watchband, as shown at 2004.

[0326] The watchband controller 2000 may have a quick launch interface 2008 (e.g., to launch applications and selectors as described herein), a touchpad 2014 (e.g., to be used as a touch mouse for GUI control in the HWC 102 display), and a display 2012. The clip 2018 may be adapted to fit a variety of watchbands so it can be used in conjunction with an independently selected watch for its functionality. In an embodiment, the clip is rotatable so that the user can position it in a desired manner. In an embodiment, the clip may be a flexible strip. In an embodiment, the flexible strip may be adapted to be stretched to attach to a hand, wrist, finger, device, weapon, or the like.

[0327] In an embodiment, the watch band controller can be configured as a removable and replaceable watch band. For example, the controller can be incorporated into the band at a certain width, segment spacing, etc., so that the watch band and its incorporated controller can be attached to the watch body. In an embodiment, the attachment can be mechanically adapted to be attached with a pin, and the watch band rotates on the pin. In an embodiment, the watch band controller can be electrically connected to the watch and / or the watch body to enable data to be transferred between the watch, the watch body, and / or the watch band controller.

[0328] The watch band controller may have 3-axis motion monitoring (e.g., via an IMU, accelerometer, magnetometer, gyroscope, etc.) to capture user motion. User motion may then be interpreted for gesture control.

[0329] In an embodiment, the watch band controller may include fitness sensors and a fitness computer. The sensors may track heart rate, calories burned, stride length, distance covered, etc. The data may then be compared to performance goals and / or standards for user feedback.

[0330] Another aspect of the present invention relates to visual display technology involving micro-Doppler ("mD") target tracking features ("mD signatures"). MD is a radar technology that uses a series of angle-dependent electromagnetic pulses that are broadcast into the environment and the returning pulses captured. Changes between the broadcast and returned pulses indicate changes in the shape, distance, and angular position of objects or targets in the environment. These changes provide a signal that can be used to track and identify targets using the mD signature. Each target or target type has a unique mD signature. Based on mD technology, shifts in the radar pattern can be analyzed in the time and frequency domains to derive information about the type of target present (e.g., whether a person is present), the target's motion, the target's relative angular position, and the distance to the target. By selecting a frequency for the mD pulse relative to known objects in the environment, the pulse can penetrate known objects, enabling information about the target to be collected even when the target is visually obscured by known objects. For example, a pulse frequency can be used that will penetrate concrete buildings, enabling the identification of people within the building. Multiple pulse frequencies can also be used in mD radars, enabling different types of information to be collected about objects in the environment. Furthermore, mD radar information can be combined with other information, such as captured images of the environment or range measurements, which are analyzed together to provide improved object recognition and improved target identification and tracking. In embodiments, the analysis can be performed on the HWC or the information can be transmitted to a remote network for analysis, with the results transmitted back to the HWC. Distance measurements can be provided by laser ranging, structured lighting, stereo depth maps, or sonar measurements. Images of the environment can be captured using one or more cameras capable of capturing images using visible, ultraviolet, or infrared light. The mD radar can be attached to the HWC, located nearby (e.g., in a vehicle), wirelessly associated with the HWC, or remotely located. Maps or other previously determined information about the environment can also be used to analyze the mD radar information. Embodiments of the present invention relate to visualizing mD features in a useful manner.

[0331] Figure 21 The FOV 2102 of the HWC 102 is illustrated from the wearer's perspective. As described elsewhere herein, the wearer has a see-through FOV 2102 in which the wearer views nearby surroundings, such as Figure 21 As described elsewhere herein, the wearer can also see displayed digital content presented within a portion of the FOV 2102. Figure 21The embodiment illustrated in FIG2 indicates that the wearer can see buildings and other surrounding elements in the environment, as well as digital content representing the trajectory or travel path of bullets being fired by different people in the area. The surroundings are viewed through the transparency of FOV 2102. The trajectories are presented via a digital computer display, as described elsewhere herein. In one embodiment, the presented trajectories are based on mD features that are collected in real time and transmitted to the HWC. The mD radar itself can be on or near the wearer of the HWC 102, or it can be located remotely from the wearer. In one embodiment, the mD radar scans the area, tracks and identifies targets (such as bullets), and transmits the trajectories to the HWC 102 based on their location.

[0332] exist Figure 21 In the embodiment illustrated in FIG, there are several tracks 2108 and 2104 presented to the wearer. The tracks transmitted from the mD radar can be associated with GPS locations, and the GPS locations can be associated with objects in the environment (such as people, buildings, vehicles, etc.), both in latitude and longitude perspectives and in altitude perspectives. The locations can be used as markers for the HWC so that the tracks as presented in the FOV can be associated or fixed in space relative to the markers. For example, if the friendly fire track 2108 is determined by the mD radar to have originated from the upper right window of the building on the left, as in FIG, the friendly fire track 2108 may be associated with the upper right window of the building on the left, as in FIG. Figure 21 As shown in FIG, a virtual marker can be set on or near a window. When the HWC views a window of a building, for example, through its camera or other sensor, the track can then be virtually anchored using the virtual marker on the window. Similarly, a marker can be set near the end location or other flight location of the friendly fire track 2108 (such as the upper left window of the center building on the right), as shown in FIG. Figure 21 This technique fixes the track in space so that it appears to be fixed to an environmental location independent of where the wearer is looking. So, for example, when the wearer's head turns, the track appears to be fixed to the marked location.

[0333] In embodiments, certain user locations can be known and thus identified within the FOV. For example, the shooter of friendly fire trajectory 2108 may be a known friendly soldier, and thus their location can be known. This location can be known based on their GPS location, which is based on a mobile communication system on their person, such as another HWC 102. In other embodiments, the friendly soldier may be marked by another friendly soldier. For example, if the friendly soldier's location in the environment is known through visual contact or transmitted information, the wearer of HWC 102 can use gestures or external user interface 104 to mark the location. If the friendly soldier's location is known, the origin location of the friendly fire trajectory 2108 can be color-coded or otherwise distinguished from unidentified tracks on the displayed digital content. Similarly, the enemy fire trajectory 2104 can be color-coded or otherwise distinguished on the displayed digital content. In embodiments, an additional, distinguished appearance may be provided on the displayed digital content for unknown tracks.

[0334] In addition to the contextually relevant trajectory appearance, the trajectory color or appearance can vary from origin to destination. This path appearance change can be based on an mD feature. The mD feature can indicate, for example, that the bullet is slowing down as it travels, and this slowing pattern can be reflected in the FOV 2102 as a color or pattern change. This can create an intuitive understanding of the shooter's location. For example, the origin color can be red, indicating high velocity, and can change to yellow along the trajectory's path, indicating a slowing trajectory. This pattern change can also be different for friendly forces, enemy forces, and unknown fighters. For example, enemy forces' trajectory can change from blue to green relative to friendly forces' trajectory.

[0335] Figure 21 An embodiment is shown in which a user sees the environment through the FOV and can also see color-coded trajectories that depend on bullet velocity and fighter type, where the trajectories are fixed in position in the environment independent of the wearer's perspective. Other information such as distance, range, range ring, time of day, date, engagement type (e.g., hold, stop fire, fall back, etc.) can also be displayed in the FOV.

[0336] Another aspect of the present invention relates to mD radar technology that tracks and identifies targets through other objects such as walls (generally referred to as through-wall mD) and visualization techniques related thereto. Figure 2222 illustrates a through-the-wall mD visualization technique in accordance with the principles of the present invention. As described elsewhere herein, an mD radar scanning the environment may be local or remote to the wearer of the HWC 102. The mD radar may identify a visible target (e.g., a person) 2204 and then track the target as it moves behind a wall 2208. The tracking may then be presented to the wearer of the HWC 102, such that digital content reflecting the target and its movement (even behind a wall) is presented in the FOV 2202 of the HWC 102. In an embodiment, the target may be represented in the FOV by an avatar when out of the visible field of view to provide the wearer with an image representing the target.

[0337] mD target recognition methods can identify a target based on its vibrations or other small movements. This can provide a personalized signature for the target. In the case of humans, this can lead to the personal identification of previously characterized targets. Cardio, heartbeat, lung expansion, and other small movements within the body can be unique to a person. If these attributes are pre-identified, they can be matched in real time to provide personal identification of the person within FOV 2202. A person's mD signature can be determined based on the person's location. For example, a database of personal mD signature attributes could include mD signatures for people standing, sitting, lying down, running, walking, jumping, and so on. This can improve the accuracy of personal data matching when tracking targets using mD signature technology in the field. When personally identifying a person, specific identification of the person's identity can be presented within FOV 2202. This indication can include color, shape, shading, name, or an indication of the person's type (e.g., enemy, friendly, etc.) to provide the wearer with intuitive, real-time information about the person being tracked. This can be very useful in scenarios where more than one person is in the area of ​​the tracked person. If only one person is personally identified in an area, this person or the person's avatar can be presented differently from other persons in the area.

[0338] Figure 23Illustration of an mD-scanned environment 2300. An mD radar can scan an environment in an attempt to identify objects within it. In this embodiment, the mD-scanned environment shows two vehicles 2302a and 2302b, an enemy soldier 2309, two friendly soldiers 2308a and 2308b, and a firing trajectory 2318. Each of these objects can be individually identified or identified by type. For example, vehicles 2302a and 2302b can be identified as a tank and a heavy truck using mD features. Enemy soldier 2309 can be identified as a type (e.g., enemy soldier) or more individually (e.g., by name). Friendly soldier can be identified as a type (e.g., friendly soldier) or more individually (e.g., by name). Firing trajectory 2318 can be characterized, for example, by weapon type or projectile type.

[0339] Figure 23a Two separate HWC 102 FOV display techniques are illustrated in accordance with the principles of the present invention. FOV 2312 illustrates a map view 2310 in which the mD-scanned environment is presented. Here, the wearer has a perspective of the mapped area, so they are able to understand all tracked targets in the area. This allows the wearer to traverse the area with awareness of the targets. FOV 2312 illustrates a heads-up view used to provide the wearer with an augmented reality-style view of their environment close to the wearer's.

[0340] One aspect of the present invention relates to suppressing extraneous or stray light. As discussed elsewhere herein, eye glow and face glow are two artifacts that develop from such light. Eye glow and face glow can be caused by image light escaping from the optics module. When a user is viewing a bright display image with an HWC, this escaping light is visible, particularly in dark environments. Light escaping through the front of the HWC is visible as eye glow because it is visible in the area of ​​the user's eyes. Eye glow can appear as a small version of the display image the user is viewing. Light escaping from the bottom of the HWC shines onto the user's face, cheeks, or chest, making these parts of the user appear to glow. Both eye glow and face glow can increase the user's visibility and emphasize the use of the HWC, which may be perceived negatively by the user. As such, reducing eye glow and face glow is beneficial. Suppressing extraneous or stray light is particularly important in combat scenarios (e.g., the mD trajectory presentation scenario described herein) and certain gaming scenarios.

[0341] Involved Figure 6The disclosure of shows an example in which a portion of the image light passes through combiner 602 so that the light shines onto the user's face, thereby illuminating a portion of the user's face, which is generally referred to herein as face glow. Face glow is caused by any portion of the light from the HWC that illuminates the user's face.

[0342] An example of a source of faceglow can be from a wide cone of angles associated with the image light incident on the combiner 602. The combiner can include a holographic mirror or a notch mirror, where a narrow band of high reflectivity is matched to the wavelength of the light generated by the light source. The wide cone of angles associated with the image light corresponds to the field of view provided by the HWC. Typically, the reflectivity of the holographic and notch mirrors decreases as the cone angle of the incident light increases above 8 degrees. As a result, for a 30-degree field of view, most of the image light is able to pass through the combiner and cause faceglow.

[0343] Figure 24 Illustration showing a light trap 2410 for face glow light. In this embodiment, an extension of the HWC's outer shielding lens is coated with a light-absorbing material in areas where the focused light that causes face glow is absorbed in the light trap 2410. The light-absorbing material can be black, or it can be a filter designed to absorb only specific wavelengths of light provided by the light source(s) in the HWC. Additionally, the surface of the light trap 2410 can be textured or fibrous to further improve absorption.

[0344] Figure 25 The illustration shows the optical system for a HWC that includes an external absorptive polarizer 2520 to block light from face glow. In this embodiment, the image light is polarized and, as a result, the light that causes face glow is similarly polarized. The absorptive polarizer is oriented about the transmission axis so that the light of face glow is absorbed and not transmitted. In this case, the rest of the imaging system in the HWC may not require polarized image light and the image light can be polarized at any point before the combiner. In an embodiment, the transmission axis of the absorptive polarizer 2520 is oriented vertically so that external glare from the water (S polarized light) is absorbed and, accordingly, the polarization of the image light is selected to be horizontal (S polarization). Thus, the image light that passes through the combiner 602 and then impinges on the absorptive polarizer 2520 is absorbed. Figure 25 In FIG, the absorptive polarizer 2520 is shown outside the shielding lens, alternatively, the absorptive polarizer 2520 can be positioned inside the shielding lens.

[0345] Figure 26The diagram illustrates an optical system for a HWC including a film with an absorptive notch filter 2620. In this case, the absorptive notch filter absorbs a narrow band of light selected to match the light provided by the optical system's light source. As a result, the absorptive notch filter is opaque to faceglow light and transparent to the rest of the visible spectrum, allowing the user to have a clear view of their surroundings. A triple notch filter suitable for this scenario is available from Iridian Spectral Technologies, Ottawa, ON: http: / / www.ilphotonics.com / cdv2 / Iridian-Interference%20Filters / New%20filters / Triple%20Notch%20Filter.pdf.

[0346] In embodiments, combiner 602 may include a notch mirror coating to reflect light of wavelengths in the image light, and notch filter 2620 may be selected to correspond to the wavelength of light provided by the light source and the narrow band of high reflectivity provided by the notch mirror. In this manner, image light not reflected by the notch mirror is absorbed by notch filter 2620. In embodiments of the present invention, the light source may provide one narrow band of light for monochromatic imaging or three narrow bands of light for panchromatic imaging. The notch mirror and associated notch filter will then each provide one narrow band or three narrow bands of high reflectivity and absorption, respectively.

[0347] Figure 27 Included is a micro-louvered film 2750 to block light from face glow. Micro-louvered film is sold by 3M as, for example, ALCF-P, and is typically used as a privacy filter for computers. See http: / / multimedia.3m.com / mws / mediawebserver mwsId=SSSSSuH8gc7nZxtUoYxlYeevUqel7z HvTSevTSeSSSSSS--&fn=ALCF-P ABR2 Control Film DS.pdf. Microlouver films transmit light within a narrow angle (e.g., 30 degrees from normal and absorb light beyond 30 degrees from normal). Figure 27 , the micro-louvered film 2750 is positioned such that face glow light 2758 is incident on the micro-louvered film 2750 more than 30 degrees from normal, while see-through light 2755 is incident on the micro-louvered film 2750 within 30 degrees of normal. As such, face glow light 2758 is absorbed by the micro-louvered film, and see-through light 2755 is transmitted such that the user has a bright see-through view of the surrounding environment.

[0348] We now return to the description of eye imaging techniques. Various aspects of the present invention relate to various methods for imaging the eye of a person wearing HWC 102. In embodiments, techniques for imaging the eye using optical paths involving an "off" state and a "no power" state, as described in detail below, are described. In embodiments, techniques for imaging the eye using optical configurations that do not involve reflecting an eye image off a DLP mirror are described. In embodiments, unstructured light, structured light, or controlled lighting conditions are used to predict the position of the eye based on light reflected off the front of the wearer's eye. In embodiments, a reflection of a presented digital content image is captured as it reflects off the wearer's eye, and the reflected image can be processed to determine the quality (e.g., sharpness) of the presented image. In embodiments, the image can then be adjusted (e.g., focused differently) to improve the quality of the presented image based on the image reflection.

[0349] Figure 28a 、 28b 28a and 28c show diagrams of various positions of the DLP mirrors. Figure 28a The DLP mirror is shown in an “on” state 2815. With the mirror in the “on” state 2815, illumination light 2810 is reflected along an optical axis 2820 that extends into the lower optical module 204. Figure 28b The DLP mirror is shown in an "off" state 2825. With the mirror in the "off" state 2825, illumination light 2810 is reflected along an optical axis 2830 that is substantially to the side of optical axis 2820 so that the "off" state light is directed toward a dark light trap as has been described elsewhere herein. Figure 28c The DLP mirror is shown in a third position, which occurs when no power is applied to the DLP. This "no power" state differs from the "on" and "off" states in that the mirror edge is not in contact with the substrate and is therefore less precisely positioned. Figure 28c All DLP mirrors are shown in an "unpowered" state 2835. The "unpowered" state sets the voltage to zero simultaneously by both the "open" and "closed" contacts for the DLP mirrors, and as a result, the mirrors return to an unstressed position where they are in the plane of the DLP platform, as shown in FIG. Figure 28c Although not typically done, it is also possible to apply a "no power" state to individual DLP mirrors. When the DLP mirrors are in the "no power" state, they do not contribute to the image content. Instead, as in Figure 28cAs shown in FIG, when the DLP mirror is in the "off" state, illumination light 2810 is reflected along an optical axis 2840 that is between optical axes 2820 and 2830 associated with the "on" and "off" states, respectively, and as such, this light does not contribute to the displayed image as a bright pixel or a dark pixel. However, this light can contribute scattered light into the lower optical module 204, and as a result, the displayed image contrast can be reduced or artifacts that detract from the image content can be created in the image. Therefore, in embodiments, it is generally desirable to limit the time associated with the "off" state to the time when an image is not being displayed, or to reduce the time associated with having the DLP mirror in the "off" state, so that the effects of scattered light are reduced.

[0350] Figure 29 An embodiment of the present invention is shown that can be used to display digital content images to the wearer of the HWC 102 and capture an image of the wearer's eye. In this embodiment, light from the eye 2971 passes back through the optics module in the lower module 204, the solid corrective wedge 2966, at least a portion of the light passes through the partially reflective layer 2960, the solid illumination wedge 2964, and is reflected by the plurality of DLP mirrors on the DLP 2955 in the "off" state. The reflected light then passes back through the illumination wedge 2964 and at least a portion of the light is reflected by the partially reflective layer 2960 and the light is captured by the camera 2980.

[0351] For comparison, illumination light 2973 from light source 2958 is also shown being reflected by partially reflective layer 2960. The angle of illumination light 2973 is such that, when the DLP mirror is in the "on" state, it reflects illumination light 2973 to form image light 2969, which substantially shares the same optical axis as the light from the wearer's eye 2971. In this way, an image of the wearer's eye is captured in a field of view that overlaps with the field of view for the displayed image content. In contrast, light reflected by the DLP mirror in the "off" state forms dark light 2975, which is directed substantially to the side of image light 2969 and the light from the eye 2971. Dark light 2975 is directed toward light trap 2962, which absorbs the dark light to improve the contrast of the displayed image, as described above in this specification.

[0352] In an embodiment, the partially reflective layer 2960 is a reflective polarizer. Light reflected from the eye 2971 can then be polarized (e.g., using an absorptive polarizer between the upper module 202 and the lower module 204) with respect to the polarization orientation of the reflective polarizer before entering the corrective wedge 2966 so that the light reflected from the eye 2971 can be substantially transmitted by the reflective polarizer. A quarter wave retarder layer 2957 is then included adjacent to the DLP 2955 (as previously described). Figure 3b 2964) so ​​that light reflected from the eye 2971 passes through the quarter-wave retarder layer 2957 once before being reflected by the plurality of DLP mirrors in the "unpowered" state, and then passes through a second time after being reflected. By passing through the quarter-wave retarder layer 2957 twice, the polarization state of the light from the eye 2971 is reversed so that when it is incident on the reflective polarizer, the light from the eye 2971 is then substantially reflected toward the camera 2980. By using a partially reflective layer 2960 (which is a reflective polarizer) and polarizing the light from the eye 2971 before entering the corrective wedge 2964, the losses caused by the partially reflective layer 2960 are reduced.

[0353] Figure 28c 10. The example in which the DLP mirrors are simultaneously in a "no power" state is shown. This mode of operation can be particularly useful when the HWC 102 is first worn on the wearer's head. When the HWC 102 is first worn on the wearer's head, it is not necessary to display an image yet. As a result, the DLP can be in a "no power" state with all DLP mirrors and an image of the wearer's eyes can be captured. Iris recognition technology or other eye pattern recognition technology can then be used to compare the captured image of the wearer's eyes to a database to determine, for example, the wearer's identity.

[0354] In by Figure 29In a further illustrated embodiment, all DLP mirrors are placed in a "de-energized" state for a portion of the frame time (e.g., 50% of the frame time for the displayed digital content image), and the capture of the eye image is synchronized to occur at the same time and for the same duration. By reducing the time the DLP mirrors are in the "de-energized" state, the time during which light is scattered by the DLP mirrors in the "de-energized" state is reduced, so that the wearer does not perceive a change in the quality of the displayed image. This is possible because DLP mirrors have response times on the order of microseconds, while a typical frame time for a displayed image is approximately 0.016 seconds. This method of capturing an image of the wearer's eye can be used periodically to capture repeated images of the wearer's eye. For example, the eye image can be captured for 50% of the frame time of every 10 frames displayed to the wearer. In another example, the eye image can be captured for 10% of the frame time of every frame displayed to the wearer.

[0355] Alternatively, the "off" state can be applied to a subset of the DLP mirrors (e.g., 10% of the DLP mirrors) while another subset is busy generating image light for the content to be displayed. This enables the capture of eye image(s) during the display of digital content to the wearer. The DLP mirrors used for eye imaging can, for example, be randomly distributed across the area of ​​the DLP to minimize the impact on the quality of the digital content being displayed to the wearer. For example, to improve the displayed image perceived by the wearer, the individual DLP mirrors that enter the "off" state for capturing each eye image can be varied over time, such as in a random pattern. In yet another embodiment, the DLP mirrors that enter the "off" state for eye imaging can be coordinated with the digital content in such a way that the "off" mirrors are removed from portions of the image where less resolution is required.

[0356] In such a Figure 9 and 29 In the embodiment of the invention illustrated in , the reflective surface provided by the DLP mirrors in both cases does not preserve the wavefront of the light from the wearer's eye, so that the image quality of the captured image of the eye is somewhat limited. This may still be useful in some embodiments, but it is somewhat limited. This is because the DLP mirrors are not constrained to be in the same plane. Figure 9 In the embodiment illustrated in , the DLP mirrors are tilted so that they form rows of DLP mirrors that share a common plane. Figure 29 In the embodiment illustrated in , the individual DLP mirrors are not positioned exactly in the same plane because they are not in contact with the substrate. Figure 29Examples of advantages of associated embodiments are: first, the camera 2980 can be positioned between the DLP 2955 and the illumination light source 2958 to provide a more compact upper module 202. Second, the polarization state of the light reflected from the eye 2971 can be the same as the polarization state of the image light 2969, so that the optical paths of the image light and the light reflected from the eye can be the same in the lower module 204.

[0357] Figure 30 A diagram illustrating an embodiment for displaying an image to the wearer and simultaneously capturing an image of the wearer's eye is shown, wherein light from the eye 2971 is reflected by a partially reflective layer 2960 toward a camera 3080. The partially reflective layer 2960 can be an optically flat layer so that the wavefront of the light from the eye 2971 is preserved, and as a result, a higher quality image of the wearer's eye can be captured. Furthermore, since the DLP 2955 is not included in the optical path for the light from the eye 2971, and Figure 30 The eye imaging process shown in does not interfere with the displayed image, so the image of the wearer's eye can be captured independently from the displayed image (e.g., independent of the timing used for the image light, the impact on resolution, or the pixel count).

[0358] exist Figure 30 In the embodiment illustrated in FIG, partially reflective layer 2960 is a reflective polarizer, illumination light 2973 is polarized, light from the eye 2971 is polarized, and camera 3080 is positioned behind polarizer 3085. The polarization axes of illumination light 2973 and light from the eye are oriented perpendicular to the transmission axis of the reflective polarizer, so that they are both substantially reflected by the reflective polarizer. Illumination light 2973 passes through quarter-wave layer 2957 before being reflected by the DLP mirror in DLP 2955. The reflected light passes through quarter-wave layer 2957, causing the polarization states of image light 2969 and dark light 2975 to be inverted compared to illumination light 2973. As such, image light 2969 and dark light 2975 are substantially transmitted by the reflective polarizer. The DLP mirror, in its "on" state, provides image light 2969 along an optical axis extending into lower optical module 204 to display an image to the wearer. At the same time, the DLP mirror in the "off" state provides dark light 2975 along an optical axis that extends to the side of the upper optics module 202. In the region where the dark light 2975 is incident on the corrective wedge 2966 on the side of the upper optics module 202, the absorptive polarizer 3085 is positioned with its transmission axis perpendicular to the polarization axis of the dark light and parallel to the polarization axis of the light from the eye, so that the dark light 2975 is absorbed and light from the eye 2971 is transmitted to the camera 3080.

[0359] Figure 31A diagram showing another embodiment of a system for displaying an image and simultaneously capturing an image of a wearer's eye is shown, similar to the one in FIG. Figure 30 The diagram is similar to that shown in . Figure 31 The difference in the system shown in is that the light from the eye 2971 undergoes multiple reflections before being captured by the camera 3180. To enable multiple reflections, a mirror 3187 is provided behind the absorptive polarizer 3185. Thus, the light from the eye 2971 is polarized before entering the corrective wedge 2966 about a polarization axis that is perpendicular to the transmission axis of the reflective polarizer including the partially reflective layer 2960. In this way, the light from the eye 2971 is reflected a first time by the reflective polarizer, a second time by the mirror 3187 and a third time by the reflective polarizer before being captured by the camera 3180. Although the light from the eye 2971 passes through the absorptive polarizer 3185 twice, it is substantially transmitted by the absorptive polarizer 3185 because the polarization axis of the light from the eye 2971 is oriented parallel to the polarization axis of the light from the eye 2971. In combination with Figure 30 The system described is the same as Figure 31 The system shown in FIG includes an optically flat partially reflective layer 2960 that preserves the wavefront of light from the eye 2971 so that a higher quality image of the wearer's eye can be captured. Furthermore, since the DLP 2955 is not included in the optical path for light reflected from the eye 2971, and Figure 31 The eye imaging process shown in does not interfere with the displayed image, so the image of the wearer's eye can be captured independently from the displayed image.

[0360] Figure 32A diagram shows a system for displaying an image and simultaneously capturing an image of the wearer's eye, including a beam splitter plate 3212 comprising a reflective polarizer held in air between a light source 2958, a DLP 2955, and a camera 3280. Both illumination light 2973 and light from the eye 2971 are polarized about a polarization axis perpendicular to the transmission axis of the reflective polarizer. As a result, both illumination light 2973 and light from the eye 2971 are substantially reflected by the reflective polarizer. Illumination light 2873 is reflected by the reflective polarizer toward the DLP 2955 and is split into image light 2969 and dark light 3275, depending on whether the respective DLP mirrors are in the "on" or "off" state. By passing through the quarter-wave layer 2957 twice, the polarization state of illumination light 2973 is inverted compared to the polarization states of image light 2969 and dark light 3275. As a result, image light 2969 and dark light 3275 are then substantially transmitted by the reflective polarizer. The absorptive polarizer 3285 at the side of the beam splitter plate 3212 has a transmission axis perpendicular to the polarization axis of the dark light 3275 and parallel to the polarization axis of the light from the eye 2971, so that the dark light 3275 is absorbed and the light from the eye 2971 is transmitted to the camera 3280. Figure 30 In the system shown, Figure 31 The system shown in FIG includes an optically flat beam splitter plate 3212 that preserves the wavefront of the light from the eye 2971 so that a higher quality image of the wearer's eye can be captured. Furthermore, since the DLP 2955 is not included in the optical path for the light from the eye 2971, and Figure 31 The eye imaging process shown in does not interfere with the displayed image, so the image of the wearer's eye can be captured independently from the displayed image.

[0361] The polarization state of the light from the eye 2971 needs to be opposite to the polarization state of the image light 2969 (as in Figure 30 、 31 32) need to be used with a lower module that includes a combiner that will reflect both polarization states. As such, these upper modules 202 are best suited for use with lower modules 204 that include a combiner that is reflective regardless of polarization state, examples of which are shown in FIG. Figure 6 、 8a , 8b, 8c and 24-27.

[0362] exist Figure 33In yet another embodiment shown in FIG, the beam splitter plate 3222 consists of a reflective polarizer on the side facing the illumination light 2973 and a short-pass dichroic mirror on the side facing the light from the eye 3271 and the camera 3280. An absorptive surface 3295 is provided to capture the dark light 3275 and the camera 3280 is positioned in an opening of the absorptive surface 3295. In this way, it is possible to Figure 32 The system works in the case of unpolarized light from the eye 3271.

[0363] In embodiments involving capturing an image of the wearer's eye, the light used to illuminate the wearer's eye can be provided by several different sources, including: light from the displayed image (i.e., image light); light from the environment passing through a combiner or other optical devices; light provided by a dedicated eye light, etc. Figure 34 and 34a A diagram of dedicated eye illumination light 3420 is shown. Figure 34 An illustration from a side view is shown in which a dedicated eye illumination light 3420 is positioned at a corner of the combiner 3410 so that it does not interfere with the image light 3415. When the wearer is viewing a displayed image provided by the image light 3415, the dedicated eye illumination light 3420 is directed so that the eye illumination light 3425 illuminates the eye box 3427 where the eye 3430 is located. Figure 34a A diagram is shown showing the perspective from the wearer's eye to illustrate how the dedicated eye illumination light 3420 can be positioned at the corner of the combiner 3410. While the dedicated eye illumination light 3420 is shown at the upper left corner of the combiner 3410, other locations along one of the edges of the combiner 3410 or other optical or mechanical components are possible. In other embodiments, more than one dedicated eye light 3420 with different locations can be used. In an embodiment, the dedicated eye light 3420 is infrared light that is invisible to the wearer (e.g., 800 nm) so that the eye illumination light 3425 does not interfere with the displayed image perceived by the wearer.

[0364] Figure 35 A series of diagrams of captured eye images showing eye glint (i.e., light reflected off the front of the eye) produced by a dedicated eye light are shown. In this embodiment of the invention, the captured images of the wearer's eye are analyzed to determine the relative positions of the iris 3550, pupil, or other portion of the eye and the eye glint 3560. The eye glint is a reflected image of the dedicated eye light 3420 when the dedicated light is used. Figure 35The relative positions of the eye glint 3560 and iris 3550 for various eye positions are illustrated. By providing the dedicated eye light 3420 in a fixed position, combined with the fact that the human eye is essentially spherical, or at least a reliably repeatable shape, the eye glint provides a fixed reference point within the displayed image or within a see-through view of the surrounding environment, with which the determined position of the iris can be compared to determine where the wearer is looking. By positioning the dedicated eye light 3420 at a corner of the combiner 3410, the eye glint 3560 is formed away from the iris 3550 in the captured image. As a result, the positions of the iris and eye glint can be determined more easily and accurately during analysis of the captured image because they do not interfere with each other. In further embodiments, the combiner includes an associated cutoff filter that prevents infrared light from the environment from entering the HWC, and the camera is an infrared camera, so that the eye glint is provided only by light from the dedicated eye light. For example, the combiner can include a low-pass filter that passes visible light while absorbing infrared light, and the camera can include a high-pass filter that absorbs visible light while passing infrared light.

[0365] In an embodiment of the eye imaging system, the lens for the camera is designed to account for the optics associated with the upper module 202 and the lower module 204. This is achieved by designing the camera to include optics in the upper module 202 as well as optics in the lower module 204, so as to produce a high MTF image of the wearer's eye at the camera's image sensor. In another embodiment, the camera lens is provided with a large depth of field to eliminate the need to focus the camera to achieve a sharp image of the eye. A large depth of field is typically provided by high f / # lenses (e.g., f / # > 5). In this case, the reduced light collection associated with high f / # lenses is compensated for by including a dedicated eye light, enabling a bright image of the eye to be captured. Furthermore, the brightness of the dedicated eye light can be modulated and synchronized with the capture of the eye image, so that the dedicated eye light has a reduced duty cycle and reduces the brightness of infrared light on the wearer's eye.

[0366] In a further embodiment, Figure 36aAn illustration of an eye image being used to identify the wearer of an HWC is shown. In this case, an image of the wearer's eye 3611 is captured and analyzed for a pattern of recognizable features 3612. The pattern is then compared to a database of eye images to determine the wearer's identity. After the wearer's identity has been verified, the operating mode of the HWC and the types of images, applications, and information to be displayed can be adjusted and controlled consistent with the determined wearer's identity. Examples of adjustments to the operating mode based on who the wearer is determined to be include: making different operating modes or feature sets available, shutting down or sending messages to an external network, allowing guest features and applications to run, etc.

[0367] In another embodiment using eye imaging, the sharpness of a displayed image is determined based on eye glint generated by reflection of the displayed image from the surface of the wearer's eye. By capturing an image of the wearer's eye 3611, eye glint 3622 (a small version of the displayed image) can be captured and analyzed for sharpness. If the displayed image is determined to be unsharp, automatic adjustment of the focus of the HWC optics can be performed to improve sharpness. This ability to measure the sharpness of the displayed image at the surface of the wearer's eye can provide a very accurate measure of image quality. The ability to measure and automatically adjust the focus of the displayed image is particularly useful in augmented reality imaging, where the focus distance of the displayed image can change in response to changes in the environment or changes in the wearer's method of use.

[0368] One aspect of the present invention relates to controlling the HWC 102 by interpreting eye images. In an embodiment, eye imaging technology (such as those described herein) is used to capture an eye image or series of eye images for processing. The image(s) may be processed to determine a user-intended action, a predetermined reaction by the HWC, or other action. For example, the image may be interpreted as a positive user-controlled action for an application on the HWC 102. Alternatively, the image may cause the HWC 102 to react in a predetermined manner, such as to ensure that the HWC 102 always operates safely, intuitively, and the like.

[0369] Figure 373708 . Figure 37 3708 ), a virtual target line can be established to project what the wearer may be looking towards or viewing in the environment. The virtual target line can be used in conjunction with images captured by a camera on HWC 102 that images the surrounding environment in front of the wearer. In embodiments, the field of view of the camera capturing the surrounding environment matches or can be matched (e.g., digitally) to FOV 3708 to make comparisons clearer. For example, if the camera captures images of the surroundings at an angle that matches FOV 3708, the virtual line can be processed (e.g., in 2D or 3D, depending on the camera's image capabilities and / or image processing) by projecting what surrounding objects align with the virtual target line. If multiple objects are located along the virtual target line, a focal plane can be established for each of the objects, allowing digital content to be placed in an area within FOV 3708 that aligns with the virtual target line and falls within the focal plane of the object of interest. The user can then see the digital content when focusing on objects in the environment that are in the same focal plane. In an embodiment, objects aligned with the virtual target line may be established by comparison with mapping information of surrounding objects.

[0370] In embodiments, digital content aligned with a virtual target line may not be displayed in the FOV until the eye position is correctly located. This may be a predetermined process. For example, the system may be configured so that specific segments of digital content (e.g., advertisements, directions, object information, etc.) appear when the wearer is viewing certain objects in the environment. Virtual target lines may be developed that virtually connect the wearer's eyes to object(s) in the environment (e.g., buildings, parts of buildings, signs on buildings, GPS locations, etc.), and the virtual target lines may be continuously updated based on the wearer's position and viewing direction (e.g., as determined by GPS, an electronic compass, an IMU, etc.) and the location of the objects. When the virtual target line indicates that the wearer's pupils are substantially aligned with, or are about to align with, the virtual target line, the digital content may be displayed in the FOV 3704.

[0371] In embodiments, time spent looking along a virtual target line and / or a particular portion of FOV 3708 may indicate the wearer's interest in an object in the environment and / or the digital content being displayed. If no digital content is displayed for a predetermined period of time spent looking in one direction, digital content may be presented in the area of ​​FOV 3708. Time spent looking at an object may be interpreted as, for example, a command to display information about the object. In other embodiments, content not related to the object may be presented, as an indication of the person's relative inactivity. In embodiments, digital content may be positioned close to the virtual target line, but not in line with it, so that the wearer's view of the surroundings is not obstructed, but the information enhances the wearer's view of the surroundings. In embodiments, time spent looking along a target line in the direction of displayed digital content may be an indicator of interest in the digital content. This may be used as a conversion event in advertising. For example, advertisers may pay more for ad placement if the wearer of HWC 102 views a displayed advertisement for a certain period of time. As such, in embodiments, the time spent viewing an advertisement, as assessed by comparing eye position to content placement, a target line, or other appropriate location, may be used to determine a conversion rate or other compensation amount due for a presentation.

[0372] One aspect of the present invention relates to removing content from the FOV of the HWC 102 when the wearer of the HWC 102 clearly wants to see the surrounding environment clearly. Figure 38 The illustration shows a scenario where an eye image suggests that the eye has moved or is moving quickly, and therefore digital content 3804 in FOV 3808 is removed from FOV 3808. In this example, the wearer may look to one side quickly, indicating that there is something in the environment to that side that has caught the wearer's attention. Eye movement 3802 may be captured by eye imaging technology (e.g., as described herein), and if the movement matches a predetermined movement (e.g., speed, rate, pattern, etc.), the content may be removed from view. In embodiments, eye movement is used as one input, and HWC movement indicated by other sensors (e.g., an IMU in the HWC) may be used as another indication. These various sensor movements can be used together to map events that should cause a change in the content being displayed in the FOV.

[0373] Another aspect of the present invention relates to determining a focal plane based on the wearer's eye convergence. Eyes generally converge slightly, and converge more when a person focuses very closely on an object. This is generally referred to as convergence. In embodiments, convergence is calibrated for the wearer. That is, the wearer can be guided through a focal plane exercise to determine how much the wearer's eyes converge at various focal planes and viewing angles. This convergence information can then be stored in a database for later reference. In embodiments, a general table can be used without a calibration step, or if the person skips the calibration step. Both eyes can then be periodically imaged to determine convergence, in an attempt to understand what focal plane the wearer is focusing on. In embodiments, the eyes can be imaged to determine a virtual target line, and then the eye convergence can be determined to establish the wearer's focus, and digital content can be displayed or altered based on this.

[0374] Figure 39 The illustration shows a scenario where digital content is moved 3902 within one or both of the FOVs 3908 and 3910 to align with the convergence of the eyes as determined by pupil movement 3904. In an embodiment, by moving the digital content to maintain alignment, the overlapping nature of the content is maintained so that the object appears appropriate to the wearer. This can be important in scenarios where 3D content is being displayed.

[0375] One aspect of the present invention relates to controlling the HWC 102 based on events detected through eye imaging. For example, a wearer moving their eyes in a certain pattern, blinking, flickering, etc. can control an application of the HWC 102. Eye imaging (e.g., as described herein) can be used to monitor the wearer's eye(s) and initiate application control commands upon detecting a predetermined pattern.

[0376] One aspect of the present invention relates to monitoring the health of a person wearing HWC 102 by monitoring the wearer's eye(s). Calibration can be performed so that the normal performance of the wearer's eyes under various conditions (e.g., lighting conditions, image light conditions, etc.) can be documented. The wearer's eyes can then be monitored for changes in their performance through eye imaging (e.g., as described herein). Changes in performance can indicate health concerns (e.g., concussion, brain injury, stroke, blood loss, etc.). If data indicating a change or event is detected, it can be transmitted from HWC 102.

[0377] One aspect of the present invention relates to verifying the security and access of certain computer assets (e.g., the HWC itself and associated computer systems) through eye images. As discussed elsewhere herein, eye images can be compared with eye images of known persons to confirm the identity of the person. Eye images can also be used to confirm the identity of the person wearing the HWC 102 before allowing the person to connect or share files, streams, information, etc.

[0378] Various use cases for eye imaging are possible based on the techniques described herein. One aspect of the present invention relates to the timing of eye image capture. The timing of eye image capture and the frequency of multiple eye image capture can vary depending on the use case for the information collected from the eye images. For example, capturing eye images to identify the user of an HWC may only be required when the HWC is powered on or when the HWC determines that the HWC has been placed on the wearer's head to control the security of the HWC and associated information displayed to the user. The position of the earhorn (or other portion of the HWC), stress, movement pattern, or orientation of the HWC can be used to determine that a person has placed the HWC on their head with the intent to use it. These same parameters can be monitored in an effort to understand when the HWC has been removed from the user's head. This can enable scenarios where capturing eye images to identify the wearer can only be completed when a change in wearing conditions is identified. In a comparative example, capturing eye images to monitor the wearer's health may require capturing images periodically (e.g., every few seconds, every few minutes, every few hours, every few days, etc.). For example, when images are being used to monitor the wearer's health, eye images may be captured at minute intervals when detected movement indicates the wearer is exercising. In another comparative example, capturing eye images to monitor the wearer's health for long-term effects only requires capturing eye images monthly. Embodiments of the present invention involve selecting the rate and timing of eye image capture corresponding to the selected usage scenario associated with the eye images. These selections can be made automatically, as in the exercise example above where movement indicates exercise, or they can be set manually. In further embodiments, the selection of the rate and timing of eye image capture is automatically adjusted depending on the mode of operation of the HWC. The selection of the rate and timing of eye image capture can further be selected in accordance with input characteristics associated with the wearer (including age and health status) or the wearer's sensed physical condition (including heart rate, blood chemistry, and eye blink rate).

[0379] Figure 40Illustrated is an embodiment in which digital content presented in a see-through FOV is positioned based on the speed at which the wearer is moving. When the person is not moving, as measured by sensor(s) in the HWC 102 (e.g., IMU, GPS based tracking, etc.), the digital content may be presented at the stationary person content position 4004. The content position 4004 is indicated as being in the middle of the see-through FOV; however, this is intended to illustrate that the digital content is positioned within the see-through FOV where it is generally desirable to know that the wearer is not moving, and as such the wearer's surrounding see-through view can be obstructed to some extent. Thus, the stationary person content position or neutral position may not be centered in the see-through FOV; it may be positioned somewhere in the see-through FOV that is deemed desirable and sensor feedback may cause the digital content to be shifted from the neutral position. Movement of the digital content for rapidly moving persons is also depicted in FIG. Figure 40 4008 as the person turns their head to the side, and then moves back as the person turns their head back in. For a slowly moving person, the head movement may be more complex and as such the movement of digital content outside the see-through FOV can follow a path such as that shown by content position 4010.

[0380] In embodiments, the sensor that assesses the wearer's movement may be a GPS sensor, an IMU, an accelerometer, or the like. The content position may shift from a neutral position to a position toward a side edge of the field of view as forward motion increases. The content position may shift from a neutral position to a position toward the top or bottom edge of the field of view as forward motion increases. The content position may shift based on a threshold velocity of the assessed motion. The content position may shift linearly based on the velocity of the forward motion. The content position may shift non-linearly based on the velocity of the forward motion. The content position may shift outside the field of view. In embodiments, if the velocity of motion exceeds a predetermined threshold, the content is no longer displayed and is redisplayed once the forward motion slows.

[0381] In embodiments, content position may generally be referred to as shifting; it should be understood that the term shifting encompasses processes where movement from one location to another, either within a see-through FOV or outside the FOV, is visible to the wearer (e.g., the content appears to move slowly or quickly and the user perceives the movement itself), or where movement from one location to another may not be visible to the wearer (e.g., the content appears to jump in a discontinuous manner or the content disappears and then reappears in a new location).

[0382] Another aspect of the present invention involves removing content from the field of view or shifting it to a position within the field of view, which enhances the wearer's view of their surroundings when a sensor causes an alarm command to be issued. In embodiments, the alarm can be triggered by a sensor or combination of sensors sensing a condition above a threshold. For example, if an audio sensor detects a loud sound of a certain pitch, content in the field of view can be removed or shifted to provide the wearer with a clearer view of their surroundings. In addition to shifting content, in embodiments, an indication of why the content has been shifted can be presented in the field of view or provided to the wearer via audio feedback. For example, if a carbon monoxide sensor detects high concentrations in an area, content in the field of view can be shifted to the side of the field of view or removed from the field of view, and an indication of the presence of high carbon monoxide concentrations in the area can be provided to the wearer. This new information, when presented in the field of view, can be similarly shifted within or outside the field of view depending on the wearer's movement speed.

[0383] Figure 41 41. The illustration shows how content may be shifted from a neutral position 4104 to an alert position 4108. In this embodiment, the content is shifted outside of the see-through FOV 4102. In other embodiments, the content may be shifted as described herein.

[0384] Another aspect of the present invention involves identifying various vectors and headings associated with the HWC 102, along with sensor inputs, to determine how to position content within the field of view. In an embodiment, the wearer's speed of movement is detected and used as input for content position, and based on the speed, content can be positioned relative to the movement vector or heading (i.e., the direction of movement) or the field of view vector or heading (i.e., the direction of the wearer's field of view). For example, if the wearer is moving very quickly, content can be positioned within the field of view relative to the movement vector, as the wearer will only be looking to their side periodically and for short periods of time. As another example, if the wearer is moving slowly, content can be positioned relative to the field of view heading, as the user may be more freely shifting their view from side to side.

[0385] Figure 42The illustration shows two examples in which motion vectors can affect content positioning. Motion vector A 4202 is shorter than motion vector B 4210, indicating that the person associated with motion vector A 4202 is moving at a lower forward speed and / or acceleration than the person associated with motion vector B 4210. Each person is also indicated as having a view vector or heading 4208 and 4212. View vectors A 4208 and B 4210 are identical from a relative perspective. The white area within the black triangle in front of each person indicates how much time each person may spend looking in a direction not aligned with the motion vector. Time spent looking away from angle A 4204 is indicated as being greater than time spent looking away from angle B 4214. This is likely because motion vector speed A is lower than motion vector speed B. Typically, the faster a person moves forward, the more likely they are to look in the forward direction. FOVs A 4218 and B 4222 illustrate how content can be aligned based on motion vectors 4202 and 4210 and field of view vectors 4208 and 4212. FOV A 4218 is illustrated as presenting content aligned with field of view vector 4220. This may be due to the lower speed of motion vector A 4202. This may also be due to the prediction that viewing from angle A will take a longer amount of time 4204. FOV B 4222 is illustrated as presenting content aligned with motion vector 4224. This may be due to the higher speed of motion vector B 4210. This may also be due to the prediction that viewing from angle B will take a shorter amount of time 4214.

[0386] Another aspect of the present invention relates to reducing the rate at which the position of content within the field of view changes. Figure 43 , the field of view vector may undergo a rapid change 4304. This rapid change may be an isolated event or it may be made at or near a time when other field of view vector changes are occurring. The wearer's head may be turning back and forth for some reason. In embodiments, rapid, continuous changes in the field of view vector may cause a reduced rate 4308 of position change of content within the FOV 4302. For example, content may be positioned with respect to the field of view vector, as described herein, and rapid changes in the field of view vector may typically cause rapid content position changes; however, because the field of view vector is continuously changing, the rate of position change with respect to the field of view vector may be reduced, slowed, or stopped. The rate of position change may be modified based on the rate of change of the field of view vector, an average of field of view vector changes, or otherwise modified.

[0387] Another aspect of the invention relates to presenting more than one content simultaneously in the field of view of the see-through optical system of the HWC 102 and positioning one content with a viewing orientation and one content with a moving orientation. Figure 44Two FOVAs A 4414 and B 4420 are shown, corresponding to the two identified field of view vectors A 4402 and B 4404, respectively. Figure 44 Also illustrated are objects in the environment 4408 at positions relative to view vectors A 4402 and B 4404. When a person is looking along view vector A 4402, environmental object 4408 can be seen at position 4412 through view field A 4414. As shown, content aligned with the view vector is presented as TEXT close to environmental object 4412. Simultaneously, other content 4418 is presented in view field A 4414 at positions aligned with the motion vector. As the speed of movement increases, content 4418 may shift as described herein. When the person's view vector is view vector B 4404, environmental object 4408 is not visible in view field B 4420. As a result, view-aligned content 4410 is not presented in view field B 4420; however, motion-aligned content 4418 is presented, still depending on the speed of movement.

[0388] Figure 45 An example data set is shown for a person moving through an environment on a path starting at a moving heading of 0 degrees and ending at a moving heading of 114 degrees, during which time the moving speed varies from 0 m / sec to 20 m / sec. It can be seen that the view heading changes on each side of the moving heading as the person moves from side to side. A large change in view heading occurs when the moving speed is 0 m / sec while the person is standing still, followed by a step change in the moving heading.

[0389] Embodiments provide a process for determining a display orientation that takes into account the manner in which a user moves through an environment and provides a display orientation that makes it easy for the user to find displayed information, while also providing an unobstructed perspective view of the environment in response to different movements, speeds of movement, or different types of information being displayed.

[0390] Figure 46 This diagram illustrates the see-through view that can be seen when using a HWC, where information is overlaid onto a see-through view of the environment. Trees and buildings are actually in the environment, and text is displayed on the see-through display so that it appears to be overlaid on the environment. In addition to textual information (such as instructions and weather information), some augmented reality information is shown, which relates to nearby objects in the environment.

[0391] In an embodiment, the display heading is determined based on the speed of movement. At low speeds, the display heading may be substantially the same as the field of view heading, while at high speeds, the display heading may be substantially the same as the direction of movement. In an embodiment, as long as the user remains stationary, the display information is presented directly in front of the user and the HMD. However, as the speed of movement increases (e.g., exceeds a threshold or continuously increases, etc.), the display heading becomes substantially the same as the direction of movement, regardless of the direction the user is looking, so that when the user is looking in the direction of movement, the display information is directly in front of the user and the HMD and is not visible when the user is looking to the side.

[0392] Rapid changes in view heading can be followed by slower changes in display heading to provide a reduced response to head rotation. Alternatively, the display heading can be substantially the time-averaged view heading, so that the display information is presented at an heading that is in the middle of a series of view headings over a period of time. In this embodiment, if the user stops moving their head, the display heading gradually becomes the same as the view heading, and the display information moves to a display field of view in front of the user and the HMD. In embodiments, this process delays the effect of the time-averaged view heading on the display heading when there is a high rate of view heading change. In this way, the effect of rapid head movement on the display heading is reduced, and the positioning of the display information within the display field of view is laterally stabilized.

[0393] In another embodiment, the display heading is determined based on the speed of movement, wherein at high speeds, the display heading is substantially the same as the movement heading. At medium speeds, the display heading is substantially the same as the time-averaged field of view heading, so that rapid head turns are damped out and the display heading is in the middle of back-and-forth head movements.

[0394] In yet another embodiment, the type of information being displayed is included in determining how information should be displayed. Augmented reality information connected to objects in the environment is given a display orientation that substantially matches the field of view orientation. In this way, as the user turns their head, the augmented reality information comes into view associated with objects in the see-through view of the environment. Simultaneously, information not connected to objects in the environment is given a display orientation determined based on the type of movement and the speed of the movement as previously described in this specification.

[0395] In yet another embodiment, when the movement speed is determined to be above a threshold, the displayed information moves downward in the display field of view so that an upper portion of the display field of view has less information or no information displayed to provide the user with an unobstructed see-through view of the environment.

[0396] Figure 47 and 48A diagram showing a see-through view including overlaid display information. Figure 47 Shown immediately from and in Figure 46 The perspective view shown in FIG is associated with a perspective view heading after a rapid change in the perspective view heading, wherein the change in the perspective view heading comes from a head turn. In this case, the display heading is delayed. Figure 48 It shows how the display heading catches up with the field of view heading at a later time. The augmented reality information remains in a position within the display field of view where an association with objects in the environment can be easily made by the user.

[0397] Figure 49 A diagram showing an example of a see-through view including overlaid display information that has been shifted downward in the display field of view to provide an unobstructed see-through view in the upper portion of the see-through view. At the same time, augmented reality labels have been maintained in position within the display field of view so they can be easily associated with objects in the environment.

[0398] In a further embodiment, in an operational mode (such as when a user is moving around an environment), digital content is presented to the side of the user's see-through FOV so that the user can view the digital content simply by turning their head. In this case, when the user is looking straight ahead, such as when the movement orientation matches the field of view orientation, the see-through FOV does not include the digital content. The user then accesses the digital content by turning their head to the side, whereupon the digital content moves laterally into the user's see-through FOV. In another embodiment, the digital content is ready for presentation and will be presented if an indication for digital content presentation is received. For example, the information may be ready for presentation and, if a predetermined position or field of view orientation of the HWC 102 is reached, the content may then be presented. The wearer may look to the side and the content may be presented. In another embodiment, the user may cause the content to move into an area in the field of view by looking in one direction for a predetermined period of time, blinking, winking, or displaying some other pattern that can be captured by eye imaging technology (e.g., as described elsewhere herein).

[0399] In yet another embodiment, an operating mode is provided in which a user can define a viewing orientation in which the associated see-through FOV includes or excludes digital content. In one example, this operating mode can be used in an office environment, where digital content is provided within the FOV when the user is looking at a wall, while the FOV is unobstructed by digital content when the user is looking toward a hallway. In another example, digital content is provided within the FOV when the user is looking horizontally, but is removed from the FOV when the user is looking downward (e.g., toward a desk or cell phone).

[0400] Another aspect of the present invention relates to the collection and use of eye position and field of view heading information. Head-mounted computing using motion heading, field of view heading, and / or eye position predictions (sometimes referred to herein as "eye heading") can be used to identify what the wearer of the HWC 102 is clearly interested in, and this information can be captured and used. In embodiments, the information can be characterized as it is viewed, as it clearly relates to what the wearer is viewing. The viewing information can be used to develop a personal profile for the wearer, which can indicate what the wearer tends to view. Viewing information from several or even many HWCs 102 can be captured so that group or crowd viewing tendencies can be established. For example, if the motion heading and field of view heading are known, a prediction of what the wearer is viewing can be made and used to generate part of a crowd profile or a personal profile. In another embodiment, if the eye heading and position, field of view heading, and / or motion heading are known, a prediction of what is being viewed can be made. Prediction can involve understanding what is proximal to the wearer, and this can be achieved by establishing the wearer's location (e.g., via GPS or other positioning technology) and establishing what mapped objects are known in the area. Prediction may involve interpreting images captured by other sensors or cameras associated with the HWC 102. For example, if a camera captures an image of a sign, and the camera is aligned with the field of view, prediction may involve assessing the likelihood that the wearer is looking at the sign. Prediction may involve capturing images or other sensory information and then performing object recognition analysis to determine what is being viewed. For example, the wearer may be walking down the street, and a camera in the HWC 102 may capture an image, and a processor onboard or remote from the HWC 102 may recognize a face, object, sign, image, etc. and may determine that the wearer may have been looking at it or looking toward it.

[0401] Figure 50 A cross-section of an eyeball of a wearer of a HWC is shown with a focal point that can be associated with the eye imaging system of the present invention. Eyeball 5010 includes an iris 5012 and a retina 5014. Because the eye imaging system of the present invention provides coaxial eye imaging with the display system, it is possible to capture an image of the eye from a perspective directly in front of the eye and in line with where the wearer is looking. In embodiments of the present invention, the eye imaging system can focus on the wearer's iris 5012 and / or retina 5014 to capture an image of the interior portion of the eye, including the retina 5014, or the outer surface of the iris 5012. Figure 50Light rays 5020 and 5025 are shown associated with capturing an image of the iris 5012 or retina 5014, respectively, with optics associated with the eye imaging system focused at the iris 5012 or retina 5014, respectively. Illumination light can also be provided in the eye imaging system to illuminate the iris 5012 or retina 5014. Figure 51 An illustration of an eye including an iris 5130 and a sclera 5125 is shown. In embodiments, an eye imaging system can be used to capture an image including the iris 5130 and a portion of the sclera 5125. The image can then be analyzed to determine the color, shape, and pattern associated with the user. In further embodiments, the focus of the eye imaging system is adjusted to enable the capture of an image of the iris 5012 or the retina 5014. The illumination light can also be adjusted to illuminate the iris 5012 or the retina 5014 through the pupil of the eye. The illumination light can be visible light to enable the capture of the color of the retina 5014 or iris 5012, or it can be ultraviolet light (e.g., 340 nm), near infrared light (e.g., 850 nm), or mid-wave infrared light (e.g., 5000 nm) to enable the capture of hyperspectral characteristics of the eye.

[0402] Figure 53The display system including the eye imaging system is shown. The display system includes a polarized light source 2958, a DLP 2955, a quarter-wave film 2957, and a beam splitter 5345. The eye imaging system includes a camera 3280, an illumination light 5355, and a beam splitter 5345. The beam splitter 5345 can be a reflective polarizer on the side facing the polarized light source 2958 and a hot mirror on the side facing the camera 3280. The hot mirror reflects infrared light (e.g., wavelengths of 700 to 2000 nm) and transmits visible light (e.g., wavelengths of 400 to 670 nm). The beam splitter 5345 can be composed of multiple laminated films, a substrate film with a coating, or a rigid transparent substrate with films on either side. By providing a reflective polarizer on one side, light from the polarized light source 2958 is reflected toward the DLP 2955, where it passes through the quarter-wave film 2957 once, is reflected by the DLP mirror in accordance with the image content being displayed by the DLP 2955, and then passes back through the quarter-wave film 2957. In doing so, the polarization state of the light from the polarized light source is changed, causing it to be transmitted by the reflective polarizer on the beam splitter plate 5345 and image light 2971 to pass into the lower optics module 204, where the image is displayed to the user. Simultaneously, infrared light 5357 from the illumination light 5355 is reflected by the hot mirror, passing it into the lower optics module 204, where it illuminates the user's eye. A portion of the infrared light 2969 is reflected by the user's eye, and this light passes back through the lower optics module 204, is reflected by the hot mirror on the beam splitter plate 5345, and is captured by the camera 3280. In this embodiment, image light 2971 is polarized, while infrared light 5357 and 2969 may be unpolarized. In an embodiment, illumination light 5355 provides two different infrared wavelengths and eye images are captured in pairs, where the paired eye images are analyzed together to improve the accuracy of user identification based on iris analysis.

[0403] Figure 54 A diagram showing a further embodiment of a display system with an eye imaging system is shown. Figure 53 In addition to the features of , this system includes a second camera 5460. Wherein the second camera 5460 is provided to capture an image of the eye in a visible wavelength. Illumination of the eye can be provided by the displayed image or by see-through light from the environment. Portions of the displayed image can be modified to provide improved illumination of the user's eye when the image of the eye is to be captured, such as by increasing the brightness of the displayed image or increasing white areas within the displayed image. Further, for the purpose of capturing the image of the eye, the modified displayed image can be briefly presented and the display of the modified image can be synchronized with the capture of the image of the eye. As in Figure 54, visible light 5467 is polarized when it is captured by the second camera 5460 because it passes through the beam splitter 5445 and the beam splitter 5445 is a reflective polarizer on the side facing the second camera 5460. In such an eye imaging system, a visible light eye image can be captured by the second camera 5460 at the same time as an infrared eye image is captured by the camera 3280. The characteristics of the cameras 3280 and the second camera 5460, and the associated respective images captured, can differ in resolution and capture rate.

[0404] Figure 52a and 52b Figure 2 shows a captured image of an eye where the eye is illuminated with a structured light pattern. Figure 52a , an eye 5220 is shown with a projected structured light pattern 5230, where the light pattern is a grid of lines. A light pattern such as 5230 can be created by Figure 53 The light source 5355 shown in FIG is provided by including a diffraction device or a refraction device to modify the light 5357, as is known to those skilled in the art. The visible light source can also be directed to the Figure 54 The second camera 5460 shown in FIG. 5460 may include diffraction or refraction to modify the light 5467 to provide a light pattern. Figure 52b The diagram shows how the structured light pattern 5230 is distorted into 5235 when the user's eye 5225 is looking to the side. This distortion comes from the fact that the human eye is not spherical in shape, but rather the iris protrudes slightly from the eyeball to form a bulge in the area of ​​the iris. As a result, when an image of the eye is captured from a fixed position, the shape of the eye and the associated shape of the reflected structured light pattern differ depending on which direction the eye is pointing. The change in the structured light pattern can then be analyzed in the captured eye image to determine the direction the eye is looking.

[0405] The eye imaging system can also be used to assess aspects of a user's health. In this case, the information obtained from analyzing captured images of the iris 5012 differs from the information obtained from analyzing captured images of the retina 5014. Images of the retina 5014 are captured using light 5357 that illuminates the inner portion of the eye, including the retina 5014. Light 5357 can be visible light, but in embodiments, it is infrared light (e.g., wavelength 1 to 5 microns), and the camera 3280 is an infrared light sensor (e.g., an InGaAs sensor) or a low-resolution infrared image sensor, which is used to determine the relative amount of light 5357 absorbed, reflected, or scattered by the inner portion of the eye. Much of the absorbed, reflected, or scattered light can be attributed to the material in the inner portion of the eye, including the retina, where densely packed blood vessels with thin walls are present, resulting in the absorption, reflection, and scattering caused by the material composition of blood. These measurements can be taken automatically while the user is wearing the HWC, at regular intervals, after a recognized event, or when prompted by external communication. In a preferred embodiment, the illuminating light is near-infrared or mid-infrared (e.g., 0.7 to 5 micron wavelength) to reduce the chance of thermal damage to the wearer's eyes. In another embodiment, polarizer 3285 is anti-reflective coated to reduce any reflections of light 5357, light 2969, or light 3275 from this surface, thereby increasing the sensitivity of camera 3280. In a further embodiment, light source 5355 and camera 3280 together comprise a spectrometer, wherein the relative intensity of light reflected by the eye is analyzed across a range of narrow wavelengths within the wavelength range provided by light source 5355 to determine the characteristic spectrum of light absorbed, reflected, or scattered by the eye. For example, light source 5355 can provide a wide range of infrared light to illuminate the eye, and camera 3280 can include a grating for laterally dispersing the reflected light from the eye into a series of narrow wavelength bands captured by a linear photodetector, so that the relative intensity can be measured by wavelength and the characteristic absorption spectrum for the eye can be determined over the wide infrared range. In a further example, light source 5355 can provide a series of narrow wavelengths of light (ultraviolet, visible, or infrared) to sequentially illuminate the eye, and camera 3280 includes a photodetector selected to measure the relative intensities of the series of narrow wavelengths in a series of sequential measurements, which together can be used to determine a characteristic spectrum of the eye. The determined characteristic spectrum is then compared to known characteristic spectra for different materials to determine the material composition of the eye. In yet another embodiment, illumination light 5357 is focused onto retina 5014 and a characteristic spectrum of retina 5014 is determined, and the spectrum is compared to known spectra for materials that may be present in the user's blood. For example, among visible light wavelengths, 540 nm is useful for detecting hemoglobin, and 660 nm is useful for distinguishing oxyhemoglobin.In a further example, in the infrared, a variety of materials including glucose, urea, ethanol, and controlled substances can be identified, as is known by those skilled in the art. Figure 55 Shown are a series of example spectra for various controlled substances measured using a form of infrared spectroscopy (ThermoScientific Application Note 51242 by C. Petty, B. Garland, and Mesa Police Department Forensic Laboratory, which is hereby incorporated herein by reference). Figure 56 An infrared absorption spectrum for glucose is shown (Hewlett-Packard Company 1999, G. Hopkins, G. Mauze; "In-vivo NIR Diffuse-reflectance Tissue Spectroscopy of Human Subjects," which is hereby incorporated by reference herein). U.S. Patent 6,675,030 (which is hereby incorporated by reference herein) provides a near-infrared blood glucose monitoring system that includes infrared scanning of a body part (such as a foot). U.S. Patent Publication 2006 / 0183986 (which is hereby incorporated by reference herein) provides a blood glucose monitoring system that includes optical measurement of the retina. Embodiments of the present invention provide a method for automatically measuring a specific material in a user's blood by illuminating the iris of the wearer's eye with one or more narrow wavelengths and measuring the relative intensity of light reflected by the eye to identify a relative absorption spectrum and comparing the measured absorption spectrum to a known absorption spectrum for the specific material, such as illuminating at 540 and 660 nm to determine the level of hemoglobin present in the user's blood.

[0406] Another aspect of the present invention relates to the collection and use of eye position and field of view heading information. Head-mounted computing using motion heading, field of view heading, and / or eye position predictions (sometimes referred to herein as "eye heading") can be used to identify what the wearer of the HWC 102 is clearly interested in, and this information can be captured and used. In embodiments, the information can be characterized as it is viewed, as it clearly relates to what the wearer is viewing. This viewing information can be used to develop a personal profile for the wearer, which can indicate what the wearer tends to view. Viewing information from several or more HWCs 102 can be captured so that group or crowd viewing tendencies can be established. For example, if the motion heading and field of view heading are known, a prediction of what the wearer is viewing can be made and used to generate part of a crowd profile or a personal profile. In another embodiment, if the eye heading and position, field of view heading, and / or motion heading are known, a prediction of what is being viewed can be made. Prediction can involve understanding what is proximal to the wearer, and this can be achieved by establishing the wearer's location (e.g., via GPS or other positioning technology) and establishing what mapped objects are known in the area. Prediction may involve interpreting images captured by other sensors or cameras associated with the HWC 102. For example, if a camera captures an image of a sign, and the camera is aligned with the field of view, prediction may involve assessing the likelihood that the wearer is looking at the sign. Prediction may involve capturing images or other sensory information and then performing object recognition analysis to determine what is being viewed. For example, the wearer may be walking down the street, and a camera in the HWC 102 may capture an image, and a processor onboard or remote from the HWC 102 may recognize a face, object, sign, image, etc. and may determine that the wearer may have been looking at it or looking toward it.

[0407] In another embodiment, a method is provided for identifying changes in the focus distance associated with a user's eyes by measuring changes in the size of eye glints. The ability to identify changes in the focus distance of a user's eyes can be useful for determining what the user is looking at in the surrounding environment when using an HMD that provides a see-through view of the surrounding environment being used. Identifying changes in the focus distance of the user's eyes can also be useful for automatic display mode selection (e.g., selecting whether a displayed image should be bright or dimmed) by determining whether the user is looking at the displayed image content (where the focus distance of the user's eyes matches the focus distance of the displayed image) or the user is looking at the surrounding environment (where the focus distance of the user's eyes differs from the focus distance of the displayed image).

[0408] The ocular lens in the human eye changes spherical radius to provide changes in accommodation or focusing distance (see S. Plainis, W. Charman, I. Pallikaris, “The Physiologic Mechanism of Accomodation”, Cataract & Refractive Surgery Today Europe, April 2014, pp 23-29). Figure 56e A graph showing the measured anterior and posterior spherical radii of a 29 year old human eye over a range of diopters of accommodation from 0 to 6, as presented in the paper by Plainis. This change in accommodation corresponds to a change in focus distance from infinity to approximately 0.16 meters based on the diopters change in the focal length of the eye's lens, where the relationship between the focal length and diopters of the lens is given in Equation 1 below.

[0409] FL = 1 / δ.

[0410] Where FL is the focal length of the lens in meters, and δ is the diopter rating of the lens. The front surface of the eye lens is covered by the cornea and iris, so that the surface of the eye lens is not exposed to the environment. However, changes in the spherical radius of the eye lens are accompanied by changes in the spherical radius of the outer surface of the cornea. These changes in the spherical radius of the cornea affect the size of the reflected eye glint that can be seen on the surface of the eye. As the spherical radius of the eye lens decreases to provide a higher level of accommodation, the outer spherical radius of the cornea also decreases, which reduces the size of the reflected eye glint. Similarly, as the spherical radius of the eye lens increases to provide a lower level of accommodation, the outer spherical radius of the cornea increases, which increases the size of the reflected eye glint. Therefore, measuring the size of eye glint can be used to identify changes in the focusing distance of the user's eye. The measurement of the size of eye glint can be provided from an image of the user's eye captured by an eye camera.

[0411] In order to provide reliable detection of the size of an eye glint, a light source of constant size should be provided, which is then reflected by the outer surface of the cornea in the form of an eye glint. Figure 56a and 56b Example eye glints 5612 and 5614 generated from a reflected illumination source, such as an LED, are shown, which is used to illuminate the user's eyes for eye imaging, such as for purposes such as eye tracking or iris recognition. In this case, the LED is a circular illumination source, so the eye glint is also circular. Figure 56a A larger diameter eye glint 5612 is shown within the image of the eye 5610, corresponding to a larger diameter eye glint 5612 than that indicated by Figure 56bThe smaller diameter eye glint 5614 shown in FIG. 5 shows a larger focus distance indicated by the smaller focus distance. The LED illumination source can provide visible wavelength light or infrared wavelength light to provide eye glint and provide illumination for eye imaging, so long as the eye camera in the HMD is capable of capturing images of the eye that include the wavelengths provided by the illumination source (e.g., as described in the eye imaging and eye illumination systems disclosed herein). Figure 56c and 56d Another example of eye glints 5616 and 5618 is shown where the illumination source is a displayed image such that eye glints 5616 and 5618 are rectangular in shape. Figure 56c Shows the ratio Figure 56d The eye blink 5618 shown in the larger eye blink 5616 indicates Figure 56c The focusing distance of the eye 5610 is greater than Figure 56c 56. The focusing distance of the eye 5610 shown in FIG. In embodiments, the light emitted to reflect from the eye may be in a known pattern so that changes in the pattern can be assessed for changes in focus.

[0412] Measuring the magnitude of eye glint from a known illumination source is readily accomplished using an eye camera in an HMD (e.g., as disclosed herein). The resolution of the magnitude of the eye glint is determined by the angular size of the pixels in the eye image from the eye camera (e.g., the eye camera field of view / the horizontal number of pixels in the eye image). Thus, it is advantageous to have more pixels in a smaller field of view in the eye camera. This method of identifying changes in focus distance associated with the user's eye can be used with any type of optics for an HMD, including, for example, refractive optics, reflective optics, holographic optics, beam splitter optics, waveguide optics, grating optics, or multi-reflector optics, as long as the optics include an eye camera for imaging the user's eye.

[0413] In further embodiments, the illumination source providing the reflected eye glint may provide structured light having a pattern. The size or spacing of the pattern may then be used to identify changes in focus distance.

[0414] In another embodiment, the identified change in the focus distance of the user's eyes is used to identify a convergence distance for the displayed image. Providing a convergence distance and focus distance for the displayed image that matches the focus distance of the user's eyes can be used to display images that are viewed intensively, such as movies. Conversely, a convergence distance and focus distance for the displayed image that differ from the focus distance of the user's eyes can be used to provide displayed images that are not viewed intensively, such as battery indicators, directional information, assembly instructions, or augmented reality objects.

[0415] In embodiments, glint size and placement measurements may be used in conjunction with other techniques, such as convergence measurements, to determine the user's focus distance.

[0416] Figure 57 A scenario is shown in which a person is walking with a head-mounted HWC 102. In this scenario, the person's geospatial location 5704 is known via a GPS sensor (which can be another positioning system), and their movement heading, field of view heading 5714, and eye heading 5702 are known and can be recorded (e.g., by the system described herein). Objects and people are present in this scenario. Person 5712 can be identified by the wearer's HWC 102 system, mapped (e.g., the person's GPS location can be known or identified), or otherwise known. The person may be wearing identifiable clothing or equipment. For example, the clothing may have a certain style, and the HWC can identify that style and record that it is being viewed. The scenario also includes mapped objects 5718 and identified objects 5720. As the wearer moves through the scenario, field of view heading and / or eye heading can be recorded and transmitted from the HWC 102. In embodiments, the time that field of view heading and / or eye heading remains in a particular position can be recorded. For example, if a person appears to look at an object or person for a predetermined period of time (eg, 2 seconds or longer), the information may be conveyed as gaze duration information as an indication that the person may have been interested in the object.

[0417] In embodiments, field of view heading may be used in conjunction with eye heading, or eye heading and / or field of view heading may be used independently. Field of view heading is a good predictor of where the wearer is looking, as many times the eyes are looking forward in the same general direction as field of view heading. In other scenarios, eye heading may be a more desirable metric, as eye and field of view headings are not always aligned. In embodiments herein, examples may be provided with the term "eye / field of view" heading, indicating that either or both eye heading and field of view heading may be used in the example.

[0418] Figure 58The diagram illustrates a system for receiving, developing, and utilizing movement heading, field of view heading, eye heading, and / or duration information from HWC(s) 102. A server 5804 may receive heading or gaze duration information (referred to as duration information 5802) for processing and / or utilization. The heading and / or gaze duration information may be used to generate a personal profile 5808 and / or a group profile 5810. Personal profile 5718 may reflect the wearer's general viewing tendencies and interests. Group profile 5810 may be a collection of heading and duration information from different wearers to create an impression of general group viewing tendencies and interests. Group profile 5810 may be divided into different groups based on other information (such as gender, likes, dislikes, biometric information, etc.) to distinguish certain groups from others. This may be useful in advertising, as advertisers may be interested in what male adult athletes (as opposed to young females) are generally viewing. Profiles 5808 and 5810, along with the original orientation and persistence information, can be used by retailers 5814, advertisers 5818, trainers, and the like. For example, an advertiser may place an ad in an environment and may be interested in knowing how many people viewed the ad, how long they viewed the ad, and where they went after viewing it. This information can be used as conversion information to assess the value of the ad and, therefore, the payment to be received for the ad.

[0419] In one embodiment, this process involves collecting eye and / or field of view heading information from multiple head-mounted computers (HWCs) that are brought into proximity with objects in an environment. For example, a number of people may be walking through an area, each of whom may be wearing a HWC capable of tracking the position of the wearer's eyes and, possibly, their field of view and movement. The various HWC-wearing individuals may then walk, ride, or otherwise come into proximity with an object in the environment (e.g., a store, a sign, a person, a vehicle, a box, a bag, etc.). As each person passes by or otherwise approaches the object, an eye imaging system can determine whether the person is looking toward the object. All of this eye / field of view heading information can be collected and used to form an impression of how the crowd is reacting to the object. A store may be having a sale and, therefore, may display a sign indicating this. Store owners and managers may be very interested in knowing whether anyone is looking at their sign. The sign can be set as an object of interest in the area, and as people navigate near the sign (perhaps determined by their GPS location), the eye / field of view heading determination system can record information relative to the environment and the sign. Once or when eye / view heading information is collected and an association between eye heading and a sign is determined, feedback can be sent back to store owners, managers, advertisers, etc. as an indication of how attractive their signage is. In embodiments, the effectiveness of a signage in attracting a person's attention, as indicated by eye / view heading, can be considered a conversion metric and influence the economic value of one or more sign placements.

[0420] In an embodiment, a map of an environment with an object (e.g., a sign) can be generated by mapping the positions and movement paths of people in a crowd as they navigate around the object. Layered onto this map can be indications of various eye / field orientations. This can be useful in instructing the wearer about their viewing of the object. The map can also include indications of how long people viewed the object from various locations in the environment and where they went after viewing the object.

[0421] In an embodiment, the process involves collecting multiple eye / field orientations from a head-worn computer, where each of the multiple eye / field orientations is associated with a different predetermined object in the environment. This technique can be used to determine which of the different objects attracts a person's attention more. For example, if there are three objects placed in an environment, and a person enters the environment and navigates their way through it, they may look at one or more of the objects and their eye / field orientation may remain on one or more of the objects longer than on the others. This can be used to create or refine a person's personal attention profile, and / or it can be used in conjunction with data from other such people regarding the same or similar objects to determine the impression of how a group or population reacts to the object. Testing advertisements in this way can provide good feedback on the effectiveness of the advertisements.

[0422] In an embodiment, the process may involve capturing eye / field of view heading once there is substantial alignment between the eye / field of view heading and an object of interest. For example, a person with an HWC may be navigating through an environment and once the HWC detects substantial alignment, or an impending planned occurrence of substantial alignment, between the eye / field of view heading and an object of interest, that occurrence and / or duration may be recorded for use.

[0423] In embodiments, a process may involve collecting eye / field of view heading information from a head-worn computer and collecting captured images from the head-worn computer, which are taken substantially simultaneously with the time the eye / field of view heading information is captured. These two pieces of information may be used in combination to gain an understanding of what the wearer is looking at and may be interested in. The process may further involve correlating the eye / field of view heading information with objects, people, or other things found in the captured images. This may involve processing the captured images to look for objects or patterns. In embodiments, gaze duration or duration may be measured and used in conjunction with image processing. This process may still involve object and / or pattern recognition, but it may also involve attempting to identify what a person is looking at over a period of time by more specifically identifying portions of an image in conjunction with image processing.

[0424] In an embodiment, the process may involve setting predetermined eye / field of view headings based on predetermined geospatial locations and using them as triggers. In the event that a head-worn computer enters a geospatial location and the eye / field of view heading associated with the head-worn computer is aligned with the predetermined eye / field of view heading, the system may collect the fact that there is a significant alignment and / or the system may record information identifying how long the eye / field of view heading remains substantially aligned with the predetermined eye / field of view heading to form persistence statistics. This may eliminate or reduce the need for image processing because the triggers can be used without having to image the area. In other embodiments, image capture and processing are performed in conjunction with the triggers. In an embodiment, the triggers may be a series of geospatial locations with corresponding eye / field of view headings, so that many locations can be used as triggers that indicate when a person enters an area close to an object of interest and / or when the person actually appears to be looking at the object.

[0425] In embodiments, eye imaging can be used to capture images of both of the wearer's eyes to determine the amount of eye convergence (e.g., using techniques described elsewhere herein) in order to gain an understanding of what focal plane the wearer is focusing on. For example, if the convergence measurement suggests the focal plane is within 15 feet of the wearer, then even if the eye / field of view heading may be aligned with an object more than 15 feet away, it can be determined that the wearer is not looking at the object. If the object is within the focal plane implied by 15 feet, then it can be determined that the wearer is looking at the object. Figure 59The illustration shows environmentally location-locked digital content 5912 indicating the location of a person 5902. In this disclosure, the term "Blue Force" is generally used to refer to a member or team member whose geospatial location is known and can be used. In embodiments, "Blue Force" is a term used to refer to members of a tactical armed team (e.g., police force, special forces, security forces, military forces, national security forces, intelligence forces, etc.). In many embodiments herein, one member may be referred to as the lead Blue Force member or first Blue Force member, and in many of the described embodiments, it is this member who is wearing the HWC. It should be understood that this terminology is used to aid the reader and facilitate clarity in various scenarios, and that other members of the Blue Force or other individuals may have HWCs 102 with similar capabilities. In this embodiment, the first person is wearing a head-worn computer 102 with a see-through field of view ("FOV") 5914. The first person is able to see through the FOV to view their surroundings, and digital content can also be presented within the FOV, allowing the first person to view their actual surroundings in a digitally augmented view through the FOV. The location of the other blue force personnel is known and is indicated at a location within the building at point 5902. This location is known in three dimensions (longitude, latitude, and altitude), which may have been determined by GPS in conjunction with altimeters associated with the other blue force personnel. Similarly, the location of the first person wearing HWC 102 is also known, as in Figure 59 5908 in the HWC 102. In this embodiment, the first person's compass heading 5910 is also known. With the compass heading 5910 known, the angle at which the first person is viewing their surroundings can be estimated. A virtual target line can be established in three-dimensional space between the first person's position 5908 and the other person's position 5902 and released from the HWC 102 near the FOV 5914. The three-dimensionally oriented virtual target line can then be used to present environmentally position-locked digital content in the FOV 5914 that indicates the other person's position 5902. The environmentally position-locked digital content 5902 can be positioned within the FOV 5914 so that the first person, who is wearing the HWC 102, perceives the content 5902 as being locked in position within the environment and marking the other person's position 5902.

[0426] The three-dimensionally positioned virtual target line can be periodically recalculated (e.g., every millisecond, every second, every minute, etc.) to reposition the ambient location-locked content 5912 to remain aligned with the virtual target line. This can create the illusion that the content 5912 remains positioned within the environment at a point associated with the other person's location 5902, independent of the location 5908 of the first person wearing the HWC 102 and independent of the compass heading of the HWC 102.

[0427] In embodiments, context-locked digital content 5912 may be positioned with a target 5904 positioned between a first person's location 5908 and another person's location 5902. A virtual target line may intersect the object 5904 before intersecting the other person's location 5902. In embodiments, context-locked digital content 5912 may be associated with the object intersection point 5904. In embodiments, the intersecting object 5904 may be identified by comparing the two person's locations 5902 and 5908 with obstacles identified on a map. In embodiments, the intersecting object 5904 may be identified by processing images captured from a camera or other sensor associated with HWC 102. In embodiments, the digital content 5912 may have an appearance indicating that it is positioned at the other person's location 5902 and at the location of the intersecting object 5904, providing a clearer indication of the other person's location 5902 within the FOV 5914.

[0428] Figure 60 The illustration shows how and where digital content can be positioned within the FOV 6008 based on a virtual target line between the position 5908 of a first person wearing the HWC 102 and the positions 5902 of other persons. In addition to positioning content in a position aligned with the virtual target line within the FOV 6008, the digital content can be presented so that it is focused by the first person when the first person focuses on a certain plane or distance in the environment. Presented object A 6018 is digitally generated content, presented as an image at content position A 6012. Position 6012 is based on the virtual target line. Presented object A 6018 is presented not only along the virtual target line but also at focal plane B 6014, so that when the first person's eye 6002 focuses on something in the surrounding environment at the distance of focal plane B 6014, the content at position A 6012 in the FOV 6008 is focused by the first person. Setting the focal plane for the presented content provides for content that was not previously in focus when the eye 6002 is focused on the set focal plane. In embodiments, this allows content at location A to not be presented when the compass of the HWC indicates that the first person is looking in the direction of the other person 5902, but rather that the content will only be in focus when the first person is focused in the direction of the other person 5902 and at the focal plane of the other person 5902.

[0429] Rendered object B 6020 is aligned with a different virtual target line than rendered object A 6018. Rendered object B 6020 is also rendered at content location B 6004 at a different focal plane than content location A 6012. Rendered content B 6020 is rendered at a different focal plane, indicating that other person 5902 is physically at a different distance. If the focal planes are sufficiently different, the content at location A will be focused at a different time than the content at location B because the two focal planes require different focus from the eye 6002.

[0430] Figure 61 The illustration shows several Blue Force members at various viewpoints from the perspective of a first person. In embodiments, relative position, distance, and obstructions can cause the digital content indicating the location of other individuals to be altered. For example, if the other individual is visible to the first person through the first person's field of view, the digital content may be locked onto the other individual's location and may be of a type indicating that the other individual's location is being effectively marked and tracked. If the other individual is in relatively close proximity but cannot be seen by the first person, the digital content may be locked onto an intersecting object or area, and the digital content may indicate that the other individual's actual location cannot be seen, but that the marker is generally tracking the other individual's general location. If the other individual is not within a predetermined proximity or is otherwise more significantly obscured from the first person's view, the digital content may generally indicate the direction and area in which the other individual is located, and the digital content may indicate that the other individual's location is not closely identified or tracked by the digital content, but that the other individual is in the general area.

[0431] Continue to refer Figure 61 , several Blue Force members are presented at various locations within the area where the first person is located. The primary Blue Force member 6102 (also generally referred to as the first person, or the person within whose HWC the FOV is located, for purposes of illustration) can directly see the Blue Force members in the open field 6104. In embodiments, the digital content provided within the primary Blue Force member's FOV may be based on a virtual target line and virtually locked to an environmental location indicating the Blue Force member's open field location 6104. The digital content may also indicate that the open field Blue Force member's location is marked and being tracked. The digital content may change form if a Blue Force member becomes obscured from the primary Blue Force member's view or otherwise becomes unavailable for direct viewing.

[0432] Blue force member 6108 is obscured from the primary blue force member's 6102 view by an obstacle that is in close proximity to obscured member 6108. As depicted, obscured member 6108 is within the building, but near one of the front walls. In this scenario, digital content provided within primary member 6102's FOV can indicate the general location of obscured member 6108. The digital content can also indicate that while the other member's position is fairly well-marked, it is obscured and therefore not as precise as if the member were in direct view. Furthermore, the digital content can be virtually position-locked to a feature on the exterior of the building where the obscured member is located. This can make environmental locking more stable and also provide an indication of a person whose location is somewhat unknown.

[0433] Blue force member 6110 is obscured by multiple obstacles. Member 6110 is in a building and there is another building 6112 between primary member 6102 and obscured member 6110. In this scenario, the digital content in the primary member's FOV will be spatially quite lacking in the actual obscured member, and as such the digital content may need to be presented in a manner that indicates the general direction obscured member 6110 is in but that the digital marker is not a reliable source for the specific location of obscured member 6110.

[0434] Figure 62

[0014] Yet another method for locating digital content within the FOV of a HWC is illustrated, wherein the digital content is intended to indicate the location of another person. This embodiment is incorporated herein by reference. Figure 62 5902 ). The primary additional element in this embodiment is the additional step of verifying the distance between a first person 5908 (a person wearing an HWC with a digital content presentation of the location's FOV) and the other person at location 5902. Here, a rangefinder may be included in the HWC and measures distance at an angle, represented by a virtual target line. If the rangefinder detects an object obscuring the path of the virtual target line, the digital content presentation in the FOV may indicate this (e.g., as described elsewhere herein). If the rangefinder confirms the presence of a person or object at the end of the angle defined by the virtual target line and the specified distance, the digital content may indicate that the appropriate location has been marked, as described elsewhere herein.

[0435] Another aspect of the present invention relates to predicting the movements of blue force members to maintain a suitable virtual marker of the blue force member's location. Figure 63Illustrated is a scenario in which a primary Blue Force member 6302 is using HWC 102 to track the positions of other Blue Force members through an augmented environment, as described elsewhere herein (e.g., as described in conjunction with the above figures). The primary Blue Force member 6302 may have knowledge of a tactical movement plan 6308. The tactical movement plan may be maintained locally (e.g., on HWC 102, where the plan is shared among Blue Force members) or remotely (e.g., on a server and transmitted to HWC 102, or transmitted to a subset of HWCs 102 for sharing by HWCs 102). In this case, the tactical plan involves a group of Blue Force members moving generally in the direction of arrow 6308. The tactical plan may affect the presentation of digital content within the FOV of the primary Blue Force member's HWC 102. For example, the tactical plan may assist in predicting the positions of other Blue Force members, and the virtual target line may be adjusted accordingly. In an embodiment, areas within the tactical movement plan may be shaded or colored, or otherwise marked with digital content within the FOV, enabling the primary blue force member to manage his activities with respect to the tactical plan. For example, he may be aware that one or more blue force members are moving toward the tactical path 6308. He may also be aware that movements within the tactical path do not appear to be associated with blue force members.

[0436] Figure 63 Also illustrated is that the internal IMU sensor in the HWC worn by the blue force member can provide guidance 6304 on the member's movements. This is helpful in identifying when the GPS position should be updated and therefore the position of the virtual marker in the FOV. This can also be helpful in assessing the validity of the GPS position. For example, if the GPS position is not updating, but there is significant IMU sensor activity, the system may question the accuracy of the identified position. The IMU information can also be useful to help track the member's position in situations where GPS information is unavailable. For example, if the GPS signal is lost, dead reckoning can be used, and the virtual marker in the FOV can indicate both the indicated movements of the team member and indicate that position identification is not ideal. The current tactical plan 6308 can be periodically updated, and the updated plan also further refines what is presented in the FOV of the HWC 102.

[0437] Figure 64A blue force tracking system according to the principles of the present invention is illustrated. In an embodiment, a blue force HWC 102 may have a directional antenna that transmits a relatively low-power directional RF signal so that other blue force members within range of the relatively low-power signal can receive it and assess the direction and / or distance of the signal based on the strength and changing strength of the signal. In an embodiment, tracking such RF signals can be used to alter the presentation of virtual markers of personnel locations within the FOV of the HWC 102.

[0438] Another aspect of the present invention relates to monitoring the health of Blue Force members. Each Blue Force member can be automatically monitored for health and stressful events. For example, a member can be equipped with a watchband or other wearable biometric monitoring device as described elsewhere herein, and the device can continuously monitor biometric information and predict health concerns or stressful events. As another example, an eye imaging system as described elsewhere herein can be used to monitor pupil dilation compared to normal conditions to predict head trauma. Each eye can be imaged to detect differences in pupil dilation that indicate head trauma. As another example, an IMU in HWC 102 can monitor a person walking to a door to look for changes in posture, which can be indicative of head or other trauma. For example, biometric feedback from a member indicating health or stress concerns can be uploaded to a server for sharing with other members, or the information can be shared with local members. Once shared, digital content in the FOV indicating the location of the person experiencing the health or stressful event can include an indication of the health event.

[0439] Figure 65 The illustration shows a scenario in which a primary Blue Force member 6502 is monitoring the location of a Blue Force member 6504 who has experienced a health event and has caused a health alert to be sent from HWC 102. As described elsewhere herein, the FOV of the primary Blue Force member's HWC 102 may include an indication of the location of the Blue Force member 6504 who has a health concern. The digital content in the FOV may also include an indication of the health condition associated with the location indication. In embodiments, non-biometric sensors (e.g., IMU, camera, rangefinder, accelerometer, altimeter, etc.) may be used to provide health and / or situational status to the Blue Force team or other local or remote personnel interested in the information. For example, if one of the Blue Force members is detected to have quickly hit the ground from a standing position, an alert may be sent indicating a person has collapsed, is in distress and has been forced to go down, has been hit, etc.

[0440] Another aspect of the present invention relates to virtually marking various previous actions and events. Figure 66As depicted in [ 66] , techniques described elsewhere herein can be used to construct a virtual previous movement path 6604 of a Blue Force member. This virtual path can be displayed as digital content in the field of view of the primary Blue Force member 6602 using methods described elsewhere herein. As the Blue Force member moves along path 6604, they may have virtually placed event markers 6608 so that when another member views that location, the markers can be displayed as digital content. For example, a Blue Force member can check and clear an area, then use an external user interface or gesture to indicate that the area has been cleared. This location will then be virtually marked and shared with other Blue Force members. Later, when someone wants to know if the location has been checked, they can view the location information. As indicated elsewhere herein, if the location is visible to the member, the digital content can be displayed in a manner that indicates that specific location. If the location is not visible from the person's perspective, the digital content may be somewhat different because it may not specifically mark the location.

[0441] Returning to the optical configuration, another aspect of the present invention relates to an optical configuration that provides digitally displayed content to the eyes of a person wearing a head-mounted display (e.g., as used in HWC 102) and allows the person to see through the display so that the digital content is perceived by the person as enhancing a see-through view of the surrounding environment. The optical configuration may include a variable-transmission optical element aligned with the person's see-through view, allowing the transmission of the see-through view to be increased or decreased. This is helpful in scenarios where a person would prefer or be better provided with a high-transmission see-through view, while in the same HWC 102, the person would prefer or be better provided with less see-through transmission. Lower see-through transmission can be used in bright conditions and / or where higher contrast is desired for digitally presented content. The optical system may also include a camera that images the surrounding environment by receiving light reflected from the surrounding environment off the optical element aligned with the person's see-through view of the surroundings. In embodiments, the camera may be further aligned in a dark light well so that light reflected and / or transmitted in the direction of the camera that is not captured by the camera is captured to reduce stray light.

[0442] In an embodiment, a HWC 102 is provided that includes a camera that is aligned coaxially with the direction the user is looking. Figure 67A diagram of an optical system 6715 including an absorptive polarizer 6737 and a camera 6739 is shown. An image source 6710 can include a light source, a display, and a reflective surface, as well as one or more lenses 6720. Image light 6750 is provided by the image source 6710, with a portion of the image light 6750 being reflected by a partially reflective combiner 6735 toward the user's eye 6730. Simultaneously, a portion of the image light 6750 can be transmitted by the combiner 6735, causing the image light to be incident on the absorptive polarizer 6737. In this embodiment, the image light 6750 is polarized light, with the polarization state of the image light 6750 oriented relative to the transmission axis of the absorptive polarizer 6737 such that the incident image light 6750 is absorbed by the absorptive polarizer 6737. In this manner, face glow caused by escaping image light 6750 is reduced. In embodiments, the absorptive polarizer 6737 includes an anti-reflective coating to reduce reflections from the surface of the absorptive polarizer 6737.

[0443] Figure 67Further shown is a camera 6739, which is used to capture images of the environment in the direction the user is looking. Camera 6739 is positioned behind an absorptive polarizer 6737 and below a combiner 6735, such that a portion of light 6770 from the environment is reflected by the combiner 6735 toward the camera 6739. The light 6770 from the environment can be unpolarized, so that a portion of the light 6770 from the environment reflected by the combiner 6735 passes through the absorptive polarizer 6737 and is captured by the camera 6739. As a result, the light captured by the camera has a polarization state opposite to that of the image light 6750. Furthermore, camera 6739 is aligned relative to the combiner 6735 such that the field of view associated with camera 6739 is coaxial with the display field of view provided by the image light 6750. Simultaneously, a portion of scene light 6760 from the environment is transmitted by the combiner 6735 to provide a see-through view of the environment to the user's eye 6730. The displayed field of view associated with image light 6750 typically coincides with the see-through field of view associated with scene light 6760, and thus the field of view of camera 6739 and the see-through field of view are at least partially coaxial. By attaching camera 6739 to the lower portion of optical system 6715, the field of view of camera 6739, as shown by light 6770 from the environment, moves as the user moves their head, so that the image captured by camera 6739 corresponds to the area of ​​the environment the user is viewing. By coaxially aligning the camera field of view with the displayed image and the user's view of the scene, it is possible to provide augmented reality images with improved alignment with objects in the scene. This is because the captured image from camera 6739 provides an accurate representation of the user's view of the scene from their perspective. For example, if the user sees an object in the scene in the middle of the see-through view of the HWC, the object will be in the middle of the image captured by the camera, and any augmented reality image associated with the object can be in the middle of the displayed image. As the user moves their head, the relative positions of objects seen in the perspective view of the scene will change, and the position of the augmented reality image can be changed within the displayed image in a corresponding manner. When a camera 6739 is provided for each of the user's eyes, an accurate representation of the 3D scene view can also be provided. This is a significant advantage provided by the present invention, as images captured by cameras located in the frame of the HWC (e.g., between the eyes or in the corners) capture images that are laterally offset from the user's perspective of the scene, and as a result, it is difficult to align the augmented reality image with objects in the scene as seen from the user's perspective.

[0444] exist Figure 67In the optical system 6715 shown in FIG, an absorptive polarizer 6737 simultaneously acts as a light trap for escaping image light 6750, a light blocker for image light 6750 for camera 6739, and a window for light from the environment 6770 to the camera 6739. This is possible because the polarization state of the image light 6750 is perpendicular to the transmission axis of the absorptive polarizer 6737, while the light from the environment 6770 is unpolarized, so that a portion of the light from the environment 6770, which is of the opposite polarization state as the image light, is transmitted by the absorptive polarizer 6737. The combiner 6735 can be any partially reflective surface, including a simple partial mirror, a notch mirror, and a holographic mirror. The reflectivity of the combiner 6735 can be selected to be greater than 50% (e.g., 55% reflectivity and 45% transmittance across the visible wavelength band), whereby the majority of the image light 6750 will be reflected toward the user's eye 6730 and the majority of the light from the environment 6770 will be reflected toward the camera 6739, such that the system will provide a brighter displayed image, a brighter captured image, with a darker see-through view of the environment. Alternatively, the reflectivity of the combiner 6735 can be selected to be less than 50% (e.g., 20% reflectivity and 80% transmittance across the visible wavelength band), whereby the majority of the image light 6750 will be transmitted by the combiner 6735 and the majority of the light from the environment 6770 will be transmitted to the user's eye 6730, such that the system will provide a brighter see-through view of the environment while providing a darker displayed image and a darker captured image. In this way, the system can be designed to facilitate the intended use by the user.

[0445] In embodiments, combiner 6735 is planar, having optical flatness sufficient to enable sharp displayed images and sharp captured images, such as a flatness less than 20 wavelengths of light within the visible spectrum. However, in embodiments, combiner 6735 may be curved, in which case both the displayed and captured images will be distorted, and this distortion will have to be digitally corrected by an associated image processing system. In the case of a displayed image, the image is digitally distorted by the image processing system in a direction opposite to the distortion caused by the curved combiner, so that the two distortions cancel each other out and the user sees an undistorted displayed image as a result. In the case of a captured image, the captured image is digitally distorted after capture to offset the distortion caused by the curved combiner so that the image appears undistorted after image processing.

[0446] In an embodiment, the combiner 6735 is an adjustable partially reflective mirror, wherein the reflectivity can be changed by the user or automatically to better function within different environmental conditions or different use cases. The adjustable partially reflective mirror can be an electrically controllable mirror, such as, for example, the e-Transflector available from Kent Optronics (http: / / www.kentoptronics.com / mirror.html), wherein the reflectivity can be adjusted based on an applied voltage. The adjustable partially reflective mirror can also be a fast switchable mirror (e.g., a switching time of less than 0.03 seconds), wherein the perceived transparency is derived from the duty cycle of the mirror rapidly switching between the reflective state and the transmissive state. In an embodiment, the image captured by the camera 6739 can be synchronized to occur when the fast switchable mirror is in the reflective state so as to provide an increased amount of light to the camera 6739 during image capture. As such, the adjustable partially reflective mirror allows the transmittance of the partially reflective mirror to be changed corresponding to ambient conditions, eg the transmittance can be low when the environment is bright and the transmittance can be high when the environment is dark.

[0447] In further embodiments, combiner 6735 comprises a hot mirror coated on the side facing camera 6739, wherein visible wavelengths of light are substantially transmitted while a spectral wavelength band of infrared light is substantially reflected, and camera 6739 captures an image that includes at least a portion of the infrared wavelengths of light. In these embodiments, image light 6750 comprises visible wavelengths of light, and a portion of the visible wavelengths of light is transmitted by combiner 6735, where it is then absorbed by absorptive polarizer 6737. Scene light 6760 comprises a portion of visible wavelengths of light and is also transmitted by combiner 6735 to provide the user with a see-through view of the environment. Light from the environment 6770 comprises visible wavelengths of light and infrared wavelengths of light. A portion of the visible wavelengths of light, along with substantially all of the infrared wavelengths of light within the spectral wavelength band associated with the hot mirror, is reflected by combiner 6735 toward camera 6739, thereby passing through absorptive polarizer 6737. In embodiments, the camera 6739 is selected to include an image sensor that is sensitive to infrared wavelengths of light, and the absorptive polarizer 6737 is selected to substantially transmit infrared wavelengths of light of both polarization states (e.g., an ITOS XP44 polarizer, which transmits both polarization states of light having wavelengths above 750 nm: see http: / / www.itos.de / english / polarisatoren / linear / linear.php), so that an increased % of infrared light is captured by the camera 6739. In these embodiments, the absorptive polarizer 6737 acts as a light trap for escaping image light 6750 and thereby blocks image light 6750 in visible wavelengths from the camera 6739, while simultaneously acting as a window to the camera 6739 for infrared wavelength light 6770 from the environment.

[0448] By aligning the camera's field of view coaxially with the displayed image and the user's view of the scene, it is possible to provide an augmented reality image with improved alignment with objects in the scene. This is because the captured image from the camera provides an accurate representation of the user's view of the scene from their perspective. In an embodiment, the camera captures an image of the scene coaxially aligned with the user's view, and the processor then identifies objects in the captured image and identifies the field of view position for the objects, which can be compared to the displayed field of view relative position, so that digital content is then displayed relative to the object's position.

[0449] Another aspect of the present invention relates to an optical assembly for use with a reflective display, wherein the reflective display is illuminated by a front light arranged to direct illumination at an angle of approximately 90 degrees to an effective reflective surface of the reflective display. In an embodiment, the optical assembly is lightweight, small, and produces a high-quality image in a head-mounted see-through display.

[0450] Figure 68A cross-sectional illustration of a compact optical display assembly for HWC 102, in accordance with the principles of the present invention, is provided, along with illustrative light rays used to illustrate how light passes through the assembly. The display assembly consists of upper and lower optics. The upper optics include a reflective image source 6810, a quarter-wave film 6815, a field lens 6820, a reflective polarizer 6830, and a polarized light source 6850. The upper optics converts illumination light 6837 into image light 6835. The lower optics include a beam splitter plate 6870 and a rotationally curved partial reflector 6860. The lower optics delivers the image light to the user wearing the HWC 102. The compact optical display assembly provides the user with image light 6835 that conveys a display image, as well as ambient light 6865 that provides a see-through view of the environment, such that the user sees the display image superimposed on the view of the environment.

[0451] In the upper optical device, linearly polarized light is provided by a polarized light source 6850. The polarized light source 6850 can include one or more lamps, such as LEDs, QLEDs, laser diodes, fluorescent lamps, etc. The polarized light source 6850 can also include a backlight assembly with a light scattering surface or diffuser to evenly spread the light across the output area of ​​the polarized light source. A light control film or light control structure can also be included to control the distribution (also known as the cone angle) of the light provided by the polarized light source 6850. The light control film can include, for example, a diffuser, an elliptical diffuser, a prismatic film, and a lenticular lens array. The light control structure can include a prism array, a lenticular lens, a cylindrical lens, a Fresnel lens, a refractive lens, a diffractive lens, or other structures that control the angular distribution of the illumination light 6837. The output surface of the polarized light source 6850 is a polarizer film that is used to ensure that the illumination light 6837 provided to the upper optical device is linearly polarized.

[0452] The illumination light 6837 provided by the polarized light source 6850 is reflected by the reflective polarizer 6830. The reflective polarizer 6830 and the polarizer on the output surface of the polarized light source 6850 are oriented so that their respective transmission axes are perpendicular to each other. As a result, most of the illumination light 6837 provided by the polarized light source 6850 is reflected by the reflective polarizer 6830. In addition, the reflective polarizer 6830 is angled so that the illumination light 6837 is reflected toward the reflective image source 6810, thereby illuminating the reflective image source 6810, as shown in FIG. Figure 68 As shown in .

[0453] Illumination light 6837 passes through field lens 6820 and is then incident on reflective image source 6810. Illumination light 6837 is then reflected by reflective image source (also referred to elsewhere herein as a reflective display) 6810. Reflective image source 6810 can include a liquid crystal on silicon (LCOS) display, a ferroelectric liquid crystal on silicon (FLCSO) display, a reflective liquid crystal display, a cholesteric liquid crystal display, a bistable nematic liquid crystal display, or other such reflective displays. The display can be a monochrome reflective display using sequential red / green / blue illumination light 6837 or a full-color display using white illumination light 6837. Reflective image source 6810 locally alters the polarization state of illumination light 6837 corresponding to the pixel-by-pixel image content displayed by reflective image source 6810, thereby forming image light 6835. Wherein, if reflective image source 6810 is a normally white display, regions of image light 6835 corresponding to bright areas of the image content end up with a polarization state opposite to that of the illumination light, and dark areas of image light 6835 end up with the same polarization state as illumination light 6837 (it should be noted that the present invention can be used with normally black displays that provide the opposite effect on polarization in the image light). As such, image light 6835 initially reflected by reflective image source 6810 has a mixed polarization state on a pixel-by-pixel basis. Image light 6835 then passes through field lens 6820, which modifies the distribution of image light 6835 while preserving the wavefront to match the requirements of the underlying optics (such as, for example, magnification and focusing). When image light 6835 passes through reflective polarizer 6830, bright areas of image light 6835 having a polarization state opposite to that of illumination light 6837 are transmitted through reflective polarizer 6830, and dark areas of image light 6835 having the same polarization state as illumination light 6837 are reflected back toward polarized light source 6850. As a result, after passing through reflective polarizer 6830, image light 6835 is linearly polarized with a single polarization state in all pixels of the image, but now with different intensities from pixel to pixel. Thus, reflective polarizer 6830 first acts as a reflector for illumination light 6837 and then again acts as an analyzer polarizer for image light 6835.

[0454] As such, the optical axis of illumination light 6837 coincides with the optical axis of image light 6835 between reflective polarizer 6830 and reflective image source 6810. Both illumination light 6837 and image light 6835 pass through field lens 6820, but in opposite directions. The field lens acts to expand illumination light 6837 so that it illuminates the full active area of ​​reflective image source 6810, and also expands image light 6835 so that it fills eyebox 6882 after passing through the compact optical display system. By overlapping the portion of the compact optical display assembly associated with illumination light 6837 and the portion of the compact optical display assembly associated with image light 6835, the overall size of the compact optical display assembly is reduced. Given that the focal length associated with field lens 6820 requires some space within the compact optical display assembly, reflective polarizer 6830 and polarized light source 6850 are located in space that would otherwise be unused, making the overall size of the display assembly more compact.

[0455] The reflective polarizer 6830 can be a relatively thin film (e.g., 80 microns) or thin plate (e.g., 0.2 mm), as in Figure 68 . The reflective polarizer 6830 can be a wire grid polarizer, such as available from Asahi Kasei under the WGF name, or a multilayer dielectric film polarizer, such as available from 3M under the DBEF name. As previously described, the reflective polarizer 6830 has two functions. First, the reflective polarizer 6830 reflects the illumination light 6837 provided by the polarized light source 6850 and redirects the illumination light 6837 toward the reflective image source 6810. Second, the reflective polarizer 6830 acts as an analyzer polarizer for the image light 6835, thereby converting the image light 6835 in mixed polarization states above the reflective polarizer 6830 into linearly polarized light having a single polarization state below the reflective polarizer 6830. While the illumination light 6837 incident on the reflective polarizer 6830 is incident on a relatively small portion of the reflective polarizer 6830, the image light 6835 is incident on the majority of the area of ​​the reflective polarizer 6830. Thus, the reflective polarizer 6830 extends at least across the entire area of ​​the field lens 6820 and may extend across as in Figure 686820 and beam splitter 6870. Furthermore, reflective polarizer 6830 is angled, at least in the portion where illumination light 6837 is incident, to redirect illumination light 6837 toward reflective image source 6810. However, in a preferred embodiment, because reflective polarizers (such as wire grid polarizers) can be relatively insensitive to angle of incidence, reflective polarizer 6830 is a flat surface that is angled to redirect illumination light 6837 toward reflective image source 6810, wherein the flat surface extends substantially across the entire area between field lens 6820 and beam splitter 6870 in one continuous flat surface to facilitate manufacturing. The film or sheet of reflective polarizer 6870 can be left at the edges to position it at the desired angle and flatten the surface.

[0456] In this article about Figures 68 to 71 The described systems and methods have numerous advantages. By avoiding grazing angles of illumination light 6837 and image light 6835 at all surfaces in a compact optical display assembly, light scattering within the assembly is reduced, resulting in a higher contrast image presented to the user's eye 6880 with a darker black. Furthermore, the reflective image source 6810 can include a compensating retarder film 6815, as known to those skilled in the art, to enable the reflective image source 6810 to provide a higher contrast image with more uniform contrast across the area of ​​the displayed image. Furthermore, by providing an optical display assembly comprised primarily of air, the weight of the compact optical display assembly is significantly reduced. By using coincident optical axes for the illumination light 6837 and image light 6835 and overlapping the illumination light 6837 and image light 6835 for a substantial portion of the optical display assembly, the overall size of the compact optical display assembly is reduced. The coincident optical axes are provided by passing the illumination light 6837 and image light 6835 in opposite directions through the field lens 6820. To maintain a uniform polarization state for the illumination light 6837, the field lens 6820 is made of a low-birefringence material, such as glass or plastic (such as OKP4 available from Osaka Gas Chemicals). By positioning the polarized light source 6850 and the associated illumination light 6837 below the field lens 6820, and by folding the optical paths of both the illumination light 6837 at the reflective polarizer 6830 and the image light 6835 at the beam splitter 6870, the overall height of the compact optical display assembly is greatly reduced. For example, the overall height of the compact optical display assembly can be less than 24 mm, as measured from the reflective image source 6810 to the bottom edge of the rotationally curved partial reflector 6860 for a display providing a 30-degree diagonal field of view with a 6×10 mm eyebox.

[0457] Preferably, the light control structure in the polarized light source 6850 includes a positive lens, such as, for example, a positive Fresnel lens, a positive diffractive lens, or a positive refractive lens. Positive Fresnel lenses or positive diffractive lenses are preferred because they can be very thin. The illumination light 6837 is thereby focused to form a smaller area or pupil at the reflective polarizer 6830, which has a direct relationship with the area of ​​the eye box 6882 at the other end of the optical device, where the image light 6835 is provided to the user's eye 6880, as shown in FIG. Figure 68 , as shown in FIG. A positive lens concentrates illumination light 6837 from polarized light source 6850 in both intensity and angular distribution to match the etendue of the optical system, thereby filling the eyebox with image light 6835. By using a positive lens to converge the light from polarized light source 6850 provided to reflective polarizer 6830, and then using field lens 6820 to expand illumination light 6837 to illuminate the active area of ​​reflective image source 6810, efficiency is improved because illumination light 6837 is delivered substantially only where it is needed to form image light 6835. Furthermore, illumination light 6837 outside the pupil can be controlled by the positive lens and clipped by the masking edge of the positive lens. By focusing illumination light 6837 and clipping light outside the pupil, illumination light 6837 is prevented from striking adjacent surfaces at grazing angles in a compact optical display assembly, reducing light scatter and thereby improving contrast in the image presented to the user's eye 6880 by providing a darker black.

[0458] It should be pointed out that although Figure 68 、 69 70 show an optical arrangement in which illumination light 6837 is provided from behind a rotationally curved partial reflector 6860, but other optical arrangements are possible within the present invention. The position of the polarized light source 6850 can be changed, for example, to be at the side of the rotationally curved partial reflector 6860, with the reflective polarizer 6830 oriented to receive the illumination light 6837 from that side and reflect the illumination light toward the reflective image source 6810 (not shown).

[0459] In a further embodiment, a portion of the image light 6835 reflected back toward the polarized light source 6850 is recycled in the polarized light source 6850 to improve the efficiency of the polarized light source 6850. In this case, a diffuser and a reflective surface are provided behind the polarized light source 6850 so that the polarization of the light is disturbed and reflected back toward the reflective polarizer 6830.

[0460] In yet another embodiment, another reflective polarizer is provided in the polarized light source 6850 and behind the previously disclosed linear polarizer. Wherein the respective transmission axes of the reflective polarizer and the linear polarizer are parallel to each other. The other reflective polarizer then reflects light back into the backlight, which has a polarization state that will not be transmitted by the linear polarizer. The light reflected back into the backlight passes through a diffuser associated with the polarized light source 6850, where the polarization state is disturbed and re-emitted, thereby recycling the light and improving efficiency.

[0461] In another embodiment, a system according to the principles of the present invention includes an eye imaging system. Figure 69 is an illustration of a compact optical display assembly that includes an eye imaging camera 6992 that captures an image of a user's eye 6880 that is coaxial with the display image provided to the user so that a full image of the user's iris can be reliably captured. The eye imaging camera 6992 is reflected into the lower optics by a reflective polarizer 6930 that includes a notch mirror coating facing the eye imaging camera 6992 that reflects wavelengths of light captured by the eye imaging camera 6992 (e.g., near infrared wavelengths) while transmitting wavelengths associated with image light 6835 (e.g., visible light wavelengths). Figure 69The eye ray 6995 shown in FIG. 1 illustrates how the field of view associated with the eye-imaging camera 6992 is relatively narrow, as it is multiply reflected by the lower optics to capture an image of the user's eye 6880. However, in order for the eye-imaging camera 6992 to be able to focus on the user's eye 6880, the eye-imaging camera 6992 needs to have a very close focus distance (e.g., 35 mm). Furthermore, the field of view and focus distance of the eye-imaging camera must account for the effect of reducing the optical power provided by the rotationally curved partial mirror 6860. To increase the efficiency of capturing light reflected from the user's eye 6880 and thereby enable brighter images of the eye, the rotationally curved partial mirror 6860 can be coated with a partial mirror coating that acts as a full mirror in the wavelengths captured by the eye-imaging camera 6992. For example, the coating can reflect 50% of the visible light associated with the image light and 90% of the near-infrared light associated with the eye light 6995. Wherein the reflection and associated changes in polarization state are similar to those associated with image light 6835, but in reverse order, this is because eye light rays 6995 originate from the user's eye 6880. An LED or other micro-lamp is provided adjacent to the user's eye 6880 to illuminate the user's eye 6880, wherein the wavelength associated with the LED or other micro-lamp is different from the wavelength associated with the image light 6835, such as, for example, a near-infrared wavelength (e.g., 850 nm, 940 nm, or 1050 nm). Alternatively, image light 6835 is used to illuminate the user's eye 6880 and a reflective polarizer 6930 having a low extinction ratio in reflection (e.g., a reflection extinction ratio of <15) is used to cause some of the eye light rays to be reflected toward the eye-imaging camera 6992.

[0462] In an alternative embodiment, the reflective and partially reflective surfaces can extend laterally to the sides of the area used to display images to the user. In this case, the eye imaging camera can be located adjacent to the field lens and pointed in a direction to provide a view from the field lens. Figure 70 The beam splitter and the rotating curved partial reflector shown in the figure are used to image the user's eyes after reflection. Figure 7068 is a diagram showing an eye imaging camera 7092 positioned to the side of the field lens 6820 and reflective polarizer 6830. The eye imaging camera 7092 is oriented so that the field of view captured by the eye imaging camera 7092 includes the user's eye 6880, as illustrated by eye light ray 7095. The quarter-wave film 6890 is also extended laterally to alter the polarization state of the eye light 7095 in the same manner as the polarization state of the image light is altered, so that the eye light passing through the beam splitter 6870 and the quarter-wave film 6890 is partially reflected by the rotationally curved partial mirror 6860 and then reflected by the beam splitter 6870 and then captured by the eye imaging camera 7092. By positioning the eye imaging camera 7092 to the side of the field lens 6820 and reflective polarizer 6830, the complexity of the optics associated with displaying images to the user is reduced. In addition, the space available for the eye imaging camera 7092 is increased because interference with the display optics is reduced. By positioning the eye imaging camera 7092 adjacent to the display optics, the eye image is captured nearly coaxially with the display image.

[0463] In yet another embodiment, a system in accordance with the principles of the present invention includes a field lens having an internal reflecting polarizer and one or more surfaces having optical power. Figure 71 7121 is an illustration of the upper optics including a field lens 7121 consisting of an upper prism 7122 and a lower prism 7123. The upper prism 7122 and the lower prism 7123 can be molded or ground and polished. A reflective polarizer 7124 is inserted on a flat surface between the upper prism 7122 and the lower prism 7123. The reflective polarizer 7124 can be a wire grid polarizer film or a multilayer dielectric polarizer as previously mentioned. The reflective polarizer 7124 can be bonded in place with a transparent UV-curable adhesive having the same refractive index as the upper prism 7122 or the lower prism 7123. Typically, the upper prism 7122 and the lower prism 7123 will have the same refractive index. The upper prism 7122 includes an angled surface for the illumination light 6837 to be provided to illuminate the reflected image source 6810. The illumination light is provided by a light source including a lamp (such as an LED), a backlight 7151, a diffuser 7152, and a polarizer 7153 as previously described. The lower prism 7123 includes a curved surface on the exit surface for controlling the wavefront of the image light 6835 supplied to the lower optical device. The upper prism may also include a curved surface on the upper surface next to the reflective image source 6810, as shown in FIG. Figure 716810. A plurality of reflective polarizers are provided, as shown in FIG. 68, for manipulating the chief ray angle of light at the surface of the reflective image source 6810. Illuminating light 6837 is polarized by a polarizer 7153 before entering the upper prism 7122. The transmission axes of the polarizer 7153 and the reflective polarizer 7124 are perpendicular to each other so that the illumination light 6837 is reflected by the reflective polarizer 7124, causing the illumination light to be redirected toward the reflective image source 6810. The polarization state of the illumination light 6837 is then altered by the reflective image source 6810 corresponding to the image content to be displayed, as previously described, and the resulting image light 6835 then passes through the reflective polarizer 7124 to form the bright and dark areas associated with the image displayed to the user's eye 6880.

[0464] In another embodiment, Figure 71 The field lens 7121 of FIG. 71 includes a polarizing beam splitter cube comprising two prisms, an upper prism 7122 and a lower prism 7123. In this case, the reflective polarizer 7124 is replaced by a polarization-sensitive coating so that light of one polarization state (typically, S-polarized light) is reflected and light of the other polarization state is transmitted. The illumination light 6837 is then provided with a polarization state that is reflected by the coating, and the image light is provided with a polarization state that is transmitted by the coating. As in FIG. Figure 71 , the beam splitter cube includes one or more curved surfaces in the upper prism 7122 or the lower prism 7123. The beam splitter cube can also include one or more angled surfaces into which the illumination light is supplied. The angled surfaces can include light control structures such as a microlens array to improve the uniformity of the illumination light 6837, or a lenticular lens array to collimate the illumination light 6837.

[0465] In yet another embodiment, Figure 71 The curved surface(s) or angled surface(s) illustrated in the figure can be molded onto a rectangular shaped beam splitter cube by the following steps: casting a UV curable material (e.g., UV curable acrylic) onto the flat surface of the beam splitter cube; placing a transparent mold with a cavity having the desired curvature onto the flat surface to force the UV curable material into the desired curvature; and applying UV light to cure the UV curable material. The beam splitter cube can be made of a material having the same or a different refractive index as the UV curable material.

[0466] In further embodiments, a polarization-sensitive reflective coating (such as a dielectric partially reflective mirror coating) can replace the Figure 686810 ). In this case, the reflective films and plates comprising reflective polarizer 6830 and beam splitter 6870 include polarization-sensitive coatings that substantially reflect light having one polarization state (e.g., S polarization) while substantially transmitting light having another polarization state (e.g., P polarization). Since the illumination light source includes polarizer 7153, the illumination light 6837 is one polarization state and it is not important that the reflective polarizer 7124 is sensitive to the polarization state in reflection; the polarization state only needs to be maintained and uniformly presented on the surface of the reflective image source 6810. However, it is important that the reflective polarizer 7124 is highly sensitive to the polarization state in transmission (e.g., extinction ratio > 200) to act as an effective polarization analyzer and to provide a high-contrast image (e.g., contrast ratio > 200) to the user's eye 6880.

[0467] In a further embodiment, Figure 71 The field lens 7121 shown in FIG. 7 can include a reflective polarizer 7124 having a curved surface (not shown) rather than a flat surface, and wherein the reflective polarizer 7124 is not a film but rather a polarization-sensitive coating, a printed wire grid polarizer, or a molded wire grid pattern (which is then metallized). In this case, the upper prism 7122 and the lower prism 7123 are fabricated as a matched pair having paired curved surfaces that together form the surface of the reflective polarizer. wherein the polarization-sensitive coating, printed wire grid, or molded wire grid pattern is applied to the paired curved surface associated with either the upper prism 7122 or the lower prism 7123, and a transparent adhesive is applied to the other paired surface to bond the upper prism 7122 and the lower prism 7123 together to form the field lens 7121 with an internally curved reflective polarizer 7121.

[0468] Another aspect of the present invention relates to making and providing an optical element for use in a see-through computer display system.In an embodiment, an optical element is lightweight, low cost, and of high optical quality.

[0469] In head-mounted displays, beam splitters can be used to direct illumination light from a light source toward a reflective image source (such as LCOS or DLP). A low-weight beam splitter with a flat partially reflective surface is desirable to provide good image quality. This flat partially reflective surface is particularly important when an eye camera is used for eye imaging...

Claims

1. A wearable head device comprising: frame; a plurality of optical modules housed in the frame, each of the plurality of optical modules including a respective reflective surface; an electronic component disposed proximate the frame and between respective reflective surfaces of two of the plurality of optical modules, the electronic component including a processor, the electronic component configured to be located within a first distance from a head of a user of the wearable head device; as well as a heat sink disposed between respective reflective surfaces of two optical modules of the plurality of optical modules and further configured to be located at a second distance from the user's head, the second distance being greater than the first distance, the heat sink being thermally coupled to the processor, Further comprising progressively tinted lenses, wherein: The progressively tinted lens includes a first portion associated with a first transmittance, the first portion being disposed in front of a portion of an upper optical module relative to the user's head to at least partially conceal the upper optical module, the upper optical module being included in a first of the two optical modules, and The progressively tinted lens further includes a second portion associated with a second transmittance higher than the first transmittance, the second portion being disposed in front of a lower optical module included in a first of the two optical modules to provide a see-through view of a surrounding environment.

2. The wearable head device according to claim 1 , wherein a first optical module of the two optical modules comprises: A folded optical element configured to provide light to a downwardly reflective surface, wherein: the lower reflective surface being configured to receive image light associated with a displayed image, The lower reflective surface is further configured to reflect at least a portion of the received image light toward the user's eyes, and The lower reflective surface is further configured to transmit a portion of scene light from the user's surroundings to provide a view of the surroundings and a view of the displayed image simultaneously.

3. The wearable head device according to claim 2, wherein: The folded optical element comprises at least two substantially flat reflective surfaces; Each of the substantially flat reflective surfaces is rotated along an optical axis of the respective substantially flat reflective surface and relative to the other substantially flat reflective surface, and The at least two substantially flat reflective surfaces are configured to provide light associated with the displayed image to the eyes of the user of the wearable head device. 4 . The wearable head device of claim 1 , wherein the first portion of the progressively tinted lens is disposed in front of at least a portion of the electronic component relative to the user's head.

5. A method for operating a wearable head device, comprising: dissipating heat via a heat sink disposed between two optical modules among a plurality of optical modules housed in a frame of the wearable head device, each of the plurality of optical modules including a respective reflective surface, the heat sink being thermally coupled to the processor, in: The wearable head device includes an electronic component disposed adjacent to a frame, the electronic component configured to be located within a first distance from a head of a user of the wearable head device and further disposed between respective reflective surfaces of two optical modules of the plurality of optical modules; and The electronic component includes a processor; and The heat sink is configured to be located at a second distance from the user's head, the second distance being greater than the first distance, in: The wearable head device further includes a progressively tinted lens; The progressively tinted lens includes a first portion associated with a first transmittance, the first portion being disposed in front of a portion of an upper optical module relative to the user's head to at least partially conceal the upper optical module, the upper optical module being included in a first of the two optical modules, and The progressively tinted lens further includes a second portion associated with a second transmittance higher than the first transmittance, the second portion being disposed in front of a lower optical module included in a first optical module of the two optical modules to provide a see-through view of a surrounding environment.

6. The method according to claim 5, further comprising: providing image light from a folded optical element of a first of the two optical modules to a lower reflective surface of the first optical module, wherein the image light is associated with a display image; reflecting at least a portion of the image light toward an eye of the user via the lower reflective surface; as well as A portion of scene light from the user's surroundings is transmitted through the lower reflective surface to provide a view of the surroundings and a view of the display image simultaneously.

7. The method according to claim 6, wherein: The folded optical element comprises at least two substantially flat reflective surfaces; Each of the substantially planar reflective surfaces is rotated along an optical axis of the respective substantially planar reflective surface and relative to the other substantially planar reflective surface; and The method further includes providing light associated with the displayed image to an eye of the user of the wearable head device via the substantially flat reflective surface.

8. The method of claim 5, wherein the first portion of the progressively tinted lens is positioned in front of at least a portion of the electronic component relative to the user's head.

Images