Eyewear with pupillary distance estimation

Abstract

An eyewear apparatus includes a sensor for measuring a deformation of an eyewear frame to estimate an interpupillary distance (IPD) of an eyewear user. The sensor is used to determine a head width (HB) of the user and estimate an IPD of the user. The processor displays an image on the display of the eyewear according to the estimated IPD to improve virtual object rendering, thereby improving an augmented reality (AR) viewing experience while reducing variability adjustment conflicts (VAMs). User profile data, such as age and gender, may be used to generate a more accurately estimated IPD of the user.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Application Serial No. 18 / 133,870, filed April 12, 2023, the contents of which are incorporated by reference in their entirety. TECHNICAL FIELD

[0002] The subject matter relates to eyewear devices, such as smart eyewear with a camera and a see-through display. BACKGROUND

[0003] Eyewear devices, such as smart eyewear, headsets, and headgear available today, integrate a camera, a see-through display, and an antenna. Such devices have various electronic components and sensors. BRIEF DESCRIPTION OF DRAWINGS

[0004] The accompanying drawings depict one or more implementations by way of example, and not by way of limitation. In the drawings, like reference numerals indicate similar or like elements.

[0005] FIG. 1A is a side view of an example hardware configuration of an eyewear device showing an optical assembly with an image display and applying a field of view adjustment to a user interface presented on the image display based on head or eye motion detected by the user;

[0006] FIG. 1B is FIG. 1A is a cross-sectional top view of a temple of the eyewear device of

[0007] FIG. 2A is a back view of an example hardware configuration of an eyewear device used in a system to identify a user of the eyewear device, the eyewear device including an eye scanner on a frame;

[0008] FIG. 2B is a back view of another example hardware configuration of an eyewear device used in a system to identify a user of the eyewear device, the eyewear device including an eye scanner on a temple;

[0009] FIG. 2C and FIG. 2D is a back view of an example hardware configuration of an eyewear device including two different types of image displays.

[0010] FIG. 3 shows FIG. 2A is a back perspective view of the eyewear device of

[0011] FIG. 4 is a cross-sectional view throughFIG. 3 Cross-sectional view of infrared emitter and frame intercept of eyewear device of

[0012] FIG. 5 Illustration depicting detection of eye gaze direction;

[0013] FIG. 6 Illustration depicting detection of eye position;

[0014] FIG. 7 Illustration depicting an example of visible light captured by visible light camera as a raw image;

[0015] FIG. 8A Rear view of example eyewear device depicting strain gauge sensors on the bridge of the frame;

[0016] FIG. 8B Rear view of example eyewear device depicting strain gauge sensors on the frame above the optical assembly;

[0017] FIG. 8C Rear view of example eyewear device depicting strain gauge sensors on the temples;

[0018] FIG. 9 Block diagram of electronic components of eyewear device;

[0019] FIG. 10 Waveform of strain gauge sensor data for a characteristic event;

[0020] FIG. 11 Method of using strain gauge sensors to control eyewear device;

[0021] FIG. 12A Top view of example eyewear device depicting no distortion of the frame when worn on a user's head;

[0022] FIG. 12B Top view of example eyewear device depicting distortion of the frame when worn on a user's head;

[0023] FIG. 13A Graph of nominal interpupillary distance (IPD) covering 34% of the U.S. population dataset;

[0024] FIG. 13B Graph of interpupillary distance (IPD) according to determined head width covering 45% of the U.S. population dataset; and

[0025] FIG. 14 Flowchart showing a method of adjusting a display to improve a user's viewing experience by estimating the user's IPD from distortion of the frame. DETAILED DESCRIPTION

[0026] Eyeglasses including sensors for measuring deformation to estimate inter-pupillary distance (IPD) of a user. The sensors are used to determine head width (HB) to estimate IPD. A processor displays images on a display of the eyeglasses according to the estimated IPD to improve virtual object rendering to improve virtual augmented reality (AR) viewing experience while reducing vergence-accommodation conflict (VAM). User profile data such as age and gender can be used to generate more accurate estimated IPD.

[0027] Additional objects, advantages, and novel features of the examples will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or can be learned by practice of the examples. The objects and advantages of the present subject matter can be realized and attained by means of the instrumentalities particularly pointed out in the appended claims.

[0028] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the relevant technology. However, it will be apparent to one skilled in the art that the present technology can be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present teaching.

[0029] The term "coupled" as used herein means any logical, optical, physical, or electrical connection, link, or the like by which signals or light from one system element are passed to another coupled element. Elements or devices coupled are not necessarily directly connected to one another and can be separated by intermediate components, elements, or communication media that can modify, manipulate, or carry the light or signals unless otherwise described.

[0030] For purposes of illustration and discussion, the orientation of the eyeglasses device, associated components, and any complete device incorporating, such as the eye scanner and camera shown in any of the accompanying drawings, is given by way of example only. In operation for a particular variable optical processing application, the eyeglasses device can be oriented in any other direction that is suitable for the particular application of the eyeglasses device; for example, upward, downward, sideways, or any other orientation. Furthermore, any directional terms, such as front, back, in, out, toward, left, right, lateral, longitudinal, up, down, top, bottom, top, bottom, and side, as used in the scope of this document, are used by way of example only and are not limited to the orientation or direction of any optical or component of an optical device as constructed or otherwise described herein.

[0031] Reference will now be made in detail to the examples shown in the accompanying drawings and discussed below.

[0032] FIG. 1A is a side view of an example hardware configuration of the eyewear device 100, including an optical assembly 180B (e.g., positioned on the right side of the eyewear device 100 as shown) with an image display 180D. FIG. 2A The eyewear device 100 includes a plurality of visible light cameras 114A-B (e.g., positioned on the left and side of the eyewear device 100 as shown) that form a stereo camera, with the visible light camera 114B located on the temple portion 110B. FIG. 7

[0033] The visible light cameras 114A-B (e.g., positioned on the left and side of the eyewear device 100 as shown) have image sensors that are sensitive to the visible light range of wavelengths. Each of the visible light cameras 114A-B has a different forward-facing coverage angle, for example, the visible light camera 114B has a depicted coverage angle 111B. The coverage angle is the range of angles at which the image sensors of the visible light cameras 114A-B pick up electromagnetic radiation and generate images. Examples of such visible light cameras 114A-B include high resolution complementary metal-oxide-semiconductor (CMOS) image sensors and video graphics array (VGA) cameras capable of, such as, 640p (e.g., 640 x 480 pixels, totaling 0.3 megapixels), 720p, or 1080p. Image sensor data from the visible light cameras 114A-B, as well as geo-location data, are captured together, digitized by an image processor, and stored in memory.

[0034] To provide stereoscopic vision, the visible light cameras 114A-B can be coupled to an image processor (element 912) for digital processing with timestamps of when the scene images are captured. The image processor 912 includes circuitry for receiving signals from the visible light cameras 114A-B and processing the signals from the visible light cameras 114A-B into a format suitable for storage in memory (element 934). The timestamps can be added by the image processor 912 or other processor that controls the operation of the visible light cameras 114A-B. The visible light cameras 114A-B allow the stereo camera to mimic human binocular vision. The stereo camera provides for the reproduction of three-dimensional images (element 758A-B) based on two captured images (element 758A-B) from the visible light cameras 114A-B, respectively, with the same timestamp. FIG. 9 FIG. 9 FIG. 7 FIG. 7 ​​​​the ability to generate three-dimensional images 715 (e.g., element 715 of FIG. 7). Such three-dimensional images 715 allow for immersive, realistic experiences, e.g., for virtual reality or video games. For stereoscopic vision, a pair of images 758A-B is generated at a given moment in time - one for each visible light camera 114A-B. When the pair of generated images 758A-B from the forward-facing coverage angles 111A-B of the visible light cameras 114A-B are stitched together (e.g., by the image processor 912), depth perception is provided by the optical assembly 180A-B.

[0035] In one example, the user interface field of view adjustment system includes the eyewear device 100. The eyewear device 100 includes a frame 105, a temple portion 110B extending from a side 170B of the frame 105, and a see-through image display 180D including an optical assembly 180B (e.g., element 180 of FIG. 1) to present a graphical user interface to a user. The eyewear device 100 includes a visible light camera 114A connected to the frame 105 or the temple portion 110A to capture images of a scene. The eyewear device 100 also includes another visible light camera 114B connected to the frame 105 or the other temple portion 110B to capture (e.g., at least substantially simultaneously with the visible light camera 114A) another image of the scene that overlaps with the image portion. Although not shown in FIG. 1, the user interface field of view adjustment system also includes a processor 932 coupled to the eyewear device 100 and connected to the visible light cameras 114A-B, a memory 934 accessible to the processor 932, and programming in the memory 934, e.g., in the eyewear device 100 itself or another part of the user interface field of view adjustment system. FIG. 2A-2B FIG. 1A-1B In one example, the user interface field of view adjustment system includes the eyewear device 100. The eyewear device 100 includes a frame 105, a temple portion 110B extending from a side 170B of the frame 105, and a see-through image display 180D including an optical assembly 180B (e.g., element 180 of FIG. 1) to present a graphical user interface to a user. The eyewear device 100 includes a visible light camera 114A connected to the frame 105 or the temple portion 110A to capture images of a scene. The eyewear device 100 also includes another visible light camera 114B connected to the frame 105 or the other temple portion 110B to capture (e.g., at least substantially simultaneously with the visible light camera 114A) another image of the scene that overlaps with the image portion. Although not shown in FIG. 1, the user interface field of view adjustment system also includes a processor 932 coupled to the eyewear device 100 and connected to the visible light cameras 114A-B, a memory 934 accessible to the processor 932, and programming in the memory 934, e.g., in the eyewear device 100 itself or another part of the user interface field of view adjustment system.

[0036] In one example, the user interface field of view adjustment system includes the eyewear device 100. The eyewear device 100 includes a frame 105, a temple portion 110B extending from a side 170B of the frame 105, and a see-through image display 180D including an optical assembly 180B (e.g., element 180 of FIG. 1) to present a graphical user interface to a user. The eyewear device 100 includes a visible light camera 114A connected to the frame 105 or the temple portion 110A to capture images of a scene. The eyewear device 100 also includes another visible light camera 114B connected to the frame 105 or the other temple portion 110B to capture (e.g., at least substantially simultaneously with the visible light camera 114A) another image of the scene that overlaps with the image portion. Although not shown in FIG. 1, the user interface field of view adjustment system also includes a processor 932 coupled to the eyewear device 100 and connected to the visible light cameras 114A-B, a memory 934 accessible to the processor 932, and programming in the memory 934, e.g., in the eyewear device 100 itself or another part of the user interface field of view adjustment system. FIG. 1A FIG. 1B In one example, the user interface field of view adjustment system includes the eyewear device 100. The eyewear device 100 includes a frame 105, a temple portion 110B extending from a side 170B of the frame 105, and a see-through image display 180D including an optical assembly 180B (e.g., element 180 of FIG. 1) to present a graphical user interface to a user. The eyewear device 100 includes a visible light camera 114A connected to the frame 105 or the temple portion 110A to capture images of a scene. The eyewear device 100 also includes another visible light camera 114B connected to the frame 105 or the other temple portion 110B to capture (e.g., at least substantially simultaneously with the visible light camera 114A) another image of the scene that overlaps with the image portion. Although not shown in FIG. 1, the user interface field of view adjustment system also includes a processor 932 coupled to the eyewear device 100 and connected to the visible light cameras 114A-B, a memory 934 accessible to the processor 932, and programming in the memory 934, e.g., in the eyewear device 100 itself or another part of the user interface field of view adjustment system. FIG. 2B FIG. 9 In one example, the user interface field of view adjustment system includes the eyewear device 100. The eyewear device 100 includes a frame 105, a temple portion 110B extending from a side 170B of the frame 105, and a see-through image display 180D including an optical assembly 180B (e.g., element 180 of FIG. 1) to present a graphical user interface to a user. The eyewear device 100 includes a visible light camera 114A connected to the frame 105 or the temple portion 110A to capture images of a scene. The eyewear device 100 also includes another visible light camera 114B connected to the frame 105 or the other temple portion 110B to capture (e.g., at least substantially simultaneously with the visible light camera 114A) another image of the scene that overlaps with the image portion. Although not shown in FIG. 1, the user interface field of view adjustment system also includes a processor 932 coupled to the eyewear device 100 and connected to the visible light cameras 114A-B, a memory 934 accessible to the processor 932, and programming in the memory 934, e.g., in the eyewear device 100 itself or another part of the user interface field of view adjustment system. FIG. 9 ​​​elements 934). The programming executed by the processor 932 configures the eyewear device 100 to perform functions, including functions to present an initial display image of a display image sequence via the see-through image display 180C-D, the initial display image having an initial field of view corresponding to an initial head direction or an initial eyewear gaze direction FIG. 5 elements 230).

[0037] The processor 932 executes programming further configures the eyewear device 100 to detect motion of a user of the eyewear device by (i) tracking head motion of the user’s head via a head motion tracker (e.g., elements 109) or (ii) tracking eye motion of the user’s eyes of the eyewear device 100 via an eye motion tracker (e.g., elements 213). The processor 932 executes programming further configures the eyewear device 100 to determine a field of view adjustment to the initial field of view of the initial display image based on the detected motion of the user. The field of view adjustment includes a successive field of view corresponding to a successive head direction or a successive eye direction. The processor 932 executes programming further configures the eyewear device 100 to generate a successive display image of the display image sequence based on the field of view adjustment. The processor 932 executes programming further configures the eyewear device 100 to present the successive display image via the see-through image display 180C-D of the optical assembly 180A-B. FIG. 1B FIG. 2B 、 FIG. 5

[0038] FIG. 1B is a cross-sectional top view of the temple of the eyewear device 100 of FIG. 1A FIG. 1 1. The configuration and placement of the visible light camera 114A is substantially similar to the visible light camera 114B, except connected and coupled on the side 170A. As shown, the eyewear device 100 includes the visible light camera 114B and a circuit board, which can be a flexible printed circuit board (PCB) 140. A hinge 126B connects the temple portion 1 10B to the temple 125B of the eyewear device 100. In some examples, components of the visible light camera 114B, the flexible PCB 140, or other electrical connectors or contacts can be located on the temple 125B or the hinge 126B.

[0039] ​​As shown, the eyewear device 100 has a head motion tracker 109 that includes, for example, an inertial measurement unit (IMU). An inertial measurement unit is an electronic device that measures and reports a body's specific force, angular rate, and sometimes the magnetic field surrounding the body using a combination of accelerometers and gyroscopes, sometimes using magnetometers as well. An inertial measurement unit works by using one or more accelerometers to detect linear acceleration and one or more gyroscopes to detect rate of rotation. A typical configuration of an inertial measurement unit contains one accelerometer, gyroscope, and magnetometer for each of the three axes: a horizontal axis for left-right movement (X), a vertical axis (Y) for up-down movement, and a depth or distance axis for forward-backward movement (Z). The magnetometer measures the direction (e.g., south, north, etc., similar to a compass that generates a heading reference) and magnitude of the magnetic field. The three accelerometers detect acceleration along the aforementioned horizontal, vertical, and depth axes, which can be defined relative to the ground, the eyewear device 100, or a user wearing the eyewear device 100. The gyroscope measures angular rate about the previously mentioned X, Y, and Z axes.

[0040] The eyewear device 100 detects motion of a user of the eyewear device 100 by tracking head motion of the user's head via the head motion tracker 109. The head motion includes a change in head orientation on a horizontal axis, a vertical axis, or a combination thereof relative to an initial head orientation during presentation of an initial display image on the image display. In one example, tracking head motion of the user's head via the head motion tracker 109 includes measuring, via the inertial measurement unit 109, an initial head orientation on a horizontal axis (e.g., X-axis), a vertical axis (e.g., Y-axis), or a combination thereof (e.g., lateral or diagonal motion). Tracking head motion of the user's head via the head motion tracker 109 also includes measuring, via the inertial measurement unit 109, a continuous head orientation on the horizontal axis, the vertical axis, or the combination thereof during presentation of the initial display image.

[0041] Tracking head motion of the user's head via the head motion tracker 109 also includes determining a change in head orientation based on both the initial head orientation and the continuous head orientation. Detecting motion of a user of the eyewear device 100 also includes determining, in response to tracking head motion of the user's head via the head motion tracker 109, that the change in head orientation exceeds a deviation angle threshold on the horizontal axis, the vertical axis, or the combination thereof. The deviation angle threshold is between about 3° to 10°. As used herein, the term "about" when referring to an angle means ±10% of the base amount.

[0042] Changes along the horizontal axis slide three-dimensional objects (such as characters, Bitmoji, application icons, etc.) in and out of the field of view by, for example, hiding, un-hiding, or otherwise adjusting the visibility of the three-dimensional objects. Changes along the vertical axis, for example, when the user looks up, in one example, display weather information, the time of day, the date, calendar appointments, etc. In another example, when the user looks down on the vertical axis, the eyewear device 100 can power down.

[0043] The temple portion 110B includes a temple body 211 and a temple cover, where in FIG. 1B Disposed inside the temple portion 110B are various interconnected circuit boards, such as PCBs or flexible PCBs, that include controller circuitry for the visible light camera 114B, one or more microphones 130, one or more speakers 132, low-power wireless circuitry (e.g., for wireless short-range network communication via Bluetooth™), high-speed wireless circuitry (e.g., for wireless local area network communication via Wi-Fi).

[0044] The visible light camera 114B is coupled to or disposed on the flexible PCB 140 and is covered by a visible light camera cover lens that is aimed through one or more openings formed in the temple portion 110B. In some examples, the frame 105 connected to the temple portion 110B includes one or more openings for the visible light camera cover lens. The frame 105 includes a forward-facing side that is configured to face outward and away from the user’s eye. The openings for the visible light camera cover lens are formed on and through the forward-facing side. In this example, the visible light camera 114B has a face-out coverage angle 111B that has a line of sight or perspective with the user’s eye (e.g., the right eye) of the eyewear device 100. The visible light camera cover lens can also be adhered to a face-out surface of the temple portion 110B with the openings formed to have a face-out coverage angle but in a different outward-facing direction. The coupling can also be achieved indirectly via an intermediate component.

[0045] The visible light camera 114A is connected to the see-through image display 180C of the optical assembly 180A to generate a background scene for the continuous display image. Another visible light camera 114B is connected to the see-through image display 180D of the optical assembly 180B to generate another background scene for the continuous display image. The background scenes partially overlap to present a three-dimensional observable area for the continuous display image.

[0046] A flexible PCB 140 is disposed inside the temple portion 110B and coupled to one or more other components housed in the temple portion 110B. Although shown as being formed on a circuit board on the temple portion 110B, the visible light camera 114B may be formed on a circuit board on the temple portion 110A, temple 125A-B, or frame 105.

[0047] FIG. 2A This is a rear view of an example hardware configuration of an eyeglass device 100, which includes an eye scanner 113 located on a frame 105 for determining the eye position and gaze direction of the wearer / user of the eyeglass device 100. FIG. 2A As shown, the glasses device 100 is configured for wear by a user. FIG. 2A The example shown is eyeglasses. Eyeglasses device 100 can take other forms and may include other types of frames; for example, headwear, headphones, or helmets.

[0048] In the example of eyeglasses, eyeglass device 100 includes a frame 105, which includes a lens ring 107A connected to a lens ring 107B via a bridge 106 adapted for use by a user's nose. Lens rings 107A-B include corresponding apertures 175A-B that house corresponding optical elements 180A-B, such as lenses and perspective displays 180C-D. As used herein, the term lens refers to a sheet of transparent or translucent glass or plastic having curved and flat surfaces that cause light to converge / diverge or cause little or no convergence / divergence.

[0049] Although shown as having two optical elements 180A-B, the eyeglasses device 100 may include other arrangements, such as a single optical element, depending on the application or intended user. As further shown, the eyeglasses device 100 includes temple portions 110A adjacent to side 170A of the frame 105 and temple portions 110B adjacent to side 170B of the frame 105. Temple portions 110A-B may be integrated into the frame 105 on the respective sides 170A-B (as shown), or implemented as separate components attached to the frame 105 on the respective sides 170A-B. Alternatively, temple portions 110A-B may be integrated into temples 125A-B or attached to other components (not shown) of the frame 105.

[0050] exist FIG. 2AIn the example of eyeglasses device 100, eye scanner 113 includes infrared emitter 115 and infrared camera 120. Visible light cameras typically include a blue light filter to block infrared light detection, and in one example, infrared camera 120 is a visible light camera, such as a low resolution video graphics array (VGA) camera (e.g., 640 x 480 pixels, for a total of 0.3 megapixels), with the blue light filter removed. Infrared emitter 115 and infrared camera 120 are co-located on eyeglass frame 105, for example, both are shown as connected to the upper portion of rim 107A. Eyeglass frame 105 or one or more of temple portions 110A-B include a circuit board (not shown) that includes infrared emitter 115 and infrared camera 120. Infrared emitter 115 and infrared camera 120 can be connected to the circuit board by, for example, soldering.

[0051] Other arrangements of infrared emitter 115 and infrared camera 120 can be implemented, including arrangements where infrared emitter 115 and infrared camera 120 are both located on rim 107B, or at different locations on eyeglass frame 105, for example, infrared emitter 115 is located on rim 107A and infrared camera 120 is located on rim 107B. In another example, infrared emitter 115 is located on eyeglass frame 105 and infrared camera 120 is located on one of temple portions 110A-B, or vice versa. Infrared emitter 115 can be connected to any location on eyeglass frame 105, temple portion 110A, or temple portion 110B to emit a pattern of infrared light. Similarly, infrared camera 120 can be connected substantially anywhere on eyeglass frame 105, temple portion 110A, or temple portion 110B to capture at least one reflection change in the pattern of infrared light emission.

[0052] Infrared emitter 115 and infrared camera 120 are arranged to face inward toward a user’s eyes with a portion or all of the field of view of the eye in order to identify a corresponding eye position and gaze direction. For example, infrared emitter 115 and infrared camera 120 are positioned directly in front of the eyes, on the upper portion of eyeglass frame 105, or in temple portions 110A-B at either end of eyeglass frame 105.

[0053] FIG. 2B is another example hardware configuration of eyeglasses device 200. In this example configuration, eyeglasses device 200 is depicted as including eye scanner 213 on temple 210B. As shown, infrared emitter 215 and infrared camera 220 are co-located on temple 210B. It should be understood that eye scanner 213 or one or more components of eye scanner 213 can be located on temple 210A and other locations of eyeglasses device 200, such as eyeglass frame 105. Infrared emitter 215 and infrared camera 220 are similar to infrared emitter 115 and infrared camera 120, respectively, and are described in more detail above. FIG. 2Athe infrared emitter and infrared camera in FIGS. 1-2, but the eye scanner 213 can vary to be sensitive to different wavelengths of light, as previously described in FIG. 2A

[0054] Similar to FIG. 2A , the eyewear device 200 includes a frame 105 that includes a rim 107A connected to a rim 107B via a bridge 106; and the rims 107A-B include respective apertures that house respective optical assemblies 180A-B that include see-through displays 180C-D.

[0055] FIG. 2C -D is a rear view of an example hardware configuration of the eyewear device 100 that includes two different types of see-through image displays 180C-D. In one example, the see-through image displays 180C-D of the optical assemblies 180A-B include integrated image displays. As shown in FIG. 2C , the optical assemblies 180A-B include any suitable type of display matrix 180C-D, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a waveguide display, or any other such display. The optical assemblies 180A-B also include one or more optical layers 176 that can include lenses, optical coatings, prisms, mirrors, waveguides, optical strips, and other optical components in any combination. The optical layers 176A-N can include prisms that are of a suitable size and configuration and that include a surface (e.g., first) for receiving light from the display matrix and another surface (e.g., second) for emitting light to the user’s eye. The prisms of the optical layers 176A-N extend over all or at least a portion of the respective apertures 175A-B formed in the rims 107A-B to allow the user to see the other surface of the prisms when the user’s eyes are looking through the corresponding rims 107A-B. The surface of the prisms of the optical layers 176A-N that receives light faces upward from the frame 105 and the display matrix covers the prisms such that photons and light emitted by the display matrix impinge on the surface. The prisms are sized and shaped such that the light rays refract within the prisms and are directed by the other surface of the prisms of the optical layers 176A-N toward the user’s eye. In this regard, the other surface of the prisms of the optical layers 176A-N can be convex to direct the light rays toward the center of the eye. The prisms can optionally be sized and shaped to magnify the image projected by the see-through image displays 180C-D, and the light travels through the prisms so that the image viewed from the other surface is larger in one or more dimensions than the image emitted from the see-through image displays 180C-D.

[0056] In another example, the see-through image displays 180C-D of the optical assemblies 180A-B include waveguide displays as FIG. 2D ​The illustrated projection image display. The optical assembly 180A-B includes a laser projector 150, which is a three-color laser projector using a scanning mirror or galvanometer. During operation, a light source such as the laser projector 150 is disposed in or on one of the temple pieces 125A-B of the eyewear device 100. The optical assembly 180A-B includes one or more optical strips 155A-N that are spaced apart across a lens width of the optical assembly 180A-B or across a lens depth between a front surface and a back surface of the lens.

[0057] As a photon projected by the laser projector 150 travels through the lens of the optical assembly 180A-B, the photon encounters an optical strip 155A-N. When a particular photon encounters a particular optical strip, the photon is diverted to the user’s eye or on to the next optical strip. The combination of modulation of the laser projector 150 and modulation of the optical strips can control a particular photon or beam of light. In an example, the processor controls the optical strips 155A-N by issuing mechanical, acoustic, or electromagnetic signals. Although shown with two optical assemblies 180A-B, the eyewear device 100 can include other arrangements such as a single or three optical assemblies, or the optical assemblies 180A-B can have different arrangements depending on the application or intended user of the eyewear device 100.

[0058] As FIG. 2C-2D further shown in FIG. 1, the eyewear device 100 includes a temple piece portion 110A adjacent to the side 170A of the frame 105 and a temple piece portion 110B adjacent to the side 170B of the frame 105. The temple piece portions 110A-B can be integrated into the frame 105 on the respective sides 170A-B (as shown), or implemented as separate components that are attached to the frame 105 on the respective sides 170A-B. Alternatively, the temple piece portions 110A-B can be integrated into the temple pieces 125A-B that are attached to the frame 105.

[0059] In one example, the see-through image display includes a see-through image display 180C and a see-through image display 180D. The eyewear device 100 includes apertures 175A-B that accommodate the respective optical assemblies 180A-B. The optical assembly 180A includes the see-through image display 180C (e.g., FIG. 2C a display matrix or optical strips and a projector not shown). The optical assembly 180B includes the see-through image display 180D (e.g., FIG. 2CThe display matrix (not shown) or optical strip 155A-N and projector 150. The continuous field of view of the continuously displayed image includes an angle of view between approximately 15° and 30° measured horizontally, vertically, or diagonally, and more specifically 24°. A continuously displayed image with a continuous field of view represents a combined three-dimensional observable area that is visible by stitching together two displayed images presented on an image display.

[0060] As used herein, “viewing angle” describes the angular range of the field of view associated with the displayed image presented on each of the image displays 180C-D of the optical components 180A-B. “Coverage angle” describes the angular range of the image that the lens of the visible light camera 114A-B or the infrared camera 220 can image. Typically, the image circle produced by the lens is large enough to completely cover the film or sensor, and may include some vignetting (i.e., the brightness or saturation of the image decreases towards the periphery compared to the center of the image). If the coverage angle of the lens does not fill the sensor, the image circle will be visible, typically with strong vignetting towards the lens rim, and the effective viewing angle will be limited to the coverage angle. “Field of view” is intended to describe the field of view of the observable area seen by the user of the eyewear device 100 through the image presented on the image displays 180C-D of the optical components 180A-B via his or her eyes. The image display 180C of the optical components 180A-B may have a field of view with a coverage angle between 15° and 30°, for example 24°, and a resolution of 480x480 pixels.

[0061] FIG. 3 It shows FIG. 2A A rear perspective view of the eyewear device. The eyewear device 100 includes an infrared emitter 215, an infrared camera 220, a front frame 330, a rear frame 335, and a circuit board 340. FIG. 3 As can be seen, the upper part of the lens rim of the eyeglasses device 100 includes a front part 330 and a rear part 335. An opening for the infrared emitter 215 is formed on the rear part 335.

[0062] As shown in the circular cross-section 4 of the upper middle portion of the lens rim, a circuit board, serving as a flexible PCB 340, is sandwiched between the front portion 330 and the rear portion 335 of the lens frame. The temple portion 110A is also shown in more detail attached to the temple 125A via a hinge 126A. In some examples, components of the eye tracker 213, including the infrared emitter 215, the flexible PCB 340, or other electrical connectors or contacts, may be located on the temple 125A or the hinge 126A.

[0063] FIG. 4 It is a cross-sectional view through the infrared emitter 215 and the lens frame, corresponding to FIG. 3 The circular cross-section of the eyeglasses device is 4. FIG. 4The multiple layers of the eyewear device 100 are shown in cross-section, as illustrated, the frame includes a frame front 330 and a frame back 335. A flexible PCB 340 is disposed on the frame front 330 and connected to the frame back 335. The infrared emitter 215 is disposed on the flexible PCB 340 and covered by an infrared emitter cover lens 445. For example, the infrared emitter 215 is reflowed to the back of the flexible PCB 340. Reflow melts solder paste to connect two components by subjecting the flexible PCB 340 to controlled heat to attach the infrared emitter 215 to one or more contact pads formed on the back of the flexible PCB 340. In one example, reflow is used to surface mount the infrared emitter 215 on the flexible PCB 340 and electrically connect the two components. However, it should be appreciated that, for example, through-holes can be used to connect leads from the infrared emitter 215 to the flexible PCB 340 via interconnects.

[0064] The frame back 335 includes an infrared emitter opening 450 for the infrared emitter cover lens 445. The infrared emitter opening 450 is formed on the back-facing side of the frame back 335, which is configured to face inward toward the user’s eye. In this example, the flexible PCB 340 can be connected to the frame front 330 via a flexible PCB adhesive 460. The infrared emitter cover lens 445 can be connected to the frame back 335 via an infrared emitter cover lens adhesive 455. The coupling can also be achieved indirectly via an intermediate component.

[0065] In one example, the processor 932 utilizes the eye tracker 213 to determine an eye gaze direction 230 of the wearer’s eye 234 as shown, and an eye position 236 of the eye 234 within the eyebox as shown. FIG. 5 FIG. 6 The eye tracker 213 is a kind of scanner that uses infrared light illumination (e.g., near-infrared, shortwave infrared, midwave infrared, longwave infrared, or far-infrared) to capture images of changes in reflection of infrared light from the eye 234 to determine the gaze direction 230 of the pupil 232 of the eye 234, and the position 236 of the eye relative to the see-through display 180D.

[0066] FIG. 7 An example is depicted in which visible light is captured with a camera. The visible light is captured by a visible light camera 114A having a visible light camera field of view 111A as a raw image 758A. The visible light is captured by a visible light camera 114B having a visible light camera field of view 111B (with overlap 713 with the field of view 111A) as a raw image 758B. Based on processing of the raw image 758A and the raw image 758B, a three-dimensional depth map 715 of the three-dimensional scene is generated by the processor 932, hereinafter referred to as the image.

[0067] ​FIG. 8A A rear view of an example hardware configuration of eyewear device 100 is depicted, including a strain gauge sensor 802A located on the bridge 106 of the frame 105. The strain gauge sensor 802A measures the force applied to the eyewear device 100 by the user's head by measuring the deformation of the frame 105. The processor 932 receives signals from the strain gauge 802A and determines when one or more characteristic events occur (see FIG. 10 ). The sampling rate of the strain gauge 802A is selected by the processor 932 to meet the sensitivity and power requirements of the eyewear device 100. For example, when the eyewear device 100 is in a sleep mode where the eyewear device 100 is not worn and the displays 180C-D are powered off, the sampling rate is set to a lower rate, such as 20 Hz, to reduce power consumption. When the eyewear device 100 is powered on and in use by a user, the processor sampling rate of the strain gauge 802A is set to a higher sampling rate, such as 50 Hz, to improve performance and reduce the effects of the strain gauge sensor 802A's perception noise. Other sampling rates can be used to improve the performance of the sensor in various applications. In another example, the strain gauge 802A senses strain at a lower sampling rate when the processor 932 is in a sleep mode and at a higher sampling rate when the processor 932 is powered on.

[0068] FIG. 8B A perspective rear view of an example hardware configuration of eyewear device 100 is depicted, with a respective strain gauge 802B located on the frame 105 above each optical assembly 180A-B.

[0069] FIG. 8C A perspective rear view of an example hardware configuration of eyewear device 100 is depicted, with a strain gauge 802C located in the temple 125A.

[0070] The temperature of the strain gauge sensors 802A-C can affect the accuracy of the measurements generated by the strain gauge sensors 802A-C. In one example, the strain gauge sensors 802A-C are placed on the eyewear device 100 in a location that is nominal from temperature fluctuations caused by electronic components and the user, such as away from the processor 932, to provide consistent sensor measurements. In another example of the eyewear device 100, the strain gauge sensors 802A-C are placed near heat generating electronic components. A temperature sensor 804, such as a thermocouple, is placed near the strain gauge sensors 802A-C so the processor 932 can calibrate the strain gauge sensors 802A-C measurements according to the measured temperature near the strain gauge sensors 802A-C. An example of a strain gauge sensor 802A-C is a linear strain gauge sensor, such as the SGD-7 / 1000-LY13 linear strain gauge sensor available from Omega Engineering Inc. located in Norwalk, CT.

[0071] The processor 932 uses measurements from the strain gauge sensors 802A-C to determine, for example, a head size of a user wearing the eyewear device 100. In one example, the processor 932 uses measurements from the strain gauge sensors 802A-C while the user is wearing the eyewear device 100 and compares the measurements to a head size database 970 stored in the memory 934 of the eyewear device 100. In another example, the head size database 970 is stored at a location remote from the eyewear device 100 and accessed via the network 995. When multiple users share a single eyewear device 100, measurements of user head size, such as head width, can be used to identify a particular user or a particular type of user.

[0072] In one example, if the sensor measurements indicate that the eyewear device 100 is bent 1 degree near the strain gauge sensors 802A-C, the processor 932 detects that an adult is wearing the eyewear device 100. If the sensor measurements indicate that the eyewear device 100 is bent 0.5 degrees near the strain gauge sensors 802A-C, the processor 932 detects that a child is wearing the eyewear device 100. The processor 932 can operate the eyewear device 100 differently depending on the determined user attributes, such as adult or child, or parent and child in a family. In one example, if a child is determined by the processor 932 to be using the eyewear device 100, features of the eyewear device 100 can be customized, or content limited, such as limiting access to adult material.

[0073] FIG. 9 A high-level functional block diagram is depicted that includes example electronic components disposed in the eyewear device 100 / 200. The electronic components shown include the processor 932, the memory 934, and see-through image displays 180C and 180D that include embedded antennas.

[0074] The memory 934 includes instructions that include computer readable code for execution by the processor 932 to implement the functionality of the eyewear device 100 / 200, including instructions (code) for the processor 932 to control in the image 715. The processor 932 receives power from a battery (not shown) and executes instructions stored in the memory 934, or integrated with the on-chip processor 932, to perform the functionality of the eyewear device 100 / 200 and to communicate with external devices via wireless connections.

[0075] The user interface adjustment system 900 includes a wearable device that is the eyewear device 100 (e.g., in FIG. 2BThe user interface adjustment system 900 includes the eyewear device 100, which includes the eye tracking system 210, the visible light cameras 114A-B, the infrared emitter 215, and the infrared camera 220. The user interface adjustment system 900 also includes a mobile device 990 and a server system 998 connected via various networks. The mobile device 990 can be a smartphone, tablet, laptop, access point, or any other such device capable of connecting with the eyewear device 100 using the low-power wireless connection 925 and the high-speed wireless connection 937. The mobile device 990 is connected to the server system 998 and the network 995. The network 995 can include any combination of wired and wireless connections.

[0076] The eyewear device 100 includes at least two visible light cameras 114A-B (e.g., one associated with each side 170A-B). The eyewear device 100 also includes see-through image displays 180C-D of the optical assembly 180A-B (e.g., one associated with each side 170A-B). In the present disclosure, the image displays 180C-D are optional. The eyewear device 100 also includes an image display driver 942, an image processor 912, a low-power circuit 920, and a high-speed circuit 930. The components for the eyewear device 100 are shown in FIG. 1. FIG. 9 The components shown in FIG. 1 are located on one or more circuit boards, such as a PCB or flexible PCB located in the temples. Alternatively or additionally, the depicted components can be located in the temples, frame, hinges, or nose bridge of the eyewear device 100. The visible light cameras 114A-B can include digital camera elements such as complementary metal-oxide-semiconductor (CMOS) image sensors, charge-coupled devices, lenses, or any other respective visible or light capturing elements that can be used to capture data, including images of a scene with unknown objects.

[0077] The eye tracking programming 945 implements user interface field of view adjustment instructions, including causing the eyewear device 100 to track, via the eye tracker 213, eye movements of a user’s eyes of the eyewear device 100. Other implemented instructions (functions) cause the eyewear device 100 to determine, based on the detected eye movements of the user corresponding to successive eye directions, a field of view adjustment to an initial field of view of an initial display image. Another implemented instruction generates, based on the field of view adjustment, successive display images of a sequence of display images. The successive display images are presented as visible output to the user via the user interface. This visible output appears on the see-through image displays 180C-D of the optical assembly 180A-B, which are driven by the image display driver 942 to present the sequence of display images, including the initial display image with the initial field of view and the successive display images with the successive fields of view.

[0078] As FIG. 9As shown, the high-speed circuit 930 includes a high-speed processor 932, a memory 934, and a high-speed wireless circuit 936. In this example, an image display driver 942 is coupled to the high-speed circuit 930 and operated by the high-speed processor 932 to drive the image displays 180C-D of the optical assemblies 180A-B. The high-speed processor 932 can be any processor capable of managing the high-speed communications and operations of any general computing system required by the eyewear device 100. The high-speed processor 932 includes the processing resources required to manage high-speed data transmissions using the high-speed wireless circuit 936 to a wireless local area network (WLAN) connection 937. In certain examples, the high-speed processor 932 executes an operating system of the eyewear device 100, such as a LINUX operating system or other such operating system, and the operating system is stored in the memory 934 for execution. Among other responsibilities, the high-speed processor 932 executing the software architecture of the eyewear device 100 is used to manage data transmissions with the high-speed wireless circuit 936. In certain examples, the high-speed wireless circuit 936 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, also referred to herein as Wi-Fi. In other examples, other high-speed communication standards can be implemented through the high-speed wireless circuit 936.

[0079] The low-power wireless circuit 924 and the high-speed wireless circuit 936 of the eyewear device 100 can include short-range transceivers (Bluetooth™) and wireless wide area, local area, or wide area network transceivers (e.g., cellular or WiFi). The mobile device 990, including the transceivers that communicate via the low-power wireless connection 925 and the high-speed wireless connection 937, can be implemented using the details of the architecture of the eyewear device 100, as can other elements of the network 995.

[0080] The memory 934 includes any storage device capable of storing a variety of data and applications, including color maps, camera data generated by the visible light cameras 114A-B and the image processor 912, and images generated for display on the see-through image displays 180C-D of the optical assemblies 180A-B by the image display driver 942, among other things. While the memory 934 is shown as integrated with the high-speed circuit 930, in other examples, the memory 934 can be a separate element of the eyewear device 100. In certain such examples, electrical routing lines can provide a connection from the image processor 912 or the low-power processor 922 to the memory 934 through a chip that includes the high-speed processor 932. In other examples, the high-speed processor 932 can manage addressing of the memory 934 such that the low-power processor 922 will initiate the high-speed processor 932 at any time a read or write operation involving the memory 934 is required.

[0081] Server system 998 may be one or more computing devices as part of a service or network computing system, for example, including a processor, memory, and a network communication interface to communicate with glasses device 100 via network 995, either directly or via mobile device 990, through high-speed wireless circuit 936. Glasses device 100 is connected to a host computer. In one example, glasses device 100 communicates wirelessly directly with network 995 without using mobile device 990, such as using a cellular network or WiFi. In another example, glasses device 100 is paired with mobile device 990 via high-speed wireless connection 937 and connected to server system 998 via network 995.

[0082] The output components of the eyeglasses device 100 include visual components, such as those used with... FIG. 2C Image displays 180C-D (e.g., displays such as liquid crystal displays (LCDs), plasma display panels (PDPs), light-emitting diode (LED) displays, projectors, or waveguides) of optical components 180A-B described in -D. Image displays 180C-D of optical components 180A-B are driven by image display driver 942. Output components of eyeglass device 100 also include acoustic components (e.g., speakers), haptic components (e.g., vibration motors), other signal generators, etc. Input components of eyeglass device 100, mobile device 990, and server system 998 may include: alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens or other haptic input components that provide the position and force of a touch or touch gesture), and audio input components (e.g., microphones), etc.

[0083] The eyewear device 100 may optionally include additional peripheral device elements. Such peripheral device elements may include biometric sensors, additional sensors, or display elements integrated with the eyewear device 100. For example, peripheral device elements may include any I / O components, including output components, motion components, position components, or any other such components described herein.

[0084] For example, biometric components of user interface field adjustments can include components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), and identifying people (e.g., voice identification, retinal scanning, face identification, fingerprint identification, or electroencephalogram-based identification) and the like. Motion components include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. Position components include location sensor components generating location coordinates (e.g., a Global Position System (GPS) receiver component), WiFi or Bluetooth™ transceivers generating positioning system coordinates, altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), and orientation sensor components (e.g., magnetometers), and the like. Such positioning system coordinates can also be received from a mobile device 990 via low-power wireless circuitry 924 or high-speed wireless circuitry 936 over wireless connections 925 and 937.

[0085] According to some examples, an "application," "a plurality of applications," is one or more programs that execute functions defined in the programs. The one or more applications can be created using a variety of programming languages such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In one specific example, a third-party application (e.g., an application developed by an entity other than the vendor of the particular platform) can operate in a mobile operating system such as the IOS™, ANDROID™, WINDOWS® Phone, or other mobile operating systems. In this example, the third-party application can invoke API calls provided by the operating system to facilitate the functionality described herein.

[0086] FIG. 10 is located, as FIG. 8AThe strain gauge sensor 802A in the bridge 106 of the eyewear device 100 is shown in a measured waveform 1000 for various characteristic events. The waveform 1000 shows several characteristic events, including an event 1002 when the user places the eyewear device 100 on the user's head, an event 1004 when the user presses a button on the eyewear device 100, an event 1006 when the user taps the eyewear device 100, an event 1008 when the user's head rotates, and an event 1010 when the user removes the eyewear device 100 from the user's head. The waveform 1000 also shows when the device 100 is worn by the user, such as between the characteristic events 1002 and 1010. This is seen as an increase in baseline measurements between the 10 and 30 second markers compared to the 0 to 5 second and 32 to 37 second markers. The processor 932 identifies the characteristic events from the measured signals generated by the strain gauge sensors 802A-C. In one example, a characteristic event database 960 is stored in the memory 934 and the processor 932 compares the measurements of the strain gauge sensors 802A-C to the characteristic event database 960A to determine the occurrence of a characteristic event. In another example, the characteristic event database 960 is stored at a location remote from the eyewear device 100 and is accessed via the network 995.

[0087] The identification of certain characteristic events can be used to trigger certain functions of the eyewear device 100. For example, the characteristic event 1002 of putting on the eyewear device 100 causes the processor 932 to turn on the displays 180C-D on the eyewear device 100, and the characteristic event 1010 of taking off the eyewear device 100 causes the processor 932 to turn off the displays 180C-D. Other functions that can be controlled by the characteristic events include the processor 932 playing and pausing music, the processor 932 changing a social status (such as online and offline), and increasing and decreasing the processor 932 power consumption (such as by controlling the sampling rate).

[0088] The characteristic event 1002 of the user placing the eyewear device 100 on the user's head lasts for approximately 2-3 seconds. While the user is putting on the eyewear device 100, the processor 932 identifies the start of the characteristic event 1002 to turn on the power to the eyewear device 100 and the displays 180C-D. This provides a smooth user experience by turning on the power to the eyewear device 100 at the same time the user is putting on the eyewear device 100.

[0089] The processor 932 can utilize the head motion tracker 109 to filter out spurious characteristic events measured by the strain gauge sensors 802A-C. For example, if a user of the glasses device 100 moves in a specific manner, such as jumping, the strain gauge sensors 802A-C can generate readings that resemble a characteristic event (such as removing the glasses device 100 from the user's head). The processor 932 uses measurements from the accelerometer of the head motion tracker 109 to verify the occurrence of the characteristic event.

[0090] FIG. 11 Flowchart 1100 shows a method for switching the power supply of the eyeglasses device 100 on and off based on the measurement values ​​of strain gauge sensors 802A-C.

[0091] At block 1102, while the glasses device 100 is in sleep mode, the processor 932 uses strain gauge sensors 802A-C to measure the strain of the glasses device 100, where the displays 180C-D are powered off and the glasses device 100 has minimal power consumption. The sampling rate of the strain gauge 802A is selected by the processor 932 to meet the sensitivity and power requirements of the glasses device 100. The obtained strain measurement values ​​are as follows: FIG. 10 The waveform 1000 is shown in the figure. The processor 932 performs strain measurements at a lower sampling rate (such as 20 Hz) to reduce power consumption.

[0092] At block 1104, processor 932 compares strain gauge measurements with characteristic event database 960 at a lower sampling rate. In one example, characteristic event database 960 is stored in memory 934, and processor 932 performs signal processing to determine the occurrence of characteristic events by comparing strain gauge measurements with characteristic event database 960. In another example, characteristic event database 960 is stored at a location remote from eyeglass device 100 and accessed by processor 932 via network 995.

[0093] At block 1106, processor 932 identifies characteristic events by performing signal processing on the measurements generated by strain gauges 802A-C. This is accomplished by processor 932 comparing the strain gauge measurements with the characteristic event database 960 to identify matches, such as... FIG. 10 The characteristic events shown are 1002, 1004, 1006, 1008, and 1010. The identification of these specific characteristic events is used to trigger certain functions of the glasses device 100. FIG. 10In one example shown, the processor 932 detects a substantially constant strain gauge measurement over a period of time (e.g., having been given time to stabilize), and then identifies a feature event, such as feature event 1002, by comparing the detected potential feature event to feature events in the feature event database and identifying a match. In another example, the processor 932 detects a changing strain gauge measurement over time to detect a waveform feature, such as during the time periods corresponding to feature events 1004, 1006, and 1008.

[0094] At block 1108, the processor 932 determines that the feature event 1002 is associated with the eyewear device 100 worn on the user's head, and powers on the eyewear device 100. Powering on the eyewear device 100 includes activating the display 180C-D and operating the processor 932 in a normal use mode. In the normal use mode, the processor samples the strain gauge sensors at an increased sampling rate (e.g., 60 cycles per second or higher).

[0095] At block 1110, the processor 932 identifies another feature event by processing measurements from the strain gauge sensors 802A-C, including comparing the sensed measurements to the feature event database 960. The additional feature events include event 1004 when the user presses a button on the eyewear device 100, event 1006 when the user taps the eyewear device 100, event 1008 when the user's head rotates, and event 1010 when the user removes the eyewear device 100 from the user's head.

[0096] At block 1112, in one example, the processor 932 determines that the feature event 1010 is associated with the eyewear device 100 being removed from the user's head, and then places the eyewear device 100 in a sleep mode. The sleep mode includes the processor 932 turning off the display 180C-D, and the processor 932 operating in a power saving mode at a reduced sampling rate.

[0097] Other feature events can be detected as described above, such as detecting the user pressing a button, tapping the eyewear device 100 (such as a display or control input), or a rotation of the user's head; and other actions can be taken by the processor 932 in response to the detected feature events.

[0098] The IPD of the user wearing the eyewear device 100 is estimated by the processor 932 to control imaging and other eyewear functions, such as improving virtual object rendering to improve the AR viewing experience while reducing VAM. The user has a head breadth (HB), where users with larger HB tend to have larger IPD. Larger HB causes the temples 125A and 125B of the eyewear device 100 to bend outward, and in turn, causes the frame 105 to bend and spatially distort. The processor 932 determines HB based on the determined frame bend, and then determines IPD based on HB.

[0099] FIG. 12A A top view of an example hardware configuration of the eyewear device 100 is depicted, including a strain gauge sensor 802A located on the bridge 106 of the frame 105. The eyewear device 100 is shown as being worn by a user having a HB, where the temples 125A and 125B contact the head of the user, where HB does not cause a bend or spatial distortion of the frame 105, collectively referred to as distortion. During manufacturing, the spacing between the extended temples 125A and 125B is set to a predetermined temple width, e.g., 145 mm. During manufacturing, the original orientation of the front-facing fields of view (FOVs) 111A and 111B of the visible light cameras 114A and 114B, respectively, is set relative to each other. FIG. 12A In the example, the predetermined temple width and HB are the same, 145 mm. When HB is less than the predetermined temple width, there is no frame bend or spatial distortion. The front-facing FOVs 111A and 111B of the visible light cameras 114A and 114B, respectively, are in the original orientation set relative to each other during manufacturing. As described with reference to FIG. 7, the visible light cameras 114A and 114B produce original images 758A and 758B that overlap at 713, where the baseline overlap 713 is defined during manufacturing. FIG. 7

[0100] The processor 932 first determines HB using a head size database 970 that includes data relating frame bend to HB, based on sensor measurements of the strain gauge sensor 802A or calculations by a computer vision (CV) algorithm 972, and then determines IPD using an IPD database 974 that includes data relating HB to IPD. In the example, the head size database 970 includes a function that relates HB to frame bend, and the IPD database 974 includes a function that relates HB to IPD. The processor 932 determines HB based on the sensor measurements of the strain gauge sensor 802A or calculations by the CV algorithm 972, and then determines IPD based on HB. FIG. 12A ​In the illustrated example, since the user HB is at or less than a predetermined temple width, such as 145 mm, the frame 105 is not curved or spatially deformed, and the IPD is set to correlate with a nominal IPD, such as 61 mm. If user demographic information, such as the user’s age and gender, is input into the eyewear device 100, the IPD can be updated to match statistical data in the IPD database 974 based on the user’s demographic information. In one example, users in the age range of 40 to 55 years old can have a greater HB than users in the age range of 25 to 40 years old. In one example, the IPD database 974 and the user demographic information are stored in the memory 934 of the eyewear device 100. In another example, the IPD database 974 and the user demographic information are stored in the remote server system 998.

[0101] FIG. 12B A top view of the eyewear device 100 worn by a user with a HB greater than a predetermined temple width is depicted, and results in a curvature or spatial deformation of the frame 105. In one example, the user’s HB is greater than the predetermined temple width, such as 155 mm. As a result of the user’s HB curving the frame 105, the front-facing fields of view (FOVs) 111 A and 11 IB of the visible light cameras 114A and 114B, respectively, are in a modified orientation relative to one another. As illustrated, the FOVs 111 A and 11 IB are oriented inward toward one another. The modified orientation of the visible light cameras 114A and 114B produces the original images 758A and 758B such that the overlap 713 is modified, e.g., it has an area that is less than the area of the baseline overlap 713. The CV algorithm 972 executed by the processor 932 determines the HB based on the characteristics of the modified overlap 713, such as the reduced area. The CV algorithm 972 can also use a visual-inertial odometry (VIO) system that samples points in the scene images and gives a series of stereo frames to estimate the curvature of the frame 105. In one example, if the processor 932 determines that the area of the modified overlap 713 is 1% less than the baseline overlap 713, the CV algorithm 972 determines that there is a 1-degree curvature in the frame 105. In another example, the processor 032 can use orientation data from the IMUs associated with the cameras 114A and 114B to determine the modified overlap 713.

[0102] In one example, a computer vision (CV) algorithm 972 determines a value of the frame bend as a function of the overlap 713. A head size database 970 includes data relating frame bend to HB. The processor 932 uses the head size database 970 to determine the head width of the user from the determined frame bend value. In one example, if the CV algorithm 972 determines a 1 degree bend in the frame 105, the processor 932 uses the head size database 970 to determine that the HB of the user is 160 mm, and then references the IPD database 974 to estimate the IPD based on the HB, such as 66 mm. If the CV algorithm 972 determines a 0.5 degree bend in the frame 105, the processor 932 uses the head size database 970 to determine that the HB is 150 mm, and then references the IPD database 974 to estimate the IPD based on the HB, such as 64 mm.

[0103] In another example, the strain gauge 802A provides a measured value of the frame bend, as described with reference to FIG. 8C the measured value is used by the processor 932 to measure the HB of the user. In one example, if the strain gauge 802A measures a 1 degree bend in the frame 105, the processor 932 uses the head size database 970 to determine that the HB of the user is 160 mm, and then references the IPD database 974 to estimate the IPD value to be 66 mm. In another example, if the strain gauge 802A measures a 0.5 degree bend in the frame 105, the processor 932 determines that the HB of the user is 150 mm, and then references the IPD database 974 to estimate the IPD value to be 64 mm. Any other strain gauge location described previously can be used to measure the bend of the frame 105 to determine the HB of the user.

[0104] FIG. 13A A static IPD value of 64 mm is shown, which is not a function of HB. At this static value, 34% of a sample population of 16,215 people representative of the U.S. population is encompassed by the 64 mm IPD value with a bandwidth of 4 mm. FIG. 13B A statistical improvement in the determined IPD value as a function of HB, implemented by the CV algorithm 972 and the strain gauge 802A, is shown. The determined IPD value is shown as a linear function of HB, with a bandwidth of 4 mm, encompassing 45% of a sample population of 16,215 people representative of the U.S. population. The determined IPD value as a function of HB has a 32% improvement compared to the static IPD value of 64 mm.

[0105] FIG. 13BThe set IPD value is shown for HB below 145 mm. In one example, for a user with HB below 145 mm, no lens frame 105 bending or spatial distortion occurs, resulting in a default IPD value of 64 mm. For a user with HB greater than 145 mm, the lens frame 105 bends and results in spatial distortion determined by the processor 932, and the processor 932 updates the IPD value based on the determined HB.

[0106] FIG. 14 is a flowchart 1400 of a method of a processor estimating a user’s IPD value.

[0107] At block 1402, the processor 932 determines the bending of the lens frame 105. In one example, the bending and spatial distortion of the lens frame 105 is measured by the strain gauges 802A. In another example, the bending and spatial distortion of the lens frame 105 is determined by the CV algorithm 972 using image data from the cameras 114A and 114B and the IMU, such as creating the overlapping 713 raw images 758A and 758B, as described above. In another example, the processor 932 determines that there is no bending and spatial distortion of the lens frame 105 due to the user HB being below a minimum measurable threshold.

[0108] At block 1404, the processor 932 uses the head size database 970 with data relating lens frame bending to HB as described above to determine the user HB. The processor 932 determines the user HB using the measurements from the strain gauges 802A or using the CV algorithm 972 based on the determined lens frame bending. In one embodiment, the head size database 970 includes statistical data of user profile data, such as HB of users with different ages and genders. The processor 932 uses the statistical data of user profile data and the determination of the lens frame bending to determine a more accurate estimate of the HB.

[0109] At block 1406, the processor 932 estimates the user IPD using the IPD database 974 by relating the determined HB to IPD. In one example, the processor 932 adjusts the IPD value to the average value of the determined IPD range. In another example, the processor 932 prompts the user on the displays 180C and 180D to adjust the IPD value from the range determined from HB. A bar slider for continuous adjustment of the IPD value, or a prompt for a few options for discrete adjustment of the IPD value, is presented to the user. Virtual objects are rendered on the displays 180C and 180D to allow the user to choose the most comfortable IPD value for the virtual object rendering. In another example, the user adjusted IPD value is saved to the user profile of the eyewear device 100. FIG. 13B

[0110] ​At block 1408, the processor 932 adjusts the displays 180C and 180D based on the determined IPD value to improve virtual object rendering, such as better aligning rendered objects (e.g., the user’s hand). The adjusted IPD value also reduces the VAM of the displays 180C and 180D. In one example, when the processor 932 determines a different frame bend value, the method starts again at block 1402. When rendering three-dimensional (3D) objects for stereoscopic vision, the cameras 114A and 114B render one view for the left eye and one view for the right eye. Knowing the IPD, the distance between the two virtual cameras can be adjusted accordingly to match the user’s actual IPD. Otherwise, the displays 180C and 180D use the stored IPD value of the user profile.

[0111] It will be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions by those skilled in the respective field of the art, unless otherwise specifically defined herein. Relation terms, such as first and second, and the like, can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “includes,” “including,” “contains,” “containing,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, includes, contains, or contains elements or steps not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by “comprises a,” “has,” “has a,” or “includes” with “one,” “the,” or “an” does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, includes, or contains the element.

[0112] Unless otherwise stated, any and all measurements, values, ratings, positions, magnitudes, sizes and other specifications that are set forth in this specification are approximate, not exact. These amounts are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain. For example, unless expressly stated otherwise, a parameter value or like quantity can vary by as much as ±10% from the stated amount.

[0113] Furthermore, in the detailed description of embodiments, it can be seen that the individual features can be combined in various examples in one embodiment. The method of disclosure is not to be interpreted as reflecting the intention that the claimed examples require more features than are explicitly stated in each claim. Rather, as reflected by the following claims, the subject matter is to be protected in the scope of less than all features of any single disclosed embodiment. Accordingly, the claims are hereby incorporated into the detailed description of embodiments, in which each claim stands as a separate example.

[0114] While there have been described herein what are considered to be the best modes and other examples of the application, it is understood that various modifications can be made to the described embodiments and that the teachings herein can be employed in various and numerous applications, only some of which have been described herein. It is intended that the following claims be construed to include all such modifications and variations as falling within the true scope of the present concept.

Claims

1. An eyeglasses device, the eyeglasses device comprising: Picture frames; Processor, the processor being configured to: Determine the deformation of the picture frame; as well as The interpupillary distance (IPD) of the user is estimated based on the determined frame deformation. and A display coupled to the frame is configured to display an image based on the estimated IPD.

2. The eyeglasses device according to claim 1, wherein, The processor is configured to determine the frame deformation when the user wears the glasses.

3. The eyeglasses device according to claim 2, wherein, The processor is configured to determine the user’s head width (HB) and to determine the estimated IPD based on the HB.

4. The eyeglasses device of claim 3 further includes a sensor coupled to the frame and configured to generate a sensor signal indicating the HB.

5. The eyeglasses device according to claim 4, wherein, The sensor includes a strain gauge sensor configured to measure the strain of the mirror frame.

6. The eyeglasses device according to claim 4, wherein, The sensor includes at least one camera, which is configured to generate camera images to determine the frame distortion using the images.

7. The eyeglasses device according to claim 6, wherein, The processor is configured to use computer vision (CV) algorithms to determine the frame distortion using the camera images.

8. The eyeglasses device of claim 3, further comprising a data table configured to correlate the frame deformation with the HB.

9. The eyewear device of claim 3, further comprising a data table configured to correlate the HB with the estimated IPD.

10. The eyeglasses device according to claim 9, wherein, The estimated IPD is configured to be adjustable by the user.

11. A method of using an eyeglasses device, the eyeglasses device including a frame, a processor configured to determine deformation of the frame when worn by a user, and a display coupled to the frame, the display being configured to display an image, the method including the processor: Determine the deformation of the picture frame; The interpupillary distance (IPD) of the user is estimated based on the determined frame deformation. as well as The image is displayed based on the estimated IPD.

12. The method according to claim 11, wherein, The processor determines the deformation of the eyeglass frame when the user wears the glasses.

13. The method according to claim 12, wherein, The processor determines the user's head width (HB) and determines the estimated IPD based on the HB.

14. The method according to claim 13, wherein, The eyewear device includes a sensor coupled to the frame and generates a sensor signal indicating the HB.

15. The method according to claim 14, wherein, The sensor includes a strain gauge sensor that measures the strain of the mirror frame.

16. The method of claim 14, wherein, The sensor includes at least one camera that generates camera images.

17. The method according to claim 16, wherein, The processor is configured to use computer vision (CV) algorithms to determine the frame distortion using the camera images.

18. The method according to claim 13, wherein, The processor uses a data table to correlate the frame deformation with the HB, and uses another data table to correlate the HB with the estimated IPD.

19. A non-transitory computer-readable medium storing program code, which, when executed by a processor of an eyeglasses device including a frame and a display coupled to the frame and configured to display an image, is operable to configure the eyeglasses device to perform the following steps: Determine the deformation of the picture frame; The interpupillary distance (IPD) of the user is estimated based on the determined frame deformation. as well as The image is displayed based on the estimated IPD.

20. The non-transitory computer-readable medium according to claim 19, wherein, The eyewear device includes a sensor coupled to the frame, and the sensor is configured to generate a sensor signal indicating the user's head width (HB), wherein the code operates to determine an estimated IPD based on the HB.

Images