Transparent insert identification

Abstract

The present disclosure relates to transparent insert identification. Various implementations disclosed herein include devices, systems, and methods that use a marking on a transparent surface (e.g., a prescription lens insert for an HMD) to identify information (e.g., prescription parameters) about the transparent surface. In some implementations, the marking does not interfere with eye tracking through or using the transparent surface to view virtual content or a physical environment. In some implementations, image data is acquired from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device. A code is then identified in the image data, where the code is detectable by the image sensor on the transparent surface without interfering with functionality of the electronic device involving the transparent surface. In some implementations, content is provided at the electronic device based on the identified code, where the content is viewable through the transparent surface.

Description

Technical Field

[0001] The present disclosure generally relates to electronic devices, such as head-mounted devices (HMDs), that can be used with attachable lenses. Background Art

[0002] People sometimes need prescription glasses to see clearly, but wearing them in an HMD can be uncomfortable. Summary of the Invention

[0003] Various embodiments disclosed herein include devices, systems, and methods for identifying information about a transparent surface using machine-readable markings on a transparent surface. For example, the transparent surface may be a prescription lens insert for an HMD, and the information may be prescription parameters for the prescription lens insert. In some cases, clip-on prescription lenses are attached to the HMD (e.g., between a display in the HMD and the user's eyes), but in order to operate properly, the HMD needs to know which prescription lenses are currently attached. In some embodiments, the markings are configured to not interfere with "normal" use of the transparent surface, e.g., while the user is looking through the transparent surface or performing eye tracking during use of the transparent surface. In some embodiments, the markings on the transparent surface are not visible to the user of the HMD. In some embodiments, the markings on the transparent surface are generally visible to the eye tracking image sensor and / or are only visible to the eye tracking image sensor at selective times.

[0004] Various embodiments disclosed herein include devices, systems, and methods for providing machine-readable markings on attachable corrective lenses to identify which specific corrective lens or lenses are attached to an HMD. For example, a right corrective lens and a left corrective lens may have different prescriptions, and based on the markings, the HMD can determine and / or notify the user whether the right corrective lens and the left corrective lens are interchangeable. For another example, multiple users of the HMD may have their own corrective lenses, and based on the markings, the HMD can determine and / or notify the user whether an incorrect prescription lens has been attached. Furthermore, the markings on the corrective lenses can be used to ensure that the corrective lenses are attached to the HMD in the correct position and orientation. Furthermore, the markings on the corrective lenses can be used to identify whether the user is color blind or a specific type of color blindness, which allows the HMD to perform corrective processing (e.g., increase contrast or modify selective colors).

[0005] Generally speaking, one innovative aspect of the subject matter described herein can be embodied as a method comprising the steps of acquiring image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device. In some implementations, a code is identified in the image data, wherein the code is detectable by the image sensor on the transparent surface without interfering with functionality of the electronic device related to the transparent surface. Content is then provided at the device based on the identified code, wherein the content is viewable through the transparent surface.

[0006] Generally speaking, one innovative aspect of the subject matter described herein can be embodied as a system including an image sensor configured to acquire image data corresponding to a transparent surface attached to an electronic device. In some implementations, a display is configured to present content at the electronic device for viewing through the transparent surface, the transparent surface including a depiction of a code, wherein the code is detectable on the transparent surface by the image sensor without interfering with viewing the display through the transparent surface. Furthermore, a processor is configured to provide the content on the display based on the code. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] So that the present disclosure may be understood by those skilled in the art, a more detailed description may be had with reference to aspects of certain exemplary implementations, some of which are illustrated in the accompanying drawings.

[0008] Figure 1 Diagrams of example electronic devices are shown, according to some implementations.

[0009] Figure 2 is a diagram illustrating exemplary markings provided by corrective lenses according to some implementations.

[0010] Figure 3 is a diagram illustrating the FOV of an image sensor of an electronic device relative to an attached corrective lens according to some implementations.

[0011] Figures 4A to 4B An exemplary configuration of an electronic device for marking corrective lenses including excitation and emission spectra is shown, according to some implementations.

[0012] Figures 5A to 5B An exemplary configuration of an electronic device for marking a corrective lens including an exposure spectrum and a detection spectrum is shown according to some implementations.

[0013] Figures 6A to 6B Example configurations of electronic devices for marking corrective lenses using a projection surface are shown, according to some implementations.

[0014] Figure 7is a flowchart showing an example method of using a mark on a transparent surface to identify information about the transparent surface, in accordance with some implementations.

[0015] Figure 8 An example electronic device is shown, in accordance with some implementations.

[0016] In accordance with common practice the various features can not be drawn to scale in the attached drawings. Accordingly, the dimensions of the various features can be arbitrarily expanded or reduced for the clarity of presentation. Also, some of the drawings can not depict all of the components of a given system, method or device. Finally, like reference numerals can be used to denote like features throughout the specification and figures. DETAILED DESCRIPTION

[0017] Numerous specific details are described to provide a thorough understanding of the example implementations shown in the figures. However, the embodiments herein are not limited to the specific details described herein. Rather, one skilled in the relevant art will recognize that other aspects, or variations, can be possible without departing from the scope of the disclosure. Also, well-known systems, methods, components, devices and circuits have not been described in exhaustive detail so as to not obscure like aspects of the example implementations described herein.

[0018] Various implementations disclosed herein include devices, systems, and methods of providing machine-readable marks using a transparent surface (e.g., a prescription lens insert for an HMD) to identify information (e.g., prescription parameters) about the transparent surface. In some implementations, the transparent surface is attached to an electronic device. The transparent surface is to be used for both visual observation by a user of the electronic device and sensing of the machine-readable marks. In some implementations, the machine-readable marks are imperceptible (e.g., invisible) to the user when looking through the transparent surface at a physical environment or virtual content.

[0019] In some implementations, the machine-readable marks embed data into a corrective lens (e.g., a clip-on prescription lens) for an HMD. For example, the marks store decodable information (e.g., prescription, calibration, metadata, etc.) related to the corrective lens that can be sensed by an eye tracking image sensor that looks at the eyes of a user of the HMD through the corrective lens attached to the HMD. In some implementations, the marks and eye tracking information (e.g., glints) are detected in different portions of an image acquired by the eye tracking image sensor.

[0020] Figure 1Figure 1 shows an exemplary electronic device according to some implementations. In some implementations, the electronic device 100 includes an eye tracking system for detecting eye position and eye movement. In some implementations, images captured by the eye tracking system can be analyzed to detect the position and movement of the user's 115 eyes, or to detect the gaze direction of one or both eyes. In some implementations, the eye tracking system can detect other information about the eyes, such as blinks (e.g., rate or duration), pupil dilation, or saccadic movements. In addition, the gaze point estimated from the eye tracking images can enable gaze-based interaction with content shown on a near-eye display of the electronic device 100.

[0021] like Figure 1 As shown, HMD 100 includes a housing 101 (or casing) that houses various components of HMD 100. In some implementations, housing 101 holds HMD 100 in place on the face of user 115 (e.g., around the eyes of user 115).

[0022] Housing 101 houses display 110, which displays images and emits light toward or onto the eyes of user 115. In various implementations, display 110 emits light through an eyepiece having one or more lenses 112, which refract the light emitted by display 110, causing the display to appear to user 115 at a virtual distance greater than the actual distance from the eyes to display 110. To enable user 115 to focus on display 110, in various implementations, the virtual distance is at least greater than the minimum focal length of the eyes (e.g., 7 cm). Furthermore, to provide a better user experience, in various implementations, the virtual distance is greater than 1 meter.

[0023] Housing 101 also houses a tracking system that includes one or more illumination / light sources 122, an image sensor 124, and a controller 180. One or more light sources 122 emit light onto the eyes of user 115, which is reflected as a light pattern (e.g., one or more flashes, such as circles) that can be detected by image sensor 124 (e.g., a camera). Based on the light pattern, controller 180 can determine eye tracking characteristics of user 115. For example, controller 180 can determine the gaze direction of one or both eyes of user 115. As another example, controller 180 can determine the blink state of user 115 (eyes open or closed). As another example, controller 180 can determine saccadic motion, pupil center, pupil size, or focus. Thus, in various embodiments, light is emitted by one or more light sources 122, reflected from the eyes of user 115, and detected by image sensor 124. In some embodiments, light from the eyes of user 115 is reflected by a mirror or passes through an optical device, such as a lens or eyepiece, before reaching image sensor 124.

[0024] In some implementations, the display 110 emits light in a first wavelength range, and the one or more light sources 122 emit light in a second wavelength range. Similarly, the image sensor 124 detects light in the second wavelength range. In some implementations, the first wavelength range is a visible wavelength range (e.g., a wavelength range of approximately 400 nm to 700 nm within the visible spectrum), and the second wavelength range is a near-infrared wavelength range (e.g., a wavelength range of approximately 700 nm to 1400 nm within the near-infrared spectrum). In some implementations, the light source 122 and the image sensor 124 use overlapping wavelengths when illuminating the eye for eye / gaze tracking. Alternatively, the light source 122 and the image sensor 124 use the same spectrum to illuminate the eye for eye / gaze tracking while the user 115 looks through corrective lenses at the display 110 that shows content using the visible spectrum.

[0025] like Figure 1 As shown, corrective lenses 200 can be attached to HMD 100. In some implementations, corrective lenses 200 are attached using housing 101 of HMD 100. Corrective lenses 200 can help user 115 accurately see objects in physical environment 105. Additionally, corrective lenses 200 can help user 115 accurately see information using display 110. However, in order for HMD 100 to accurately provide content to user 115 using image sensor 124, HMD 100 needs to know the prescription (or other information) regarding corrective lenses 200. One way for HMD 100 to access information regarding one or more attached corrective lenses 200 is to incorporate indicia in or on the corrective lenses 200. In some implementations, a sensor in HMD 100, such as image sensor 124, is used to detect machine-readable indicia on corrective lenses 200. For example, markings on a corrective lens can be captured by the image sensor 124, and the information contained therein can be decoded by the controller 180 and used to modify the operation of the HMD 100 (e.g., based on clip-on corrective lens prescription parameters). In some implementations, the markings on the corrective lens 200 are not visible to the user 115. In some implementations, the markings on the corrective lens 150 are not normally visible to the image sensor 124, but are visible to the image sensor 124 only at selective times. In some implementations, the machine-readable marking used to embed data in the corrective lens 200 includes a material that is undetectable in a first state (e.g., transparent) and provides a detectable pattern in a second state to form a temporarily visible / detectable marking.

[0026] Figure 2 FIG is a diagram illustrating an exemplary corrective lens 200 including machine-readable markings according to some implementations. Figure 2As shown, machine-readable indicia 250 (magnified to be visible) is disposed on a peripheral region of the corrective lens 200. In some implementations, the size of the indicia 250 is below a size detectable by the human eye. Alternatively, the contrast or spatial frequency of the indicia 250 in the corrective lens 200 is imperceptible to the human eye. In some implementations, the indicia 250 is formed on the corrective lens 200 using a subtractive process (e.g., etching) or using an additive process (e.g., tinting, deposition). In some implementations, the indicia 250 is formed at multiple locations on the corrective lens 200 (e.g., for redundancy). In some implementations, the indicia 250 is in or near the surface closest to the HMD 100 (e.g., image sensor 124) and, therefore, further away from the focus of the user 115, for example, covered by a protective coating. Alternatively, the marking 250 can be on an inner layer (e.g., a sub-surface) of the corrective lens 200 at the surface closest to the eye, which can be closest to the focal length of the image sensor 124 (e.g., focused on the eye) and away from the (hyper)focal length of the eye, which is at the distance of the virtual image of the display 110 (e.g., at a distance of 1 m).

[0027] When attached to the HMD 100, the marker 250 is within the field of view (FOV) of a sensor (e.g., an image sensor) of the HMD 100. In some implementations, the marker 250 is outside the area of ​​the corrective lens 200 used for viewing the foveated rendering of the display 110. Figure 2 As shown, the marker 250 is located outside the area of ​​the corrective lens 200 used when the user 115 views the display 110 of the HMD 100. The marker 250 is used to encode information used by the HMD 100. For example, the marker 250 can encode the prescription parameters (e.g., myopia, hyperopia) of the prescription corrective lens 200. Alternatively, the position of the marker 250 on the corrective lens 200 can be used to determine the pose (e.g., 3D position and 3D orientation) of the corrective lens 200 when attached to the HMD 100.

[0028] Figure 3is an example image sensor of an electronic device relative to the FOV of an attached corrective lens, according to some implementations. In some implementations, the image sensor 124 has a single FOV (e.g., FOV 300a) relative to the attached corrective lens 200. For example, the single FOV is used for eye tracking functionality and detection of a marker on the corrective lens 200. In other implementations, the image sensor 124 has multiple FOVs relative to the corrective lens 200. For example, each different FOV can be used to detect a different marker of multiple markers on the corrective lens 200. In some implementations, the multiple FOVs of the image sensor 124 differ in size, magnification (e.g., magnification or demagnification), orientation (e.g., horizontal, vertical, or both), or a combination thereof. As shown, Figure 3 The image sensor 124 has a first FOV 300a (e.g., for detecting the marker 250) and a different second FOV 300b (e.g., for eye tracking functionality), as shown. As described with respect to the image sensor 124, the light source 122 in the HMD 100 can include multiple different emission fields, the HMD 100 can include multiple light sources 122 each having a different emission field, or a combination thereof. In some implementations, the emission of the light source 122 is selectively filtered to modify the light spectrum.

[0029] In some implementations, the machine-readable markers used to embed data in the corrective lens 200 include a material that is transparent when not excited and emits a detectable pattern of light to form a temporary visible / detectable marker when excited. In some implementations, the markers 450 (e.g., 450a, 450b) include a fluorescent material having an excitation wavelength / spectrum and an emission wavelength / spectrum. The excitation and emission spectra can be very narrow. In one implementation, the excitation and emission spectra of the markers 450 on the corrective lens 200 are not visible to the user 115 or the image sensor 124. In another implementation, the excitation and emission spectra of the markers 450 on the corrective lens 200 are not visible to the user 115, but the emission spectrum of the markers 450 is detectable by the image sensor 124.

[0030] As Figure 4AAs shown, the transparent fluorescent material forming marker 450a has an excitation spectrum that does not overlap with the incident illumination (e.g., visible spectrum) seen by user 115 or the detectable spectrum of eye-tracking image sensor 124. However, the emission spectrum of marker 450a is within the detectable spectrum of eye-tracking image sensor 124. In such implementations, marker 450a can be detected at any time because marker 450a is not visible to user 115, but additional illumination source 422 (e.g., excitation illumination) can be added to HMD 100. In some implementations, marker 450a and eye-tracking information are detected in different portions of an image from image sensor 124. Alternatively, marker 450a and eye-tracking information can at least partially overlap in an image from image sensor 124. In other implementations, marker 450a can be detected in one or more images captured by image sensor 124 when user 115 blinks, looks away, or when eye-tracking functionality is not being used by HMD 100. In one implementation, the marker 450a may be detected when eye tracking is enabled and content is displayed (or not displayed) in a particular area of ​​the display 110. Alternatively, the marker 450a may be detected when eye tracking is enabled using one or more blank frames (e.g., the light source 122 is turned off) synchronized with the image sensor 124. For example, the detectable spectrum of the eye tracking image sensor 124 may be a spectrum around 850nm or 950nm wavelength. Although Figure 4A An implementation is shown where the emission spectrum of marker 450a is within the detectable spectrum of eye-tracking image sensor 124, but embodiments are not limited to such implementations.

[0031] For example, Figure 4B As shown, the transparent fluorescent material forming the marker 450b can have an excitation spectrum and an emission spectrum that do not overlap with the incident illumination (e.g., visible spectrum) seen by the user 115 or the detectable spectrum of the eye-tracking image sensor 124. In such implementations, an additional illumination source 422 (e.g., excitation illumination) and an additional image sensor 424 that can detect the emission spectrum of the marker 450b can be added to the HMD 100. Thus, the fluorescent marker 450b can be detected at any time without interfering with the operation of the HMD 100. For example, the illumination source 422 can be UV spectrum, and the image sensor 424 can be NIR / IR spectrum, which is different from the image sensor 124. In some implementations, when the excitation illumination is UV, the marker 450 is detected when the HMD 100 is not in use (e.g., to avoid exposure to UV illumination).

[0032] In one implementation, the indicia 450 a has an excitation spectrum provided by the illumination source 122 that is within a first portion of the detectable spectrum of the image sensor 124 , and the emission spectrum of the indicia 450 a is within a second portion of the detectable spectrum of the image sensor 124 .

[0033] In some implementations, the emission spectrum of the fluorescent material used for the marker 450 overlaps with the visible spectrum, and the information emitted by the fluorescent marker 450 will be in the visible spectrum of the user 115. In such implementations, an additional visible light filter (e.g., a notch filter in the corrective lens 200) between such a marker 450 and the eye of the user 115 can be used to hide emissions from the fluorescent marker 450 that appear in the visible spectrum from being visible to the user 115. Alternatively, when the excitation spectrum of the marker 450 overlaps with the visible spectrum, detection occurs when the user 115 looks away or the HMD 100 is not in use (e.g., based on timing).

[0034] In some implementations, the markings used to embed data in the corrective lens 200 include a material that is initially transparent but has variable reflectivity properties. In some implementations, the machine-readable markings 550 (e.g., 550a, 550b) have an exposure wavelength / spectrum and a detection wavelength / spectrum. In some implementations, the markings 550 used to embed data in the corrective lens 200 include reversible photochromic materials that change their reflective properties from transparent to opaque (e.g., detectably darker) over a portion of the electromagnetic spectrum (e.g., within the detection spectrum of the detection image sensor) when exposed to a higher energy spectrum / illumination (e.g., UV illumination).

[0035] Thus, when the reversible photochromic material is exposed to high-energy illumination and the external light source is activated within the detection spectrum of the detection image sensor / camera, the photochromic reaction spatially darkens the external light transmission through the markings 550 in the corrective lens to reveal the embedded information. In some implementations, when the photochromic reaction is achieved using UV illumination, the markings 550 are detected when the HMD 100 is not in use (e.g., to avoid exposing the skin to UV illumination).

[0036] like Figure 5AAs shown, the reversible photochromic material forming the mark 550a has an exposure spectrum from the exposure illumination source 522 (e.g., outside the visible spectrum, IR, UV illumination) that does not overlap with the incident illumination seen by the user 115 (e.g., visible spectrum) or the detectable spectrum of the eye-tracking image sensor 124. However, the detection spectrum of the mark 550a is within the spectrum of the eye-tracking image sensor 124. In such implementations, the mark 550a can be detected at any time because the mark 550a is not visible to the user 115, but additional illumination sources 522 (e.g., excitation illumination) can be added to the HMD 100. In some implementations, the mark 550a and the eye tracking information are detected in different parts of the image from the image sensor 124. Alternatively, the exposure mark 550a illuminated by the eye-tracking light source 122 can be detected by the image sensor 124 when the user 115 looks away, blinks, when the image sensor 124 is not in an actively illuminated eye tracking situation, or when the HMD 100 is not in use. Although Figure 5A An implementation is shown where the detection spectrum of marker 550a is within the detectable spectrum of eye-tracking image sensor 124, but embodiments are not limited to such implementations.

[0037] For example, Figure 5B As shown, the marker 550b may have an excitation spectrum and a detection spectrum that do not overlap with the incident illumination (e.g., visible spectrum) seen by the user 115 or the detectable spectrum of the eye-tracking image sensor 124. In addition, the marker 550b has a detection light source and a sensor that do not overlap with the incident illumination (e.g., visible spectrum) seen by the user 115 or the detectable spectrum of the eye-tracking image sensor 124. However, an exposure illumination source 522 and an additional illumination source 526 and an image sensor 524 for detecting the exposure marker 550b are added to the HMD 100. In such a specific implementation, the reversible photochromic marker 550b can be detected at any time without interfering with the operation of the HMD 100. For example, the exposure illumination source can be in the UV spectrum, and the detection light source / image sensor can be in the NIR / IR spectrum different from the image sensor 124 or in the visible spectrum that is shielded from the user 115.

[0038] In some implementations, the machine-readable mark used to embed data in the corrective lens includes a reflective holographic optical element embedded in or attached to the corrective lens. Thus, the holographic mark in the corrective lens 200 is illuminated by a light source (e.g., an incoherent LED) and detected by an image sensor in the HMD 100. For example, the light source can be an additional light source placed at a different location (e.g., away from the eye-tracking light source 122) to reveal the holographically stored data (e.g., a Denisyuk hologram) and route it to the viewing direction of the image sensor 124. In such implementations, the holographic mark can be detected at any time because, although an additional illumination source (e.g., holographic illumination) can be added to the HMD 100, the holographic mark is not visible to the user 115. In some implementations, the holographic mark and the eye-tracking information are detected in different portions of the image from the image sensor 124. In one implementation, an additional illumination source (e.g., a holographic illumination source) and an additional image sensor for detecting the illuminated holographic mark are added to the HMD 100. In some implementations, the reference beam angle of the illuminated holographic mark is limited (eg, 15°, 30°, 45°).

[0039] In some implementations, the machine-readable marking used to embed data in the corrective lens 200 comprises a material that is transparent when not illuminated but generates a detectable light pattern on the projection surface when illuminated to form a temporarily visible / detectable marking. In some implementations, the caustic marking 650 in the corrective lens 200 comprises a computationally caustic material that uses illumination and detection wavelengths / spectra.

[0040] In some implementations, the machine-readable mark used to embed data in the corrective lens 200 includes a calculated caustic material that forms the caustic mark 650. In some implementations, the information in the caustic mark 650 becomes visible to the caustic mark image sensor when illuminated by a caustic light source (e.g., a specific light source at a specific angle) on a projection surface 660 (e.g., a detection surface at a specific distance) that is spaced apart from the corrective lens 200 (e.g., the caustic mark 650). Figure 6A As shown, eye tracking light source 122 (e.g., IR LED) illuminates caustic marker 650 in corrective lens 200, and eye tracking image sensor 124 (or another camera 624) detects information (e.g., a light and dark pattern) visible on the skin surface of user 115 (e.g., becomes projection surface 660). Alternatively, as Figure 6BAs shown, an additional caustic light source 622 in the HMD 100 can be used to illuminate the caustic marker 650 so that the projected information is detected on the projection surface 660 by the image sensor 124 or an additional image sensor 624. In such implementations, the projected information from the marker 650 can be detected at any time because the marker 650 is not visible to the user 115. Alternatively, the caustic marker 650 is detected when the HMD 100 is not in use. In this case, the projection surface 660 can be a surface on which the HMD 100 is placed (e.g., a flat surface such as a table, a desktop, or a nearby flat surface such as a vertical surface or wall). Again, when the caustic marker 650 is illuminated by the corresponding illumination source, the invisible information in the caustic marker 650 is made visible on the projection surface to the image sensor of the HMD 100. In some implementations, the caustic marker 650 is relatively small, transparent, and can be located in a peripheral region on the corrective lens 200. For example, the caustic marker 650 is a polished uneven segment of the corrective lens 200.

[0041] In some implementations, the machine-readable marker in the corrective lens includes (1) a marker (e.g., size or wavelength) that is always detectable but not perceptible by the human eye, such as a very small but detectable etching in the corrective lens, (2) a marker that is detectable in the emission spectrum after being illuminated by a different excitation spectrum / illumination (e.g., fluorescent), (3) a marker that is detectable in the illumination / detection spectrum after being illuminated by a different exposure spectrum / illumination (e.g., photochromic), (4) a marker that is detectable when illuminated in a single illumination / detection spectrum (e.g., reflective hologram), or (5) a marker that is detected using a spaced-apart projection surface when illuminated in a single illumination / detection spectrum (e.g., computed caustic).

[0042] In some implementations, the marker in the corrective lens 200 can include information such as the prescription of the lens, the curvature of the lens, the cylinder power, the sphere power, the prism information, etc. In one implementation, the calibration data for each lens can be included in the marker on the corrective lens 200. For example, the calibration data for each lens provides the actual surface profile of the lens (e.g., lens defects) that provides additional data and allows the HMD 100 to more accurately accommodate each individual corrective lens. The calibration data for each lens can be an order of magnitude greater data than the prescription parameters.

[0043] In some implementations, the HMD 100 uses the information in the markings of the corrective lens 200 to adjust the rendering process of the display 110, for example, to reduce or correct distortions. In addition, the HMD 100 uses the information in the markings of the corrective lens 200 to verify the identity of the particular attached corrective lens 200, the user 115, or the spatial positioning of the attached corrective lens 200. For example, the iris of the user 115 can be used to identify the user 115 and verify the attached corrective lens 200. As another example, a small displacement (e.g., right, left, up, or down) in the spatial positioning of the attached corrective lens 200 can be identified from the markings in the corrective lens 200 and corrected using the rendering process of the display 110. Alternatively, when a large displacement (e.g., exceeding a threshold) in the spatial positioning of the attached corrective lens 200 is detected, a warning to reattach the corrective lens can be provided.

[0044] In some implementations, the markings in the corrective lens 200 can be detected once (e.g., when the HMD 100 is powered on, during initialization, or when placed on the head of the user 115), repeatedly (e.g., 1 time per second or 1 time per minute), when eye tracking is not used, or at a point in time when the corrective lens 200 is mounted to the HMD 100 (e.g., using one or more of a detected vibration, a “click,” a special orientation, or an instruction (e.g., “please detect the attached prescription lens”)). In one implementation, the markings in the corrective lens 200 are detected when an event (e.g., impact, temperature change, during a single instance or over time) that can have shifted the position of the attached corrective lens 200 occurs. In some implementations, the detection of the markings in the corrective lens occurs in milliseconds (e.g., 10 ms, 30 ms, etc.). In some implementations, the markings in the corrective lens 200 are detected using an additional illumination source when the HMD 100 is not on the head of the user 115.

[0045] In some implementations, the markings in the corrective lens 200 are on the frame of the corrective lens 200 or on a combination of the frame and the transparent surface of the corrective lens 200. For example, certain portions (e.g., the interior or the exterior) of the frame of the corrective lens 200 are visible to the image sensor of the HMD 100 and thus can be used for the machine-readable markings as variously described herein.

[0046] In some implementations, the markings in the corrective lens 200 are a combination of controlled active illumination and reactive materials that are transparent in the visual and image sensor spectrum when not excited. In some implementations, the image sensor 124 is sensitive to wavelengths within the visible spectrum, but can alternatively be sensitive to NIR, IR, UV spectrum, etc.

[0047] In some implementations, the machine-readable markings (e.g., markings 250, 450, 550, 650) in the corrective lens 200 can be generated using additive methods, subtractive methods, or embedded within the surface of the corrective lens 200 (e.g., beneath a protective coating). Furthermore, the markings in the corrective lens 200 can be located in only a portion of the corrective lens 200 or can be repeatedly generated at multiple locations in the corrective lens 200 or the frame of the corrective lens 200.

[0048] Various embodiments disclosed herein include devices, systems, and methods for providing machine-readable markings on attachable corrective lenses to identify which specific corrective lens(es) are attached to an HMD. For example, a right-side corrective lens and a left-side corrective lens may have different prescriptions, and thus, if the right and left corrective lenses are swapped, the HMD user can be notified. As another example, multiple users of an HMD may have their own corrective lenses, and the HMD user can be notified if an incorrect prescription lens is attached. Furthermore, markings on the corrective lenses are used to ensure proper positioning of the corrective lenses attached to the HMD.

[0049] In various implementations, the image sensor 124 is a frame / shutter-based camera that generates images of the user's 115 eyes at a particular point in time or at multiple points in time at a frame rate. Each image includes a matrix of pixel values ​​corresponding to pixels of the image, where the pixels correspond to positions of a light sensor matrix of the camera. In specific implementations, each image is used to measure or track pupil dilation or pupil position by measuring changes in pixel intensity associated with one or both of the user's pupils. In various implementations, the image sensor 124 is an event camera that includes multiple light sensors at multiple corresponding positions (e.g., a light sensor matrix) that generates an event message indicating a specific position of a particular light sensor in response to the particular light sensor detecting a change in light intensity.

[0050] In some implementations, the HMD 100 uses computer vision techniques and a combination of sensors to track the position of the HMD 100 in the physical environment. In some implementations, visual inertial odometry (VIO) or simultaneous localization and mapping (SLAM) tracks the 6DOF movement of the HMD 100 in the physical environment 105 in real time (e.g., 3DOF of spatial (xyz) motion (translation) and 3DOF of angular (pitch / yaw / roll) motion (rotation)). In some implementations, the VIO recalculates or updates the position of the electronic device in the physical environment 105 between each frame refresh on the display of the HMD 100. For example, the VIO recalculates the position of the HMD 100 10 times, 30 times, or 60 times per second or more. In some implementations, the VIO tracks position using visual sensors (e.g., a camera system) by matching points in the real world to pixels on an image sensor in each frame, using depth sensors, and / or using an inertial system (e.g., an accelerometer and gyroscope, an inertial measurement unit (IMU), etc.). In some implementations, a color image can be used. Alternatively, in some implementations, a grayscale image can be used.

[0051] Figure 7 is a flow chart illustrating an exemplary method for identifying information about a transparent surface using markings on a transparent surface. For example, the transparent surface may be a prescription lens insert for an HMD, and the information may be prescription parameters of the prescription lens insert. In some implementations, the markings are configured to not interfere with "normal" use of the transparent surface, such as viewing through the transparent surface or performing eye tracking during use of the transparent surface. In some implementations, method 700 is performed by a device (e.g., Figure 8 The method 700 may be performed by an electronic device 800 (e.g., an electronic device 800). The method 700 may be performed using an electronic device, or by multiple devices communicating with each other. In some implementations, the method 700 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 700 is performed by a processor executing code stored in a non-transitory computer-readable medium (e.g., a memory). In some implementations, the method 700 is performed by an electronic device having a processor.

[0052] At block 710, method 700 acquires image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device. In some implementations, the electronic device is an HMD, and the transparent surface is a prescription lens attachment to the HMD. For example, the transparent surface can be an insertable prescription lens, a removable prescription lens, a clip-on prescription lens, etc. In some implementations, the image sensor data can be one or more images, each image including at least a portion of the transparent surface. In some implementations, the image sensor is a visible light image sensor, an IR image sensor, a NIR image sensor, or a UV image sensor. The image sensor can capture additional data, such as depth data.

[0053] At block 720, method 700 identifies a code in the image data, wherein the code is detectable by an image sensor on a transparent surface without interfering with functionality of an electronic device involving the transparent surface. In some implementations, the code can be read without interfering with a user viewing a physical environment or an XR environment (e.g., content) through the transparent surface while using an HMD. In some implementations, the code can be read without interfering with eye tracking functionality implemented by the HMD when the transparent surface is attached. In some implementations, the code can be (1) a mark on / in the transparent surface that is invisible to the human eye, such as a very small etching on a portion of the transparent surface, (2) a mark detectable in an emission spectrum after being illuminated by different excitation spectra / illuminations (e.g., fluorescence), (3) a mark detectable in an illumination / detection spectrum after being illuminated by different exposure spectra / illuminations (e.g., photochromic), (4) a mark detectable when illuminated in a single illumination / detection spectrum (e.g., a reflection hologram), or (5) a mark detected using a spaced-apart projection surface (e.g., computational caustics).

[0054] At box 730, method 700 provides content at the electronic device based on the identified code, where the content can be seen through the transparent surface. In some specific implementations, the content is provided or rendered based on the identified prescription in the identified code. In some specific implementations, providing the content may involve adapting the manner in which the content is rendered based on the identified code. For example, providing the content based on the identified code may involve modifying the displayed image to compensate for lens distortion based on the prescription in the identified code (e.g., transparent surface). Alternatively, the identified code (e.g., prescription) is used for appropriate rendering, such as reducing or avoiding pupil movement, which is an artifact when a straight line may appear to jitter (e.g., when the user 115 turns their head). For another example, providing the content based on the identified code can verify the 3D position and orientation of the transparent surface attached to the HMD.

[0055] In some implementations, the method 700 detects attachment (e.g., an attach action) of the transparent surface to the electronic device. In some implementations, detecting attachment of the transparent surface to the electronic device is based on audio data, IMU data, verbal communication, proximity sensors, visual sensors, instructions, etc. For example, upon detecting attachment of the transparent surface to the electronic device, image data is acquired from an image sensor that includes the transparent surface to identify a code on the transparent surface.

[0056] In some implementations, the method 700 identifies the code of the transparent surface only once. For example, the code of the transparent surface is identified prior to use of the electronic device. Alternatively, the code of the transparent surface is identified during a blink of an eye of a user of the electronic device. In some implementations, the identified code of the transparent surface is repeatedly or periodically detected. For example, the identified code of the transparent surface is detected each time the electronic device is enabled.

[0057] In some implementations, blocks 710-730 are repeatedly performed. In some implementations, the techniques disclosed herein can be implemented on a smartphone, tablet, or wearable device such as an HMD with an optically see-through display or an opaque display.

[0058] A physical environment refers to the physical world that people can interact with and / or sense without aid of electronic systems. A physical environment refers to the physical world that people can sense and / or interact with without aid of electronic devices. A physical environment can include physical features such as physical surfaces or physical objects. For example, a physical environment corresponds to a physical park that includes physical trees, physical buildings, and physical people. People can directly sense and / or interact with a physical environment such as through sight, touch, hearing, taste, and smell. In contrast, an extended reality (XR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic devices. For example, an XR environment can include augmented reality (AR) content, mixed reality (MR) content, virtual reality (VR) content, etc. In the case of an XR system, a subset of a person’s physical motions, or representations thereof, are tracked, and, in response, one or more characteristics of one or more virtual objects simulated in the XR system are adjusted in a manner consistent with at least one physical law. For example, an XR system can detect head movements, and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. As another example, an XR system can detect movements of an electronic device (e.g., a mobile phone, a tablet, a laptop, etc.) that presents an XR environment, and, in response, adjust graphical content and an acoustic field presented to the person in a manner similar to how such views and sounds would change in a physical environment. In some cases (e.g., for accessibility reasons), an XR system can adjust characteristics of graphical content in an XR environment in response to representations of physical motions (e.g., voice commands).

[0059] There are many different types of electronic systems that enable people to sense and / or interact with various XR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system may have an integrated opaque display and one or more speakers. Alternatively, a head-mounted system may be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Instead of an opaque display, a head-mounted system may have a transparent or translucent display. The transparent or translucent display may have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In some embodiments, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology that projects graphic images onto a person's retina. Projection systems can also be configured to project virtual objects into a physical environment, such as as a hologram or onto a physical surface.

[0060] In some implementations, the electronic device that presents the XR environment is a single device that can be handheld (e.g., a mobile phone, tablet, laptop, etc.) or wearable (e.g., a watch, head-mounted device (HMD), etc.). In some implementations, the functionality of the electronic device is implemented via two or more communication (e.g., wired or wireless) devices (e.g., additionally including an optional base station). Other examples include laptop computers, desktop computers, servers, or other such devices that include additional capabilities in terms of power, CPU power, GPU power, storage power, memory power, etc.

[0061] Figure 8is a block diagram of an exemplary device 800. While some specific features are shown, those skilled in the art will appreciate from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the implementations disclosed herein. To this end, as a non-limiting example, in some embodiments, the electronic device 800 includes one or more processing units 802 (e.g., microprocessors, ASICs, FPGAs, GPUs, CPUs, processing cores, etc.), one or more input / output (I / O) devices and sensors 806, one or more communication interfaces 808 (e.g., USB, FIREWIRE, THUNDERBOLT, IEEE 802.3x, IEEE 802.11x, IEEE802.16x, GSM, CDMA, TDMA, GPS, IR, BLUETOOTH, ZIGBEE, SPI, I2C, or similar types of interfaces), one or more programming (e.g., I / O) interfaces 810, one or more displays 812, one or more internal or external-facing sensor systems 814, memory 820, and one or more communication buses 804 for interconnecting these and various other components.

[0062] In some implementations, the one or more communication buses 804 include circuits that interconnect and control communications between system components. In some implementations, the one or more I / O devices and sensors 806 include at least one of the following: an inertial measurement unit (IMU), an accelerometer, a magnetometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, or one or more depth sensors (e.g., structured light, time of flight, etc.), etc.

[0063] In some implementations, one or more displays 812 are configured to render content to a user. In some implementations, one or more displays 812 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light-emitting field-effect transistor (OLET), organic light-emitting diode (OLED), surface-conduction electron emitter display (SED), field emission display (FED), quantum dot light-emitting diode (QD-LED), microelectromechanical system (MEMS), or similar display types. In some implementations, one or more displays 812 correspond to waveguide displays such as diffraction, reflective, polarization, holographic, etc. For example, electronic device 800 may include a single display. As another example, electronic device 800 may include a display for each eye of the user.

[0064] In some implementations, the one or more internally facing or externally facing sensor systems 814 include an image capture device or array that captures image data or an audio capture device or array that captures audio data (e.g., a microphone). The one or more image sensor systems 814 may include one or more RGB cameras (e.g., with a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor), a monochrome camera, an IR camera, etc. In various implementations, the one or more image sensor systems 814 also include an illumination source that emits light, such as a flash. In some implementations, the one or more image sensor systems 814 also include an on-camera image signal processor (ISP) that is configured to perform a plurality of processing operations on the image data.

[0065] Memory 820 includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid-state memory devices. In some implementations, memory 820 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 820 optionally includes one or more storage devices remotely located from one or more processing units 802. Memory 820 includes non-transitory computer-readable storage media.

[0066] In some implementations, the memory 820 or a non-transitory computer-readable storage medium of the memory 820 stores an optional operating system 830 and one or more instruction sets 840. The operating system 830 includes processes for handling various basic system services and for performing hardware-related tasks. In some implementations, the instruction set 840 includes executable software defined by binary information stored in the form of electrical charge. In some implementations, the instruction set 840 is software that can be executed by one or more processing units 802 to implement one or more of the techniques described herein.

[0067] In some implementations, the instruction set 840 includes a marker detector 842 that can be executed by the processing unit 802 to detect markers on a transparent surface according to one or more techniques disclosed herein, and use the detected markers to identify information about the transparent surface. For example, the transparent surface can be a prescription lens insert for an HMD, and the information can be prescription parameters of the prescription lens insert.

[0068] Although instruction set 840 is shown as residing on a single device, it should be understood that in other implementations, any combination of elements may be located in separate computing devices. Figure 8More functionality is used as a functional description of the various features present in particular implementations, as opposed to structural diagrams of the implementations described herein. As will be appreciated by those of ordinary skill in the art, items shown separately could be combined and some items could be separated. For example, actual number of instructions and the division of

[0069] It is to be understood that the implementations described above are cited by way of example, and that the present disclosure is not limited to the particulars of the implementations set forth above. Rather, the scope includes both combinations and sub-combinations of the various features described above, as well as modifications and variations thereto which would occur to those skilled in the art, having the benefit of the foregoing description, who seeks to implement the disclosure.

[0070] Those of ordinary skill in the art will realize that the systems, methods, components, devices and circuits described herein are not limited to the specific implementations described. Moreover, other effective aspects and / or variants are not included in the description described herein. Therefore, a number of details are described in order to provide a thorough understanding of the example aspects illustrated in the drawings. Moreover, the drawings merely illustrate some example implementations of the present disclosure and therefore should not be taken to be limiting.

[0071] While this specification contains many specifics, these specifics should not be construed as limiting the scope of any invention or potentially patentable subject matter in any way. Some features described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination. Moreover, although certain features can be described as being implemented in one combination, one or more features from a combination can be left out of the combination, and the combination can be claimed as a new combination whether or not the one or more features are present in the claimed combination.

[0072] Similarly, although operations are shown in a particular order in the accompanying drawings, this should not be understood as requiring that such operations be performed in a sequential order or in the particular order shown, or that all of the operations shown be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. In addition, the division of each system component in the above-described embodiment should not be understood as requiring such division in all embodiments, and it should be understood that the program components and system can generally be integrated together in a single software product or packaged in multiple software products.

[0073] Thus, specific embodiments of the subject matter have been described. Other embodiments are also within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. Furthermore, the processes illustrated in the accompanying drawings do not necessarily require the specific order shown or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0074] The embodiments of the subject matter and operations described in this specification may be implemented in digital electronic circuits or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more thereof. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a computer storage medium for execution by a data processing device or for controlling the operation of the data processing device. Alternatively or in addition, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more thereof, or included in a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device. Furthermore, although a computer storage medium is not a propagated signal, a computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagated signal. The computer storage medium can also be, or be included in, one or more separate physical components or media (eg, multiple CDs, disks, or other storage devices).

[0075] The term "data processing device" encompasses all types of devices, equipment and machines for processing data, such as programmable processors, computers, systems on a chip, or multiple or combinations of the foregoing. The device may include a dedicated logic circuit (e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit)). In addition to hardware, the device may also include a code for creating an execution environment for the computer program under consideration, such as a code constituting a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform operating environment, a virtual machine, or a combination of one or more of them. The device and the execution environment can implement various different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures. Unless otherwise specifically stated, it should be understood that throughout the specification, discussions using terms such as "processing," "calculating," "calculating," "determining," and "identifying" refer to the actions or processes of a computing device, such as one or more computers or similar electronic computing devices, which manipulate or convert data represented as physical electronic quantities or magnetic quantities within the memory, registers, or other information storage devices, transmission devices, or display devices of a computing platform.

[0076] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems that access stored software that programs or configures the computing system from a general-purpose computing device to a dedicated computing device that implements one or more specific implementations of the subject matter of the present invention. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained herein in software used to program or configure a computing device.

[0077] The specific implementation of the method disclosed herein can be performed in the operation of such a computing device. The order of the blocks presented in the above examples can be changed, for example, the blocks can be reordered, combined and / or divided into sub-blocks. Certain blocks or processes can be executed in parallel. The operations described in this specification can be implemented as operations performed by a data processing device on data stored on one or more computer-readable storage devices or received from other sources.

[0078] The use of "suitable for" or "configured to" herein is intended to be open and inclusive language, and does not exclude devices that are adapted or configured to perform additional tasks or steps. Furthermore, the use of "based on" is intended to be open and inclusive, as a process, step, calculation, or other action that is "based on" one or more stated conditions or values ​​may, in practice, be based on additional conditions or values ​​beyond those stated. The headings, lists, and numbers included herein are for ease of explanation only and are not intended to be limiting.

[0079] It will also be understood that, although the terms "first," "second," etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description as long as all occurrences of "first node" are consistently renamed and all occurrences of "second node" are consistently renamed. A first node and a second node are both nodes, but they are not the same node.

[0080] The terms used herein are merely for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms "a" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and cover any and all possible combinations of one or more of the associated listed items. It will also be understood that the term "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0081] As used herein, the term “if” may be interpreted to mean “when the precondition is true” or “when the precondition is true” or “in response to determining” or “upon determining” or “in response to detecting” that the precondition is true, depending on the context. Similarly, the phrase “if it is determined that [the precondition is true]” or “if [the precondition is true]” or “when [the precondition is true]” is to be interpreted to mean “upon determining that the precondition is true” or “in response to determining” or “upon determining” that the precondition is true or “when detecting that the precondition is true” or “in response to detecting” that the precondition is true, depending on the context.

Claims

1. A method for identifying information about a transparent surface, comprising: At the processor: acquiring image data from an image sensor of an electronic device, the image data corresponding to a transparent surface of the electronic device; identifying a code in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with functionality of the electronic device involving the transparent surface; and providing content at the device based on the identified code, wherein the content is viewable through the transparent surface, wherein the method further comprises: detecting a blink of an eye by a user of the electronic device; acquiring one or more frames of data from the image sensor during the blink; and The code on the transparent surface is detected in the one or more frames of data from the image sensor.

2. The method according to claim 1, wherein The recognized code identifies prescription parameters of the transparent surface, a user of the electronic device, or a position and orientation of the transparent surface relative to the electronic device.

3. The method of claim 1 , further comprising detecting the code based on detecting an action of attaching the transparent surface to the electronic device, wherein Detecting the attachment is based on audio data, IMU data, voice communication, proximity sensors, vision sensors, or commands.

4. The method according to claim 1, wherein The code includes markings in the transparent surface that are not perceptible to the human eye.

5. The method according to claim 1, wherein The code comprises an excitable fluorescent label visible at a predetermined emission wavelength or wavelength band.

6. The method according to claim 1, wherein The code comprises an excitable photochromic mark visible at a predetermined detection wavelength or wavelength band.

7. The method according to claim 1, wherein The code comprises a reflective holographic mark visible from a limited range of angular positions.

8. The method according to claim 1, wherein The code includes calculated caustics marks visible at a preset detection wavelength or wavelength band on a projection surface spaced apart from the transparent surface.

9. A method for identifying information about a transparent surface, comprising: At the processor: acquiring image data from an image sensor of an electronic device, the image data corresponding to a transparent surface of the electronic device; identifying a code in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with functionality of the electronic device involving the transparent surface; periodically detecting the code generated using active lighting when eye tracking or blink detection is not being performed by the electronic device or the electronic device is not in use; and Content is provided at the device based on the identified code, wherein the content is viewable through the transparent surface.

10. An electronic device comprising: an image sensor configured to acquire image data corresponding to a transparent surface attached to the electronic device; a display configured to present content at the electronic device for viewing through the transparent surface, the transparent surface including a depiction of a code, wherein the code is detectable on the transparent surface by the image sensor without interfering with viewing of the display through the transparent surface; and a processor configured to provide content on the display based on the code, wherein the processor is further configured to perform operations comprising: detecting a blink of an eye by a user of the electronic device; acquiring one or more frames of data from the image sensor during the blink; and The code on the transparent surface is detected in the one or more frames of data from the image sensor.

11. The electronic device according to claim 10, wherein: The code includes markings in the transparent surface that are not perceptible to the human eye.

12. The electronic device according to claim 10, wherein: The code includes an excitable fluorescent material that is visible at a predetermined emission wavelength or wavelength band when the electronic device is not in use.

13. The electronic device according to claim 10, wherein: The code comprises an excitable photochromic material visible at a predetermined detection wavelength or wavelength band.

14. The electronic device according to claim 10, wherein: The code comprises a reflective holographic optical element visible from a limited range of angular positions.

15. The electronic device according to claim 10, wherein The code includes calculated caustics, and the calculated caustics are visible at a preset detection wavelength or wavelength band on a projection surface spaced apart from the transparent surface.

16. The electronic device according to claim 10, wherein The electronic device includes a head-mounted device.

17. The electronic device of claim 10, further comprising tracking the eyes or gaze of a user of the electronic device using the image sensor.

18. The electronic device according to claim 10, wherein The recognized code identifies prescription parameters of the transparent surface, a user of the electronic device, or a position and orientation of the transparent surface relative to the electronic device.

19. A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising: acquiring image data from an image sensor of an electronic device, the image data corresponding to a transparent surface attached to the electronic device; identifying a code in the image data, wherein the code is detectable on the transparent surface by the image sensor without interfering with functionality of the electronic device involving the transparent surface; and providing content at the device based on the identified code, wherein the content is viewable through the transparent surface, wherein the operations further comprise: detecting a blink of an eye by a user of the electronic device; acquiring one or more frames of data from the image sensor during the blink; and The code on the transparent surface is detected in the one or more frames of data from the image sensor.

Images