Wearable multi-functional ophthalmic device for monitoring oculomotor components

Abstract

Wearable multi-functional ophthalmic devices and associated methods for monitoring oculomotor components are described. The described technology enables seamless integration of multiple primary diagnostic functions, including fundus image capture, accommodation response, and eye tracking, along with the required illumination units and visual stimuli display. The subsystems work together to illuminate the eye pupil and retina features, to render visual stimuli through an integrated display, to record the ocular responses to oculomotor demands, and to capture fundus images.

Description

WEARABLE MULTI-FUNCTIONAL OPHTHALMIC DEVICE FOR MONITORINGOCULOMOTOR COMPONENTSCROSS-REFERENCE TO RELATED APPLICATION(S)

[0001] This application claims priority to the provisional application with serial number 63 / 653,647 titled “A WEARABLE MULTI-FUNCTIONAL OPHTHALMIC DEVICE FOR MONITORING OCULOMOTOR COMPONENTS,” filed May 30, 2024 The entire contents of the above noted provisional application are incorporated by reference as part of the disclosure of this document.TECHNICAL FIELD

[0002] The technology described in this patent document relates to ophthalmic devices and associated methods for measuring monitoring oculomotor components.BACKGROUND

[0003] Accurate and clear visual perception requires meticulous coordination of oculomotor (OM) components, including Eye Movement (EM), Accommodation (ACC), and Pupil Size (PS). This coordination involves continuous monitoring of aspects like target direction, distance, and ambient light, which are called oculomotor demands (ODs) - and the synchronized adjustment of eye position, focus, and pupil size, known as oculomotor responses (ORs). Visual development irregularities, eye strain, oculomotor system injuries, and aging can create persistent, significant OR / OD mismatches, leading to various eye diseases. Clinically, the objective is to reduce or eliminate these mismatches to maintain optimal visual function. The surveillance of the aforementioned oculomotor components is very crucial in ocular pathology diagnostics.SUMMARY

[0004] The techniques disclosed herein can be implemented in various embodiments related to a wearable multi-functional ophthalmic device for monitoring oculomotor components. The disclosed methods, systems and devices seamlessly integrate multiple primary diagnostic functions — fundus image capture, accommodation response, and eye tracking, along with the required illumination units and visual stimuli display. The methods, systems, and devices include features to illuminate the eye pupil and retina features, to render visual stimuli through an integrated display, to record the ocular responses to oculomotor demands, and to capture fundus images. Various embodiments detailing associated lens designs, their imaging performance,simulations, and experimental results to validate the concept are described herein.BRIEF DESCRIPTION OF THE DRAWINGS

[0005] FIG. 1 illustrates two example optical configurations for implementing eye movement tracking based on pupil-corneal reflection methods.

[0006] FIG. 2 illustrates an example table-top fundus camera configuration for implementing eye tracking based on monocular indirect ophthalmoscopy.

[0007] FIG. 3 illustrates an example smartphone-based fundus camera design.

[0008] FIG. 4 illustrates examples of fundus illumination including trans-pupillary illumination and trans-pars-planar illumination.

[0009] FIG. 5 A shows a schematic diagram of an autorefractor based on Scheiner’s principle.

[0010] FIG. 5B illustrates a schematic diagram of an accommodation measurement device based on wavefront aberrometry.

[0011] FIG. 6A illustrates a stand-alone fundus imaging system with a fixation target.

[0012] FIG. 6B illustrates a smartphone-based fundus imaging system with a fixation target.

[0013] FIG. 7A illustrates a functionality integrated eye-tracking camera.

[0014] FIG. 7B illustrates a systematic integrated eye-tracking camera and micro-display optical layout.

[0015] FIG. 8 illustrates a block diagram of an example multi-functional ophthalmic device including subsystems and their functionalities in accordance with some embodiments.

[0016] FIG. 9 illustrates example schematics showing the optical conjugates for the subsystems described in FIG. 8.

[0017] FIG. 10 depicts a schematic layout of a wearable multi-functional ophthalmic device for monitoring oculomotor components in accordance with an example embodiment.

[0018] FIG. 11 illustrates reflectance and transmittance as a function of wavelength for two mirror coatings associated with an X-cube of FIG. 10 in accordance with an example embodiment.

[0019] FIG. 12 illustrates a schematic layout of a wearable multi-functional ophthalmic device for monitoring oculomotor components in accordance with another example embodiment.

[0020] FIG. 13 illustrates reflectance and transmittance as a function of wavelength for two mirror coatings associated with prisms of FIG. 12 in accordance with an example embodiment.

[0021] FIG. 14 illustrates reflectance and transmittance as a function of wavelength for analternative coating scheme in accordance with an example embodiment.

[0022] FIG. 15 illustrates a schematic layout of a wearable multi-functional ophthalmic device for monitoring oculomotor components in accordance with another example embodiment.

[0023] FIG. 16 illustrates reflectance and transmittance as a function of wavelength for coatings associated with prisms of FIG. 15 in accordance with an example embodiment.

[0024] FIG. 17 illustrates a schematic layout of a wearable multi-functional ophthalmic device for monitoring oculomotor components in accordance with another example embodiment.

[0025] FIG. 18 illustrates different techniques for combining retina illumination and imaging paths for fundus imaging in accordance with some example embodiments.

[0026] FIG. 19 illustrates example optical diagrams to demonstrate illumination effect of different LED conjugate planes for configuration each LED in an LED array is placed on the optical axis of its corresponding condenser lenslet element.

[0027] FIG. 20 illustrates an example optical diagram to demonstrate illumination effect of LED array when each LED in the array is placed on the optical axis of its corresponding condenser lenslet element and is conjugated at the center coverage spot.

[0028] FIG. 21 illustrates an example layout of a wearable multi-functional ophthalmic device for monitoring oculomotor components based on the configuration of FIG. 17 and panel (d) of FIG. 18 in accordance with an example embodiment.

[0029] FIG. 22 illustrates example optical design layouts associated with fundus imaging path, display path, and retina illumination path for a wearable multi-functional ophthalmic device in accordance with an example embodiment.

[0030] FIG. 23 A illustrates modulation transfer function (MTF) of a wearable multi-functional ophthalmic device evaluated at an entrance pupil path in accordance with an example embodiment.

[0031] FIG. 23B illustrates MTF of a wearable multi-functional ophthalmic device evaluated a display path in accordance with an example embodiment.

[0032] FIG. 24 illustrates a mechanical housing of a multifunctional fundus imaging device in accordance with an example embodiment.

[0033] FIG. 25 illustrates color fidelity of the display path associated with multifunctional fundus imaging device in accordance with an example embodiment.

[0034] FIG. 26 illustrates captured images that demonstrate the efficacy of a fundus imagingsubsystem in capturing detailed images of the fundus in accordance with an example embodiment.

[0035] FIG. 27 illustrates images that demonstrate eye-tracking capabilities of a multifunctional fundus imaging device in accordance with an example embodiment.

[0036] FIG. 28 illustrates a block diagram of a device 2800 that can be used to implement certain aspects of the disclosed technology.DETAILED DESCRIPTION

[0037] To facilitate understanding of the embodiments described herein, the following provides an overview of the various technologies applicable for measuring the aforementioned oculomotor components.

[0038] Technology related to monitoring eye movements and eye pupil size is known as eye tracking technology. Scientists have been investigating techniques to monitor eye movements for decades and various eye movement tracking technologies have been developed and adopted in many disciplines as fundamental research instruments. Among these different tracking methods, an image-based pupil-corneal reflection approach, which utilizes the various eye features captured by a camera, is considered as the most practical method for non-contact measurement of eye movements and eye pupil size. It takes advantage of the spectral properties of the eye under near infrared (NIR) illumination. The eye is often illuminated with one or multiple NIR light sources such as LEDs to create IR-illuminated eye images, and the images are analyzed through sophisticated algorithms to extract two types of image features: the structural features of the eye, such as eye pupil and limbus, and the artificial features formed by the reflections of the light sources from the anterior cornea or reflection from the surfaces of the eye lens. Based on this pupil- corneal reflection method, eye movement tracking technology is commercially available as a stand-alone system such as Tobii Pro Glasses.

[0039] FIG. 1 illustrates two distinct optical configurations employed for implementing the pupil-corneal reflection method for eye movement tracking. The two configurations mainly differ by the relative positioning of the NIR light sources and the eye-tracking camera for capturing eye images. The first configuration is called dark pupil eye-tracking which is illustrated in panel (a) of FIG. 1. In this configuration, the NIR LED light sources are positioned off the optical axis of the camera, leading the pupil to act as a "light trap." As a result, the pupil appears dark against the illuminated iris, facilitating the detection and tracking of eye features from the captured eyeimages. The second configuration is called bright pupil eye-tracking which is illustrated in panel (b) of FIG. 1. In this configuration, the NIR LED light sources and the eye-tracking camera are aligned on the same optical axis, causing the pupil to appear brightly illuminated owing to the retroreflective properties of the retina.

[0040] Besides eye tracking technology, the second critical diagnostic tool in ophthalmology is fundus imaging, providing insights into the retina, optic disc, and other structures at the back of the eye and facilitating the early recognition, diagnosis, and ongoing tracking of numerous eye diseases and conditions. The optical design of fundus cameras is based on the principle of monocular indirect ophthalmoscopy which requires an illumination path and an imaging path. As illustrated by a schematic layout in FIG. 2, a traditional table-top fundus camera consists of a sequence of optical components including objective lens, relay lens, ophthalmic lens, condensing lens, beam splitter, annular aperture, LED array, polarizers, and imaging detector. The components work together to direct the illumination light through the pupil of the eye, collecting light reflected from the retinal surface and relaying it to the objective lens to form an image of the retina on an image detector. A beamsplitter is usually used to add in the optical path for the illumination subsystem. Traditional table-top fundus cameras represent a cornerstone in retinal imaging, but they are subject to several limitations, including being too bulky, sophisticated, and costly. As a result, such instruments are mainly used in high-end clinical settings and require skilled personnel to operate. In recent years, miniature or handheld fundus cameras have emerged to address these specific limitations. FIG. 3 showcases a smartphone-based fundus camera design. The majority design of miniature handheld fundus camera systems incorporates an ophthalmic lens to initially image the fundus. Following this, a relay system is employed to transfer the captured real image from the ophthalmic lens directly to the imaging lens and the sensor of a smartphone. The emergence of smartphone-based fundus imaging marks a pivotal shift, particularly for its portability, cost efficiency, and inherent connectivity that aligns seamlessly with telemedicine practices.

[0041] As shown in both FIGS. 2 and 3, a retina illumination unit is a critical sub-system required for any fundus imaging device. The anatomical structure of the human eye presents inherent challenges in delivering adequate illumination to the fundus for imaging purposes. Various techniques have been developed to effectively illuminate the fundus area. FIG. 4 illustratesexamples of fundus illumination including trans-pupillary illumination and trans-pars-planar illumination: trans-pupillary illumination (panel (al)), and its associated field of view (FOV) (panel (a2)); trans-pars-planar illumination (panel (bl)), and its associated FOV (panel (b2)). Traditional tabletop fundus cameras use trans-pupillary illumination shown in panel (al) of FIG. 4. This method directs light through the peripheral zone of the pupil, where the light source is optically conjugated to the pupil, creating an annular illumination pattern at the periphery of the pupil, while the fundus imaging path is achieved through the central region of the pupil. This design ensures spatial separation between the paths of illumination and imaging light to mitigate reflections from the cornea and crystalline lens. Specifically, the peripheral zone of the pupil is utilized for illumination, while the central area is reserved for imaging, creating a reflection-free visualization of the fundus. As depicted in the optical schematic in panel (a2) of FIG. 4, such an arrangement requires a sufficiently large pupil diameter to create a buffer zone for preventing an overlap between the paths of illumination and imaging light on the cornea and crystalline lens surfaces. This method often requires pupil dilation to accommodate wide-field fundus photography effectively. Another approach to fundus illumination known as trans-pars-planar illumination has been developed, as depicted in panel (bl) of FIG. 4. This method is able to utilize the entire pupil for imaging purposes, thereby enabling nonmydriatic wide-field fundus photography. The process involves directing illumination light through the pars-plana, where it subsequently becomes diffused by the sclera, achieving homogeneous illumination across the intraocular space as shown in panel (b2). This technique naturally bypasses corneal reflections and can utilize the full aperture of the pupil for imaging. As a result, it facilitates wide-field fundus photography without the need for pharmacological pupil dilation.

[0042] The third key diagnostic tool in ophthalmology is related to accommodation tests that measures ACC in several different ways, such as tests of accommodative amplitude, relative accommodation, accommodative facility, accommodation lag, or accommodative responses to accommodative stimuli. One of the state-of-the-art devices for measuring accommodation is the autorefractor. FIG. 5 A shows a schematic diagram of an autorefractor based on Scheiner’s principle. Typically, the system projects two LEDs onto the pupillary plane and they effectively act as a modified Scheiner pinhole by virtue of the narrow beams of light produced by the small aperture pinhole located at the focal point of the objective lens. The LEDs would double inappearance if there’s any refractive error in the eye. Following this, the retinal image of the LEDs would get reflected off the retina and exit the eye. The outgoing light is then redirected by a beamsplitter towards a dual photodetector. As the LED apparatus shifts back and forth, the spacing between the two images on the photodetector changes. A single, centered LED image on both photodetectors indicates the absence of refractive error, with the LED’s position denoting the specific refractive error for that meridian. For astigmatism assessments, four LEDs are utilized, allowing for the measurement of power perpendicular to the meridian being examined.

[0043] FIG. 5B shows a schematic diagram of an accommodation measurement device based on wavefront aberrometry. Wavefront sensing autorefractors measure the eye's refractive error by analyzing the way light is altered as it enters the eye and reflects off the retina. The process begins with a collimated infrared light beam being projected into the eye as the patient stares and focuses on a target image. The collimated beam then passes through the eye's refractive elements and reflects off the retina. The reflected light is captured by the autorefractor's lenslet array, and sophisticated algorithms are used to calculate the refractive error based on the deviation of the reflected light from an expected pattern. This involves determining spherical and cylindrical errors, and the axis of astigmatism. The device iteratively adjusts its optical elements to minimize aberrations in the reflected light, ultimately providing a prescription that simulates the corrective lenses needed for optimal vision correction. The autorefractor is able to shift the focus of the accommodative stimulus image at different frequencies to measure the ACC components mentioned above.

[0044] In addition to monitoring these oculomotor components, another key function essential for an ocular diagnostic device is to provide an optical method to present visual stimuli and accommodation targets, which thereby enables the synchronous rendering of ODs. In stand-alone fundus cameras, as shown in FIG. 6A, simple fixation targets such as simple illuminated cross targets are rendered through a path shared with the retinal illumination. Similarly, in smartphonebased fundus cameras, as shown in FIG. 6B, the fixation target is usually positioned on the side of the device. This design leverages the synchronized accommodation reflex of the human visual system to help fixate the accommodation of the eye being tested. In both cases, it is usually impossible to render sophisticated visual stimuli beyond the purpose of diagnostic purpose.

[0045] In recent years, the rapid development of virtual reality (VR) and augmented reality(AR) display technologies has facilitated the integration of eye-tracking technology with VR or AR displays. An integrated eye-tracked display system can display two-dimensional (2D) images or stereoscopic three-dimensional (3D) images as a classical VR or AR display does, while additionally tracking the gaze direction of the user, leading to a wearable platform far beyond the capability of a traditional device for ocular pathology diagnostics. FIGS. 7A and 7B illustrate two different approaches for an eye-tracked display: the functionality integration approach (FIG. 7A) and a systematic integration approach (FIG. 7B). In a system utilizing the functionality integration approach, the eye tracker and the display are brought together by stacking the optical components required for each function without co-optimization. In a system utilizing the systematic approach, the optical systems for eye-tracking and display are co-optimized as one single instrument by maximally sharing optical components, leading to more compact and robust solutions. As shown by the example in FIG. 7B, the eye illumination path, the eye imaging path, and the virtual display path are integrated into a partially shared optical system in this case, a single freeform eyepiece.

[0046] Despite significant advancements and commercial availability of numerous stand-alone devices designed to measure the different ORs, existing solutions — whether in commercial markets or within research domains — typically are unable to simultaneously measure multiple OR components. Instead, a typical practice is to obtain separate measurements through different instruments. Such isolated OR assessments may not accurately reflect problems related to abnormal coordination among the OR components, which can compromise diagnosis and effective treatment and limit the utility of these devices. For instance, an integrated eye-tracked display discussed earlier offers the ability to render sophisticated visual stimuli and excel in monitoring eye movements and measuring pupil dynamics with high precision but falls short in the functionality to perform fundus imaging or to measure ACC. The portable smartphone-based systems offer easy access to fundus imaging but fall short in capabilities such as EM tracking and PS measurement, which are crucial parameters for comprehensive OR assessment.

[0047] There is a notable gap for wearable devices capable of capturing high-quality fundus images, continuously monitoring eye movements, accommodation response, and pupil size, and rendering sophisticated, dynamic visual stimuli on demand. The present invention aims to address this gap.

[0048] This patent document discloses methods, designs, and embodiments related to awearable multi-functional ophthalmic device for monitoring oculomotor components. This device seamlessly integrates multiple primary diagnostic functions — fundus image capture, accommodation response, and eye tracking, along with the required illumination units and visual stimuli display. The subsystems work together to illuminate the eye pupil and retina features, to render visual stimuli through an integrated display, to record the ocular responses to oculomotor demands, and to capture fundus images.

[0049] Implementations of the disclosed technology can capture high-quality fundus images, continuously monitor ocular responses such as eye movements, accommodation response, and pupil size, and render sophisticated, dynamic visual stimuli on demand through a VR display. Achieving this goal requires a seamless integration of multiple imaging functions such as fundus imaging, eye pupil imaging, and VR display imaging as well as multiple active illumination functions such as retinal illumination for fundus imaging and eye pupil illumination for eye imaging. This integrated system is designed to comprehensively manage and accurately quantify both OD and OR parameters, as mentioned earlier. The disclosed embodiments offer a multifunctional solution that enhances diagnostic capabilities, enriching user interaction and experience while addressing a broad spectrum of ophthalmic assessment needs.

[0050] FIG. 8 shows a block diagram of an example multi-functional ophthalmic device based on the disclosed technology for monitoring oculomotor components to illustrate its subsystems and associated functionalities.

[0051] The example device of FIG. 8 device uniquely combines all or a subset of the four primary imaging functions — real-time eye imaging for eye tracking and pupil measurements, non- invasive fundus imaging, non-invasive measurement of accommodation response, and an immersive virtual reality (VR) display, along with the required eye pupil and retina illumination units. More specifically, the eye pupil illumination unit provides adequate and nearly uniform illumination around the eye pupil area to facilitate the capture of high-quality eye images which are then utilized for eye-tracking and pupil size measurements. To prevent from causing pupillary reflex and interfering with normal visual functions, it is typically implemented with near infrared light emitting diodes (IR-LED). As illustrated in FIG. 1, the IR-LEDs may be configured off the optical axis of the eye-imaging camera for dark pupil eye tracking (panel (a) in FIG. 1) or be coaxial with the eye-imaging camera for bright pupil eye tracking (panel (b) in FIG. 1).

[0052] Associated with the eye pupil illumination unit of FIG. 8 is the eye-tracking unit that captures high-quality images of the eye pupil, corneal reflections and the features surrounding the eye pupil and the images are then analyzed through sophisticated image-processing algorithms to track ocular responses such as pupil position and size as well as gaze direction. The next key functional unit is the retina illumination unit which provides sufficient and uniform illumination at the back of the retina to facilitate fundus imaging. To ensure eye safety and achieve color imaging, the retina illumination unit typically utilizes pulsed NIR in combination with pulsed broadband light sources. The power rating of these light sources is typically much higher than those for eye pupil illumination. As illustrated in FIG. 4, the retina illumination may be induced either through trans-pupillary illumination (panel (a)) or trans-pars-planar illumination (panel (b)).

[0053] Associated with the retinal illumination unit of FIG. 8 is the fundus imaging unit that captures high-quality, broadband images of the fundus. The next important functional unit is the eye accommodation measurement unit and its associated illumination unit to measure the spherical power of an accommodated eye lens. The accommodation measurement unit and its associated illumination unit can be adapted from the well-known Scheiner’s Principle as illustrated by FIG. 5.

[0054] Finally, instead of using simple, static fixation targets to present stimuli and accommodation targets, an VR display path is implemented in our system, which is particularly effective in presenting sophisticated and on-demand visual stimuli and content, facilitating the measurement of eye movement and accommodation responses during extended examination periods. The addition of the VR display path also opens opportunities for rendering visual stimuli for potential treatment to oculomotor disorders.

[0055] FIG. 9 shows example schematics illustrating the optical conjugates for the subsystems described in FIG. 8. Specifically, panel a(l) shows eye imaging for eye-tracking optical conjugate, panel (a2) shows eye illumination for eye-tracking optical conjugate, panel (bl) shows fundus imaging optical conjugate, panel (b2) shows fundus illumination optical conjugate, panel (c) shows VR display optical conjugate, panel (dl) shows accommodation measurement detector optical conjugate and panel (d2) shows accommodation measurement illumination optical conjugate. The eye imaging sub-system for eye-tracking must be optically conjugated to the eye pupil to ensure uniform precise monitoring of eye movements and pupil size (panel (al)), whilethe eye pupil illumination subsystem is preferred to be nearly collimated (i.e. infinity conjugation) at the eye pupil to provide nearly uniform illumination around the eye pupil area for capturing high-quality eye images (panel (a2)). Conversely, the fundus imaging sub-system requires optical conjugation to the retina to facilitate detailed visualization of retinal structures (panel (bl)), while the retina illumination subsystem typically focuses the source near the pupil plane to provide nearly uniform illumination at the retina for capturing high-quality retina images (panel (b2)). Panel (c) in FIG. 9 illustrates a typical VR display optical path that requires focusing the light from a microdisplay on the retina to render visual stimuli, while panels (dl) and (d2) illustrate the optical paths for the accommodation measurement sub-system and its associated illumination both of which requires optical conjugation to the retina.

[0056] These differences in conjugation require maintaining separate optical paths for each function, while ensuring the overall system remains compact. For instance, the optical conjugates required for the eye pupil illumination path and the retina illumination path are located at different positions. Implementations of the disclosed embodiments provide an innovative optical architecture to integrate these imaging and illumination functions that considers these requirements and maximizes the benefits of path sharing.

[0057] The disclosed embodiments may integrate the distinct imaging and illumination functions as described in FIG. 8 within a confined structure and space. A fully integrated system with the capabilities outlined in FIG. 8 can be constructed by combining up to 7 different optical paths (FIG. 9) that are directed toward or from the eye optics and by utilizing path multiplexing. Some implementations of integrated systems based on the disclosed technology integrate multiple beamsplitters and off-axis optical paths to achieve a compact and efficient design.

[0058] Additionally, a head-mounted wearable device based on the disclosed technology can be designed such that the integrated system not only remains lightweight and unobtrusive but also ensures comfort and usability for extended periods. Some example designs minimize weight and volume by maximizing path sharing and simplifying the optical architecture and mechanical housing. Some head-mounted wearable devices based on the disclosed technology include integrated systems designed such that all necessary functionalities are integrated into a small form factor without compromising the system’s performance or user experience. Finally, the integrated design of the system can be made cost-effective while enhancing portability and quality.

[0059] FIG. 10 depicts the schematic layout of a first example embodiment for a wearable multi-functional ophthalmic device for monitoring oculomotor components. The optical system of this embodiment encompasses seven distinct optical paths: eye pupil illumination and accommodation illumination paths (illustrated in a gray solid line with an arrow pointing toward the eye pupil), retina illumination (in gray long dash line with arrow pointing toward retina), VR display path (in gray thick line with arrow pointing toward retina), fundus imaging path (in black long dash line with arrow pointing toward fundus camera), and the eye-tracking and accommodation imaging paths (in black solid line with arrow pointing toward NIR sensors). The ray paths dedicated to illumination and display are denoted by lines with arrows pointing toward the pupil or retina, while those paths for imaging functions are indicated by lines with arrows pointing toward their corresponding detectors. The fundus imaging path is designed to capture retinal images on demand, leveraging illumination provided by the retina illumination path. Simultaneously, the eye pupil illumination path maintains consistent and uniform lighting on the pupil, aiding the eye-tracking path in its ongoing monitoring of eye position and pupil size. Given the fact that the illumination path for eye accommodation measurement shares the same wavelength bandwidth and similar conjugate requirements to that of eye pupil illumination, the paths for these two functions are combined. Moreover, the VR display path plays a crucial role in presenting visual stimuli to the eye, thus enhancing the capabilities of our device in providing a thorough oculomotor response (OR) assessment.

[0060] The key enabler to the embodiment shown in FIG. 10 is the innovative design of the X-cube beamsplitter through which all of the seven optical paths in FIG. 10 are combined together in a compact fashion. The X-cube beamsplitter is composed of two orthogonal surfaces SI and S2 where the surface SI has a hot mirror coating and the surface S2 has a cold mirror coating. FIG. 11 illustrates the reflectance and transmittance of the two mirror coatings with respect to wavelengths are plotted in panel (a) and panel (b), respectively. The cold mirror coating reflects light with wavelengths less than Xi and transmits light with wavelengths greater than Xi , while the hot mirror coating reflects light with wavelengths greater than X2 and transmits light with wavelengths shorter than X2. In a typical embodiment, X2 is greater than Xi.

[0061] This X-cube naturally combines the optical paths from three orthogonal directions based on their spectral bands: the band less than Xi, the band between Xi and X2, and the bandgreater than X2. For instance, Xi can be chosen to around 650 nm and X2 be around 800 nm or longer. The Surface SI with the hot mirror coating is oriented such that it reflects the NIR light with wavelengths equal to or greater than X2 from the light sources for the eye-tracking or accommodation measurement paths toward the eye. The same surface SI also directs the same NIR light reflected by either the eye pupil or retina toward their corresponding detectors for eye pupil imaging or accommodation measurement. The Surface S2 with the cold mirror coating is oriented such that it reflects the visible light with wavelengths equal or less than Xi from the microdisplay toward the eye for forming the VR display path. The spectral band between Xi and X2, which passes through the X-cube, is used for fundus illumination and fundus imaging. Besides the advantages of offering a compact solution for combining multiple optical paths, the proposed X-cube design, which is based on spectral bands used for these illumination and imaging functions, effectively minimizes light crosstalk among the complex illumination and imaging paths.

[0062] The optical paths combined by the X-cube in FIG. 10 share an eyepiece optics. The different paths may utilize different portions of the eyepiece aperture. Among the optical paths, the NIR eye illumination path and the NIR imaging path for eye-tracking or accommodation measurement are combined before interfacing with the X-cube. Similarly, the retina illumination path and the retina imaging path are combined before interfacing with the X-cube. The eye pupil illumination unit and the eye-tracking camera unit are located on one side of the X-cube while the display unit is located on the opposite side of the X-cube. The display can be a self-emitting microdisplay such as an organic light emitting diode (OLED) display or a display engine composed of a spatial light modulator and an illumination source.

[0063] FIG. 12 depicts the schematic layout of a second example embodiment for a wearable multi-functional ophthalmic device for monitoring oculomotor components. The optical system that integrates all or a subset of the seven distinct optical paths mainly comprises two freeform prisms, prism 1 and prism 2. These two prisms embody the combined optical functions of the X- cube and eyepiece optics for the embodiment in FIG. 10, but in a much more compact form factor. Prism 1, placed adjacent to the eye of a user, is composed of three optical surfaces, SI, S2, and S3. All or a subset of these three surfaces may be freeform surfaces. Prism 2, attached to prism 1, is composed of three optical surfaces, S4, S5, and S6. All or a subset of these three surfaces may be freeform surfaces.

[0064] Among these surfaces forming prisms 1 and 2, surfaces S2 and S5 may be applied with a hot mirror coating and a cold mirror coating, respectively. The reflectance and transmittance of the two mirror coatings with respect to wavelengths are plotted in FIG. 13, panels (a) and (b), respectively.

[0065] The cold mirror coating reflects light with a wavelength less than XI and transmits light with wavelength greater than Xi, while the hot mirror coating reflects light with a wavelength greater than X2 and transmits light with wavelength shorter than X2. In a typical embodiment X2 is greater than Xi. As a result, the combination of the two prisms naturally combines the optical paths from three distinct directions based on their spectral bands: the band less than Xi, the band between XI and X2, and the band greater than X2. For instance, Xi can be chosen to around 650nm and X2 be around 800nm or longer. The spectral band greater than X2 is coupled in and out of the prism 1 through surface S3 and reflected by the surface S2. This band is used for the illumination and imaging paths for eye-tracking imaging or accommodation imaging. The light rays from a microdisplay with spectral band less than Xi, which is used for the VR display path, are coupled into the prism 2 through the surface S6, reflected by the surface S5, and subsequently transmitted through the S4 of the prism 2 and the surfaces S2 and SI of the prism 1 to project a virtual image of the display onto eye pupil. The spectral band between Xi and X2 transmits through prism 1 and prism 2 and is used for fundus illumination and fundus imaging.

[0066] FIG. 12 illustrates example light paths of these illumination and imaging functions. Similar to FIG. 10, the eye pupil illumination and accommodation illumination paths are illustrated in gray solid line with arrow pointing toward eye pupil, retina illumination is in gray long dash line with arrow pointing toward retina, the VR display path is in gray thick line with arrow pointing toward retina, the fundus imaging path is in in black long dash line with arrow pointing toward fundus camera, and the eye-tracking and accommodation imaging paths are in black solid line with arrow pointing toward NIR sensors. The ray paths dedicated to illumination and display are denoted by lines with arrows pointing toward the pupil or retina, while those paths for imaging functions are indicated by lines with arrows pointing toward their corresponding detectors.

[0067] FIG. 14 illustrates an alternative coating scheme to the previously proposed coating schematics in FIG. 13 that may be applied to the surfaces S2 and S5. In this scheme, the surface S2 is coated with a hot mirror coating, which reflects light with a wavelength greater than X2 andtransmits light with wavelengths shorter than X2. The surface S5 is coated with a beamsplitter coating, which partially reflects and transmits light with a wavelength less than Xi. The reflectance and transmittance of the two coatings with respect to wavelengths are plotted in panels (a) and (b) of FIG. 14, respectively. In a typical embodiment, X2 is greater than Xi. For instance, Xi can be chosen to be around 7800nm and X2 to be around 800 nm or longer. As a result, the virtual display path, the fundus illumination path, and the funds imaging path share the same spectral band (i.e. the band less than Xi). One noticeable advantage of this scheme is that a much broader band (including the visible band and a portion of the NIR band) can be utilized for fundus imaging, rather than a narrow band between Xi and X2.

[0068] As shown in FIG. 12, the optical paths are integrated together through these freeform prisms. The different paths may utilize different portions of the prism surfaces. Among the optical paths, the NIR eye illumination path and the NIR imaging path for eye-tracking or accommodation measurement are combined before interfacing with prism 1. Similarly, the retina illumination path and the retina imaging path are combined before interfacing with prism 2. The display can be a self-emitting microdisplay such as an organic light emitting diode (OLED) display or a display engine composed of a spatial light modulator and an illumination source.

[0069] The optical surfaces of prism 1 and prism 2 can be optimized to obtain the necessary optical power required for the corresponding optical paths. For instance, for the purpose of eyetracking, the surfaces SI, S2, and S3 will be optimized to fulfdl the requirements as a condenser lens for creating collimated and uniform illumination on the eye pupil and as an imaging optics for collecting and focusing the reflected light from the eye pupil to capture high-quality eye images. For fundus illumination and imaging, besides the surfaces SI, S2, S4, and S5 of the prisms, additional optics may be needed to combine the retina illumination and imaging functions, which will be disclosed in the description that follows.

[0070] FIG. 15 depicts the schematic layout of a third example embodiment for a wearable multi-functional ophthalmic device for monitoring oculomotor components. The optical system that integrates all or a subset of the seven distinct optical paths mainly comprises three freeform prisms, prism 1, prism 2, and prism 3. These three prisms embody the combined optical functions of the embodiment shown in FIG. 12 with an additional optical see-through function, allowing a user a direct-view of the physical world. The addition of the optical see-through path significantlydifferentiates this embodiment from the other two embodiments shown in FIG. 10 and FIG. 12, respectively. Prism 1, placed adjacent to the eye of a user, is composed of three optical surfaces, SI, S2, and S3. All or a subset of these three surfaces may be freeform surfaces. Prism 2, attached to the prism 1, is composed of three optical surfaces, S4, S5, and S6. All or a subset of these three surfaces may be freeform surfaces. Prism 3, attached to prism 2, is composed of three optical surfaces, S7, S8, and S9. All or a subset of these three surfaces may be freeform surfaces.

[0071] Among these surfaces forming prisms 1, 2, and 3, surface S2 may be applied with a hot mirror coating, and surfaces S5 and S8 may be applied with a beamsplitter coating. The hot mirror coating reflects light with a wavelength greater than Ki and transmits light with wavelengths shorter than X2. Surfaces S5 and S8 are coated with a beamsplitter coating, which partially reflects and transmits light with wavelengths less than Xi. The reflectance and transmittance of these coatings with respect to wavelengths are illustrated in FIG. 16, panels (a) and (b), respectively. In a typical embodiment, X2 is greater than Xi. For instance, Xi can be chosen to be around 7800nm and X2 be around 800 nm or longer. As a result, the virtual display path, the see-through path, the fundus illumination path, and the funds imaging path share the same spectral band (i.e. the band less than Xi).

[0072] The combination of the three prisms naturally combines the optical paths from four distinct directions. Similar to the embodiment illustrated in Fig. 12, the spectral band greater than X2 is coupled in and out of the prism 1 through surface S3 and reflected by the surface S2. This spectral band is used for the illumination path and imaging paths, which are illustrated in gray solid line with arrow pointing toward eye pupil and black solid line with arrow pointing toward NIR sensors, respectively, for eye-tracking imaging or accommodation imaging. The light rays from a micro-display with a spectral band less than Xi, which is for the VR display path illustrated in gray thick line with arrow pointing toward the retina, are coupled into prism 2 through the surface S6 and partially reflected by the surface S5, and subsequently transmitted through the S4 of the prism 2 and the surfaces S2 and SI of the prism 1 to project a virtual image of the display onto eye pupil. The light rays from the illumination sources used for fundus illumination with a spectral band less than Xi, which are in gray long dash line with arrow pointing towards the retina, are coupled into prism 3 through surface S9, partially reflected by the surface S8, are subsequently transmitted through the surfaces S7, S5, S4, S2, and SI of the three prisms, and are focused nearthe eye pupil to form trans-pupillary illumination for fundus imaging. The light rays reflected off the retina surface, which are in black long dash line with arrow pointing toward fundus camera, are transmitted through the eye lens, the surfaces SI, S2, S4, S5, and S7, reflected by the surface S8, coupled out of the prism 3 through the surface S9 and focused onto an imaging detector for fundus imaging. The light rays from a real-world scene with a spectral band less than Xi, which are illustrated by black dash-dot lines, subsequently transmit through the surfaces S8, S7, S5, S4, S2, and SI and reach the eye pupil, which enables an optical see-through path.

[0073] The optical paths are integrated together through these freeform prisms. The different paths may utilize different portions of the prism surfaces. Among the optical paths, the NIR eye illumination path and the NIR imaging path for eye-tracking or accommodation measurement are combined before interfacing with the prism 1. Similarly, the retina illumination path and the retina imaging path are combined before interfacing with the prism 3. The display can be a self-emitting microdisplay such as organic light emitting diode (OLED) display or display engine composed of a spatial light modulator and an illumination source.

[0074] The optical surfaces of the prisms 1, 2, 3 can be optimized to obtain the necessary optical power required for the corresponding optical paths. For instance, for the purpose of eyetracking, the surfaces SI, S2, and S3 will be optimized to fulfill the requirements as a condenser lens for creating collimated and uniform illumination on the eye pupil and as an imaging optics for collecting and focusing the reflected light from the eye pupil to capture high-quality eye images. For the purpose of fundus illumination and imaging, besides the surfaces SI, S2, S4, S5, S7, and S8 of the prisms, additional optics may be needed for creating trans-pupillary illumination on the eye pupil and for capturing high-quality retina images. For the purpose of VR display, the surfaces SI and S2 of Prism 1 and the surfaces S4, S5, and S6 of prism 2 are optimized to render high- resolution VR display. For the optical see-through path, the surfaces SI and S2 of prism 1, the surfaces of S4 and S5 of prism 2, and the surfaces S7 and S8 of prism 3 are jointly optimized to minimize the net optical power and optical distortion of these prisms in combination to prevent apparent distortion to a real-world scene.

[0075] FIG. 17 depicts the schematic layout of a fourth example embodiment for a wearable multi-functional ophthalmic device for monitoring oculomotor components. This example embodiment avoids the use of custom-designed X-cube with special coatings as in the schemeshown in FIG. 10 or the use of specially-designed freeform prisms as those shown in FIGS. 12 and 15. Similar to the other embodiments, the optical design of this fourth embodiment encompasses seven distinct optical paths: eye pupil illumination and accommodation illumination paths (illustrated in gray solid line with arrow pointing toward eye pupil), retina illumination (in gray long dash line with arrow pointing toward retina), VR display path (in gray thick line with arrow pointing toward retina), fundus imaging path (in black long dash line with arrow pointing toward fundus camera), and the eye-tracking and accommodation imaging paths (in black solid line with arrow pointing toward NIR sensors). The ray paths dedicated to illumination and display are denoted by lines with arrows pointing toward the pupil or retina, while those paths for imaging functions are indicated by lines with arrows pointing toward their corresponding detectors.

[0076] Instead of using a specially designed X-cube to combine three distinct spectral bands to fulfill the illumination and imaging functions, the embodiment of FIG. 17 utilizes a broadband beamsplitter that partially reflects and transmits both NIR and visible light. The optical paths are grouped by their distinctive imaging functions and distributed through the beamsplitter in three orthogonal directions: the NIR illumination and imaging paths for eye-tracking and eye accommodation measurements, the VR display path using visible spectrum, and the fundus illumination and imaging paths for retina image. For instance, the NIR eye illumination unit and NIR sensor unit are considered as one imaging function and they share one of the three light paths of the beamsplitter. Similarly, the retina illumination unit and fundus imaging unit share a second path of the three light paths of the beamsplitter. This grouping scheme makes it possible to combine these distinctive optical paths through a single beamsplitter without adding more complex combining mechanisms or increasing the complexity of the optical architecture.

[0077] In FIG. 17, the NIR eye illumination / imaging unit and the display unit are located in opposite directions in a co-linear fashion while the retina illumination / imaging units are located on a third side of the beamsplitter, facing the eyepiece. To combine these three directions through the shared broadband beamsplitter, a hot mirror is inserted between the beamsplitter and the display unit. As a result, the light rays from the NIR illumination unit (illustrated in gray solid lines) first transmit through the beamsplitter toward the hot mirror, are then reflected by the hot mirror back toward the beamsplitter, and finally reflected by the beamsplitter toward the eyepiece for pupil illumination. Similarly, the light rays scatter off the eye pupil (in black solid lines) aretransmitted through the eyepiece toward the beamsplitter, are reflected by the beamsplitter toward the hot mirror, are reflected by the hot mirror back toward the beamsplitter, and are finally transmitted through the beamsplitter toward the NIR sensor. While the fundus imaging camera is co-axal with the eyepiece optics, the fundus illumination unit is located off-axis to minimize the illumination light leakage into the camera sensor. Furthermore, to avoid crosstalk between the NIR illumination / imaging units and the fundus illumination / imaging units, a pair of orthogonal linear polarizers are placed in the NIR illumination and NIR imaging units, respectively, and a linear polarizer with the same polarization state as the one for NIR imaging paths is placed in the retina imaging path.

[0078] The optical paths in FIG. 17 are integrated together through the broadband beamsplitter and share the same eyepiece. The different paths may utilize different portions of the eyepiece. Among the optical paths, the NIR eye illumination path and the NIR imaging path for eye-tracking or accommodation measurement are combined before interfacing with the beamsplitter. Similarly, the retina illumination path and the retina imaging path are combined before interfacing with the beamsplitter. The display can be a self-emitting microdisplay such as an organic light emitting diode (OLED) display or a display engine composed of a spatial light modulator and an illumination source.

[0079] The optical schemes disclosed above, including the first, second, third, and fourth example embodiments, presented optical architectures for path multiplexing that enable the integration of as many as 8 illumination and imaging functions to develop a wearable multifunctional ophthalmic device that can capture high-quality fundus images, continuously monitor ocular responses such as eye movements, accommodation response, and pupil size, and render sophisticated, dynamic visual stimuli on demand through a VR display.

[0080] In embodiments described above, the optical paths combined by either an X-cube, freeform prism, or a broadband beamsplitter as illustrated by FIGS. 10, 12, 15, and 17. Among the optical paths, the NIR eye illumination path and the NIR imaging path for eye-tracking or accommodation measurement are combined before interfacing with combining optics such as an X-cube. Similarly, the retina illumination path and the retina imaging path are combined before interfacing with the combining optics. The display can be a self-emitting microdisplay such as an organic light emitting diode (OLED) display or a display engine composed of a spatial lightmodulator and an illumination source.

[0081] In some implementations of the disclosed embodiments, various methods for combining the NIR illumination and imaging paths for eye tracking or accommodation measurements are possible. For instance, the methods illustrated in FIG. 7 can be adopted for eye pupil illumination and eye-tracking imaging, while the methods illustrated in FIG. 5 can be adopted for eye accommodation measurements.

[0082] Described herein is an example embodiment of a compact fundus camera system for combining the illumination and imaging paths for fundus imaging as disclosed in this patent document. Besides the innovative design of the combining optics for path multiplexing, the present embodiment also discloses optical methods for combining the retina illumination and imaging paths. In the description that follows, several exemplary methods for combining these two optical paths are provided.

[0083] It is worth mentioning that the methods described below utilize the X-Cube combiner as an example and these methods can be equally applicable to the path-combining architectures illustrated in FIGS. 12, 15, or 17 or other path-combiner optics with similar path multiplexing capabilities.

[0084] FIG. 18 illustrates four different example methods and configurations for combining the retina illumination and imaging paths for fundus imaging. In some embodiments, these configurations can be implemented in the configurations of FIGS. 10, 12, 15 or 17. Panel (a) in FIG. 18 illustrates a method and configuration that adopts a scheme similar to the prior art shown in FIGS. 2 and 3 where retina illumination path (in gray long-dash lines) and the retina imaging path (in black long-dash lines) are combined through a beamplitter. An annular source is collimated by a condenser lens and the collimated light rays are reflected by a beamsplitter toward the X-cube and continue through the eyepiece to create retinal illumination. The annular source is optically conjugated to the eye pupil to provide pulsed trans-pupillary illumination on the retina as illustrated in panel (a2) of FIG. 4. In the imaging path, the reflected light by the retina passes through the eye lens, passes through the eyepiece, the X-cube, and the beamsplitter, and finally is imaged through a relay lens onto an imaging detector. A pair of orthogonally oriented linear polarizers may be inserted in the illumination and imaging paths, respectively, to improve the contrast of retinal image acquisition.

[0085] Panel (a) of FIG. 18 illustrates one example approach that uses a beamsplitter for combining the two optical paths and the separate usage of condenser optics and relay optics. The schemes shown in panels (b) through (d) offer more compact and lightweight approaches for combining the retinal illumination and imaging paths.

[0086] Notably, panel (b) in FIG. 18 illustrates an example spatially-multiplexing method and configuration for combining the retinal illumination and imaging paths where the same optics serve the dual function of a condenser lens for illumination and a relay lens for imaging. In the illumination path, the light rays (illustrated in gray long-dash lines) from an annular source are collimated by the outer peripheral of the relay / condenser lens and are directed toward the X-cube and eyepiece to provide pulsed trans-pupillary illumination. In the imaging path, the reflected light rays by the retina (illustrated in black long-dash lines) are transmitted through the eye lens and the X-cube is imaged through the inner central region of the relay / condenser lens onto an imaging detector. The annular source is placed behind the imaging detector and the relay / condenser lens is placed in front of the imaging detector. A pair of orthogonally oriented linear polarizers may be inserted in the illumination and imaging paths, respectively, to improve the contrast of retinal image acquisition.

[0087] Panel (c) in FIG. 18 illustrates another example method and configuration which is modified from the scheme shown in panel (b). In this embodiment, the condenser lens is placed between the imaging detector and the annular source and the outer peripheral of the condenser lens is solely used for the purpose of collimating the light source. In the imaging path, the reflected light rays by the retina (illustrated in black long-dash lines) transmitted through the eye lens and X-cube are directly imaged onto an imaging detector without the necessity of relay optics.

[0088] Panel (d) in FIG. 18 illustrates another example that is much more compact than the other methods and configurations of FIG. 18. The large condenser / relay optics used in the embodiments shown in panels (a) through (c) is replaced by a condenser lenslet array (see panel (d) and its inset) where the center element is the imaging aperture for retinal imaging and the elements around the central aperture are lenslets as illustrated by the 2D view illustrated by the inset image on the top right. An LED array, as illustrated by the inset of panel (d), includes by an array of LEDs and is used as the light source for illuminating the retina. Each element of the LED array corresponds to a lenslet in the condenser lenslet array. To ensure trans-pupillary illumination,each LED element is displaced from the optical axis of its corresponding condenser lenslet element as illustrated in panl (d) of FIG. 18. The light rays (illustrated in gray long-dash lines) emitted by the LED sources are collimated by their corresponding condenser lenslet and are directed toward the X-cube and eyepiece to provide pulsed trans-pupillary illumination. In the imaging path, the reflected light rays by the retina (illustrated in black long-dash lines) are transmitted through the eye lens and X-cube and are imaged through the central imaging aperture onto an imaging detector. An array of linear polarizers may be inserted in the illumination path and another orthogonally oriented linear polarizer may be inserted in front of the imaging detector to improve the contrast of retinal image acquisition.

[0089] The anatomical structure of the human eye presents inherent challenges in delivering adequate illumination to the fundus for imaging purposes. To overcome this challenge of delivering uniform illumination across the entire FOV, as mentioned earlier, each LED element is displaced from the optical axis of its corresponding condenser lenslet element as illustrated in panel (d) of FIG. 18. Furthermore, each LED is optically conjugated at a designated distance in front of the pupil, a position we refer to as the Center Coverage Spot (CCS). Direct LED placement in front of the pupil is challenging since it would interfere with the eye’s optical path. The challenge is solved through this design which effectively creates 'virtual LEDs' positioned ideally in front of the pupil at CCS for full coverage illumination.

[0090] FIG. 19 illustrates illumination effect of different LED conjugate planes — at the pupil (panel (a)); in front of the pupil (panel (b); and behind the pupil (panel (c)) — when each LED in the array is placed on the optical axis of its corresponding condenser lenslet element. FIG. 20 illustrates illumination effect of LED array when each LED in the array is placed on the optical axis of its corresponding condenser lenslet element and is conjugated at the center coverage spot. As depicted in FIG. 20, the innovative configuration described above is achieved when the bottom marginal ray from the upper LED assembly (illustrated in a gray dashed line) and the top marginal ray from the lower LED assembly (illustrated in a gray solid line) intersect at the eyepiece's focal point. This ensures that the marginal rays from both LED groups, crossing at the eyepiece's focal point, will refract at the eyepiece and approach the pupil at a zero-degree angle of incidence. Consequently, these rays will converge precisely at the center of the retina, thereby ensuring comprehensive coverage of the FOV without leaving any areas unilluminated. Furthermore, wecan also generate a larger area of illumination by making an array of those LED groups surrounding the imaging detector on the vertical and lateral planes.

[0091] To validate the feasibility and innovative approach the embodiments disclosed in this patent document, a prototype was constructed utilizing commercially available optical components and a mechanical housing fabricated with a high-precision 3D printer. FIG. 21 shows the optical schematics of the prototype design, which is based on the combination of the method shown in FIG. 17 for optical path multiplexing and the method shown in panel (d) of FIG. 18 for fundus illumination and imaging. The NIR eye illumination path and the NIR imaging path for eyetracking are combined through a polarizing beamsplitter (PBS).

[0092] The specifications and characteristics of this design are shown in Table 1. A principal design challenge was to achieve optimal optical performance utilizing only stock optical components while ensuring a sufficient working distance within the fundus imaging path to accommodate the retina illumination subsystem. This was accomplished with a working distance in the fundus path of 4.6 mm, seamlessly integrating with our specifically chosen retina illumination components. The optical design yields a 22-degree field of view for the fundus path when used with a 2 / 3-inch CMOS sensor, and a 36-degree field of view for the display path when used with a 0.7-inch OLED micro-display. The effective focal length for both paths is approximately 29 mm. System optimization was conducted using CodeV software, targeting a maximum entrance pupil diameter of 8 mm, which corresponds to the maximum average human pupil dilation to ensure design versatility. Nonetheless, performance evaluations will primarily focus on a 4 mm entrance pupil diameter, which represents typical human pupil size under indoor lighting conditions. The chosen wavelengths for the fundus path were 780 nm, 625 nm, and 530 nm, while the display path was optimized for 625 nm, 565 nm, and 475 nm. It is important to note that the eye-tracking camera is specifically designed for a monochromatic wavelength of 780 nm due to the fact that NIR light has the advantage of illuminating the eye area without inducing pupil shrinkage, which hinders fundus imaging greatly. To constrain the device's overall dimensions, a maximum lens diameter of under 30 mm was set.

[0093] Table 1. First-order lens design specifications for fundus and display path

[0094] The prototype directed to the embodiment of FIG. 21 leverages a configuration comprised entirely of optical components readily available from industry suppliers, specifically Edmund Optics and Thorlabs. As depicted in panel (a) of FIG. 22, the fundus imaging path is designed with an entrance pupil diameter of 8 mm, strategically positioned to align precisely with the user’s eye position. This alignment effectively equates the system's entrance pupil with that of the human eye. The design process took into consideration the impacts of the linear polarizer and the CMOS cover glass on optical performance. The MTF of this setup, represented in FIG. 23 A, was assessed under 4 mm entrance pupil diameter, reflecting common human pupil size under indoor lighting conditions.

[0095] Using a 5-megapixel 2 / 3” IMX264 CMOS sensor from Sony, the system achieves a Nyquist frequency of 145 Ip / mm. At a 4 mm pupil size, the fundus imaging path successfully maintains a minimum of 10% MTF up to 75 Ip / mm. However, to accommodate the physical presence of the retina illumination subsystem, a minimum working distance of 4.5 mm was necessary.

[0096] Panel (b) of FIG. 22 shows the display path with an 8 mm entrance pupil diameter. Due to the shared eyepiece between the display and fundus imaging paths, optimization efforts for enhancing the display image quality were confined to the last two lenses in the configuration. Additionally, a hot mirror is integrated into the design, positioned right before the display. This inclusion aims to reflect the NIR lights from the eye-tracking camera and eye pupil illumination subsystems.

[0097] In the MTF plots presented in FIG. 23B, the display path demonstrates the capability to sustain an MTF above 20% across nearly the entire range of Nyquist frequency at almost all field angles. The only exceptions are at the extremes of the two edge angles, specifically at the half field of view (HFOV) of 16.7 and 18 degrees radially, where the MTF slightly falls below thisthreshold.

[0098] Table 2. Lens prescription for fundus path in Fig. 22 (a)

[0099] Table 3. Lens prescription for display path in Fig. 22(b)

[0100] The retina illumination subsystem, depicted in panel (c) of FIG. 22, incorporates identical optical elements as those employed in the fundus imaging subsystem, with the addition of two unique components to complete its functionality. These additions include a 3mm rightangle mirror, sourced from Edmund Optics (Part No. 25698), and a 3mm aspherical collimation lens from LightPath Technologies (Part No. LP355660). A single configuration of this design is capable of illuminating one hemisphere within the retina. Consequently, to achieve comprehensive illumination across the entire field of view, two sets are required as shown in FIG. 19.

[0101] The mechanical housing of the multifunctional fundus imaging device, designed for precision assembly, is depicted in FIG. 24. Panel (a) provides an external view of the assembled system, showcasing its comprehensive design. Panel (b) presents a top-down cross-sectional view, illustrating the integration of all optical components within the system. Panel (c) offers an in-depth examination of the retina illumination subsystem, highlighting its specific design considerations.

[0102] A key feature of the mechanical housing is its pressed-fit assembly mechanism, which incorporates inner sleeves equipped with built-in spacers that can be inserted into outer sleeves for extra stability. The built-in spacers are engineered to enhance alignment accuracy by mitigating potential tilt within the tubes during assembly and avoiding lens drop during disassembly. This pressed-fit approach ensures that lenses are securely pressed together, eliminating unnecessary airspace between them and optimizing optical performance. Moreover, we designed all the spacers to take into account the specific curvature of each corresponding lens, thereby increasing the contact area. This thoughtful consideration not only improves the structural integrity of the assembly but also substantially reduces the likelihood of lens tilt or decentering, further optimizingthe assembled device's optical performance.

[0103] FIG. 25 exemplifies the color fidelity of the display path in the developed system. A standard colored bar target, as depicted in panel (b), was displayed on the 0.7-inch OLED microdisplay. The image captured, shown in panel (a), reveals no noticeable color discrepancies, thereby demonstrating the exceptional color accuracy of our VR display system. It is important to note the presence of slight distortion in the captured image; however, this is considered a minor issue that can be readily addressed through display calibration processes.

[0104] FIG. 26 demonstrates the efficacy of our fundus imaging subsystem in capturing detailed images of the fundus. For this evaluation, a paper target was positioned within an anatomically accurate eye model supplied by Ocular Instruments (Model OEMI-7). To mimic the optical properties of the human eye's vitreous humor, the model's cavity was filled with distilled water. Alignment was done by positioning the eye model such that its pupil matched with the entrance pupil of our imaging system, as illustrated in panel (a) of FIG. 26. The image result is depicted in panel (b). The system exhibits notably high optical performance, especially considering its exclusive use of commercially available, off-the-shelf lenses.

[0105] FIG. 27 demonstrates the eye-tracking subsystem of our device, showcasing its ability to both illuminate and monitor the eye area effectively. Using 780 nm NIR LED illumination, we initially put the eye-tracking subsystem to the test with an eye model supplied by Ocular Instruments (Model OEMI-7). Panels (al) to (cl) capture the eye-tracking images from this test, as the eye model points in different directions, using a micro-camera with focusing adjustability supplied by Estink (OV5640 Sensor). The striped reflections around the eye model are caused by the eye model's front covering plastic. We then tested the eye-tracking camera on a human subject, as depicted in panels (a2) to (c2), and the camera adeptly captured the subject’s eye movements as the subject shifted his gaze in different directions.

[0106] FIG. 28 illustrates a block diagram of a device 2800 that can be used to implement certain aspects of the disclosed technology. For example, the device of FIG. 28 can be used to receive, process, store, provide for display and / or transmit various data and signals associated with disclosed image sensors. The device 2800 comprises at least one processor 2804 and / or controller, at least one memory 2802 unit that is in communication with the processor 2804, and at least one communication unit 2806 that enables the exchange of data and information, directly or indirectly,through the communication link 2808 with other entities, devices, databases and networks. The communication unit 2806 may provide wired and / or wireless communication capabilities in accordance with one or more communication protocols, and therefore it may comprise the proper transmitter / receiver, antennas, circuitry and ports, as well as the encoding / decoding capabilities that may be necessary for proper transmission and / or reception of data and other information. The exemplary device 2800 of FIG. 28 may be integrated as part of larger component (e.g., a server, a computer, tablet, smart phone, etc.) that can be used for performing various computations, methods or algorithms disclosed herein.

[0107] The processor(s) 2804 may include central processing units (CPUs) to control the overall operation of, for example, the host computer. In certain embodiments, the processor(s) 2804 accomplish this by executing software or firmware stored in memory 2802. The processor(s) 2804 may be, or may include, one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), graphics processing units (GPUs), or the like, or a combination of such devices.

[0108] The memory 2802 can be or can include the main memory of a computer system. The memory 2802 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, or the like, or a combination of such devices. In use, the memory 2802 may contain, among other things, a set of machine instructions which, when executed by processor 2804, causes the processor 2804 to perform operations to implement certain aspects of the presently disclosed technology.

[0109] It is understood that the term “beam splitter” as found in this patent document is a nonlimiting term corresponding to any optical device configured to distribute light incident thereupon into multiple beams of light. The beam splitter, when operated in reverse, can combine multiple incident light beams of into fewer (e.g., a single beam) that exits the beam splitter. Examples of such optical devices may include an optical prism or a configuration of optical surfaces.

[0110] One aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components. The optical system includes an eyepiece lens, a display configured to present visual stimuli to an eye, a first illumination source configured to produce light for illuminating the eye, a second illuminationsource configured to produce light for illuminating the eye, and a beam splitter configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path. The optical system also includes a first imaging detector positioned along the eye tracking imaging path and configured to produce images based on light received thereon, and a second imaging detector positioned along the fundus imaging path and configured to produce images based on light received.

[0111] In one example embodiment, the beam splitter comprises a prism and includes: a first reflective layer on a first surface of the prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, and a second reflective layer on a second surface of the prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, wherein spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other.

[0112] In another example embodiment, the pupil illumination path is a path from the first illumination source, to the beam splitter, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the beam splitter, and to the eyepiece lens, the stimuli illumination path is a path from the display, to the beam splitter, and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, and to the first imaging detector, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, and to the second imaging detector. In yet another example embodiment, the first reflective layer forms a hot mirror and the second reflective layer forms a cold mirror. In still another example embodiment, the first illumination source comprises one or more near infrared (NIR) light sources and the second imaging detector is configured to detect NIR light received from the eye in response to illumination of the eye with NIR light. For instance, the images obtained at the second imaging detector enables one or both of an eye tracking assessment or an eye accommodation assessment.

[0113] Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, where theoptical system includes a display configured to present visual stimuli to an eye, a first freeform prism, a first illumination source configured to produce light for illuminating the eye along a pupil illumination path, a second freeform prism coupled to the first freeform prism, a first imaging sensor configured to produce images based on light received thereon from an eye tracking imaging path, a second illumination source configured to produce light for illuminating the eye that propagates along a retina illumination path, and a second imaging sensor configured to produce images based on light received thereon from a retina imaging path. In this optical system, the first or the second freeform prisms are configured to receive light associated with three illumination paths: (a) the pupil illumination path corresponding to the first illumination source, (b) the retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the first or the second freeform prisms also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

[0114] In one example embodiment, the pupil illumination path is a path from the first illumination source, to the first freeform prism that directs at least a portion of the light corresponding to the first illumination source in a direction of the eye, the retina illumination path is a path from the second illumination source, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the first illumination source in a direction of the eye, the stimuli illumination path is a path from the display, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the display in a direction of the eye, the eye tracking imaging path is a path of light associated with the first illumination source that is received at the first freeform prism after interaction with the eye, and to the first imaging sensor, and the fundus imaging path is a path of light associated with the second illumination source and received at the first freeform prism after interaction with the eye, to the second freeform prism, and to the second imaging sensor.

[0115] According to another example embodiment, the first freeform prism and the second freeform prism form a unit that includes: a first reflective layer on a first surface interposed in between, and is in contact with, the first and the second freeform prisms, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, and a second reflective layer on a second surface of the second freeformprism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, wherein spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other.

[0116] In another example embodiment, the first freeform prism is configured to receive light associated with the first illumination source at multiple locations along the first surface. In yet another example embodiment, the first reflective layer forms a hot mirror and the second reflective layer forms a cold mirror. In still another example embodiment, the first reflective layer forms a hot mirror and the second reflective layer includes a beam splitter coating. In one example embodiment, the optical system is configured to combine the pupil illumination path and the eye tracking imaging path before interfacing with the first freeform prism. In another example embodiment, the retina illumination path and the fundus imaging path have at least one common optical path section. In still another example embodiment, the stimuli illumination path is used to project a virtual image of the display onto the eye.

[0117] Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components that includes a display configured to present visual stimuli to an eye, a first freeform prism, a first illumination source configured to produce light for illuminating the eye along a pupil illumination path, a second freeform prism coupled to the first freeform prism, a first imaging sensor configured to produce images based on light received thereon from an eye tracking imaging path, a second illumination source configured to produce light for illuminating the eye along a retina illumination path, second imaging sensor configured to produce images based on light received thereon from a retina imaging path, and a third freeform prism coupled to the second freeform prism, the third freeform prism configured to receive light associated with a real scene and to direct light from the real scene towards the eye along an optical see-through path. In this optical system, the first freeform prism, the second freeform prism, and the third freeform prism form a configuration capable of receiving light associated with four illumination paths: (a) the pupil illumination path corresponding to the first illumination source, (b) the retina illumination path corresponding to the second illumination source, (c) a stimuli illumination path corresponding to the display, and (d) the optical see-through path corresponding to the real scene, the configuration also capable to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundusimaging path.

[0118] In one example embodiment, the pupil illumination path is a path from the first illumination source, and to first freeform prism that provides at least a portion of the light corresponding to the first illumination source in a direction of the eye, the retina illumination path is a path from the second illumination source, to the third freeform prism, to the second freeform prism, and to the first freeform prism that provides at least a portion of the light corresponding to the second illumination source in a direction of the eye, the stimuli illumination path is a path from the display, to the second freeform prism, and to the first freeform prism that provides at least a portion of the light corresponding to the display in a direction of the eye, the optical see-through path is a path from the real scene, to the third freeform prism, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the real scene in a direction of the eye, the eye tracking imaging path is a path of light associated with the first illumination source and received at the first freeform prism after interaction with the eye, to the first freeform prism, and to the first imaging sensor, and the fundus imaging path is a path of light associated with the second illumination source and received at the first freeform prism after interaction with the eye, to the second freeform prism, to the third freeform prism, and to the second imaging sensor.

[0119] According to another example embodiment, the first freeform prism, the second freeform prism, and the third freeform prism form a configuration that includes: a first reflective layer on a first surface of the freeform prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, a second reflective layer on a second surface of the second freeform prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, and a third reflective layer on a third surface of the third freeform prism, the third reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, wherein the first reflective layer is interposed in between, and is in contact with, the first and the second freeform prisms, the second reflective layer is interposed in between, and is in contact with, the second and the third freeform prisms, and spectral reflectance or transmittance properties of the first, second, and third reflective layers differ from each other.

[0120] In another example embodiment, the first freeform prism is configured to receive light associated with the first illumination source at multiple locations along the first surface. In still another example embodiment, the first reflective layer forms a hot mirror, wherein the second reflective layer and the third reflective layer each include a beam splitter coating. In yet another example embodiment, the optical system is configured to combine the pupil illumination path and the eye tracking imaging path before interfacing with the first freeform prism. In one example embodiment, the retina illumination path and the fundus imaging path have at least one common optical path section.

[0121] Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components that includes an eyepiece lens, a display configured to present visual stimuli to an eye, a beam splitter, a first polarizer, a first illumination source configured to produce light that passes through first polarizer, a reflective optical element interposed between the first illumination source and the display, wherein the reflective optical element is configured to allow light from the display that reaches the reflective optical element from a first direction to be transmitted therethrough and light that is received on the reflective optical element from a second direction opposite to the first direction to be reflected therefrom. The optical system further includes a second polarizer, a first image sensor configured to receive polarized light from the second polarizer, a second illumination source configured to provide off-axis illumination of the eye, a third polarizer, and a second image sensor configured to receive light that passes through the third polarizer, wherein the beam splitter is configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

[0122] In one example embodiment, the first illumination source is configured to produce near-infrared light and the first image sensor is configured to sense near-infrared light. In another example embodiment, the pupil illumination path is a path from the first illumination source, to the beam splitter, to the reflective optical element, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the beam splitter, and to the eyepiece lens,the stimuli illumination path is a path from the display, to the beam splitter, and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, to the reflective optical element, and to the first image sensor, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, and to the second image sensor.

[0123] According to one example embodiment, the reflective optical element comprises a reflective surface coating configured to reflect or transmit light incident thereupon based on a wavelength of the light incident thereupon. In another example embodiment, the third polarizer and the second polarizer are configured to produce, at respective outputs thereof, polarized light with an identical polarization state. In yet another example embodiment, the first polarizer is positioned along the pupil illumination path, the second polarizer is positioned along the eye tracking imaging path, and the third polarizer is positioned along the fundus imaging path. In still another example embodiment, the first illumination source is configured to produce illumination for an external area surrounding the eye that is imaged using the first image sensor, and the second illumination source is configured to produce illumination for one or more internal areas of the eye including at least a portion of a retina, that is imaged by the second image sensor.

[0124] Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components that includes an eyepiece lens, an illumination source configured to produce light for illuminating an eye, a condenser lens positioned in front of the illumination source, an imaging detector configured to produce images based on light received thereon, and a beam splitter positioned to receive light from the condenser lens and to direct at least a portion of the light received thereon in a direction of the eye, the beam splitter further configured to receive and to direct light received from the eyepiece lens in a direction of the imaging detector. In this optical system, the condenser lens, the imaging detector, and the illumination source form a configuration that allows light propagating along an illumination path for illuminating the eye to be combined with light propagating along an imaging path.

[0125] In one example embodiment, the optical system further includes a pair of orthogonal polarization elements configured to produce linearly polarized light at outputs thereof, respectively, wherein a first of the pair of orthogonal polarization elements is positioned along the illumination path and a second of the orthogonal polarization elements is positioned along theimaging path. In another example embodiment, the system also includes a relay lens positioned between the beam splitter and the imaging detector. In yet another example embodiment, the beam splitter comprises a prism and includes: a first reflective layer on a first surface of the prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, a second reflective layer on a second surface of the prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, and spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other. In still another example embodiment, the first reflective layer forms a cold mirror and the second reflective layer form a hot mirror. In one example embodiment, the beam splitter is an x-cube prism or freeform prism.

[0126] According to another example embodiment, the illumination and imaging paths are coaxial along an optical axis, the illumination source is an annular source configured to illuminate a peripheral zone of the condenser lens, the illumination path comprises a path from the illumination source, to the condenser lens to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, to a central zone of the condenser lens and to the imaging detector. In one example embodiment, the illumination and imaging paths are coaxial along an optical axis, the illumination source is an annular source configured to illuminate a peripheral zone of the condenser lens, the illumination path comprises a path from the illumination source, to the condenser lens to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, and to the imaging detector that is positioned between the beam splitter and the condenser lens.

[0127] In still another example embodiment, the illumination and imaging paths are coaxial along an optical axis, the illumination source comprises an array of light sources, the condenser lens comprises an array of condenser lenslets positioned around a periphery of the optical axis thus having an open central zone that allows light received from the beam splitter to traverse through the open central zone before reaching the imaging detector, the illumination path comprises a path from the array of the light sources, to the condenser lenslets, to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, through the open central zone, and to the imaging detector. In yet another example embodiment, theillumination source is configured to produce light for illuminating at least a section of a retina of the eye, and the imaging detector is configured to produce images of one or more sections of the retina.[00128J Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components that includes an eyepiece lens, a display configured to present visual stimuli to an eye, a first condenser lens;

[0129] a first illumination source configured to produce light that passes through the first condenser lens, an imaging lens, a first image sensor configured to receive polarized light from the imaging lens, and a reflective optical element interposed between the first image sensor and the display, wherein the reflective optical element is configured to allow light from the display that reaches the reflective optical element from a first direction to be transmitted therethrough and light that is received on the reflective optical element from a second direction opposite to the first direction to be reflected therefrom. The optical system further includes a beam splitter, a polarizing beam splitter (PBS) configured to receive light associated with the first illumination source and to direct light having a first polarization state in a direction of the beam splitter, the PBS further configured to receive light from the beam splitter and to direct light having a second polarization state orthogonal to the first polarization state in a direction of the first image sensor, a second illumination source comprising an array of light sources, a second image sensor, an array of condenser lenslets positioned around a periphery an open central zone that allows light received from the beam splitter to traverse through the open central zone before reaching the second image sensor, and a polarizer positioned between the beam splitter and the array of condenser lenslets. In this optical system the beam splitter is configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

[0130] In one example embodiment, the pupil illumination path is a path from the first illumination source, to the first condenser lens, to the PBS, to the beam splitter, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the array ofcondenser lenslets, to the polarizer, to the beam splitter and to the eyepiece lens, the stimuli illumination path is a path from the display, to the reflective optical element, to the beam splitter and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, to the PBS, to the imaging lens and to the first image sensor, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, to the polarizer, through the open central zone, and to the second image sensor. In another example embodiment, each light source in the array of light sources is associated with a respective lenslet in the array of condenser lenslets configured to provide off-axis illumination. In still another example embodiment, the array of light sources is configured to emit pulsed light. In yet another example embodiment, the first illumination source is configured to produce near-infrared light and the first image sensor is configured to sense near-infrared light. In one example embodiment, the reflective optical element comprises a reflective surface coating configured to reflect or transmit light incident thereupon based on a wavelength of the light incident thereupon. In another example embodiment, the beam splitter is an x-cube prism or freeform prism.

[0131] Another aspect of the disclosed embodiments relates to an optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components that includes a first illumination source configured to produce light for illuminating an eye, a first imaging detector configured to produce images based on light received thereon, the images tracking one or more ocular responses of the eye to light produced by the first illumination source, a second illumination source configured to produce light for illuminating a retina of the eye, a second imaging detector comprising a camera configured to produce images based on light received thereon that has interacted with the eye, a third illumination source configured to produce light for stimulating the retina of the eye such that an accommodation response by the retina of the eye to light stimulation is provoked, a measurement device configured to record the accommodation response, a display configured to present visual stimuli to the eye during operation of the wearable multi-functional ophthalmic device, and a beam splitter configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundusimaging path, wherein the beam splitter is further configured to combine illumination paths with imaging paths.

[0132] While this patent document contains many specifics, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features that are described in this patent document in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a subcombination or variation of a subcombination.

[0133] Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Moreover, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0134] It is understood that the various disclosed embodiments may be implemented individually, or collectively, in devices comprised of various optical components, electronics hardware and / or software modules and components. These devices, for example, may comprise a processor, a memory unit, an interface that are communicatively connected to each other, and may range from desktop and / or laptop computers, to mobile devices and the like. The processor and / or controller can perform various disclosed operations based on execution of program code that is stored on a storage medium. The processor and / or controller can, for example, be in communication with at least one memory and with at least one communication unit that enables the exchange of data and information, directly or indirectly, through the communication link with other entities, devices and networks. The communication unit may provide wired and / or wireless communication capabilities in accordance with one or more communication protocols, and therefore it may comprise the proper transmitter / receiver antennas, circuitry and ports, as well as the encoding / decoding capabilities that may be necessary for proper transmission and / or receptionof data and other information. For example, the processor may be configured to receive electrical signals or information from the disclosed sensors (e.g., CMOS sensors), and to process the received information to produce images or other information of interest.[00135J Various information and data processing operations described herein may be implemented in one embodiment by a computer program product, embodied in a computer- readable medium, including computer-executable instructions, such as program code, executed by computers in networked environments. A computer-readable medium may include removable and non-removable storage devices including, but not limited to, Read Only Memory (ROM), Random Access Memory (RAM), compact discs (CDs), digital versatile discs (DVD), etc. Therefore, the computer-readable media that is described in the present application comprises non-transitory storage media. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes

[0136] Only a few implementations and examples are described and other implementations, enhancements and variations can be made based on what is described and illustrated in this patent document.

Claims

CLAIMSWHAT IS CLAIMED IS:

1. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising: an eyepiece lens; a display configured to present visual stimuli to an eye; a first illumination source configured to produce light for illuminating the eye; a second illumination source configured to produce light for illuminating the eye; a beam splitter configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path; a first imaging detector positioned along the eye tracking imaging path and configured to produce images based on light received thereon; and a second imaging detector positioned along the fundus imaging path and configured to produce images based on light received.

2. The optical system of claim 1, wherein the beam splitter comprises a prism and includes: a first reflective layer on a first surface of the prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, and a second reflective layer on a second surface of the prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, wherein spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other.

3. The optical system of claim 2, whereinthe pupil illumination path is a path from the first illumination source, to the beam splitter, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the beam splitter, and to the eyepiece lens, the stimuli illumination path is a path from the display, to the beam splitter, and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, and to the first imaging detector, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, and to the second imaging detector.

4. The optical system of claim 2, wherein the first reflective layer forms a hot mirror and the second reflective layer forms a cold mirror.

5. The optical system of claim 2, wherein the first illumination source comprises one or more near infrared (NIR) light sources and the second imaging detector is configured to detect NIR light received from the eye in response to illumination of the eye with NIR light.

6. The optical system of claim 5, wherein the images obtained at the second imaging detector enables one or both of an eye tracking assessment or an eye accommodation assessment.

7. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising: a display configured to present visual stimuli to an eye; a first freeform prism; a first illumination source configured to produce light for illuminating the eye along a pupil illumination path; a second freeform prism coupled to the first freeform prism; a first imaging sensor configured to produce images based on light received thereon from an eye tracking imaging path;a second illumination source configured to produce light for illuminating the eye that propagates along a retina illumination path; and a second imaging sensor configured to produce images based on light received thereon from a retina imaging path, wherein the first or the second freeform prisms are configured to receive light associated with three illumination paths: (a) the pupil illumination path corresponding to the first illumination source, (b) the retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the first or the second freeform prisms also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

8. The optical system of claim 7, wherein the pupil illumination path is a path from the first illumination source, to the first freeform prism that directs at least a portion of the light corresponding to the first illumination source in a direction of the eye, the retina illumination path is a path from the second illumination source, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the first illumination source in a direction of the eye, the stimuli illumination path is a path from the display, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the display in a direction of the eye, the eye tracking imaging path is a path of light associated with the first illumination source that is received at the first freeform prism after interaction with the eye, and to the first imaging sensor, and the fundus imaging path is a path of light associated with the second illumination source and received at the first freeform prism after interaction with the eye, to the second freeform prism, and to the second imaging sensor.

9. The optical system of claim 7, wherein the first freeform prism and the second freeform prism form a unit that includes:a first reflective layer on a first surface interposed in between, and is in contact with, the first and the second freeform prisms, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, and a second reflective layer on a second surface of the second freeform prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon; wherein spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other.

10. The optical system of claim 9, wherein the first freeform prism is configured to receive light associated with the first illumination source at multiple locations along the first surface.

11. The optical system of claim 9, wherein the first reflective layer forms a hot mirror and the second reflective layer forms a cold mirror.

12. The optical system of claim 9, wherein the first reflective layer forms a hot mirror and the second reflective layer includes a beam splitter coating.

13. The optical system of claim 7, configured to combine the pupil illumination path and the eye tracking imaging path before interfacing with the first freeform prism.

14. The optical system of claim 7, the retina illumination path and the fundus imaging path have at least one common optical path section.

15. The optical system of claim 7, wherein the stimuli illumination path is used to project a virtual image of the display onto the eye.

16. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising:a display configured to present visual stimuli to an eye; a first freeform prism; a first illumination source configured to produce light for illuminating the eye along a pupil illumination path; a second freeform prism coupled to the first freeform prism; a first imaging sensor configured to produce images based on light received thereon from an eye tracking imaging path; a second illumination source configured to produce light for illuminating the eye along a retina illumination path; a second imaging sensor configured to produce images based on light received thereon from a retina imaging path; and a third freeform prism coupled to the second freeform prism, the third freeform prism configured to receive light associated with a real scene and to direct light from the real scene towards the eye along an optical see-through path, wherein the first freeform prism, the second freeform prism, and the third freeform prism form a configuration capable of receiving light associated with four illumination paths: (a) the pupil illumination path corresponding to the first illumination source, (b) the retina illumination path corresponding to the second illumination source, (c) a stimuli illumination path corresponding to the display, and (d) the optical see-through path corresponding to the real scene, the configuration also capable to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

17. The optical system of claim 16, wherein the pupil illumination path is a path from the first illumination source, and to first freeform prism that provides at least a portion of the light corresponding to the first illumination source in a direction of the eye, the retina illumination path is a path from the second illumination source, to the third freeform prism, to the second freeform prism, and to the first freeform prism that provides at least a portion of the light corresponding to the second illumination source in a direction of the eye,the stimuli illumination path is a path from the display, to the second freeform prism, and to the first freeform prism that provides at least a portion of the light corresponding to the display in a direction of the eye, the optical see-through path is a path from the real scene, to the third freeform prism, to the second freeform prism, to the first freeform prism that provides at least a portion of the light corresponding to the real scene in a direction of the eye, the eye tracking imaging path is a path of light associated with the first illumination source and received at the first freeform prism after interaction with the eye, to the first freeform prism, and to the first imaging sensor, and the fundus imaging path is a path of light associated with the second illumination source and received at the first freeform prism after interaction with the eye, to the second freeform prism, to the third freeform prism, and to the second imaging sensor.

18. The optical system of claim 16, wherein the first freeform prism, the second freeform prism, and the third freeform prism form a configuration that includes: a first reflective layer on a first surface of the freeform prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, a second reflective layer on a second surface of the second freeform prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon, and a third reflective layer on a third surface of the third freeform prism, the third reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon; wherein the first reflective layer is interposed in between, and is in contact with, the first and the second freeform prisms, the second reflective layer is interposed in between, and is in contact with, the second and the third freeform prisms, and spectral reflectance or transmittance properties of the first, second, and third reflective layers differ from each other.

19. The optical system of claim 18, wherein the first freeform prism is configured to receive light associated with the first illumination source at multiple locations along the first surface.

20. The optical system of claim 18, wherein the first reflective layer forms a hot mirror, wherein the second reflective layer and the third reflective layer each include a beam splitter coating.

21. The optical system of claim 16, configured to combine the pupil illumination path and the eye tracking imaging path before interfacing with the first freeform prism.

22. The optical system of claim 16, wherein the retina illumination path and the fundus imaging path have at least one common optical path section.

23. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising: an eyepiece lens; a display configured to present visual stimuli to an eye; a beam splitter; a first polarizer; a first illumination source configured to produce light that passes through first polarizer; a reflective optical element interposed between the first illumination source and the display, wherein the reflective optical element is configured to allow light from the display that reaches the reflective optical element from a first direction to be transmitted therethrough and light that is received on the reflective optical element from a second direction opposite to the first direction to be reflected therefrom; a second polarizer; a first image sensor configured to receive polarized light from the second polarizer; a second illumination source configured to provide off-axis illumination of the eye; a third polarizer; anda second image sensor configured to receive light that passes through the third polarizer, wherein: the beam splitter is configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

24. The optical system of claim 23, wherein the first illumination source is configured to produce near-infrared light and the first image sensor is configured to sense near-infrared light.

25. The optical system of claim 23, wherein: the pupil illumination path is a path from the first illumination source, to the beam splitter, to the reflective optical element, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the beam splitter, and to the eyepiece lens, the stimuli illumination path is a path from the display, to the beam splitter, and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, to the reflective optical element, and to the first image sensor, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, and to the second image sensor.

26. The optical system of claim 23, wherein the reflective optical element comprises a reflective surface coating configured to reflect or transmit light incident thereupon based on a wavelength of the light incident thereupon.

27. The optical system of claim 23, wherein the third polarizer and the second polarizer are configured to produce, at respective outputs thereof, polarized light with an identical polarization state.

28. The optical system of claim 23, wherein the first polarizer is positioned along the pupil illumination path, the second polarizer is positioned along the eye tracking imaging path, and the third polarizer is positioned along the fundus imaging path.

29. The optical system of claim 23, wherein the first illumination source is configured to produce illumination for an external area surrounding the eye that is imaged using the first image sensor, and the second illumination source is configured to produce illumination for one or more internal areas of the eye including at least a portion of a retina, that is imaged by the second image sensor.

30. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising: an eyepiece lens; an illumination source configured to produce light for illuminating an eye; a condenser lens positioned in front of the illumination source; an imaging detector configured to produce images based on light received thereon; and a beam splitter positioned to receive light from the condenser lens and to direct at least a portion of the light received thereon in a direction of the eye, the beam splitter further configured to receive and to direct light received from the eyepiece lens in a direction of the imaging detector; wherein the condenser lens, the imaging detector, and the illumination source form a configuration that allows light propagating along an illumination path for illuminating the eye to be combined with light propagating along an imaging path.

31. The optical system of claim 30, further comprising: a pair of orthogonal polarization elements configured to produce linearly polarized light at outputs thereof, respectively, wherein a first of the pair of orthogonal polarization elements ispositioned along the illumination path and a second of the orthogonal polarization elements is positioned along the imaging path.

32. The optical system of claim 30, further comprising: a relay lens positioned between the beam splitter and the imaging detector.

33. The optical system of claim 30, wherein the beam splitter comprises a prism and includes: a first reflective layer on a first surface of the prism, the first reflective layer configured to reflect or transmit light incident on the first surface based on a wavelength of the light incident thereon, a second reflective layer on a second surface of the prism, the second reflective layer configured to reflect or transmit light incident on the second surface based on a wavelength of the light incident thereon; and spectral reflectance or transmittance properties of the first and the second reflective layers differ from each other.

34. The optical system of claim 33, wherein the first reflective layer forms a cold mirror and the second reflective layer form a hot mirror.

35. The optical system of claim 30, wherein the beam splitter is an x-cube prism or freeform prism.

36. The optical system of claim 30, wherein: the illumination and imaging paths are coaxial along an optical axis, the illumination source is an annular source configured to illuminate a peripheral zone of the condenser lens, the illumination path comprises a path from the illumination source, to the condenser lens to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, to a central zone of the condenser lens and to the imaging detector.

37. The optical system of claim 30, wherein: the illumination and imaging paths are coaxial along an optical axis, the illumination source is an annular source configured to illuminate a peripheral zone of the condenser lens, the illumination path comprises a path from the illumination source, to the condenser lens to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, and to the imaging detector that is positioned between the beam splitter and the condenser lens.

38. The optical system of claim 30, wherein: the illumination and imaging paths are coaxial along an optical axis, the illumination source comprises an array of light sources, the condenser lens comprises an array of condenser lenslets positioned around a periphery of the optical axis thus having an open central zone that allows light received from the beam splitter to traverse through the open central zone before reaching the imaging detector, the illumination path comprises a path from the array of the light sources, to the condenser lenslets, to the beam splitter and to the eyepiece lens, and the imaging path comprises a path from the eyepiece lens, to the beam splitter, through the open central zone, and to the imaging detector.

39. The optical system of claim 30, wherein the illumination source is configured to produce light for illuminating at least a section of a retina of the eye, and the imaging detector is configured to produce images of one or more sections of the retina.

40. An optical system for a wearable multi-functional ophthalmic device capable of monitoring oculomotor components, comprising: an eyepiece lens; a display configured to present visual stimuli to an eye; a first condenser lens;a first illumination source configured to produce light that passes through the first condenser lens; an imaging lens; a first image sensor configured to receive polarized light from the imaging lens; a reflective optical element interposed between the first image sensor and the display, wherein the reflective optical element is configured to allow light from the display that reaches the reflective optical element from a first direction to be transmitted therethrough and light that is received on the reflective optical element from a second direction opposite to the first direction to be reflected therefrom; a beam splitter; a polarizing beam splitter (PBS) configured to receive light associated with the first illumination source and to direct light having a first polarization state in a direction of the beam splitter, the PBS further configured to receive light from the beam splitter and to direct light having a second polarization state orthogonal to the first polarization state in a direction of the first image sensor; a second illumination source comprising an array of light sources; a second image sensor; an array of condenser lenslets positioned around a periphery an open central zone that allows light received from the beam splitter to traverse through the open central zone before reaching the second image sensor; and a polarizer positioned between the beam splitter and the array of condenser lenslets, wherein: the beam splitter is configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path.

41. The optical system of claim 40, wherein:the pupil illumination path is a path from the first illumination source, to the first condenser lens, to the PBS, to the beam splitter, and to the eyepiece lens, the retina illumination path is a path from the second illumination source, to the array of condenser lenslets, to the polarizer, to the beam splitter and to the eyepiece lens, the stimuli illumination path is a path from the display, to the reflective optical element, to the beam splitter and to the eyepiece lens, the eye tracking imaging path is a path from the eyepiece lens, to the beam splitter, to the PBS, to the imaging lens and to the first image sensor, and the fundus imaging path is a path from the eyepiece lens, to the beam splitter, to the polarizer, through the open central zone, and to the second image sensor.

42. The optical system of claim 40, wherein each light source in the array of light sources is associated with a respective lenslet in the array of condenser lenslets configured to provide off- axis illumination.

43. The optical system of claim 42, wherein the array of light sources is configured to emit pulsed light.

44. The optical system of claim 40, wherein the first illumination source is configured to produce near-infrared light and the first image sensor is configured to sense near-infrared light.

45. The optical system of claim 40, wherein the reflective optical element comprises a reflective surface coating configured to reflect or transmit light incident thereupon based on a wavelength of the light incident thereupon.

46. The optical system of claim 40, wherein the beam splitter is an x-cube prism or freeform prism.

47. An optical system for a wearable multi-functional ophthalmic device capable of monitoringoculomotor components, comprising: a first illumination source configured to produce light for illuminating an eye; a first imaging detector configured to produce images based on light received thereon, the images tracking one or more ocular responses of the eye to light produced by the first illumination source; a second illumination source configured to produce light for illuminating a retina of the eye; a second imaging detector comprising a camera configured to produce images based on light received thereon that has interacted with the eye; a third illumination source configured to produce light for stimulating the retina of the eye such that an accommodation response by the retina of the eye to light stimulation is provoked; a measurement device configured to record the accommodation response; a display configured to present visual stimuli to the eye during operation of the wearable multi-functional ophthalmic device; and a beam splitter configured to receive light associated with three illumination paths: (a) a pupil illumination path corresponding to the first illumination source, (b) a retina illumination path corresponding to the second illumination source, and (c) a stimuli illumination path corresponding to the display, the beam splitter also configured to receive light associated with two imaging paths: (A) an eye tracking imaging path, and (B) a fundus imaging path, wherein the beam splitter is further configured to combine illumination paths with imaging paths.

Images