Ophthalmic test system and method

Abstract

A modular head-mounted machine for performing ophthalmic tests on a patient includes removable optical modules that can be replaced based on the ophthalmic test requirements. A miniaturized fundus camera is provided that can be removably mounted within the head-mounted machine, including mounting to the optical module. The position of the fundus camera can be automatically adjusted to align the camera with the eye of the head-mounted machine wearer. Software controls the image capture, and the captured images are assembled and combined to provide a wide-field retinal image. The optical module can have replaceable sub-components, allowing configuration for different ophthalmic tests, such as visual field testing and optical coherence tomography.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 62 / 881,120, filed on July 31, 2019, the entire contents of which are expressly incorporated by reference. Technical Field

[0003] The present invention relates to improvements in vision testing and ophthalmic imaging systems, including improved head-mounted hardware incorporating ophthalmic devices and systems using such devices. Background Art

[0004] Eye care is an important part of overall health, and numerous specialized systems have been developed to allow ophthalmologists to examine a person's eyes. Many of these devices are expensive, limiting their availability. They are also bulky and often require dedicated tables or specialized stands for mounting. Due to their size, weight, and generally unsightly appearance, dedicated space may be required in the doctor's office for these devices.

[0005] To examine patients using these devices, they are often required to position and maintain their head in a specific position while their eyes are examined. Patients with limited mobility may not be able to position their bodies in the way that specific optical tools require. This can limit the ability to provide comprehensive eye exams for these patients. Similarly, due to size and expense, bringing various specialized eye exam systems to patients who cannot visit a doctor's office can be difficult or impossible.

[0006] A common tool for eye examinations is a fundus camera for capturing retinal images. A fundus camera is a specialized low-power microscope with a camera attached. Conventional fundus camera systems have a housing that is approximately 1 to 2 feet in each dimension, with an objective lens on one side and a camera display on the other. In use, the patient sits in front of a table that supports the fundus camera system, places their chin on a chin rest attached to the system, and presses their forehead against a forehead bar. The chin rest is adjusted so that the patient's eyes are relative to the objective lens of the camera system. The camera is then focused and one or more photos are taken. During this process, the patient may reposition their forehead for comfort or because of a natural tendency to move their neck (e.g., due to stiffness). This action may cause the eyes to be misaligned with the camera, requiring the doctor to readjust the focus of the camera.

[0007] Similarly, conventional devices for performing perimetry or visual field testing are sensitive to patient head and eye movement during use, require the patient to place their head on a chin rest, and are bulky and expensive, limiting portability and availability outside of dedicated settings. For visual field testing, the user looks at the center of a bowl-shaped "perimeter" instrument. Lights or other objects are shown within the instrument, and the patient is instructed to see them. Mapping what the patient can and cannot see shows their visual field. Movement of the patient's head relative to the light field of the perimetry changes their field of view and reduces the accuracy of the test results. If the patient shifts their gaze from the central target, even if their head remains still, the results are reduced.

[0008] OCT is a non-invasive imaging technique that relies on low-coherence interferometry to generate in vivo cross-sectional images of ocular tissue. OCT is used to detect disease and provides a quantitative and repeatable method of assessing surgical and pharmaceutical interventions. Over the past decade, OCT systems have become more common in community optometric practice, and over the past decade, the performance of OCT systems has evolved with higher scan densities, faster data acquisition and processing speeds, and computerized image analysis.

[0009] Conventional devices for coherence tomography (OCT) have similar deficiencies as conventional fundus cameras and perimetry systems. When the system scans the eye, the patient must sit in front of the OCT machine, holding their head still for 5 to 10 minutes. Movement of the patient's head can cause misregistration issues. To prevent patient movement, an assistant is typically required to be present throughout the examination to monitor the patient's head orientation and correct the position as needed. OCT scanning devices can also be bulky and expensive.

[0010] There is a need for a portable and inexpensive system for performing eye examinations in a variety of environments, and which can be easily used without requiring the patient's head to be placed on a chin rest and forehead rest. It would also be advantageous if such a system could be used to perform fundus imaging, perimetry testing, and for OCT imaging. It would be a further advantage if such a system could be easily configured and reconfigured to allow for use in different types of eye examinations, such as fundus imaging, OCT imaging, and perimetry, rather than requiring a dedicated testing system for each examination. SUMMARY

[0011] These and other problems and deficiencies are addressed by a head-mounted system that a patient can wear, which includes optical modules that can be configured to support a variety of optical tests. The optical modules can be removable from the head-mounted system, allowing a module configured for one test to be replaced by a module configured for a different test. The optical modules can be made of removable sub-modules, allowing a single module to be customized for application of different optical tests. A miniature fundus camera can be movably mounted within the head-mounted system to allow adjustment of the camera position relative to the patient's eye. Multiple images captured by the fundus camera can be combined to generate a very wide-angle view of the retina. Optical displays and other illumination elements can be used for perimetry.

[0012] In one embodiment, a fundus camera includes a housing having a first end and a second end, a front side, a back side, a major axis extending from the first end and the second end, and an aperture opening on the first side. A camera assembly is mounted inside the housing. The camera has a field of view that extends from the aperture. The fundus camera includes an illumination light source configured to produce an illumination beam, an electronic image capture camera, an objective lens, and an exit mirror. The illumination beam travels along an illumination beam path from the illumination light source to the exit mirror. The exit mirror redirects the illumination beam path through the objective lens (which can be a spherical lens) and into the field of view. It also redirects light from the illumination beam that is reflected by objects in the field of view and back through the objective lens along a return light path that extends to the imaging camera. The paths of the exit light that illuminates the field of view and the incoming light from the illuminated objects in the camera can be substantially parallel.

[0013] The mirrors, such as the exit mirror, can be rotatable or pivotable to change the direction of the illumination beam path in the field of view, illuminating different portions of the patient's retina. The direction of the camera field of view can also be changed. Attenuation elements, such as beam splitters, can be used to reduce the intensity of the illumination light. Internal folding mirrors can be operable to increase the optical path while keeping the camera size small. Glare reduction elements can also be included.

[0014] The fundus camera can also include a rotatable mounting assembly configured to allow the camera within the elongated housing to be rotatably mounted within a head-mounted or other system. The rotatable mounting assembly can be on the back of the housing, adjacent to the first end of the housing, with the imaging aperture on the front of the housing, adjacent to the second end. In one embodiment, the mounting assembly includes a gear system that includes a stationary gear attached to the housing and a rotating gear connected to the stationary gear and mounted to a shaft, such that rotation of the shaft causes the camera to pivot.

[0015] In one embodiment, a head-mounted system for optical examination of a user is provided. The head-mount includes a main body configured to be worn on the face of the user. The face side of the head-mount has a forward edge configured to rest on the face of the user when the head-mount is worn. As described above, a fundus camera can be rotatably mounted and positioned within the head-mount so that it can be moved in front of the eyes of the user when the head-mount is worn. Rotating the fundus camera when the head-mount is worn allows the camera objective to be moved relative to the eyes of the user so that the camera can be repositioned as needed to image the retina. A motor can be provided to selectively rotate the fundus camera, for example under the control of a computer program. A single fundus camera can be mounted at approximately the midpoint between the left and right sides of the main body and can be rotated from a first, left position to image the left eye of a user wearing the head-mount to a second, right position to image the right eye of a user wearing the head-mount. Alternatively, separate left and right rotatably mounted fundus cameras can be provided and positioned to image the left and right eyes, respectively. A fundus camera carriage can be slidably engaged to the head-mount and can be moved between first and second positions, for example in a track. Changing the position of the carriage within the track moves the fundus camera and allows the position of the camera relative to the patient's eye to be changed. In one embodiment, the fundus camera is rotatably and slidably mounted to provide multiple degrees of freedom of position adjustment.

[0016] An eye tracking camera can be included in the head-mount, which is operable to capture images of the user's eyes, process these images to determine the position and / or direction of the line of sight of the eyes. The fundus camera can capture a plurality of images and combine them to generate a very wide field of view image of the retina. Eye tracking data captured along with the fundus camera images can be used to determine which portion of the retina has been imaged in any given image captured by the fundus camera, and this data is used to map the captured images into a combined image of the retina.

[0017] One or more visual displays can be provided in the head-mount and located behind the fundus camera. The displays can be used to present images to the person wearing the head-mount. The fundus camera can be moved between an imaging position in which the fundus camera obscures a portion of the visual display and a storage position in which the visual display is substantially unobscured by the fundus camera. The fundus camera can also be removably mounted within the head-mount main body so that the head-mount can be used with or without the fundus camera.

[0018] According to one aspect of the present invention, a computer-controlled method for imaging the fundus of a patient's retina is provided. A headset is placed on the patient's head. The headset includes a main body and a rotatably mounted fundus camera, as described above, which can be moved to different positions in front of the patient's eye. The fundus camera captures images of the user's eye. The position of the fundus camera is adjusted based on a first position to improve alignment of the camera objective lens with the eye pupil. The adjustment can be automatic or manual. Once adjusted, multiple retinal images are captured using the fundus camera.

[0019] During image capture, the user can be instructed to change the direction of their gaze so that the camera can image a different portion of the retina. These instructions can be audible. In one embodiment, the headset further includes a plurality of visible light sources arranged around an inner periphery and visible to the user. The method includes illuminating at least one of the plurality of visible light sources to indicate the desired direction of gaze.

[0020] The plurality of captured retinal images are mapped to respective locations in a retinal image map and then stitched together to form a combined wide field of view of the retina. The location of each retinal image in the retinal image map is on the portion of the retina imaged in the respective retinal image. This can be determined with reference to data indicating the position and target of the fundus camera relative to the eye and the direction of the eye's gaze, such as determined by an eye-tracking camera. Eye-tracking data captured during fundus imaging can be stored with the fundus images and used for image mapping after the fundus image capture phase is complete.

[0021] After determining that a sufficient portion of the retina has been imaged based on allowing complete mapping of the desired portion of the retina, fundus image capture can be automatically terminated. If it is determined that a portion of the retina was not successfully imaged, the computer can automatically instruct the patient to change the direction of vision so as to bring the portion of the retina that was not successfully imaged into the field of view of the fundus camera.

[0022] According to another aspect of the present invention, a wearable optical headset system is provided. The system includes a generally tubular outer frame having an interior extending between an open front and rear portion of the outer frame, wherein the front portion is configured to be pressed against a user's face and surround the user's eyes. At least one headband attached to the outer frame is configured to hold the outer frame against the user's face when the headset is worn. An optical module housing having a front surface, a rear surface, and side surfaces is provided. The optical module is configured to be removably and slidably engaged within the interior of the outer frame, with the front of the optical module visible through the front portion of the outer frame. The optical module housing encloses computer circuitry comprising a computer processor, a digital memory connected to the processor and configured to store computer software executable by the processor, and an optical component comprising at least one of an image display system and an image capture system. The optical component is electrically connected to the processor and controlled by the processor according to the stored computer software. The optical module can be secured to the headset housing using resilient clips arranged along an outer perimeter of the optical housing, the resilient clips engaging corresponding apertures in the outer frame.

[0023] A first electronic display may be mounted on a rear surface of the optical module and configured to output visual data in response to signals from the processor. A second electronic display may be mounted on an outer side of the outer frame and configured to output visual data. An electrical interface may be provided between the headset housing and the optical module to allow systems within the optical module to control the display mounted on the outer frame. Various user input devices may be provided on the optical module and / or the outer frame.

[0024] The removable optical module may include a rotatable fundus camera and an eye tracking camera configured to image the eyes of a user wearing the headset. Visible light LEDs may be positioned along the perimeter of the optical module in locations visible to the user wearing the headset.

[0025] According to another aspect, the optical module itself is modular and includes a plurality of subassemblies. Each subassembly has its own housing having a front surface and a rear surface, wherein the subassemblies can be stacked from the rear of the optical module to the front of the optical module and where each subassembly can be removably connected to an adjacent subassembly. In one embodiment, a first subassembly includes a circuit board having computer circuitry therein and a second subassembly includes a visual display viewed by a wearer of the headset. Electrical and mechanical interfaces are provided on adjacent surfaces of the subassemblies. When connected, the visual display in the second subassembly can be controlled by a processor in the first subassembly. A third subassembly can be provided that includes functional components for performing an eye examination and is mounted in the optical module in front of the second subassembly. In one embodiment, the third subassembly includes a fundus camera that is rotatably mounted to the third subassembly.

[0026] In one embodiment, the optical module includes a generally planar circuit board having a front and a back and having computer circuitry thereon. The optical assembly includes first and second visual displays, which may be micromirror displays. Each visual display is electrically connected to the circuit board. First and second lens assemblies, respectively positioned in front of the first and second displays, are operable to form a virtual image of an image presented on the respective displays. To a user wearing the headset, the virtual image appears to be at a first distance from the user that is greater than the actual distance between the user's eyes and the visual display. The lens assembly may include a liquid lens having an electrically controllable focus responsive to a signal from the computer circuit. The apparent distance of the virtual image can be changed by adjusting the focus of the liquid lens.

[0027] The display in the optical system can be a retinal image display, which includes a light emitter configured to emit a light beam. An integrating rod is positioned to receive the light beam when it is emitted and output an integrated light beam. At least one lens is configured to receive the light beam from the integrating rod and focus the light beam. A beam splitter in the focused light beam path directs a portion of the focused light beam to intersect the surface of a digital micromirror device (DMD). Light reflected from the DMD re-enters the beam splitter, and a portion passes through a projection lens, which focuses the reflected light for a user wearing the headset to view an image generated by the DMD.

[0028] The fundus camera may also be rotatably mounted to the front frame in the optical module and configured to image the eye structure of a person wearing the headset system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Further features and advantages of the present invention, as well as the structure and operation of various embodiments of the present invention, are disclosed in detail below with reference to the accompanying drawings, in which:

[0030] Figure 1 This is a simplified high-level block diagram of an ophthalmic testing system in general;

[0031] Figure 2A and 2B A modular headset system according to one aspect of the present invention is shown;

[0032] Figure 3A and 3B An optical module comprising several subcomponents is shown;

[0033] Figure 4 shows a headset system with various external user inputs and outputs mounted thereon;

[0034] Figure 5 Shows the user's Figure 4 headset.

[0035] Figure 6is a high-level block diagram of the main electronic components 600 that may be provided in an optical module;

[0036] Figures 7A-7D A specific embodiment of an optical module suitable for displaying a VR environment as part of an eye examination is shown;

[0037] Figure 8 The configuration of the optical module is shown as viewed from the side facing the user, and the optical module can be inserted into the headset bracket;

[0038] Figure 9A and 9B shows a sample image rendered on a head-mounted display as part of the viewpoint rendering process;

[0039] Figure 10A and 10B A view showing a miniaturized fundus camera module mounted in a housing;

[0040] Figure 10C is an exploded view of an optical module that can be mounted in an outer frame of a head-mounted system and includes a rotatably mounted fundus camera;

[0041] Figure 11A and 11B shows a headset system viewed from the side facing the user, with a fundus camera mounted in the headset;

[0042] Figure 12 is a high-level block diagram of an eye tracking system that includes a fundus camera;

[0043] Figure 13 is a high-level flow chart of a method for performing retinal imaging using a fundus scanning system;

[0044] Figure 14A A fundus camera having a rotatable mounting assembly is shown;

[0045] Figures 14B-14D Shown with gear mounting assembly Figure 14A Fundus camera;

[0046] Figure 14E Shows the headset installed Figure 14A Fundus camera;

[0047] Figure 15A and 15B Shows Figure 14B Top and bottom views of the gear mounting assembly

[0048] Figures 16A-16E A slidably mounted fundus camera is shown;

[0049] Figure 17 is a high-level schematic diagram of a specific configuration of a fundus camera that can be miniaturized and used in a head-mounted assembly;

[0050] Figures 18A-18C Shows top and perspective exploded views of a miniaturized fundus camera design; and

[0051] Figures 19A-19C An embodiment of a retinal image display incorporating a digital micromirror device and usable in the head-mounted assembly disclosed herein is shown; DETAILED DESCRIPTION

[0052] Figure 1 is a simplified high-level block diagram of an overall ophthalmic testing system 10 in which optical tests and data capture, such as fundus imaging, visual field testing, and OCT scanning, are performed using equipment housed in a portable headset 12 that a patient can wear. The headset 12 includes internal equipment 14 (not shown) for capturing images or data from one or both eyes of a patient wearing the headset 12. As described further below, the equipment 14 may include lenses, cameras, light emitters, visual displays, mechanical structures, electronics, and computing hardware and software for administering one or more optical tests (e.g., fundus imaging, visual field testing, OCT, and automated replication). A given headset 12 may be dedicated to a single type of test. The headset equipment components 14 may be modular, and some or all may be removed from the headset housing and replaced with different components to allow the same headset 12 to be used for a variety of different types of eye tests.

[0053] While certain computing hardware and software may be integrated within the headset 12, a separate control system 16 for headset control and image / data capture may be connected to the headset 12 and include computing hardware and software for controlling the headset components 14 during the test process. For example, the control system 16 may be used to configure the headset 12 with test parameters, initiate the test process, and receive captured test data from the headset.

[0054] The captured test data may be stored in one or more electronic data stores 18. The data store 18 may be internal or external to the control system 16 and connected directly or via a network. Any suitable data store 18 may be provided, such as an internal or external hard drive or a network / cloud based data store. A separate system 20 may also be provided with data and image processing and analysis capabilities to increase the functionality available in the control system 16. The system 20 may be located locally on the control system 16 or remotely connected via a network. In one configuration, the control system 16 is a local computer, such as a tablet, laptop or desktop computer with internal memory that serves as the data store 18, and the system 20 is a remote server with its own data storage, and a specialist may connect through the remote server to access the test data, such as a doctor viewing eye test data for remote diagnosis.

[0055] During the optical test, the patient may be required to provide input, such as indicating when they see a particular feature on a display, while wearing the headset 12. A hand controller, keyboard, remote control, or other input device 22 may be provided for this purpose. The input device 22 may be connected to a system within the headset 12 or to the control system 16 using a wired or wireless link (e.g., a plug-in USB cable or a Bluetooth connection).

[0056] According to various aspects of the present invention, using the headset system disclosed herein for eye testing provides various advantages. When worn correctly, the headset moves with the user's head, so even when the user's head moves around, the imaging and other equipment in the headset can maintain correct optical alignment with the user's eyes. This is especially useful for children who cannot remain still during an eye exam and are more prone to fidgeting, as well as for older adults with ADHD tendencies or movement disorders (including paralysis, Parkinson's disease, amyotrophic lateral sclerosis (ALS), Tourette syndrome, tremor, ataxia, dystonia, multiple system atrophy, Rett syndrome, or myoclonus).

[0057] The system allows for easier examinations of patients with limited mobility. For example, if a person is bedridden due to an injury that does not allow him or her to sit in a chair for a standard desk exam, the headset 12 can be placed on their head and a standard eye exam can be performed while the patient lies in bed.

[0058] A headset system with integrated optical testing equipment can eliminate the need for a second person to be present during the eye test and correctly place the camera tube in front of the patient's eyes (a process that can also be uncomfortable for the user). Instead, the patient can put on the headset directly.

[0059] The headset 12 can be configured to display a 3D virtual reality environment for eye examinations. Eye tracking functionality can be provided in the headset and used to monitor the position of the user’s eyes and their gaze direction during a virtual examination, either in a configuration where images are presented to the user for viewing or in other testing configurations that can not use target images. The system 10 is particularly useful for detecting early signs of dyslexia in children aged five and under, an age range in which the eyes are mostly developed.

[0060] Liquid lens technology can be incorporated into the lens system between the user’s eyes and the internal display. The optical characteristics of the liquid lens can be controlled to adjust the apparent distance of the virtual image formed by the image on the display. This is useful for testing or simulating myopia or hypermetropia. The use of liquid lenses also allows the headset system to be configured as a head-mounted phoropter. The optical power of the liquid lens can be automatically adjusted to avoid the need for another person to mechanically change the optical power of the lens looked through by the user during the test.

[0061] The headset can be configured to provide space to wear eyeglasses while performing eye examinations. For patients whose eye vision requires a large refractive correction, the device can allow the eyeglasses to be best tried on while using the ophthalmic instrument without limiting the field of view of the screen. It is also possible to enhance the immersion in the eye examination, which can improve the reliability of the test results, especially for clinical purposes.

[0062] Turning to Figure 2A and 2B , a physical structure of a modular headset system 100 is shown, which can be used as Figure 1 the headset 12 in the system as well as for other purposes. The headset system 100 includes head-mounted hardware 105, which includes an outer frame 115, one or more head straps 120, and a cable 125, which can be used to carry power, control signals, and data as needed in a given implementation.

[0063] Headgear 120 can be fixed or adjustable, and can be configured in various conventional ways to secure head-mounted system 100 to the face of a user. Headgear 120 should be operable to allow system 100 to be secured to the face of a person while distributing the bulk of the weight of system 100 around the perimeter of the user's head. The portions of system 100 that contact the face of the user can incorporate face pads, such as pads made of a breathable cloth, and the pads can be designed to further distribute the weight of the ophthalmic head-mounted system across the face and to keep the center of gravity as close to the neck as possible and to minimize slippage for comfort. In one configuration, there is a circular strap that fits around the patient's head and a second strap that fits around the patient's head. The straps can be adjustable according to the size of the patient's head, such as by using a gear mechanism that can automatically or manually tighten or loosen the straps.

[0064] Head-mounted system 100 also includes an optical module 110 that can be removably mounted within outer frame 115 and housing 130. Optical module 110 contains various head-mounted equipment components, such as lenses, cameras, light emitters, visual displays, mechanical structures, electronics, and computing hardware and software. These components are contained within housing 130. In one embodiment, housing 130 is configured to be securely fitted within the outer frame 115 of the head-mounted system, and to be easily removable by the user. Various mechanisms can be used to removably mount optical module 110 to head-mounted system 100. In the illustrated embodiment, optical module 110 is secured within outer frame 115 by a plurality of clips 135 on optical module housing 130 that engage corresponding holes 140 in outer frame 115. Other mounting mechanisms known to those skilled in the art can also be used to removably secure optical module 110 within outer frame 115, including screws, spring clips or hooks, and other structures.

[0065] A port or other interface 145 can be provided in housing 130 of optical module 110 to allow electrical connections between optical module 110 and head support 105 or other external equipment. This connection can be used to power optical module 110 and / or provide data and control connections. In one embodiment, port 145 contains electrical contacts that engage corresponding contacts on the inside of outer frame 115 when optical module 110 is installed. Although one port 145 is shown, a plurality of separate ports can be provided. Additionally, or alternatively, a plug-in cable can be used to connect port 145 on optical module 110 to electronics in head support 105. If no connection to outer frame 115 is needed, port 145 can be positioned so as to be accessible when optical module 110 is in the outer frame. A hole can be formed in outer frame 115 to provide external access to port 145.

[0066] In embodiments that are not powered through a head-mounted cable 125, an internal battery can be provided within the optical module 110. Depending on the configuration and functionality of the optical module 110, the cable 125 can not be required. For example, the optical module 110 can be powered by an internal battery, and can provide communication with an external computing system (e.g., control system 16) through Wi-Fi, Bluetooth, or other wireless connection.

[0067] Multiple optical modules 110 having different functionality can be provided in pre-assembled form, and the respective optical modules can be swapped in and out to provide different functionality to the head-mounted device. Various different optical modules 110 are presented herein. Each module 110 can include a built-in computing system with firmware that controls how the module operates, and also supports communication with the head-mounted device to adapt to the relevant imaging or ophthalmic task being delivered. The firmware can include functionality for displaying images, controlling electrical, mechanical, and optical components, capturing images, performing image processing routines, and other functionality to support the overall testing system. Offloading various image processing and control functionality to the computing system in the module 110 can reduce the data that needs to be exchanged with an external computer system.

[0068] Because the optical module 110 is removable, the same optical module 110 can be used with different head-mounted hardware 105, such that, for example, the optical module 110 can be switched between head-mounted hardware 105 that is adapted for adults and head-mounted hardware 105 that is adapted for children. Different types of ophthalmic examinations can require different functionality. The system 100 allows a first optical module 110 configured for a first type of test to be removed and replaced with a second optical module 110’ configured for a second type of test, while using the same head-mounted hardware 105. As discussed further below, the optical module 110 itself can be modular to allow some internal devices (e.g., lens assemblies and associated mechanical structures) to be removed and replaced to provide different functionality, e.g., to support different types of eye tests, while allowing other internal devices, e.g., electronics and computing hardware, to be reused.

[0069] Turning to Figure 6, showing a high-level block diagram of the main electronics 600, which can be provided within the optical module 110 and can work with various additional optical, mechanical, and other components that can also be within the module 110. In one embodiment, the optical module 110 can contain a computer processor 602 and one or more internal visual displays 604, such as OLED or LCD displays or micro-mirror projection displays, one for each eye, and used to display a virtual reality environment as part of an eye examination. The displays can be high-resolution displays with 4k resolution, i.e., 3840 x 2160 pixels or 4096 x 2160 pixels. A particular improved design of a digital micro-mirror display device suitable for installation in a head-mounted machine is further described below.

[0070] The module system 600 can include a conventional imaging camera 610 as well as one or more eye tracking cameras 612 (typically four for each eye, although fewer or greater numbers can be used). Motion sensors 606 provide measurements that allow detection of head-mounted machine motion. LEDs or other light sources 608 can be provided. Infrared LEDs can be used to illuminate a person's eyes for eye tracking purposes. Visible light LEDs can be used to signal to a person wearing the head-mounted machine and to illuminate a person's eyes for visible light camera imaging.

[0071] Digital memory 614 stores computer software executed by the processor 602, such as an operating system and software for implementing optical testing as well as storing data used and generated during operation of the system. Various user input devices 618 can be provided, including one or more of buttons, touch screen, toggle switches, dials, or other input mechanisms, as well as audio components 618 to support audio input and output, such as a microphone and headphones (or a headphone jack that will link to an attached headset).

[0072] One or more communication ports 620 are provided to allow wired or wireless connection to external devices, such as the control system 16. The VR module 600 can be configured so that its functionality can be accessed using a conventional VR head-mounted machine interface.

[0073] Different modules 110 can be specially configured for different types of optical examinations, such as OCT, autorefraction, and retinal imaging using a miniature fundus camera. The modules 110 can be configured to provide VR environment testing simulations, or to measure eye color defects, visual field examinations of a user's field of view, and to perform autorefraction functions. Each configuration can require different optical, mechanical, and other components. Additional modules 110 can provide optional functionality. While the system is discussed herein in the context of medical testing, different modules can be used for any purpose.

[0074] Appropriate internal software or firmware is stored in memory 614 to allow processor 602 to interact with and control various internal components and communicate with other system components, such as the head-mounted hardware 105 and components on external control system 16 or other external computers, to partially or fully support ophthalmic imaging or other testing functions, return captured images or other test data. Control and test software required by control system 16 or other external devices can be pre-installed in optical module 110. When an external computer is connected to optical module 110 (e.g., by a cable), software required to be run on that computer can be downloaded from optical module 110. Likewise, an externally connected computer can be used to update software on optical module 110.

[0075] For a module system 600 that includes a camera for imaging external or internal features of an eye, images can be captured by the module and processed and analyzed to identify potential problems. Captured images can be processed using software such as stitching algorithms, image processing, machine learning, and pattern recognition to generate detailed images of the eye and identify potential problems. Some image capture and initial processing can be done by processor 602. For example, software stored in memory 614 can be provided to capture images, perform some initial image processing, and determine when a sufficient number of images have been captured for a particular imaging process to be addressed. More complex processing, such as stitching and analysis, can be done on a separate computer, such as control system 16 or system 20. In alternative embodiments, external system 16 or 20 can be provided with only low-level support by computer components in module system 600 to control the overall imaging sequence.

[0076] Visual display 604 for presenting image tests to a patient wearing system 100 can have a large spectrum, such as an OLED or LCD display or a micro-mirror projection display, and have a high resolution. Display 604 can be used to present multiple images with varying degrees of color to test color acuity and the severity of color blindness. Providing higher resolution images in pixels per inch can provide better results for visual tests such as the Ishihara color blindness test. Display 604 can also be used to provide a visual field check, visual acuity, and an Amsler grid test. These aspects are discussed below. Software can be configured to allow one or more of these tests to be run. Test selection can be on-demand and some or all of the tests can be initiated in sequence for a given patient by, for example, an optometrist, depending on the screening needs for that patient.

[0077] In one embodiment, the removable optical module 110 comprises individual modular sub-components that can be combined by an end user, such as a physician or technician. Each sub-component can provide a different functionality. By replacing one or more sub-components, the entire module 110 can be configured to support different eye tests, which can require individual and incompatible optics.

[0078] Figure 3A and 3B An exploded and assembled cross-sectional view of an optical module 300 in an example embodiment is shown, having three sub-components 305, 310, 315 mounted in an outer layer or housing 320 configured to mate with the outer frame 115 of the headrest 105. The functionality of each sub-component can be different.

[0079] In one embodiment, the rear component 305 includes the main computer system components, such as the processor 602, program and data memory 614, and wireless and wired communication modules 620. The sub-component 310 includes a video display 604, such as an OLED or LED display, and can include other optical components, such as an IR emitter 608 and eye tracking camera 612. Related optics can also be included, such as lenses for the camera and display. The sub-component 315 includes additional mechanical, optical, and / or electronic elements, such as filters, additional lenses, cameras, etc., suitable for a particular eye test or other use of the optical module 110.

[0080] Mechanical alignment and connection structures can be provided to allow adjacent sub-components to be properly aligned and directly and removably coupled together. Electrical interfaces can also be provided on the sub-components to allow direct electrical and data communication between adjacent sub-components, such as to allow the computing circuitry in the sub-module 305 to drive the video display in the sub-module 310. Suitable mechanical and electrical interfaces are known to those skilled in the art for these purposes.

[0081] In one configuration, opposing mechanical structures 325a and 325b are provided on mating surfaces of adjacent sub-components, such as on the adjacent front surface 305a of the sub-component 305 and on the rear surface 310a of the sub-component 310. The mechanical structures 325a, 325b can be configured to mate for alignment purposes, such as a protrusion / slot configuration. They can be further configured to mechanically engage one another, such as through spring-biased snap members, to help hold adjacent sub-components together.

[0082] Mating electrical interfaces 330a, 330b, such as conductive pads and spring-loaded pins, can be provided on opposing mating surfaces of adjacent subcomponents to provide direct data and electrical communications between adjacent subcomponents. Other data connections, such as optical data emitter / detector interfaces may also be suitable for data communications. Alternatively or in addition, connections between subcomponents in module 300 can be made indirectly through housing 320. Connection circuits 340a can be provided on one or more sides of a submodule that will be electrically connected to corresponding connection circuits 340b inside housing 320. The connection circuits 340b in housing 320 can be provided in fixed locations or provide expansion interfaces, such as data and / or power buses, to support connections from subcomponents located at various positions within the length of housing 320.

[0083] Different subassembly configurations are possible. For example, a computer system component and a display component can be combined in a single subassembly. In some configurations, the subassemblies can be used individually, and additional functionality can be added through further submodules. For example, a dual display system with a liquid lens optical train and eye tracking can be provided for certain types of tests, such as visual field testing. Additional submodules can be fitted into place to provide additional functionality, allowing different tests to be performed. In one configuration, a miniaturized fundus camera, such as that disclosed separately below, can be added for retinal imaging.

[0084] The housing 320 can have internal stops or other structures that allow the mounted submodules to be placed in known locations along the length of the housing 320, and by extension, in predetermined locations relative to the head-mounted hardware 105, and thus on the user's face, when the entire module 300 is mounted therein. Spacers can be added between or before the submodules as needed to properly position them within the housing 320.

[0085] Each subassembly can have a designated function code that identifies its purpose, and this function code can be read by the computer subassembly 305 or other system, such as in the housing 320 or control system 16, to allow software to know which subassemblies are installed and, therefore, what testing functions are available. The system can automatically detect the installed modules and signal the activation of the appropriate software, such as software for performing fundus camera imaging, visual field testing, OCT, etc.

[0086] In addition to separate user input systems (such as Figure 1 In addition to the input terminal 22), or alternatively, the user input terminal and output terminal can be set outside the head-mounted system 100. Figure 4 A headset system is shown with various external user inputs and outputs mounted thereon. Figure 5 Shows the user's Figure 4Headset. Figure 4 and Figure 5 , one or more input buttons 415 can be provided on the outer frame 115. Signals from the buttons 415 can be coupled through the port 145 or other connection to a computer system within the optical module 110 (including in the form of the module 300), or to an external computer operating with the system 100. Additionally or alternatively, one or more input buttons 420 can be provided external to the optical module 110, with signals from those buttons 420 being processed by circuitry within the optical module 110. The buttons 415, 420 can be mechanical push button switches, touch-sensitive devices such as capacitive or pressure-sensitive areas, or other forms of input.

[0087] According to a further feature, a small video display 405 can be formed on an outer surface 425 of the outer frame 115 of the head support 105. While the screen of the display 405 is displayed on a side surface, it can also be located on a different surface of the outer frame 115, such as the top surface. Additionally or alternatively, a separate display 410 can be positioned with its screen on the outward-facing surface 430 of the optical module 110. The display screens 405, 410 can be touchscreen devices and, in this configuration, can be used instead of or in addition to the separate buttons 415, 420.

[0088] Displays 405 , 410 may be driven by a computer system within optical module 110 through a connection via port 145 or other connection, or coupled to and driven by an external computer operating with system 100 , such as via cable 125 .

[0089] Displays 405, 410 may be used to output information messages for system diagnostics, such as for the optical module 110. In one configuration, when a new optical module 110 is inserted, one or both of the display screens 405, 410 may provide outputs indicating that each component in the optical module 110 is functioning properly and that required power and data connections (e.g., to components in the head mount 105) have been established.

[0090] Displays 405, 410 can also be used to output information messages related to the ongoing visual test sequence. In one embodiment, the test software is configured to indicate the test being performed and the stage of the test. Other information, such as a visual representation of the display being shown to the patient, can be mirrored on the same or a separate display. This allows a doctor or other person to easily monitor the patient's test progress without having to access a separate computer connected to the headset.

[0091] The displays can also be used to output information related to the current module configuration. One display (e.g., the front display) can be used to output instructions on how to operate the display optics assembled within the removable module, while a second display (e.g., the side display) can output operational data such as power and battery levels, volume adjustments, and the title of the test being performed by that unit configuration.

[0092] The display can also serve as a touchscreen control device, allowing the physician to change various configuration features, such as the test being run or the position of adjustable optical features within the headset. For example, controls for adjusting the interpupillary distance of the lens elements can be provided. Images from the headset's built-in camera can also be displayed on the screen, allowing the physician to see the patient's eyes during the examination.

[0093] Figures 7A-7D A specific embodiment of an optical module 700 is shown, which is suitable for displaying a VR or other visual environment as part of an eye exam and can be assembled into the head-mounted hardware 105 or otherwise incorporated into a head-mounted VR display. Referring to the figures, the module 700 has a rear frame or housing 702. A motherboard 704 is disposed within the rear frame and contains computer circuitry for controlling various components within the optical module 700 and providing communication with external devices. Left and right images are provided by a pair of visual displays 706a, 706b. In a specific configuration, the displays 706a, 706b have edge connectors 707a, 707b that connect to corresponding interfaces 705 located on an edge (e.g., a bottom edge) of the motherboard 704. The interfaces 705 can include connection slots 705a that are parallel or perpendicular to the surface of the motherboard and are configured to receive corresponding guide edge connectors 707a, 707b on the displays to facilitate close connection and mounting to and parallel to the motherboard 704. In addition to the rigid form factor, the male edge connector can be composed of a flexible cable.

[0094] Mounted within a frame assembly 710 are a pair of lens assemblies 708a, 708b. These lens assemblies 708a, 708b are configured to create a virtual image of the image visible to a person wearing the headset on respective displays 706a, 706b, with the image appearing to the person farther away and larger than the actual display. Various types of lens assemblies can be used for each lens assembly 708a, 708b, including single lenses, composite lens structures, and regular or hybrid Fresnel lenses. An interpupillary eyepiece can be placed in front of the respective Fresnel lens to focus the display for the user. In addition to displays, micromirror displays can also be used. While two displays 706a, 706b and corresponding lens assemblies 708a, 708b are shown, some embodiments may include only a single display and / or a single lens assembly. The lens assemblies 708a, 708b can be assembled into a frame assembly 710, which includes respective display lens frames 712a, 712b mounted to a front frame 714. The various components 704 , 706 , 708 , 710 , 712 , 714 are mounted to and / or within an outer frame 716 , which may be removably mounted within the outer frame 115 of the head support 105 .

[0095] Various other components may also be included in the optical module 700, including LEDs and eye-tracking cameras. These components may be mounted on the circuit board 704 or other components. For example, the eye-tracking camera may be mounted within the rear frame 702. An eye-tracking system may be provided to collect data from the user's eyes, and an image of each eye may be generated from the camera in the tracking system. The LEDs may be mounted on portions of the frame assembly 710. Electrical connections between these components and the driver circuitry may be provided in a manner known to those skilled in the art.

[0096] Displays 706a, b can be controlled independently. Some exams may require both displays to be on. Other exams focus on only one eye, requiring one display to be on and the other off. Eye tracking features and other off-eye functions can be left on or disabled depending on the test being performed.

[0097] In certain embodiments, each lens assembly 708a, 708b includes a liquid lens element having an electronically adjustable focus. Electrical connections are provided for a lens driver circuit 709 (not shown), which may be located on the main board 704 or elsewhere. The lens driver circuit 709 provides control signals for adjusting the focus of the lenses 708a, 708b.

[0098] Figure 7C and 7DA liquid lens assembly is shown that includes left and right lens assemblies 708a and 708b. The liquid lens assembly includes mounting rings 720, 722, a static lens 724, which may be a Fresnel lens, a spacer 726, and a variable focus liquid lens 728 having a ribbon cable or other set of wires 730 for carrying electrical signals to control the liquid lens 728. Other lens configurations may also be used.

[0099] By changing the focus of the liquid lens, various static and interactive optical vision tests can be performed. Adjusting the liquid lens can also allow users with poor vision to use the VR system without the need for glasses or contact lenses. By adjusting the focus of the liquid lenses 708a, b, the apparent distance of the image displayed on the corresponding display 706a, b can also be changed. Due to the image parallax between the two images on the 3D display, this change in perceived apparent distance differs from distance perception. By providing two separate visual cues for virtual image distance, the immersiveness of the VR display can be enhanced. For example, a 3D VR image can be generated and displayed on dual displays 706a, 706b, with the positions of 3D elements in the left and right images selected to provide a desired apparent distance based on parallax. An eye-tracking camera in the headset can be used to determine the 3D element the user is viewing. The virtual distance from the user to the 3D element along the Z axis can be determined, and the focus of the liquid lenses 708a, b can be dynamically adjusted so that the 3D element the user is viewing appears to be at that virtual distance based on eye focus and parallax.

[0100] Go to Figure 8 , showing a particular configuration of an optical module 800 viewed from the side facing the user, which may be inserted into a device such as Figure 1 The head support 105 shown in the head support and can be Figures 7A-7D 1 and 2. An embodiment of the assembly shown, a modular optical module 300 or other configuration. The optical module 800 has an outer housing 802 having a clip 804 for connecting to the head mount 105. An inner housing 806 fits within the outer housing 802 and has a plurality of eye tracking cameras 808. In the embodiment shown, there are four cameras 808 for each eye. Each camera is positioned to image a respective quadrant of the user's eye. The lens assembly 810 is mounted in a frame assembly that may include left and right lenses 812 and a front frame 814. The lenses 812 may include the lenses 812 as described above with reference to FIG. Figures 7A-7D The composite liquid lens assembly in question.

[0101] A plurality of LEDs 816 facing the user are mounted in the module 800 and can be used to illuminate the user's eyes for eye tracking purposes and other reasons. Clusters of LEDs can be arranged along the perimeter of the frame assembly. For example, OLEDs and infrared LEDs, white LEDs and blue LEDs. In one embodiment, the LEDs are controlled as a group. In various embodiments, each LED cluster and / or each LED within a cluster can be controlled individually.

[0102] Figure 9A and 9B Sample individual and combined images are shown on displays 902a, 902b, respectively, each display having an image displayed thereon, and each image being intended to be viewed by the user's respective left or right eye when the user is wearing the headset. Figure 9B The combined image 904 in the figure represents what a normal user sees. Figure 9A How separate images in a VR environment are combined into a single VR environment. The images can be 3D or can reflect a flat scene at a set apparent distance from the viewer.

[0103] Software that controls VR headsets, such as Figure 7A The head-mounted system 700 shown can be used to present similar Figure 9A and 9B Images that are used for color vision testing in a virtual reality environment. Isolating a specific eye can be achieved by turning off one of the display screens, allowing a series of eye tests to be administered to one eye while the other eye sees only a dimmed screen. An internal opaque baffle extending vertically between the left and right image and optical assemblies on the user side of the headset can be provided to isolate the left and right image screens, so that each eye of the person wearing the headset sees only the image presented by its corresponding display. The baffle can be removed and installed using hook-and-loop or other fasteners or a sized friction fit, and is used when the headset is used for tests requiring eye image isolation. In one configuration, the baffle can be built into a removable front pad mounted on the user-facing side of the headset. When a series of optical tests are being run, the software can be configured to prompt the user to insert the baffle before performing eye tests requiring eye isolation, and to remove the baffle after completing these tests and before performing additional eye tests. Data from cameras within the headset, such as the eye-tracking camera 612, the vision camera 610, or other sensors, can be used to determine or signal the presence of the baffle. The testing software can then use the status of this value to determine whether a given test can be run properly.

[0104] exist Figure 9A and 9BIn the example image 902a, the image represents a planar scene. The system can display images with a specific overlap. The amount of overlap between images 902a and 902b can be a function of the apparent distance of the virtual image viewed by the user. In one test scenario, as described above, the liquid lens focus can be adjusted to provide 3D depth cues based on parallax or overlap between the images, while also adjusting the apparent distance of the virtual image based on eye focus. The overlap between the displayed images can also be adjusted to test the user's ability to blend the two images into one.

[0105] In an embodiment suitable for a VR headset, a viewpoint rendering process implemented by software and / or firmware generates the images displayed on each display of the headset. Images from one or more eye-tracking cameras 612 mounted in the headset are processed using conventional techniques to determine where the user's gaze is directed in each image. The area of ​​each image where the user's pupil is directed is rendered at a high resolution, such as at or near the maximum resolution of the display. The peripheral portion of the image where the pupil is not directed is rendered at a lower image resolution than the center of the gaze.

[0106] The viewpoint rendering can move with the position of the guided pupil or the user's line of sight on the screen. The system can be configured to receive input from the eye tracking camera 612 at a predetermined rate and refresh the image at the same rate for viewpoint rendering. The image can be displayed repeatedly, for example at a frequency of 60 to 90 Hz, which can be equal to or greater than the predetermined frequency. The reduction in image resolution based on the distance of the point in the image from the user's line of sight can be changed in a variety of ways, such as by a smooth change or a piecewise function. The resolution can be reduced in a circular pattern, a rectangular pattern, or other appropriate and desired form.

[0107] Viewpoint rendering allows for a more efficient display system because system resources are not required to generate the entire image at a high or maximum resolution level. The resources required to generate and display a high-resolution image are used for areas in the image that the user is directly looking at, which will be rendered at a high resolution. Images along the periphery of the user's line of sight are rendered at a lower resolution and require fewer system resources to generate. The resources required to display the complete image are therefore reduced. This can allow the system to, for example, be implemented using a slower processor and / or to provide images at a higher frame rate than would be possible if the entire image were rendered at full resolution. Viewpoint rendering can be used in a variety of vision testing scenarios. It can also be used in other VR environments where complex visual displays, such as interactive 3D environments, are continuously presented.

[0108] Go to Figure 19A and 19B, showing an improved design of a retinal image display (RID) incorporating a digital micromirror device (DMD) that can produce virtual images with high pixel density and large field of view. The RID can be made small enough to be incorporated into a head-mounted machine in VR form for eye testing as noted herein.

[0109] Figure 19A A first embodiment of a RID 1900 is shown. A light source 1902 produces light of various colors and can be controlled by external control circuitry (not shown). The light source 1902 can include a plurality of LEDs, including one or more of red, green, blue, and white light emitters. Other light sources can be used instead. Light 1904 produced by the light source 1902 is directed into an integrating rod 1906, which serves to homogenize the incident light 1904 and produce a rectangular light beam 1908 having a predetermined aspect ratio. A conventional integrating rod can be used. In a particular embodiment, the integrating rod has a length between ½ inch and 1 inch, such as 5 / 8 inch, and has a height and width between 1 / 8 inch and 3 / 8 inch, such as ¼ inch each, to output light having a square cross-section. The dimensions of the integrating rod 1906 can be selected to provide an exit light having an aspect ratio compatible with the micromirror device used.

[0110] The light beam 1908 exiting the integrating rod 1906 is focused using one or more lenses, such as bi-convex lenses 1910, 1912, to produce a focused light beam 1918. To reduce the overall length of the RID 1900, a pair of opposing fold mirrors 1914, 1916 can be used to redirect the light exiting the first lens 1908 to the second lens 1910. The lenses 1910, 1912 can be oriented so that their respective optical axes are parallel to each other. The fold mirrors 1914, 1916 can be mirrors. Alternatively, the mirrors 1914, 1916 can be replaced with prisms, with the flats of the prisms oriented perpendicular to the incoming light.

[0111] The focused light beam 1918 is directed to a beam splitter 1920, such as a partial mirror, which redirects a portion 1924 of the light beam to a reflective active face of a DMD 1922. The portion of the light 1918 not directed to the DMD exits the beam splitter 1920 in a different direction. A light-absorbing baffle 1927 can be provided to absorb this extra light to prevent internal reflections from affecting the image seen. The beam splitter 1920 can be a plate having a thickness of 0.5 to 0.55 millimeters and positioned so that the incoming incident light is at an angle of about 30 to 40 degrees from the reflected light reflected from the DMD.

[0112] DMD control circuitry (not shown) is used to control the micromirrors of the DMD to modulate the incident light to produce the desired image. Conventional DMD modules are commercially available and can be used, for example, a DMD chip with a 0.66-inch diagonal micromirror array having a 5.4-micromirror pitch and producing a 4K UDH resolution (840 x 2160 pixels) image. The DMD modulated light is reflected back through beamsplitter 1920, a portion 1928 passes through and into projection lens 1930, which focuses the light into the viewer’s eye 1932. Advantageously, system 1900 can produce an immersive near-eye display of virtual images with a large field of view ranging from 40 degrees to 200 degrees.

[0113] Figure 19B An optional RID module embodiment is shown in which beamsplitter 1920 comprises a total internal reflection (TIR) prism assembly 1940. Figure 19C A further view of prism assembly 1940 is shown. Prism assembly 1940 comprises a pair of right-angle prisms 1942, 1944. Prism 1942 has a prism angle A. Hypotenuse 1950 of first prism 1942 is adjacent hypotenuse 1954 of second prism 1944. Prisms 1942, 1944 are composed of different optical materials having different angles of refraction. Prism 1942 has an index of refraction Na, prism 1944 has an index of refraction Nb.

[0114] Prism assembly 1940 is positioned such that incident light 1918 enters base side 1952 of prism 1942 at an angle of incidence θin relative to a line normal to base 1952, the light exits side 1956 of prism 1944 at an angle θDMD relative to a line normal to the active mirror surface of DMD 1922.

[0115] TIR prism assembly 1940 prism (1940) directs an incoming or incident light that enters surface (1942) at an angle relative to an axis normal to surface (1942). Upon entering, the incident light enters the reflective-transmissive surface 1950 of prism 1942 (1950), then transmits out of prism 1942 TIR prism (1940) to DMD 1922 (1922). The light ray then reflects from DMD (1922) back to TIR prism assembly 1942 and through surface 1956 (1940), transmits through surface (1952) and into TIR prism (1940). The reflected light then transmits through the interface between hypotenuse 1950 of prism 1942 and hypotenuse 1954 of prism 1944 (1950 and 1954). The transmitted light passes through prism 1944, exits through entrance 1956, then transmits through surface 1958, out of TIR prism assembly 1940, where it can be directed to an eye.

[0116] The angles of the TIR prism are right angles (90 degrees), while angles "A" and "B" are complementary angles that sum to 90 degrees and can vary depending on the incident direction and the physical position of the DMD 1922. Suitable configurations for the TIR beam splitter 1940 (1922) are known to those of ordinary skill in the art.

[0117] Such as including Figure 19A and 19B The RID module of the design shown can be an integrated part of the entire VR headset, or set in a Figure 2A The RID can be packaged as a removable submodule, such as Figure 3A Submodule 310.

[0118] The brightness of an LED-illuminated projection system can be easily adjusted by modulating the current intensity of the LED.A pair of RID display units can be provided in the headset, one for each eye, where the user will see only one image.

[0119] A mechanism can be provided to adjust the relative position of the two RID units seen by the user based on the interpupillary distance (IPD) between the centers of the pupils of the two eyes. This adjustment can be manual, for example, via a knob located on or near the headset, or electronically controlled by a small servo motor located within the virtual reality headset. A camera in the headset (e.g., an eye-tracking camera at a known location within the headset) can be used to capture data from which the IPD can be calculated. The position of the RID display unit can then be automatically adjusted in response. A physical servo motor, linear actuator, or similar device can be used to adjust the position of the RID display unit until software monitoring the position of the RID unit relative to the user's eyes (e.g., as seen by the eye-tracking camera) indicates that the RID display is properly aligned with the eyes. The position of the RID unit can then be stored and used to reposition the RID display for the same user at a later stage. For a known IPD, an optical encoder can be used to position the RID display at a predetermined position. The encoder can provide absolute or relative position. In addition to the encoder, the known displacement of the RID unit for a given "tick" of the servo motor can be used to determine when the RID has been displaced the appropriate amount for the known IPD.

[0120] For example, the aforementioned headset system with a wide-angle RID display or other wide-angle display system can be used to quickly and efficiently perform visual field testing on a patient. Visual field testing is used to determine the entire area a user's eyes can see (visual field) when their gaze is focused on a single point, including where the patient's peripheral vision begins and ends, and how well they can see objects in their peripheral vision. A person's normal field of view spans approximately 120 degrees of arc. The wide-angle display headset system disclosed herein is particularly well-suited for performing automated visual field testing.

[0121] The test pattern can be presented at different locations within the field of view, and patient feedback indicating when the pattern becomes visible is detected and recorded. Using a VR headset to conduct the test allows the user to move their head while maintaining the position of the test field relative to the patient's eyes. This can reduce patient stress during the test.

[0122] There are various general test modes for testing. The modes include:

[0123] *10-2: Measure 10 degrees temporally and nasally, testing 68 points (commonly used to test macular, retinal and neuro-ophthalmological conditions and advanced glaucoma);

[0124] *24-2: Measure 24 degrees temporally and 30 degrees nasally, testing 54 points.

[0125] *24-2C: 24 degrees plus 10 degrees measured on the temporal and nasal sides, testing 64 points; and

[0126] *30-2: 76 test points measured 30 degrees nasally to the temporal side. This test pattern covers the central visual field and follows the vertical and horizontal meridians. The test point locations are equidistant from one another and spaced at a fixed number of degrees; for example, the 30-2 and 24-2 are separated by 6 degrees. The 24-2 pattern is based on the 30-2 pattern, but with the exception of the two nasal points, the outermost ring of test points is removed.

[0127] According to one aspect of the present invention, the VR headset system is programmed with multiple different test modes. The physician can manually select a given test mode to be used, or select a series from a set of modes programmed into the system. The software will present visual stimuli to the patient according to the selected test mode and record the results. Alternatively, the system can be programmed to automatically go through a series of selected tests. According to a particular feature, when multiple different tests are to be automatically performed, the system can randomly select data points from the complete set of data points available for each test, so that the various tests are essentially merged together and run in parallel. When the test results are completed, the test data can be output to a remote computer storage area, such as the control system 16 or the system 20. The data can also be stored in a local or remote data storage 18 for later reference.

[0128] The eye-tracking camera system in the headset can be used to determine when the patient's gaze is directed toward a target, such as a dot or small image in the center of their field of view. At the start of the test, the system monitors eye gaze to determine when the patient is steadily fixating on the central target. Once this condition is met, the test can begin.

[0129] During the test, the eye tracking data can also be used to determine when the patient's line of sight deviates from the central target. In one embodiment, when the software detects a line of sight deviation, the test can be paused and the patient is automatically instructed to return their line of sight to the target. The test can be resumed when this condition is met, for example by analyzing the eye tracking data. Alternatively, the eye tracking data can be used to determine the angular deviation of the user from the central target. This deviation can be used to generate a correction factor that will be applied to the deviation of the test data point being displayed. For example, if a data point for a given test deviates 30 degrees from the center, but when the patient indicates that they have seen the data point, their line of sight was deviated 10 degrees from the data point, then the detection can be considered to have applied to a data point with only a 20 degree deviation. The software would not consider that the 30 degree deviation data point has been tested, and therefore the point can be retried again. Similarly, if a line of sight deviation is detected prior to the data point being displayed, the angular position of the data point can be adjusted by the deviation so that the relative deviation of the data point when displayed is as intended. In the example above, with a 10 degree line of sight deviation detected, the 30 degree deviation data point in the direction of the deviation can be displayed at a 40 degree position so that the actual deviation of the patient is the correct 30 degrees.

[0130] A common and important eye exam involves taking a photograph of the retina using a fundus camera. Fundus optical equipment in conventional designs is bulky and expensive, particularly in systems where the camera is designed to provide a wide field of view within the eye in order to image as much of the retina as possible at once. According to one aspect of the present invention, a miniaturized fundus camera is provided. The fundus camera can be mounted within a VR-style headset, such as the dedicated configuration headset 12 or as part of the removable optical module 110, 300, and used to provide both narrow and wide field retinal imaging capabilities. As the fundus camera takes a set of images, data can be collected from the real-time video of the eye. Eye tracking and image processing can be used to combine the images captured by the camera to generate an image covering most or all of the retina.

[0131] The fundus camera components can be movably mounted within the headset to allow the camera aperture to be positioned at various locations in front of the user's eye. In one configuration, the fundus camera is mounted on a pivotable arm within the VR-style headset. Pivoting the arm allows the camera aperture to be moved in front of the eyes of the person wearing the headset so that retinal images can be captured. The camera can then be moved away and at least partially out of the field of view, for example, to allow viewing of a video display that can also be included in the headset. In a further configuration, the fundus camera is mounted on a rotatable pivot that is positioned on a track along a portion of the perimeter of the headset, providing additional degrees of freedom for positioning the camera.

[0132] A mechanical connection can be provided to allow direct manual adjustment of the position of the fundus camera within the headset. Where the fundus camera is movable, a servo motor, linear actuator, or other mechanism can be provided to allow the camera to be moved by remote control. Software can be used to correctly position the fundus camera directly in front of the user's eye. The software can process images from the eye tracking camera. Images from the camera in the fundus camera can also be used. Real-time displays of the eye tracking and / or fundus camera images can be displayed on a remote PC (or external screen display on the VR headset as shown in Figure 4 The person administering the test can use computer inputs or other inputs to manually control the position of the camera to align it with the user's eye. The software can also be configured to analyze the eye tracking camera and / or fundus camera images and use that data to automatically position the fundus camera in the correct position in front of the user's eye.

[0133] As discussed further below, during the fundus imaging system, the fundus camera will capture in vivo images of the interior of the eye during normal eye saccades or while the eye is moving. At the same time, the eye tracking camera can be used to track the direction of the patient's eye. The eye position can be recorded in a spherical, Cartesian, or another coordinate system and the position data stored in a manner that allows the eye position at a given time to be correlated with the image captured simultaneously by the fundus camera. The eye coordinate information is used to determine the portion of the retina captured in the corresponding fundus image, thereby determining which fundus images are adjacent to one another. The in vivo images are segmented and a mosaic created using the coordinate information collected from the eye tracking camera to fit the segmented images together. Image processing algorithms then combine the segmented images to create a wide field view of the visualized fundus. By combining the eye tracking data with the eye images, the eye images can be stitched together to create a complete image of the data collected; and with an expanded field of view on each eye, for example, a horizontal field of view of up to 120° and a vertical field of view of up to 135°.

[0134] Figure 10A and 10BThe front and rear views of a fundus camera module 1008 suitable for installation in a headset are shown, respectively. The camera module 1008 has an elongated housing 1040. A camera objective lens 1050 is located on the front side 1052 of the housing, through which a camera inside the housing 1040 can capture images of objects in the field of view in front of the objective lens 1050. The housing 1040 can be made small enough to allow the camera module 1008 to be installed within the VR headset 12. For example, the housing 1040 can have a length between 43 mm and 58 mm, a height between 11 mm and 22 mm, and a depth between 22 mm and 33 mm. The specific configuration of the internal components of the fundus camera is described separately below. Alternatively, rather than providing the camera 1008 in its own dedicated housing 1040, some or all of the fundus camera components can be integrated into other components of the headset.

[0135] The position of the eyes on the face varies from person to person. To address this issue, the camera module can be removably mounted within the headset 12. Various mounting techniques can be used. In one embodiment, a mounting shaft 1054 extends rearwardly from a rear surface 1056 of the housing 1045. The shaft 1054 can mate with a cylindrical hole in the headset structure to rotationally couple the camera module 1008 to the headset, allowing the camera module 1008 to be repositioned as needed to match the patient's eye position. Power and data connections to the camera components within the camera module 1008 can be established via one or more cables and data lines 1058 extending from the housing 1040 and integrated within the mounting shaft 1054, or by other means. Alternative structures known to those skilled in the art can also be used to mount the camera 1008 to the headset. For example, the camera housing 1040 can have a hole that accommodates a mounting shaft extending from a portion of the headset.

[0136] The fundus camera module 1008 can be fixedly or removably mounted to the interior of the headset. One or more camera modules 1008 can be installed for fundus imaging and then removed, allowing the headset to be used for other purposes. The fundus camera can also be configured and rotatably mounted in a position within the headset to allow the camera 1008 to be partially or completely moved out of the user's line of sight, allowing the headset to be used for other purposes while still having the fundus camera installed.

[0137] Figure 10C An exploded view of an optical module 1000 is shown that can be mounted in the outer frame 115 of a head mounted system and includes a pair of fundus camera modules 1008a, 1008b rotatably mounted to a frame assembly 1010. The illustrated configuration of the optical module 1000 is similar to Figure 7AThe optical module is shown and may include a main board 1004 that includes computer circuitry for controlling various components of the fundus cameras 1008a, b and other system components and communicating with external devices.

[0138] In some configurations, it may be useful to provide image display capabilities as well as a fundus camera system. Left and right images can be provided by a pair of visual displays 1006a, 1006b mounted behind the fundus camera 1008. During the actual fundus imaging process, no visual images should be displayed on the display because a dark interior is required to allow the user's pupil to fully dilate. However, in the case where the fundus imaging camera is not sensitive to infrared illumination (e.g., using an IR blocking filter), an eye tracking system with infrared illumination can still be used. If displays 1006a, 1006b are provided, a lens module assembly 1012 may also be included. Although two fundus cameras are shown, some embodiments may include only one camera. For example, the fundus camera can be mounted between the left and right eye areas on the frame assembly 1010 and mounted so that it can swing back and forth to be positioned in front of either eye. This is discussed further below.

[0139] Various other components may also be included in the optical module, including LEDs and an eye-tracking camera, which may be mounted on circuit board 1004 or other components. For example, the eye-tracking camera may be mounted within rear frame 1002, and the LEDs may be mounted on portions 1012a, 1012b of frame assembly 1010. Electrical connections between these components and the driver circuitry may be provided using techniques known to those skilled in the art.

[0140] Various other headset configurations with integrated fundus cameras can be provided, and various mechanisms for mounting the fundus camera (fixed or movable) within the headset can be used. In addition to being rotatably mounted, the fundus camera can also or alternatively be slidably mounted to allow adjustment of the camera position. In order to allow the fundus camera to be completely moved out of the display screen 1004, modifications may need to be made to the standard VR headset form factor. For example, the top, bottom, or sides of the housing elements 1016, 115 may need to extend upward or in other directions to provide space to accommodate the fundus cameras when they are not in use.

[0141] Figure 11A and 11b show a headset system 1100 viewed from the side facing the user, which includes a right fundus camera 1108 mounted inside the headset and corresponding to Figure 10C The assembly style of the configuration. Figure 11AFigure 11a shows the left and right fundus cameras 1108 in a retracted position, while Figure 1 lb shows the fundus cameras 1108 in a deployed position, with the objective aperture 1109 of each camera moved into position for retinal imaging. The fundus cameras 1108 as shown have the same form factor as the cameras 1008 shown in Figure 10A and 10B Figures 1a and 1 b. Alternative form factors can be used for the cameras 1108 and the overall headset design. The cameras 1108 are configured in the headset so that they can be positioned to be centered closely in front of the pupils, for example at a distance of one centimeter. In particular embodiments, the fundus cameras 1108 can be rotated 35 to 65 degrees from a retracted or retracted position to the in-use position.

[0142] The headset 1100 has a plurality of eye tracking cameras 1110, which can be infrared cameras. The frame rate of the cameras 1110 can be fixed or adjustable, for example in the range of 50 Hz to 100 Hz. In the illustrated embodiment, there are four eye tracking cameras 1110 for each eye. Each eye tracking camera 1110 is positioned to allow imaging of a respective quadrant of the user's eye. The eye tracking cameras 1110 are positioned so that when the fundus cameras 1108 are engaged, one or more of the infrared eye tracking cameras for each eye can view and record pupil movement for that eye.

[0143] The headset 1100 can include a plurality of LEDs 1116 facing the user, which can be used to illuminate the user's eyes for eye tracking purposes and other reasons. The LEDs 1116 can be arranged in clusters around the periphery of the frame assembly 1114 within the headset. A given cluster can include different types of LEDs, for example OLED and infrared LEDs, white light LEDs and blue light LEDs. The LED clusters can be configured to be controlled as a group, or each cluster of LEDs 1116 and / or each LED 1116 in a cluster can be controlled individually. Infrared LEDs can also be provided for illumination of the eye tracking cameras 1110.

[0144] During the fundus imaging phase, the eye tracking and fundus camera retinal image capture routines can run simultaneously, while allowing the user's eyes to freely roam. The pupils trace a path that is roughly similar to the path being traced on a half-dome. The eye tracking cameras 1110 periodically capture the position of the pupils, and can process the images to determine the position of the pupils and when the position of the pupils changes. The infrared reflections from the pupils are greater than the reflections from the iris and sclera, thus allowing the cameras to more easily distinguish the pupils on the surface of the eye. The images captured with the eye tracking cameras 1110 can be processed using conventional eye tracking algorithms, including corneal reflections, binocular and dark pupil tracking. Parallax compensation can also be implemented.

[0145] The fundus camera 1108 typically only images a portion of the user's retina at a time. During the image capture process, the user can be instructed to change the direction of their gaze, thereby changing the area of the retina that is imaged by the camera. The instructions can be audible and / or visual. For example, the user can be given an audio prompt, such as telling them to look up, left, right, down, or forward, as appropriate. One or more visual LEDs 1116 can also be turned on or flashed to indicate the direction in which the user should look. These LEDs can glow dimly so that they are visible but do not emit enough light to cause the user's pupils to constrict.

[0146] Multiple overlapping retinal photographs taken with the fundus camera 1108 can be combined using conventional image stitching algorithms and image recognition software to provide a single image of a large portion of the retina. Using the system disclosed herein, images can be captured and combined to generate a retinal image from a horizontal field of view of up to 120° and a vertical field of view of up to 135° behind the eye.

[0147] The portion of the retina that is imaged can be determined from information about the direction of the user's gaze and the relative position of the fundus camera. By using this data to determine how the multiple images should be positioned relative to each other before stitching, the speed and accuracy of the image stitching can be improved. The fundus camera position and eye tracking information can be stored in a separate data stream with appropriate time stamp information to allow it to be matched to the fundus camera images. Alternatively, this information can be added as metadata to each image frame captured by the fundus camera, so the recording of the fundus camera image stream will also carry this additional data. The metadata can be added to the fundus images by software executing within the headset or in an external system.

[0148] The system can be programmed to determine when sufficient images of sufficient quality of the retina have been captured so that all areas of the target retina are present in at least one image of acceptable quality. In normal circumstances, an average of 500-720 fundus image frames (24 frames per second, 30 seconds of video length) captured by the fundus camera 1108 as the user moves their eyes can provide sufficient imaging. If the processing software determines that there are areas of the retina that have not been fully imaged, additional audio and / or visual prompts can be triggered after the sequence of fundus images has been captured, requesting the patient to change the direction of their gaze.

[0149] Figure 12is a high-level block diagram of an eye tracking system 1200 incorporating a fundus camera, as described above. The system includes a head-mounted fundus camera, such as camera 1108 described above. A control system 1204 contains the hardware and software that controls the fundus camera 1108 for initial image capture and processing, and can be a processor board with appropriate hardware and software integrated into the headset and connected to the camera 1208, as described above. A remote image processing and analysis system 1206 can be connected to the control system 1204 to provide additional functionality. The control system 1204 and the analysis system 1206 are similar to Figure 1 The control system 16 and the analysis system 20 are provided.

[0150] Figure 13 1306 is a high-level flow chart of a retinal imaging method for generating a complete retinal image using the fundus scanning system disclosed herein. In an initial step, a digital image from a fundus camera is received and captured (step 1302). In addition to capturing the image, the system can also capture information about the position of the user's eyes, which is determined by analyzing eye tracking images and the position of the fundus camera when a particular image was taken. As described above, these data points can be sent to an image processing system linked to the captured images. As the user changes the direction of their gaze (e.g., in response to instructions), images continue to be captured. The captured fundus images are synchronized with the eye and camera position data. In addition to processing the eye tracking images in real time, the eye processing images can be stored with the captured fundus images and processed further at a later time. The image timestamps stored with the captured eye and fundus camera images can be used to synchronize the images.

[0151] Initial low-level image processing of the captured images is performed. This processing may include flattening to correct for radial distortion (step 1304) and general polishing / cleaning of the images using conventional image processing filters known to those skilled in the art (step 1306). Initial pattern recognition is performed to extract detail from the captured images, which is subsequently used during image stitching (step 1308). Image matching is then performed to determine the best match of descriptors between frames, and frames with similar features may be grouped together. While image alignment can be performed based solely on pattern matching, the accuracy of this alignment can be improved by using the fundus camera and eye position associated with the images as an initial alignment, followed by smaller adjustments to the alignment based on feature matching. Conventional pattern recognition software can be used to refine the alignment, for example by grouping and adjusting the image alignment based on feature matching. Other techniques known to those skilled in the art for grouping and positioning images may also be used (steps 1310, 1312). This alignment can be performed relative to a concave spherical surface using the image position and orientation relative to the retinal region represented by the image. Image quality may also be assessed prior to alignment, and images with quality values ​​below a given threshold may be discarded. For example, pictures determined to be blurry due to eye movement may be discarded.

[0152] The image capture process continues until the system determines that the area of ​​the retina to be imaged is adequately captured in one or more images (step 1314). For example, the system may continue capturing images until the unstitched but positioned images are arranged on top of each other without gaps. In one embodiment, the camera is moved and / or the user is instructed to move their eyes in a sequence designed to provide image capture of the entire area of ​​the target retina. After a preset sequence of image captures, the system may detect an imaging gap. In this case, an appropriate signal may be generated to move the camera and / or instruct the user to move their eyes to a position that allows the gap area to be imaged. Various other methods of determining that enough pictures of the correct area have been taken may be used, and one or more of the flattening, polishing, pattern recognition, pre-stitching, and image matching steps (1304, 1306, 1308, 1310, 1312) may be performed before or after this determination.

[0153] The selected and arranged images are then stitched together to create a combined image of the retina (step 1316). During the stitching process, if more than two frames need to be stitched together, bundle adjustment and blending can be performed to ensure that there are no visible seams and undesirable black areas when a frame is matched with multiple other frames. In areas where two or more frames overlap, the combined image can include a composite of the overlapping areas.

[0154] Various conventional image enhancements can be applied to improve overall image quality and enhance target details (step 1318). The final images can then be stored and analyzed to determine if there are any areas of interest. Machine learning and Al software, properly trained, can also be used to process the images and identify potential issues, diseases, or other areas of possible interest (step 1320).

[0155] According to another aspect of the application, with reference to Figure 14A The fundus camera 1400 can be rotatably mounted in the headset by a rotatable mounting assembly 1440. The mounting assembly 1440 can be driven by a motor, and the motor can be used to control the position of the fundus camera in order to place the objective lens in front of the eye and align it with the pupil. The rotatable mounting assembly 1440 includes a first portion 1450 that is attached to the fundus camera housing (e.g., at the rear wall 1402) and a second portion 1460 that is used to drive rotation of the first portion and that is attached to the headset. Various motor drive configurations can be used. For example, the motor can be mounted to the second portion 1460, and the first portion 1450 is mounted directly on the motor shaft or is connected indirectly, e.g., using gears or belts driven by the motor or a linear actuator or other device.

[0156] Optionally, the mounting device can utilize a motor or other means for causing rotation that is mounted to the fundus camera at the portion 1450, and the axis about which the camera rotates is connected to the headset, either directly or indirectly, through the second portion 1460. While the mounting assembly 1440 is shown mounted on the rear wall 1404, the rotating mounting assembly 1440 can be mounted elsewhere on the housing, e.g., on a side of the housing or even on the front of the housing.

[0157] A rotary encoder can also be integrated or otherwise coupled to the rotating motor or other element and used to generate a signal that provides an indication of the position of the fundus camera. Optionally, position markers can be printed on the housing of the fundus camera, and an optical sensor is used to detect the markers in order to determine the amount of rotation of the camera. Other ways of detecting the position and / or motion of the fundus camera can also be used. In a dual camera system, the two cameras can be moved synchronously or asynchronously.

[0158] Figure 14B and 14C A particular embodiment is shown in which a differential gear mechanism is used to allow the fundus camera to rotate about a pivot axis. Figure 14B is an illustration of a fundus camera with a gear mounting assembly 1404 that is connected to the rear wall 1402 of the fundus camera housing at a support plate 1408. The gears can be driven by a motor 1406. Figure 14C and 14DThey are Figure 14B Side and end views of an embodiment. Figure 14E Is displayed with Figure 14B X-ray image of a portion of a headset with a fundus camera. Mounting assembly 1404 allows the fundus camera 1402 to rotate about a pivot axis aligned with the axis of the motor shaft. A variety of motors can be used. In one embodiment, motor 1406 is an electric coreless vibrating DC motor. The motor can be driven by other circuitry in the headset to allow the physician to automatically or manually adjust the position of the fundus camera using a suitable software interface. Motor 1406 can be mechanically connected to the internal structure of the headset, for example, by mounting it on the Figure 10C The rotatable bracket assembly is shown within a hole in the frame assembly 1010 of the headset assembly. The mounting position of the rotatable bracket assembly on the fundus camera and within the headset can be varied. In the illustrated embodiment, the fundus camera is mounted at the upper outer corner. The fundus camera can also be mounted in various other locations. For example, the fundus camera can be mounted outside between the top and bottom, from the lower outer corner, or from any other position that allows the fundus camera to be moved so that the objective lens is in front of the user's eye.

[0159] Figure 15A and 15B A top view and a bottom view, respectively, of an embodiment of a gear mounting assembly 1404 are shown. The gear mounting assembly includes a main motor gear 1504 having motor spokes 1502 that allow the gear 1504 to be attached to a motor. The main motor gear 1504 drives two internal gears 1506 that are located within an outer ring gear 1508 that is attached to a support plate 1408. An inner surface 1514 of the support plate 1408 is secured to the rear surface 1402 of the fundus camera housing. Appropriate holes in the housing surface 1402 allow the shafts 1516 and mounting pins 1512 of the internal gears to pass through. Rotating the main motor gear 1504 causes the ring gear 1508 to rotate, thereby rotating the fundus camera, reference Figure 11A and 11B , so that the imaging objective 1109 can be positioned at any desired position along an arc centered on the pivot point. Although a separate support plate 1408 is shown, the support plate 1408 can be integrally formed in the camera housing or omitted entirely, and the ring gear 1508 can be fixed to or integrally formed in the wall of the headset housing.

[0160] An alternative mechanism for rotatably mounting the fundus camera can also be used. In one configuration, instead of a motor, a knob is located on the outside of the headset and mechanically coupled to the fundus camera. For example, the physician can manually adjust the fundus camera's position by rotating the knob while using real-time video from the camera as positioning feedback.

[0161] Rotatably mounting the fundus camera to position the objective lens along an arc can still not allow for optimal positioning. In another embodiment, referring to Figures 16A-16E Rather, the portion of the mounting assembly 1460 (which can be the motor 1406, the motor shaft, or other elements depending on the mounting assembly design, if the motor body is mounted on the fundus camera housing) is itself further movably mounted within a track that allows for lateral movement of the pivot point of the mounting assembly. By adding an additional degree of freedom to the positioning of the fundus camera, the objective lens portion of the camera can be positioned in a wider range of positions compared to a fundus camera that can only move rotationally.

[0162] Figures 16A-16D A dual camera assembly is shown, where each fundus camera 1400 has a respective mounting assembly 1610 for rotationally coupling the fundus camera to the headset such that the fundus camera 1400 can be rotated relative to the headset. The mounting assembly 1610 is slidably mounted to the headset within a respective track 1602 having a first end 1604 and a second end 1606. The mounting assembly can be positioned at a plurality of positions along the respective track 1602, such as at the first end 1604 and the second end 1606 of the track 1602 or different positions therebetween. At different positions in the track 1602, the fundus camera 1400 can be rotated to adjust its position. Figure 16A and 16B The fundus camera 1400 is shown in a first position 1604 in the respective track 1602 and two different rotational positions. Figure 16C and 16D The fundus camera 1402 is shown in a second position 1604 in the respective track 1602 and two different rotational positions.

[0163] Various mechanical drive systems known to those skilled in the art can be used to move the slidably mounted portion of the mounting assembly 1610 along the track 1604 to different positions. In one configuration, a movable belt driven by a separate motor can be used. In another configuration, a linear actuator can be used to change the position of the slot. Likewise, a worm gear or other system can be used to slide the slot mounted portion of the mounting assembly 1610 back and forth. Other motion mechanisms can also be used. The system can be configured to position the pivot point of the camera at predetermined fixed positions, such as the ends 1604, 1606 of the slot 1602, or to allow the pivot to be freely positioned at variable positions along the track 1602.

[0164] In Figure 16EIn another configuration shown, instead of using two fundus cameras, only a single fundus camera 1402 is used. The pivot axis can be located centrally, for example, along the top center of the headset, so that the fundus camera 1402 can be rotated in front of either eye. To provide additional degrees of freedom in positioning the camera, tracks 1610 can be provided to allow the fundus camera to move left and right so that it can be positioned in various additional locations on the left and right sides of the headset, similar to Figures 16D-16D The dual-camera / dual-track system shown.

[0165] In an alternative embodiment, instead of rotatably mounting the fundus camera, the mounting assembly may be slidably attached within a slot or track formed on the fundus camera, thereby allowing the position of the fundus camera relative to the mounting assembly to be adjusted along a first axis, such as in a vertical direction. The portion of the mounting assembly coupled to the housing may also be slidably mounted, allowing movement along a second axis, such as in a horizontal direction.

[0166] Figure 17 17 is a high-level schematic diagram of a fundus camera 1700 that can be miniaturized and used in the headset assembly discussed herein, and can also be used in other configurations, such as a handheld imager. The fundus camera 1700 has a housing 1702 that contains a light source 1704 and an image capture camera 1726. The light source 1704 emits light 1705, which is directed into an optical assembly 1710 that directs at least a portion of the light 1705 out of the camera through a portion 1725 and into the patient's eye 1730 (when the camera is in use and aligned). The light illuminates the retina 1732. Light reflected by the retina (or other structures within the eye) leaves the eye, passes through an objective lens portion 1725, and enters an optical assembly 1720 that directs at least a portion of the light to the camera that captures the image.

[0167] Figure 18A 1 is a top view of a design for a fundus camera 1800, which can be made small enough, for example, only a few centimeters long, to fit in a headset as described above (either by mounting it in a fixed or movable bracket). The fundus camera 1800 can also be used in systems other than the VR headset configuration discussed herein. Figure 18B and 18C yes Figure 18A Exploded perspective view of the design.

[0168] refer to Figures 18A-18CThe fundus camera 1800 components are mounted in a housing 1802 and include a light source 1804 that generates a light beam 1805. A flat glass 1806 and / or a polarizer 1808 can be positioned along the path of the light beam 1805 to reduce glare. A beam splitter 1810 is positioned along the path of the light beam 1805 and redirects the light beam 1805 and helps control light scatter and glint. A light absorbing surface 1812 can also be provided to remove glare and light reflections that arise from the beam splitter that are not desired to be output from the camera. The beam splitter can be operable to both redirect the light and reduce the intensity of the reflected light. This can allow the use of a bright LED as the light source 1804 that can be less expensive and easier to use than a low intensity LED of equivalent optical quality. A partial mirror or prism can be used as an alternative to the beam splitter.

[0169] In particular embodiments, the light source 1804 includes at least one LED that emits light at a wavelength that will excite blood vessels, the wavelength of light being from the group of the blue light region 450-500 nm, the violet light region 400-450 nm, and the ultraviolet (UV) region 200-400 nm. Multiple LEDs can be provided and control can be provided to activate them individually and in combination. The light source 1804 can include internal lenses or other optical components that are operable to focus or redirect the generated light to produce a light beam that exits the light source.

[0170] The light beam 1805 that is reflected from the beam splitter 1810 can be redirected by one or more mirrors, such as mirrors 1814 and 1816. A final mirror, such as mirror 1816, is positioned to direct the light beam 1805 at a 90 degree or other suitable angle so that the light beam 1805 exits the housing 1802. The light 1817 that is reflected from the mirror 1816 can pass through a lens 1818, such as a spherical lens that helps to compensate for eye focus and provide a wide field of view for image capture. A negative lens can also be provided to correct for aberrations. An external flat glass 1820 can be provided to protect the system from external contaminants. If the camera assembly 1800 is properly positioned, light will enter the eye of the user 1830 and illuminate at least a portion of the retina.

[0171] In the illustrated embodiment, light emitted from the light source passes through the flat glass 1806 and / or the polarizer 1808, then through the beam splitter 1810 that reduces the intensity of the light, and then the light beam is redirected by one or more mirrors 1814 and 1816. These elements can be positioned in other orders in the light beam. For example, the beam splitter can be positioned before the glass 1806 and / or the polarizer 1808.

[0172] At least one light directing mirror, such as mirror 1814, can be rotatably mounted along at least one axis to allow the direction of the light beam 1815 exiting the mirror 1814, and thus the light beam output from the camera, to be redirected. Redirecting the light beam by moving the mirror 1814 allows the system to illuminate different regions of the retina in the eye during image processing. This redirection also shifts the center of the camera's field of view, allowing for changes in the area of the eye that is imaged. Providing two movable mirrors that can be pivoted along different axes can allow the direction of the light beam to be redirected in two dimensions. The mirror positions can be automatically adjusted under the control of the system software. The eye and camera can take multiple images at fixed positions as the moving mirror is swept across the illumination of the portion of the retina visible to the camera and the imaging direction of the camera is adjusted to better capture images of those portions. The mirror positions can also be adjusted in response to changes in the position and / or line of sight direction of the eye. The combined images can provide a higher quality image with more uniform illumination.

[0173] Light 1821 reflected by the retina or other structures in the eye exits the eye, passes through the plate 1820, the lens 1818, and is redirected by the mirror 1816. A polarizing filter 1822 and a glass plate 1824 can be provided in the return light path to further reduce glare. A lens 1826 focuses the light onto an imaging camera 1826. The imaging camera 1828 can include an internal lens that is operable to focus incident light onto a sensor element. A stop or wall 1840 can be provided between the exit and entrance light paths to avoid stray light from the light source 1804 affecting the image.

[0174] For at least a portion of the camera internal light path, the exit light beam that illuminates the eye and the reflected light that is directed to the imaging camera travel along parallel paths. In the illustrated embodiment, for most of the light beam path, the axis of the light beam 1805 emitted from the light source 1804 remains parallel to the beam axis of the return light, except for the path between the beam splitter 1810 and the folding mirror 1814.

[0175] The imaging camera 1820 can comprise a conventional high resolution digital camera, for example 20 MP resolution with pixel size of 2.4 pm x 2.4 pm and frame rate of 24 fps. While an ag global shutter imaging system can be used, the rolling shutter feature advantageously allows the image capture to be not at a single instant in time, but rather by rapidly scanning the eye vertically and horizontally in succession. A high speed data port, for example USB 3.0, allows rapid transfer of data from the camera to other components in the optical module 110 or to external devices. The camera can be configured to run continuously free-running, or under control of hardware and / or software triggers. Software exposure control can be provided through appropriate APIs of the camera internal control circuitry. Those skilled in the art will know appropriate lens assemblies to provide a focused image of the retina on the imaging chip. Mechanical and / or liquid lens focusing systems can be used to compensate for distance variations of the user's eye. The camera system can capture static angle images of 45 degrees or more, and combine these to produce a retinal image of up to 120 degrees.

[0176] The fundus images captured using the system disclosed herein can be transmitted in real time to an image processing system and processed, for example as disclosed herein, to generate a complete 120 degree image of the eye.

[0177] While specific designs of switchable optical modules are disclosed herein, alternatively the system can be installed directly within the outer frame of the head support as part of a dedicated head mounted system. In this case the outer frame of the module would not be necessary and the components could instead be mounted directly into the outer frame of the head mounted system. In another configuration, a modular optical module is provided but as part of the manufacturing process the modular optical module is fixedly installed within the head support. In this way, custom VR head mounted systems with different functionality can be more easily manufactured on demand. It is also possible to provide custom head mounted systems with integrated fundus cameras, OCT systems and perimetry testing systems.

[0178] Various aspects, embodiments and examples of the present application have been disclosed and described herein. Modifications, additions and substitutions to those aspects, embodiments and examples can occur to those skilled in the art without departing from the spirit and scope of the present application as defined in the following claims.

Claims

1. A headset for optical inspection, comprising: a headset body configured to be worn on a user's face, the headset body having a front face side facing the user and a rear side opposite to the face side, a left side, a right side, a top side, and a bottom side; the facial side having a forward edge and a recess therein, the forward edge being configured to rest on the user's face when the headset is worn, the recess extending rearwardly from the forward edge and defining an open area having a rear and a peripheral side surface, the open area being adjacent the user's eyes when the headset is worn; a fundus camera comprising an imager and optics within an elongated housing, the optics including a camera objective, the fundus camera having a field of view extending outwardly from the objective along an imaging axis, the optics configured to direct light entering the camera objective to the imager; a mounting assembly rotatably coupling the housing to the headset body to permit rotation of the housing about a pivot axis, the housing extending laterally from the pivot axis into the open area, wherein the objective lens and the imaging axis are laterally offset from the pivot axis, and the fundus camera can be positioned such that the camera field of view extends from the open area toward an eye of a user; The rotation of the fundus camera moves the camera objective lens along an arc within the open area, and the rotation allows the position of the camera objective lens relative to the user's eyes to be adjusted when the user wears the headset.

2. The headset of claim 1, the mounting assembly further comprising a motor that, when activated, rotates the fundus camera housing relative to the headset body.

3. The headset according to claim 1, wherein the housing has a first end and a second end, a front side and a rear side, the camera objective lens is located on the front side adjacent to the first end of the housing, and the mounting assembly includes a first part and a second part that are rotatably coupled to each other; the first part of the mounting assembly is connected to the housing adjacent to the second end of the housing, and the second part of the mounting assembly is connected to the headset body.

4. The headset according to claim 3, wherein the second portion of the mounting assembly is connected to the headset body at a point midway between the left and right sides of the body; and when the user wears the headset, the fundus camera can be rotated from a first position suitable for imaging the left eye of the user wearing the headset to a second position suitable for imaging the right eye of the user.

5. The headset according to claim 1 , wherein the fundus camera comprises a left fundus camera rotatably mounted toward a left side of the open area and a right fundus camera rotatably mounted toward a right side of the open area, the left fundus camera and the right fundus camera being configured to respectively image a left eye and a right eye of a user when the user wears the headset.

6. The headset of claim 1 , the fundus camera further comprising a light source within the housing, the light source being configured to generate a light beam that exits the camera objective and is capable of illuminating the retina of the user's eye when the light beam passes through the pupil of the eye.

7. The headset of claim 6, wherein the fundus camera further comprises an exit mirror for directing the light source through the camera objective lens, the exit mirror being movable in at least one axis to change the direction of the light beam exiting the objective lens.

8. The headset according to claim 1, wherein the mounting assembly comprises a first portion and a second portion rotationally coupled to each other; A first portion of the mounting assembly is connected to the housing; The second portion of the mounting assembly is slidably mounted in a track formed in the headset body and is movable between a first track position and a second track position, wherein movement of the second portion within the track provides lateral movement of the fundus camera within the open area.

9. The headset of claim 1 , further comprising a plurality of eye-tracking cameras configured to capture images of a user's eyes when the user wears the headset, the eyes remaining visible to at least one eye-tracking camera as the fundus camera moves within the open area.

10. The headset of claim 1, further comprising a display located behind the fundus camera relative to a front side of the headset, the display configured to present a visible image to a user of the headset when the headset is worn.

11. The headset of claim 10 , wherein the fundus camera is movable between an imaging position, wherein the fundus camera obscures a first portion of a visual display from a user, and a stored position, wherein the fundus camera obscures a lesser amount of the visual display from the user than when the fundus camera is in the imaging position.

12. The headset according to claim 10, wherein the fundus camera is removably mounted within the headset body. 13 . The headset according to claim 12 , wherein the headset comprises an outer frame and an optical module, the optical module being removably mounted inside the outer frame, the fundus camera being mounted to the optical module.

14. A method for imaging the fundus of a patient's retina, comprising the steps of: A headset is provided, the headset comprising a headset body and a first fundus camera, the headset body having a face side, the face side having a front edge and a recess, the face side being configured to rest on a user's face when the headset is worn, the recess extending rearwardly from the front edge and defining an open area adjacent to a first eye of the user, the first fundus camera being mounted within the headset and rotatable along a first pivot axis, the first fundus camera having a first field of view along a first optical axis extending from a first camera objective lens toward the user's face, the first optical axis being laterally displaced from the first pivot axis, wherein rotation of the first fundus camera moves the first camera objective lens along an arc within the open area to change the horizontal and vertical positions of the first camera objective lens relative to the user's eye; Mounting the headset on the user's head; capturing a first image of the first eye by the first fundus camera; adjusting a rotational position of the first fundus camera based on the first image and by rotating the first fundus camera about the first pivot axis to change the alignment of the first camera objective lens with the pupil of the first eye; and After adjusting the rotational position of the first fundus camera, a first retinal image of the first eye is captured using the first fundus camera.

15. The method according to claim 14, wherein The step of adjusting the rotational position of the first fundus camera based on the first retinal image includes the steps of displaying the first image on a computer display of a computing device external to the headset, and receiving a control signal in the headset from the computing device operable to rotate the first fundus camera.

16. The method according to claim 14, wherein after adjusting the rotational position of the first fundus camera, the step of capturing the retinal image of the first eye comprises: capturing a plurality of retinal images of the first eye, each retinal image being of a respective portion of the retina; The method further includes the step of positioning each of a plurality of retinal images captured by the first fundus camera relative to a concave spherical surface, wherein an image position and orientation of each respective retinal image is associated with a retinal region represented by the respective image.

17. The method of claim 16, wherein the headset further comprises an eye-tracking camera, the method further comprising the steps of: capturing a plurality of images of the first eye with an eye-tracking camera within the headset during the step of capturing a plurality of retinal images of the first eye with the first fundus camera; analyzing images captured by an eye-tracking camera to determine the direction of gaze of the first eye; and wherein, The relative image position and orientation of each respective retinal image is a function of the determined line of sight direction and the position of the first fundus camera objective relative to the pupil of the first eye when the fundus camera captured the respective image.

18. The method according to claim 17, wherein The step of placing each of the plurality of retinal images captured by the first fundus camera further includes matching image features in the captured retinal images.

19. The method according to claim 16, further comprising the steps of: determining that a predetermined minimum portion of the retina has been imaged based on predetermined imaging criteria; After the determination, the step of capturing a plurality of retinal images with the first fundus camera is terminated.

20. The method according to claim 19, further comprising the steps of: The portion of the retina that has not been successfully imaged is automatically identified, and the patient is automatically instructed to change the direction of his or her gaze so as to bring the portion of the retina that has not been successfully imaged into the field of view of the first fundus camera.

21. The method according to claim 16, further comprising the steps of: The retinal images are stitched into a combined retinal image, and the combined retinal image is stored in a computer memory.

22. The method according to claim 16, further comprising the steps of: During the step of capturing a plurality of retinal images with the first fundus camera, the patient is instructed to change the direction of his or her gaze.

23. The method of claim 22, wherein the headset further comprises a plurality of visible light sources arranged around an inner periphery of the open area; and the step of instructing the patient to change the direction of gaze comprises illuminating at least one of the plurality of visible light sources to indicate the desired direction of gaze.

24. The method according to claim 14, wherein the headset further comprising a second fundus camera mounted within the headset, the second fundus camera being rotatable along a second pivot axis, and the second fundus camera having a second field of view along a second optical axis extending from a second camera objective lens toward a face of the user, the second optical axis being laterally displaced from the second pivot axis, wherein rotation of the second fundus camera about the pivot axis moves the second camera objective lens along an arc within the open area to change a horizontal and vertical position of the second camera objective lens relative to a second eye of the user; The method further comprises the following steps: adjusting the rotational position of each of the first fundus camera and the second fundus camera based on the interpupillary distance between the pupil center of the first eye and the pupil center of the second eye; and After the rotational positions of the first and second fundus cameras have been adjusted based on the pupil distance, a plurality of retinal images are captured using the first and second fundus cameras.

Images