Capture of grid of cardinal images for detection and monitoring of strabismus

Abstract

An eye examination apparatus comprising: a virtual reality device configured to be worn on a user's head, the virtual reality device having a housing and a display operable to display a fixation point to an eye of the user that corresponds to any one of a number of diagnostic gaze positions; and an imaging device coupled to the housing that is operable to capture an image of the eye viewing the fixation point for use in examining the eye for an ocular condition.

Description

CAPTURE OF GRID OF CARDINAL IMAGES FOR DETECTION AND MONITORING OF STRABISMUS Cross Reference To Related Applications

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 573,773, filed on April 3, 2024, the contents of which are incorporated herein by reference. Field

[0002] The subject matter of this disclosure relates to techniques for detection and monitoring of ocular pathologies, more specifically techniques for automatically capturing images of a patient’s eye and mapping the images in a format suitable for detection and monitoring of strabismus. Background

[0003] For some conditions, an eye care professional (ECP) collects a grid of photographs in which the patient looks at a center point and then outwards in forty-five- degree steps (aligned to the horizontal and vertical axes). This allows diagnosis of certain ocular pathologies such as strabismus. Strabismus refers to a misalignment of the eyes in which the eyes do not align to look at the same thing (i.e., incomitant gaze). For example, one eye is fixed on what the person intends to look at (the fixing eye) and the other eye is looking at something else (the deviated eye). An ECP may check for strabismus by holding the patient’s head still and having them follow a finger or light as it moves through various gaze positions. The ECP then compares the gaze of each eye and if the gaze of one eye is different than the other eye at one or more of the gaze positions, the presence of strabismus may be determined. This process, however, is time and personnel intensive. Additionally, it does not readily allow an ECP to identify pathologies that are dependent upon fixation of the eye. In addition, it may be difficult to synchronize the capture of the photo with the patient’s eye gaze. Such a difficulty may lead to only a summary of the test being recorded instead of photographic data being recorded. 3960-P498WO 1Summary

[0004] An aspect of the disclosure relates to an objective and repeatable technique for detecting and monitoring ocular pathologies. The technique offers the ability to easily control the eye fixation during an exam and to synchronize the capture of a photo of the eye with the patient’s eye gaze. Using a device that allows separate images to be presented to each eye (e.g., a head mounted display), fixation points are presented to which the user is instructed to attend. The fixation points may be presented binocularly or monocularly. Several times during the sequence, when the fixation point is stationary, images are captured of the eyes (e.g., by a built-in camera(s)). Those images are then displayed as a map (e.g., a rectilinear grid) for presentation where each image’s location corresponds to the location of the fixation point. The presented fixation point could be provided as an annotation. Furthermore, the images may be paired with those of the other eye even if the eyes are imaged separately, or as two separate maps of each eye. The images may be superimposed on a false image of a face to give context for the ECP. The images may be presented raw as captured or edited for clarity or emphasis (e.g., removing corneal glints, positioning the eyes in the face properly) or adjusting contrast, brightness or color of the image towards borders to better match with false images of the face. The ECP may then examine the mapped images to look at, for example, the presence or severity of strabismus based on whether the gaze of each eye is the same / different, or other ocular pathologies or conditions.

[0005] Representatively, in some aspects, the disclosure is directed to an eye examination apparatus comprising: a virtual reality device configured to be worn on a user’s head, the virtual reality device having a housing and a display operable to display a fixation point to an eye of the user that corresponds to any one of a number of diagnostic gaze positions; and an imaging device coupled to the housing that is operable to capture an image (or series of images or videos) of the eye viewing the fixation point for use in examining the eye for an ocular condition. The imaging device may include a camera integrated into the housing. In some aspects, the image captured by the imaging device is of a right eye of the user, and the apparatus further comprises a second imaging device operable to capture an image of a left eye viewing the fixation point. In some aspects, the image of the right eye and the image of the left eye are captured at a same time. In still further aspects, the image of the right eye and the image of the left 3960-P498WO 2eye are superimposed on an image of a face that is examined by an ECP to evaluate whether the ocular condition of strabismus is present. In some aspects, the fixation point is a first fixation point displayed on a first display to a right eye, the apparatus further comprises a second fixation point displayed on a second display to a left eye, and the imaging device captures an image of the right eye and the left eye viewing the first fixation point and the second fixation point, respectively. In still further aspects, the fixation point is a first fixation point corresponding to a first diagnostic gaze position, and the display is operable to display a second fixation point corresponding to a second diagnostic gaze position. The image may be a first image of the eye viewing the first fixation point, and the imaging device is further operable to capture a second image of the eye viewing the second fixation point. In some aspects, the first image and the second image are arranged in a grid used by an ECP to examine the eye for the ocular condition of strabismus. The fixation point may include a shape operable to be displayed at a number of locations on the display selected to corresponds to any one of a number of diagnostic gaze positions.

[0006] In other aspects, an eye examination system may include a virtual reality device configured to be worn on a user’s head, the virtual reality device having a housing and a display operable to display a number of fixation points to an eye of the user that correspond to a number of diagnostic gaze positions; an imaging device coupled to the housing that is operable to capture images of the eye while viewing the number of fixation points; and one or more processors communicatively coupled to the virtual reality device, the one or more processors configured to process the images and generate a grid of images to be used in examining the eye for an ocular condition. The imaging device may include a camera integrated into the housing. In some aspects, the images captured by the imaging device are of a right eye of the user, and the system further includes a second imaging device operable to capture images of a left eye viewing the number of fixation points. The images of the right eye and the images of the left eye may be captured at a same time. In still further aspects, the images of the right eye and the images of the left eye are superimposed on images of a face that are examined by an eye care professional to evaluate whether the ocular condition of strabismus is present. In some aspects, the number of fixation points are displayed in a series on a first display to a right eye, the apparatus further comprises a number of fixation points displayed in a series on a second display to a left eye, and the imaging 3960-P498WO 3device captures images of the right eye and the left eye viewing the series of fixation points. In other aspects, the images are arranged in a grid comprising at least nine images that are arranged in a three-by-three grid to correspond to the diagnostic gaze positions. The grid may be used by an eye care professional to examine the eye for the ocular condition of strabismus. In some aspects, the grid may include a first grid of images of the right eye and a second grid of images of the left eye. The fixation points may include a shape operable to be displayed at a number of locations on the display selected to corresponds to any one of the number of diagnostic gaze positions.

[0007] The above summary does not include an exhaustive list of all aspects of the present disclosure. It is contemplated that the disclosure includes all systems and methods that can be practiced from all suitable combinations of the various aspects summarized above, as well as those disclosed in the Detailed Description below and particularly pointed out in the Claims section. Such combinations may have particular advantages not specifically recited in the above summary. Brief Description of the Drawings

[0008] Several aspects of the disclosure here are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings, in which like references indicate similar elements. It should be noted that references to “an” or “one” aspect in this disclosure are not necessarily to the same aspect, and they mean at least one. Also, in the interest of conciseness and reducing the total number of figures, a given figure may be used to illustrate the features of more than one aspect of the disclosure, and not all elements in the figure may be required for a given aspect.

[0009] FIG.1 illustrates a side perspective view of a user wearing a VR headset.

[0010] FIG.2 illustrates a cross-sectional top view of the user wearing the VR headset of FIG.1.

[0011] FIG. 3 illustrates a top schematic view of an example system for performing an eye examination using the VR headset of FIG.2.

[0012] FIG. 4 illustrates a block diagram of an example operation for performing an eye examination using a VR headset. 3960-P498WO 4

[0013] FIG.5 illustrates a block diagram of an example system for performing an eye examination using a VR headset. Detailed Description

[0014] Several aspects of the disclosure with reference to the appended drawings are now explained. Whenever the shapes, relative positions and other aspects of the parts described are not explicitly defined, the scope of the invention is not limited only to the parts shown, which are meant merely for the purpose of illustration. Also, while numerous details are set forth, it is understood that some aspects of the disclosure may be practiced without these details. In other instances, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0015] The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. Spatially relative terms, such as “beneath”, “below”, “lower”, “above”, “upper”, and the like may be used herein for ease of description to describe one element's or feature's relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as “below” or “beneath” other elements or features would then be oriented “above” the other elements or features. Thus, the exemplary term “below” can encompass both an orientation of above and below. The device may be otherwise oriented (e.g., rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.

[0016] As used herein, the singular forms “a”, “an”, and “the” are intended to include the plural forms as well, unless the context indicates otherwise. It will be further understood that the terms “comprises” and / or “comprising” specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. 3960-P498WO 5

[0017] The terms “or” and “and / or” as used herein are to be interpreted as inclusive or meaning any one or any combination. Therefore, “A, B or C” or “A, B and / or C” mean “any of the following: A; B; C; A and B; A and C; B and C; A, B and C.” An exception to this definition will occur only when a combination of elements, functions, steps or acts are in some way inherently mutually exclusive.

[0018] FIG.1 depicts an eye examination system including a user (wearer) 102 wearing a virtual reality (VR) device or headset 104 which as modified below can be used to perform various ophthalmic examinations on the eyes of the wearer, including examinations to determine the presence or severity of strabismus (among other conditions). The modifications enable the use of a relatively low-cost solution in the form of an otherwise consumer grade VR headset.

[0019] Representatively, FIG. 2 is a cross-sectional top schematic view of the system of FIG.1 having an integrated eye examination system. From this view, it can be seen that when the VR headset 104 is positioned on the user’s head (as shown in FIG.1), the user’s eyes 202A, 202B are aligned with the various optical components of the VR headset 104 for performing an eye examination (e.g., an examination to monitor strabismus). Representatively, the VR headset 104 may include a housing 203 that contains, or is otherwise coupled to, the various optical components. Housing 203 is positioned over the eyes of the user and is removably secured to the head of the user using a strap as shown in FIG.1, or other similarly suitable attachment mechanism that can removably secure housing 203 onto the user’s head. In some aspects, the various optical components may include, among others, one or more VR displays 204A, 204B that operate like a traditional VR display to display an image that is viewed by a respective one of the eyes 202A-B of the user. The components may further include one or more of an imaging device 206A, 206B integrated into housing 203. Imaging device 206A, 206B may be arranged within housing 203 at any location or position relative to each of eyes 202A, 202B suitable for capturing an image of the eye 202A, 202B when viewing the image displayed on displays 204A-B. Representatively, in some aspects, imaging device 206A may be arranged in front of eye 202A (e.g., right eye) near display 204A and be used to capture an image of eye 202A. It is contemplated that the image may include a photo, video, or a series of photos of videos. Similarly, imaging device 206B may be arranged in front of eye 202B (e.g., left eye) near display 204B and be 3960-P498WO 6used to capture an image of eye 202B. Other locations and / or positions of imaging device 206A-B, however, are further contemplated. For example, in some aspects, imaging devices 206A-B may be positioned next to eyes 202A-B and reflectors may be incorporated into housing 203 to reflect an image of each eye 202A-B back to imaging devices 204A-B so that imaging devices can capture an image of each of eyes 202A-B. In this aspect, during an ocular examination or procedure for detecting the presence or severity of strabismus, images may be displayed at different locations within displays 202A-B and images of eyes 202A-B at different angles looking at the images are captured by the imaging devices 206A-B. The captured images may then be arranged in a map or grid of photographs or videos that can be evaluated by the ECP to determine the presence or severity of strabismus.

[0020] Referring now in more detail to one representative system for evaluating the presence of strabismus, FIG.3 illustrates a top schematic view of the VR headset of FIG. 2 adapted to monitor for and / or detect strabismus. Representatively, from this view, it can be understood that during the examination, each of eyes 202A-B are directed toward a respective one of displays 204A-B. Each display 204A-B may then display an image, for example in the form of a fixation point that directs the eyes to one of any of the diagnostic positions of gaze (or diagnostic gaze positions) used to assess ocular function. Representative diagnostic positions of gaze may include, but are not limited to, directing the eyes to a primary position (e.g., looking straight ahead), secondary positions (e.g., looking inward, outward, upward or downward), up / right, right, down / right, down / left, left and up / left. For example, FIG. 3 illustrates display 204A displaying images at three fixation points 302A-1, 302A-2, 302A-3 which correspond to three different diagnostic positions of gaze. Similarly, display 204B displays images at three fixation points 302B-1, 302B-2, 302B-3 which correspond to three different diagnostic positions of gaze. The positions may be, for example, a center gaze position (e.g., illustrated by fixation points 302A-2, 302B-2) followed by right / left outward gaze positions (e.g., fixation points 302A-1, 302B-1 and 302A-3, 302B-3). The different gaze positions may be approximately 45 degrees relative to the center gaze position (aligned to the horizontal and vertical axes). The fixation points 302A-1, 302A- 2, 302A-3, 302B-1, 302B-2 and 302B-3 (as well as any remaining fixation points) may be presented binocularly or monocularly in series. Several times during the sequence or series of displaying the fixation points at different locations (when the fixation point is 3960-P498WO 7stationary), images are captured of the eyes 202A-B by the imaging devices 206A-B. The imaging devices may capture the images of each of eyes 202A-B fixated on fixation points corresponding to the same diagnostic position of gaze at the same time. Imaging devices 206A-B may be, for example, built in cameras, image sensors or any other type of device suitable for capturing an image of eyes 202A-B when viewing the fixation points. In some aspects, the images may be a series of photos or videos. In some aspects, the images could be taken by the imaging devices 206A-B in the visible or in the near- infrared (IR) range. It is further contemplated that although two imaging devices 206A- B and two displays 204A-B are shown integrated into housing 203, one display may display the fixation points to both eyes and / or one imaging device may be used to capture images of both eyes when viewing the fixation points.

[0021] The images captured by imaging devices 204A-B are then displayed in a map or rectilinear grid 304 of images or tiles 306 in which the eye gaze imaged corresponds to the location of the fixation point. For example, a single image per location or fixation point could be selected and displayed in grid 304. In some aspects, grid 304 may be a three-by-three grid of at least nine images or tiles 306 as shown. In other aspects, grid 304 could be different (larger or smaller) depending on the application. In still further aspects, two maps or grids 304 corresponding to the location of fixation points viewed by each eye separately could be generated. Regardless of the number of images and / or grids, it should be understood that the images should be placed at positions corresponding to the gaze angle to provide an intuitive presentation of the direction of gaze to the ECP for assessment. For example, image tile 306A presented in grid 304 is an image of eyes 202A and 202B looking to the left (e.g., 45 degrees from a center gaze position) and viewing fixation points 302A-3 and 302B-3, respectively. Image tile 306B presented in grid 304 is an image of eyes 202A and 202B looking straight ahead and viewing fixation points 302A-2 and 302B-2 (e.g., the center gaze position). Image tile 306C presented in grid 304 is an image of eyes 202A and 202B looking to the right (e.g., 45 degrees from the center gaze position) and viewing fixation points 302A-1 and 302B-1. The remaining images or tiles represented in grid 304 illustrate images of eyes 202A-B viewing fixation points corresponding to the six remaining diagnostic positions of gaze. In some aspects, the presented fixation point could be provided as an annotation. Furthermore, the captured images may be paired with those of the fellow eye as shown in grid 304 even if eyes 202A-B are imaged 3960-P498WO 8separately. In still further aspects, the images may be superimposed on a false image of a face to give context for the ECP. The images may be presented raw as captured or edited for clarity or emphasis (e.g., removing corneal glints, positioning the eyes in the face properly) or adjusting contrast, brightness or color of image towards borders to better match with false image of a face. In still further aspects, if the fixation points are presented monocularly, two maps may be generated, one for each fixated eye (left and right). If the system also has the ability to capture eye tracking, the measured gaze angle could further be annotated on the images to facilitate evaluation by the ECP. In addition, a digital camera based system could be integrated into the system that automatically captures the images when stimuli are presented and then crops the image appropriately is further contemplated.

[0022] It is further contemplated that fixation points may be displayed as circular, dot or other basic images on display 204A-B, or they could be pictures, graphics or other interesting shapes (e.g., a spaceship, a farm animal, etc.) to better help fixate the gaze of a younger patient. Thus, the term “fixation point” should be understood as including an image of any type of shape, object or the like that can be displayed by the VR headset display for viewing and focusing a gaze of a user’s eye. The fixation points could also be dynamic and / or change over time to help maintain the attention of the patient.

[0023] FIG. 4 illustrates a block diagram of one representative process for performing an eye examination to detect strabismus using a VR headset, as previously discussed. Representatively, process 400 includes an initial operation 402 of displaying an image such as a fixation point that directs the eye gaze to any number of the cardinal diagnostic gaze positions on a display of the VR headset as previously discussed. The fixation point may be an image of any type of shape such as a dot, or any type of object, that is suitable to fix the gaze of the user at a particular diagnostic gaze position (e.g., center, 45 degrees to the left, 45 degrees to the right, etc.). Once the fixation point is displayed, the user is instructed to view the fixation point at operation 404. An image of the user’s eye at the fixation point is then captured at operation 406. In some aspects, the VR headset may automatically detect that the eye is at the fixation point (e.g., using an eye tracker) and capture the image using an imaging device such as a camera or image sensor integrated into the VR headset as previously discussed. Once the image 3960-P498WO 9of the eye (or both eyes) at the first or initial fixation point is captured, the process returns to operation 402 for display of another fixation point at a different location corresponding to another diagnostic gaze position. In some aspects, the second fixation point may be at a location approximately 45 degrees from the previous fixation point. An image of the user’s eye at the second fixation point is captured at operation 406. This process of operations 402-406 may continue until images of each of the user’s eyes at all fixation points corresponding to all diagnostic gaze positions are captured. Once all the images for each eye are captured, a grid or map based on the images is generated as previously discussed at operation 408. The grid or map may then be used by the ECP to assess the movement of the eyes and gaze angle to determine whether, for example, both eyes at each of the diagnostic gaze positions are aligned to the same fixation point, or different fixation points suggesting the presence of strabismus.

[0024] Referring now to FIG.5, FIG.5 illustrates a block diagram of an example system for measuring corneal topography using a VR headset. Representatively, to perform the eye examination (e.g., a strabismus assessment), system 500 may include headset 104 positioned on the user’s head as previously discussed. Headset 104 may be communicatively coupled to a remote computer or other electronic device 502 allowing for control of the functions of headset 104 and for performing the eye examination. Representatively, remote device 502 may be equipped with a transmitter / receiver 504 for communicating (e.g., wireless internet communication) with headset 104. Device 502 may further include a processor 506 connected to a memory 508 that stores a software application or algorithm for conducting an ocular examination (e.g., test for strabismus) as described herein. Using the software application, the medical professional may perform the operation 400 described in reference to FIG.4. For example, the headset display may present a number of fixation points to direct the user’s gaze and the camera may capture images of the user’s gaze at the different fixation points. Once acceptable data has been acquired in the form of a series of images collected by the camera, the images may be processed by the processor using a conventional algorithm designed to arrange the images into a map or grid showing the different eye positions, as previously discussed, for evaluation by the ECP for strabismus. 3960-P498WO 10

[0025] While certain aspects have been described and shown in the accompanying drawings, it is to be understood that such are merely illustrative of and not restrictive on the broad invention, and that the invention is not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those of ordinary skill in the art. The description is thus to be regarded as illustrative instead of limiting. 3960-P498WO 11

Claims

In the Claims What is claimed is:

1. An eye examination apparatus comprising: a virtual reality device configured to be worn on a user’s head, the virtual reality device having a housing and a display operable to display a fixation point to an eye of the user that corresponds to any one of a number of diagnostic gaze positions; and an imaging device coupled to the housing that is operable to capture an image of the eye viewing the fixation point for use in examining the eye for an ocular condition.

2. The apparatus of claim 1 wherein the imaging device comprises a camera integrated into the housing.

3. The apparatus of claim 1 wherein the image captured by the imaging device is of a right eye of the user, and the apparatus further comprises a second imaging device operable to capture an image of a left eye viewing the fixation point.

4. The apparatus of claim 3 wherein the image of the right eye and the image of the left eye are captured at a same time.

5. The apparatus of claim 4 wherein the image of the right eye and the image of the left eye are superimposed on an image of a face that is examined by an eye care professional to evaluate whether the ocular condition of strabismus is present.

6. The apparatus of claim 1 wherein the fixation point is a first fixation point displayed on a first display to a right eye, the apparatus further comprises a second fixation point displayed on a second display to a left eye, and the imaging device captures an image of the right eye and the left eye viewing the first fixation point and the second fixation point, respectively. 3960-P498WO 127. The apparatus of claim 1 wherein the fixation point is a first fixation point corresponding to a first diagnostic gaze position, and the display is operable to display a second fixation point corresponding to a second diagnostic gaze position.

8. The apparatus of claim 7 wherein the image is a first image of the eye viewing the first fixation point, and the imaging device is further operable to capture a second image of the eye viewing the second fixation point.

9. The apparatus of claim 8 wherein the first image and the second image are arranged in a grid used by an eye care professional to examine the eye for the ocular condition of strabismus.

10. The apparatus of claim 1 wherein the fixation point comprises a shape operable to be displayed at a number of locations on the display selected to corresponds to any one of a number of diagnostic gaze positions.

11. An eye examination system comprising: a virtual reality device configured to be worn on a user’s head, the virtual reality device having a housing and a display operable to display a number of fixation points to an eye of the user that correspond to a number of diagnostic gaze positions; an imaging device coupled to the housing that is operable to capture images of the eye while viewing the number of fixation points; and one or more processors communicatively coupled to the virtual reality device, the one or more processors configured to process the images and generate a grid of images to be used in examining the eye for an ocular condition.

12. The system of claim 11 wherein the imaging device comprises a camera integrated into the housing and the images comprise a series of photos or videos of the eye.

13. The system of claim 11 wherein the images captured by the imaging device are of a right eye of the user, and the system further comprises a second imaging device operable to capture images of a left eye viewing the number of fixation points. 3960-P498WO 1314. The system of claim 13 wherein the images of the right eye and the images of the left eye are captured at a same time.

15. The system of claim 14 wherein the images of the right eye and the images of the left eye are superimposed on images of a face that are examined by an eye care professional to evaluate whether the ocular condition of strabismus is present.

16. The system of claim 11 wherein the number of fixation points are displayed in a series on a first display to a right eye, the system further comprises a number of fixation points displayed in a series on a second display to a left eye, and the imaging device captures images of the right eye and the left eye viewing the series of fixation points.

17. The system of claim 16 wherein the images are arranged in a grid comprising at least nine images that are arranged in a three-by-three grid to correspond to the diagnostic gaze positions.

18. The system of claim 17 wherein the grid is used by an eye care professional to examine the eye for the ocular condition of strabismus.

19. The system of claim 17 wherein the grid comprises a first grid of images of the right eye and a second grid of images of the left eye.

20. The system of claim 11 wherein the fixation points comprise a shape operable to be displayed at a number of locations on the display selected to corresponds to any one of the number of diagnostic gaze positions. 3960-P498WO 14

Images