Systems and methods for improving vision in eyes of viewers with retinal disorders

Abstract

To improve vision of a viewer's eye with impaired retina.SOLUTION: A portable system and method for training an alternate retinal location of a viewer's eye with impaired retina and an assistance system for improving vision of such viewer's eye are disclosed. The portable system for training comprises an eye tracking module to provide eye information of the viewer's eye and a virtual image display module to display a virtual image centered at the alternate retinal location on the viewer's impaired retina other than centered at a fovea. The virtual image display module further comprises a first light signal generator to generate multiple first light signals and a first combiner to redirect the multiple first light signals towards the alternate retinal location, when a pupil of the viewer's eye is located approximately at the center of the viewer's eye.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] Related Applications This application claims the benefit of Provisional Application No. 63 / 209,405, filed June 11, 2021, entitled "VISION-ASISTED DEVICE FOR USERS WITH IMPAIRED RETINA," which is incorporated herein by reference in its entirety.

[0002] Additionally, the entirety of PCT International Application PCT / US20 / 59317, filed November 6, 2020, entitled "SYSTEM AND METHOD FOR DISPLAYING AN OBJECT WITH DEPTHS," is incorporated herein by reference.

[0003] The present invention relates to a system for training the eyes of a viewer with retinal disorders to improve the vision of such a viewer's eyes, and more particularly to a system for training alternative retinal positions in the eyes of a viewer with retinal disorders to improve the vision of the viewer's eyes. [Background technology]

[0004] People with retinal disorders lose vision in the central or peripheral areas of vision. People with age-related macular degeneration (AMD) lose vision in the central area of ​​vision. People with glaucoma lose vision in the peripheral area of ​​vision. The eyes of people with retinal disorders typically have a damaged macula (or macula), an oval pigmented area near the center of the retina of a person's eye. The human macula typically has a diameter of approximately 5.5 mm (0.22 inches) and is subdivided into the umbo, foveola, foveal avascular zone, fovea, parafovea, and perifoveal areas. The macula is responsible for central, high-resolution color vision possible in bright light. The fovea is responsible for sharp, central vision (also called foveal vision), which is necessary for activities in humans where visual detail is most important, such as reading and driving. The fovea is surrounded by the parafoveal zone and an outer perifoveal area.

[0005] When a person's eyes fixate an object, they typically use the fovea to aim at the object and obtain better resolution of the object's image. Therefore, the visual axis is defined as an imaginary line between the object and the fovea. As mentioned above, retinal disorders can be caused by AMD, glaucoma, or other diseases. A person with a retinal disorder can experience blurred or no vision in the central or peripheral areas of their vision. A person's vision can be improved by training the eccentric visual field (PRL) of a viewer's eye in a healthy state to respond to received light signals. Therefore, a portable system for training the PRL of a viewer's eye with a retinal disorder and an assistive system for improving the vision of a viewer's eye are desired. Summary of the Invention

[0006] The present disclosure relates to a portable system and method for training an alternative retinal position in a viewer's eye with a retinal disorder, thereby improving the viewer's vision. The viewer's retinal disorder may be caused by age-related macular degeneration (AMD), glaucoma, or other diseases. AMD patients have a degenerated macula that can cause blurred or no vision in the central region of vision. Glaucoma patients lose vision in the peripheral region rather than the central region. The vision of these patients in the central or peripheral region of vision can be improved by training an alternative retinal position in the viewer's eye, which remains healthy, to respond to received light signals. The alternative retinal position may also be referred to as the eccentric visual field (PRL). A portable system for training an alternative retinal position in a viewer's eye with a retinal disorder includes an eye tracking module and a virtual image display module. The eye tracking module provides information about the viewer's eye. Based on the eye information from the eye tracking module, the virtual image display module displays a virtual image centered at the alternative retinal position of the viewer's eye, rather than the fovea, when the pupil of the viewer's eye is located approximately at the center of the viewer's eye. The virtual image display module includes a first light signal generator and a first combiner, the first light signal generator generating a plurality of first light signals for a virtual image, and the first combiner redirecting the plurality of first light signals from the first light signal generator to alternative retinal locations in a viewer's eye to display a plurality of first pixels of the virtual image.

[0007] After an alternative retinal location of the viewer's eye has been trained to substitute the fovea for visual fixation, an assistive system and method can be used to improve the vision of the viewer's eye with retinal damage by projecting a virtual image corresponding to a target object onto the fovea and its adjacent region (in the case of a glaucoma patient) or onto the trained alternative retinal location (in the case of an AMD patient). The assistive system for improving vision includes an image capture module, a processing module, and a virtual image display module. The image capture module is configured to capture a field of view directly in front of the viewer's eye (a default target object) or a specific target object on which the viewer's eye fixates, and thus receives a plurality of image pixels. The processing module is configured to generate virtual image information related to the target object. The virtual image display module includes a first optical signal generator and a first combiner. The first optical signal generator generates a plurality of first optical signals for the virtual image based on the virtual image information provided by the processing module. For a viewer whose damage is in the macula, particularly the fovea and adjacent areas, such as an AMD patient, the first combiner redirects the first optical signals from the first optical signal generator to an alternative retinal location in the viewer's eye, rather than the fovea, to display the first pixels of the virtual image. For a viewer whose damage is in the peripheral region of vision, such as a glaucoma patient, the first combiner redirects the first optical signals to a healthy central region of the macula, including the fovea and adjacent areas.

[0008] The alternate retinal location is selected from a portion of the retina that remains healthy. Guidance for selecting the alternate retinal location includes (1) the height of the alternate retinal location and (2) the relative position of the alternate retinal location with respect to the fovea that allows binocular fixation when the eyes are turned. First, the alternate retinal location should be selected so that a first height of the alternate retinal location for the eye of a viewer with a retinal disorder is near a second height of the preferred sensing location for the other eye of a viewer with or without a retinal disorder. Second, the alternate retinal location should be selected outside the fovea of ​​the eye of a viewer with a retinal disorder so that when the viewer's eyes are fixated in the peripheral region of their field of vision, the visual axes of both eyes from either the alternate retinal location or the preferred sensing location can intersect at the object on which the viewer's eyes are fixating.

[0009] Once the alternative retinal location is selected, coordinates of the alternative retinal location are generated based on landmarks in the eye of the viewer with retinal damage to provide a precise location for the virtual image display module to project the virtual image, which landmark may be the optic disc of the eye of the viewer with retinal damage. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram illustrating one embodiment of a system for training alternative retinal positions in the eyes of a viewer with retinal disorders, according to the present invention. [Figure 2] FIG. 2 is a schematic diagram illustrating one embodiment of a virtual image display module and an eye tracking module in accordance with the present invention. [Figure 3] 2 is a schematic diagram illustrating one embodiment of a first optical signal generator and a first combiner in accordance with the present invention; [Figure 4A] 1 is a schematic diagram illustrating one embodiment of a virtual image display module that projects light signals through different light paths to form virtual images centered at alternative retinal locations, according to the present invention. FIG. [Figure 4B]1 is a schematic diagram illustrating one embodiment of a virtual image display module that projects light signals through different light paths to form virtual images centered at alternative retinal locations, according to the present invention. FIG. [Figure 4C] 1 is a schematic diagram illustrating one embodiment of a virtual image display module that projects light signals through different light paths to form virtual images centered at alternative retinal locations, according to the present invention. FIG. [Figure 5] 1 is an image illustrating one embodiment of a microperimetry image in accordance with the present invention. [Figure 6] 1 is an image illustrating one embodiment of a fundus map showing the relative positions of alternative retinal positions, the optic disc, and the fovea, in accordance with the present invention. [Figure 7A] FIG. 1 is a schematic diagram illustrating one embodiment of a portable system for training alternative retinal positions in the eyes of a viewer with retinal disorders, according to the present invention. [Figure 7B] FIG. 1 is a schematic diagram illustrating one embodiment of a portable system for training alternative retinal positions in the eyes of a viewer with retinal disorders, according to the present invention. [Figure 7C] FIG. 1 is a schematic diagram illustrating one embodiment of a portable system for training alternative retinal positions in the eyes of a viewer with retinal disorders, according to the present invention. [Figure 7D] FIG. 1 is a schematic diagram illustrating one embodiment of a portable system for training alternative retinal positions in the eyes of a viewer with retinal disorders, according to the present invention. [Figure 8] 1 is a block diagram illustrating one embodiment of an assistance system for improving vision in the eye of a viewer with retinal damage, according to the present invention. [Figure 9A] 1 is an image illustrating one embodiment of a visual field associated with glaucoma, in accordance with the present invention. [Figure 9B] 1 is an image illustrating one embodiment of a visual field associated with glaucoma, in accordance with the present invention. [Figure 9C] 1 is an image illustrating one embodiment of a visual field associated with glaucoma, in accordance with the present invention. [Figure 10]1 is a schematic diagram illustrating one embodiment of an assistance system for improving vision in the eye of a viewer with retinal damage, according to the present invention; [Figure 11] 11A-11B are schematic diagrams illustrating one embodiment of adjusting a captured image using depth information in accordance with the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] The terms used in the description set forth below are intended to be interpreted in the broadest reasonable manner, even when used in conjunction with detailed descriptions of certain specific embodiments of the technology. Although particular terms may even be emphasized below, any terms intended to be interpreted in any restrictive manner will be specifically defined as such in this detailed description section.

[0012] The present disclosure relates to a portable system and method for training an alternative retinal position in a viewer's eye with a retinal disorder, thereby improving the viewer's vision. The viewer's retinal disorder may be caused by age-related macular degeneration (AMD), glaucoma, or other diseases. AMD patients have a degenerated macula that can cause blurred or no vision in the central region of vision. Glaucoma patients lose vision in the peripheral region rather than the central region. The vision of these patients in the central or peripheral region of vision can be improved by training an alternative retinal position in the viewer's eye, which remains healthy, to respond to received light signals. The alternative retinal position may also be referred to as the eccentric visual field (PRL). A portable system for training an alternative retinal position in a viewer's eye with a retinal disorder includes an eye tracking module and a virtual image display module. The eye tracking module provides information about the viewer's eye. Based on the eye information from the eye tracking module, the virtual image display module displays a virtual image centered at an alternative retinal position in the viewer's eye, rather than centered on the fovea, when the pupil of the viewer's eye is located approximately at the center of the viewer's eye. The virtual image display module includes a first optical signal generator and a first combiner. In other words, the viewer's eyes are fixated directly ahead, and the visual axis of the viewer's eyes is approximately perpendicular to the viewer's frontal plane in that situation. The first optical signal generator generates a plurality of first optical signals for the virtual image. The first combiner redirects the plurality of first optical signals from the first optical signal generator to alternative retinal locations of the viewer's eyes to display a plurality of first pixels of the virtual image.

[0013] After an alternative retinal location of the viewer's eye has been trained to substitute the fovea for visual fixation, an assistive system and method can be used to improve the vision of the viewer's eye with a retinal disorder by projecting a virtual image corresponding to a target object onto the fovea and its adjacent region (in the case of a glaucoma patient) or onto the trained alternative retinal location (in the case of an AMD patient). The assistive system for improving vision includes an image capture module, a processing module, and a virtual image display module. The image capture module is configured to capture a field of view directly in front of the viewer's eye (a default target object) or a specific target object that one or both of the viewer's eyes fixate on, and thus receives a plurality of image pixels. In another embodiment, the image capture module also receives corresponding depths of the plurality of image pixels. The processing module is configured to generate virtual image information related to the target object. The virtual image display module includes a first optical signal generator and a first combiner. The first optical signal generator generates a plurality of first optical signals for the virtual image based on the virtual image information provided by the processing module. For a viewer whose damage is in the macula, particularly the fovea and adjacent areas, such as an AMD patient, the first combiner redirects the first optical signals from the first optical signal generator to an alternative retinal location in the viewer's eye, rather than the fovea, to display the first pixels of the virtual image. For a viewer whose damage is in the peripheral region of vision, such as a glaucoma patient, the first combiner redirects the first optical signals to a healthy central region of the macula, including the fovea and adjacent areas.

[0014] The alternate retinal location is selected from a portion of the retina that remains healthy. Multiple locations on the viewer's retina may be available to serve as the alternate retinal location. Selection from these multiple available locations may affect the likelihood of binocular fusion between the viewer's two eyes. Therefore, the alternate retinal location should be selected to facilitate binocular fusion. Guidance for selecting the alternate retinal location includes (1) the height of the alternate retinal location and (2) the relative position of the alternate retinal location with respect to the fovea, which allows binocular fixation when the eyes are turned. First, the first height of the alternate retinal location for the eye of the viewer with a retinal disorder should be selected so that it is close to the second height of the preferred sensing location for the other eye of the viewer, whether or not the viewer has a retinal disorder. In other words, the first height is approximately the same as the second height. Second, the alternate retinal location should be selected outside the fovea of ​​the viewer's eye with retinal disorders so that when the viewer's eyes are fixated in the peripheral region of their field of vision, the visual axes of both eyes from either the alternate retinal location or the preferred sensing location can intersect at the target object on which the viewer's eyes are fixating.

[0015] Once the alternative retinal location is selected, coordinates of the alternative retinal location are generated based on landmarks in the eye of the viewer with retinal damage to provide a precise location for the virtual image display module to project the virtual image, which landmark may be the optic disc of the eye of the viewer with retinal damage.

[0016] As shown in FIG. 1 , a portable system 100 for training an alternative retinal position of a viewer's eye with a retinal disorder includes an eye tracking module 110 and a virtual image display module 120. The eye tracking module 110 is configured to track the viewer's eyes and provide related eye information, such as the viewer's eye movement, pupil position, pupil size, gaze angle (field of view; visual axis), and convergence angle. The eye tracking module 110 may include a first camera 112 that tracks the eye with a retinal disorder. Based on the eye information from the eye tracking module 110, the virtual image display module 120 projects a virtual image at a predetermined alternative retinal position of the viewer's eye to provide a stimulus for training purposes when the viewer's pupil is located approximately in the center of the viewer's eye. At that moment, the viewer's eye is fixed on a point directly ahead, and the viewer's visual axis is also approximately perpendicular to the viewer's frontal plane. The virtual image may be predetermined by a doctor, a training professional, or the viewer. In one embodiment, the predetermined virtual image is a red or green cross symbol.

[0017] As described above, the eye tracking module 110 is configured to track one or both eyes of a viewer and provide related eye information, such as pupil position, pupil size, gaze angle (field of view), and convergence angle of each eye of the viewer. Such eye information may be used to determine whether the pupil of a viewer's eye is located approximately at the center of the viewer's eye with a retinal disorder. In one embodiment shown in FIG. 2 , the eye tracking module 110 may include a first camera 112 and an eye tracking reflector 114 to track the eye of a viewer with a retinal disorder. The eye tracking reflector 114 may have approximately 100% reflectivity to IR light in this embodiment. The first camera 112 may further include an IR laser diode and an IR light sensor. The eye tracking reflector 114 is positioned in an optical path between the first camera 112 and the viewer's eye. IR light generated by the IR laser diode is reflected by the eye tracking reflector 114 and then projected onto the viewer's eye. IR light reflected from the viewer's eyes returns to the IR light sensor via the eye tracking reflector 114, where eye information, including pupil position, is analyzed and determined. In another embodiment, the viewer has retinal disorders in both eyes. The eye tracking module 110 may further include a second camera 116 to track the viewer's other eye. In addition to traditional eye tracking cameras, the first camera 112 and the second camera 116 may be constructed using ultra-compact microelectromechanical systems (MEMS) technology. The first camera 112 and the second camera 116 may use infrared emitters and sensors to detect and derive various eye information. The eye tracking module 110 may also include an integrated inertial measurement unit (IMU), an electronic device that measures and reports body force, angular velocity, and possibly body orientation using a combination of accelerometers, gyroscopes, and possibly magnetometers.

[0018] The eye tracking module 110 can measure the position and size of the pupil of the viewer's eye and determine the degree or extent to which the pupil is away from the center of the viewer's eye. In one embodiment, the eye tracking device 110 receives and analyzes 60 frames of reflected IR light per second to determine the position of the pupil. If the pupil of the viewer's eye is away from the center of the viewer's eye by more than a predetermined degree, such as 0.5 degrees, the eye tracking module 110 can notify the virtual image display module 120 to take a pause action.

[0019] As shown in FIG. 3 , the virtual image display module 120 includes a first optical signal generator 10 and a first combiner 20. The first optical signal generator 10 may use, as its light source, a laser, a light-emitting diode (“LED”) including mini- and micro-LEDs, an organic light-emitting diode (“OLED”), a superluminescent diode (“SLD”), a liquid crystal on silicon (LCoS), a liquid crystal display (“LCD”), or any combination thereof. In one embodiment, the optical signal generator 10 is a laser beam scanning projector (LBS projector) that may include a light source 11 including a red light laser 15, a green light laser 16, and a blue light laser 17, a light color modifier such as a dichroic combiner and a polarization combiner, and a two-dimensional (2D) adjustable reflector 12 such as a 2D electromechanical system (“MEMS”) mirror. In another embodiment, the light source 11 may further include an IR (infrared) light laser 14. The first optical signal generator 10 may further include a collimator 13 positioned between the light source 11 and the 2D adjustable reflector 12 to more align (parallelize) the direction of movement of the optical signal in a specific direction. The collimator 13 may be a curved or convex lens. The 2D adjustable reflector 12 may be replaced with two one-dimensional (1D) reflectors, such as two 1D MEMS mirrors. The LBS projector sequentially generates and scans optical signals one by one to form a 2D virtual image with a predetermined resolution, e.g., 1280 x 720 pixels per frame. In this way, one optical signal per pixel is generated and projected toward the first combiner 20 at a time. For a user to view such a 2D virtual image from one eye, the LBS projector must sequentially generate optical signals for each pixel, e.g., 1280 x 720 optical signals, within the duration of the visual field, e.g., 1 / 18 second. Therefore, the duration of each optical signal is approximately 60.28 nanoseconds.

[0020] In another embodiment, the first light signal generator 10 may be a digital light processing projector ("DLP projector") capable of generating a 2D color image at once. Texas Instrument's DLP technology is one of several technologies that can be used to manufacture DLP projectors. An entire 2D color image frame, which may contain, for example, 1280 x 720 pixels, is projected simultaneously toward the first combiner 20.

[0021] The first combiner 20 receives the multiple optical signals generated by the first optical signal generator 10 and redirects them to an alternative retinal location in the viewer's eye, rather than the fovea. Here, the first combiner 20 can function as a reflector. The first combiner 20 can be made of glass or a plastic material such as a lens, and may be coated with a specific material, such as metal, to make it reflective. One advantage of using a reflective combiner instead of a conventional waveguide to direct the optical signals to the user's eye is that it eliminates the problem of undesirable diffraction effects, such as multiple shadows, color shifts, etc.

[0022] In another embodiment shown in FIG. 2 , the optical path of the virtual image display module 120 can be designed to further include an auxiliary first combiner 25. The optical signal generated by the first optical signal generator 10 is projected toward the first combiner 20, which redirects the optical signal to the auxiliary first combiner 25, which further redirects the optical signal to an alternative retinal position of the viewer's eye other than the fovea. In addition, the virtual image display module 120 may further include a safety reflector 122 and a safety sensor 124 disposed between the first combiner 20 and the auxiliary first combiner 25. In one embodiment, the reflectivity of the reflector 122 is approximately 10%, allowing approximately 90% of the optical signal to pass through. The safety sensor 124 receives the reflected optical signal from the reflector 122 and measures its intensity. If the intensity of the light signal exceeds a predetermined value, the safety sensor 124 will, for safety reasons, notify the first light signal generator 10 to power off the light source or prevent the light signal from being projected into the viewer's eyes to avoid eye injury.

[0023] In one embodiment, to precisely control the position of the viewer's eye onto which the first optical signal is projected, the six-degree-of-freedom first combiner 20 and auxiliary first combiner 25 may be independently adjusted by moving along and / or rotating about a horizontal axis (or pitch axis, X-axis), a vertical axis (or longitudinal axis, Y-axis), and / or a depth axis (or vertical axis, Z-axis) to a specific degree, such as by rotating by 5 degrees. The horizontal axis may be set to lie along the direction of the interpupillary line. The vertical axis may be set to lie along the midline of the face and perpendicular to the horizontal direction. The depth direction (or vertical axis, Z-axis direction) may be set to lie perpendicular to the frontal plane and perpendicular to both the horizontal and vertical directions. More specifically, the first combiner 20 and the auxiliary first combiner 25 can be rotated about a horizontal axis to move the projection position of the optical signal above or below the viewer's retina, rotated about a vertical axis to move the projection position of the optical signal to the left or right of the viewer's retina, and / or moved along a depth axis to adjust the pupil distance.

[0024] As described above, the virtual image display module 120 projects a virtual image at a predetermined alternative retinal position of the viewer's eye based on the eye information from the eye tracking module 120 when the viewer's pupil is located approximately at the center of the viewer's eye, providing a stimulus for training purposes. At that moment, the viewer's eye is fixated on a point directly ahead, and the viewer's visual axis is also approximately perpendicular to the viewer's frontal plane. The visual axis is an imaginary line that passes through the pupil and connects the fixation point to the fovea of ​​the viewer's eye. This is the most natural and easiest fixation point for a viewer. As a result, the viewer does not need to rotate their eyeballs to train the alternative retinal position. Such fixation training of the eyes of a viewer with retinal disorders, such as an AMD patient, can enable the viewer's eyes with retinal disorders to look directly ahead while maintaining fixation, without having to turn their head to one side to view the center of the image. The eye tracking module 120 can detect the position and size of the pupil of the viewer with retinal disorders and then determine whether the pupil is located at the center of the viewer's eye. The virtual image display module 120 projects the light signal to a predetermined alternate retinal position when the pupil is located at the center of the viewer's eye. The virtual image display module 120 may suspend projection when the pupil deviates from the center of the viewer's eye by a predetermined range, e.g., 1 degree, because in that situation the light signal is projected to a different location than the alternate retinal position intended for training. When the pupil deviates from the center of the viewer's eye by a certain range, the light signal may not even be able to pass through the pupil because the system is calibrated to project the light signal to a fixed location for the viewer.

[0025] As shown in FIGS. 4A-4C, the virtual image display module 120 can project the light signals forming the virtual image 440 toward an alternative retinal location 420 via different light paths. Specifically, the virtual image is projected onto a region of the viewer's retina that is centered at the alternative retinal location 420, rather than centered on the fovea 410. In one embodiment, the virtual image may have 921,600 pixels in a 1280 x 720 array. The light signals that collectively form the virtual image can be considered as a light beam. Based on the light path of the center of the light beam, the light paths of the projection of the light signals can be divided into three categories. In FIG. 4A, the light signals that form the virtual image 440 are projected through an approximately central portion of the pupil 430; in FIG. 4B, the light signals that form the virtual image 440 are projected through an upper portion of the pupil 430; and in FIG. 4C, the light signals that form the virtual image 440 are projected through a lower portion of the pupil 430. Alternatively, the optical signals forming the virtual image 440 may be projected through the right or left portion of the pupil 430. Projecting the optical signals forming the virtual image through a substantially central portion of the pupil may have several advantages. First, the virtual image is less likely to be partially blocked when the pupil is small due to strong ambient light. Second, the angle of incidence is generally smaller to project the optical signals of the virtual image to alternative retinal locations. The first combiner 20 and / or the auxiliary first combiner 25 can be adjusted to project the optical signals through selected optical paths.

[0026] System 100 can further include fundus perimetry 130 to perform visual field testing by generating a "retinal sensitivity map" of the amount of light perceived at specific portions of the retina in the viewer's eye. To reduce redundancy, fundus perimetry 130 can share light source 11 and some optical components with virtual image display module 120. In one embodiment shown in FIG. 3, fundus perimetry 130 includes light source 11, a set of optical components 131, a light intensity sensor 136, and a perimetry controller 138. The set of optical components 131 may include three reflectors 132, 133, and 134 to guide light reflected from the viewer's eye to light intensity sensor 136, which may be a CCD (charge-coupled device). Perimetry controller 138 can receive electrical signals from light intensity sensor 136 and generate a retinal sensitivity map, such as that shown in FIG. 5, that provides information for a physician to select an alternative retinal location. The alternative retinal location may be selected based on some guidance to facilitate visual fixation. In one embodiment, fundus perimetry 130 may be microperimetry or scanning laser ophthalmoscopy (SLO).

[0027] As described above, for patients with retinal disorders, an alternative retinal location is selected from a portion of the retina that remains healthy. Multiple locations on the viewer's retina may be available to serve as alternative retinal locations. As shown in FIG. 5, the microperimetry map typically illustrates the degree of health of the viewer's retina using color, with green representing a healthy (fully functional) state, yellow representing a partially damaged state that may still function to some extent (partially functional), and red representing a damaged (non-functional) state. Thus, the color of each small square in FIG. 5 represents the level of retinal function at each particular location. Typically, green represents fully functional; yellow represents partially functional; and red represents non-functional. Selecting an alternative retinal location from these multiple available healthy locations for training may affect the likelihood of binocular fusion between the viewer's two eyes, for example, one AMD eye and one normal eye, or both AMD eyes. Therefore, the alternative retinal location must be selected to facilitate binocular fusion. Guidance for selecting an alternate retinal location includes (1) the height of the alternate retinal location and (2) the relative position of the alternate retinal location with respect to the fovea, which allows binocular fixation when the eyes change direction. First, the first height of the alternate retinal location for the eye of a viewer with a retinal disorder should be selected so that it is close to the second height of the preferred sensing location for the other eye of a viewer with or without a retinal disorder. Binocular fixation may occur more easily if the alternate retinal location for the eye of a viewer with a retinal disorder is at approximately the same height as the preferred sensing location of the other eye of the viewer, such as the fovea of ​​a normal eye. In other words, the first height is approximately the same as the second height. Second, the alternate retinal location should be selected outside the fovea of ​​the eye of a viewer with a retinal disorder so that when the viewer's eyes are fixed in the peripheral region of the field of view, the visual axes of both eyes from either the alternate retinal location or the preferred sensing location can intersect at the target object on which the viewer's eyes are fixating.

[0028] Once the alternate retinal location 630 is determined, 2D coordinates are generated to pinpoint the location of the alternate retinal location based on landmarks. In one embodiment shown in Figure 6, the optic disc 610 of the viewer's eye is used as a landmark to derive the location of the fovea 620. Then, assuming the fovea 620 is the origin with coordinates (0,0), the coordinates of the alternate retinal location 630 can be obtained.

[0029] The system 100 may further include a processing module 140 that executes a training program for the viewer. The processing module 140 may include a processor and memory to serve as a computing power center for other modules of the system 100, such as the eye tracking module 110 and the virtual image display module 120. A training application / software can be installed on the processing module 140 to provide the training program to the viewer. The training program can be customized for each individual. Additionally, the system 100 is portable, allowing the viewer to easily complete the training at home. In one embodiment, a training session lasts approximately 15 minutes. The time the viewer blinks may not be counted toward the training session time. An artificial intelligence (AI) model can be used to determine whether a blink has occurred. The shape, size, and color of the virtual image used for training, such as a red or green cross or a red or green circle, may be selected from the program. If the viewer's pupils are likely to frequently wander off-center at the beginning of training, larger virtual images can be used for training. If the viewer's pupils are likely to fixate directly ahead for longer periods of time, smaller virtual images can be used for training. The training program can record all relevant data detected during the training session and generate a training report. All relevant training data and reports may be remotely uploaded to an information system at a clinic or hospital for medical evaluation.

[0030] System 100 may further include a feedback module 150 configured to provide feedback to the viewer when the viewer's pupils are away from the center of the viewer's eyes by a predetermined amount, e.g., 0.5 degrees or more, based on the eye information from eye tracking module 110. In other words, when the viewer's eyes are no longer fixating directly ahead and the visual axis of the viewer's eyes is no longer perpendicular to the viewer's frontal plane, feedback module 150 may provide audio and / or visual feedback to guide the viewer's pupils back to the center of the eyes. Visual guidance may include visible indicators that indicate the direction in which the viewer's eyes should move, such as flashing arrows indicating the direction in which the viewer's pupils should move. Such visual guidance may be displayed by virtual image display module 120. Audio guidance may include audio feedback indicating the direction of the viewer's eye movement, which may be provided by a speaker.

[0031] System 100 may further include an interface module 160 that allows the viewer to control various functions of system 100. Interface module 160 may be operated by voice, hand gestures, finger / foot movements, and may be in the form of pedals, a keyboard, a mouse, knobs, switches, a stylus, buttons, sticks, a touchscreen, etc.

[0032] As shown in FIGS. 7A-7D , portable system 100 may further include a frame 170 including a base 171, a chin holder 172, a forehead rest 173, and a tablet connector 174, in addition to a light engine 175 including eye tracking module 110, virtual image display module 120, fundus perimetry module 130, and process module 140. The height of chin holder 172 is adjustable. The relative position of forehead rest 173 can be adjusted toward or away from the viewer. In one embodiment, the size of system 100 including frame 170 is approximately 50-65 cm in height, 30 cm in width, and about 30 cm in depth. Additionally, in one embodiment, the weight of system 100 including frame 170 is approximately 3 kg.

[0033] After the viewer's eye with an alternative retinal location has been trained for visual fixation by the portable system 100, the viewer can use the system 200 to improve the vision of the eye with the retinal disorder by projecting a virtual image corresponding to a target object to the trained alternative retinal location of the viewer's eye with the retinal disorder. As shown in FIG. 8 , the system 200 for improving vision includes an image capture module 210, a processing module 220, and a virtual image display module 230. The image capture module 210 is configured to receive a plurality of image pixels and corresponding depths of the target object 205. In one embodiment, the image capture module 210 captures the field of view directly in front of both eyes of the viewer as the target object. In other words, the viewing angle of the image capture module 210 is perpendicular to the frontal plane of the viewer wearing the auxiliary system 200. The processing module 220 generates virtual image information related to the target object. The virtual image display module 230 displays the virtual image in the eye of the viewer with the retinal disorder based on the virtual image information. For viewers with retinal damage in the macula, particularly the fovea and its adjacent regions, such as those with AMD, the virtual image display module 230 can project a virtual image centered at an alternative retinal location in the viewer's eye rather than at the fovea. For viewers with retinal damage in the peripheral region of vision, such as those with glaucoma, the virtual image display module 230 can project a virtual image centered at the healthy central region of the macula, including the fovea and its adjacent regions. In this situation, as shown in FIGS. 9A-9C, the virtual image can be shrunk to a smaller size due to the smaller portion of the healthy retina in the central region that can receive and respond to optical signals. As a result, a shrunk virtual image with the same field of view, albeit smaller in size, is perceived as if the object were originally captured by the image capture module 210. FIG. 9A shows the field of view perceived by the viewer's healthy eye. FIG. 9B shows the field of view perceived by the viewer's eye with glaucoma.9C illustrates the field of view perceived by the eye of a viewer with glaucoma when the virtual image display module 230 projects a foreshortened virtual image of a target object onto the foveal region of the eye of the viewer with retinal damage. To avoid interference with natural light from the environment, the system 200 can reduce or block natural light from entering the eye of the viewer with retinal damage. As a result, the eye of the viewer with retinal damage perceives primarily or almost exclusively the virtual image projected by the virtual image display module 230. The virtual image perceived by the eye of the viewer with retinal damage and the actual image perceived by the other eye of a viewer with healthy vision can be at least partially fused into a single image. Binocular fusion can also occur when both eyes of a viewer have retinal damage and each receive a virtual image from the virtual image display module 230.

[0034] The assistance system 200 for improving vision may further include an eye tracking module 240 and an interface module 250. Similar to the eye tracking module 110 in the training system 100, the eye tracking module 240 in the assistance system 200 may be configured to track one or both of the viewer's eyes and provide related eye information, such as the viewer's eye movement, pupil position, pupil size, gaze angle (field of view; visual axis), and convergence angle. The eye tracking module 240 may further include cameras 242, 244 to determine a target object based on the viewer's eye fixation. The interface module 250 enables the viewer to control various functions of the system 200. The interface module 250 may be operated by voice, hand gestures, or finger movements and may be in the form of a pedal, keyboard, mouse, knob, switch, stylus, button, stick, touchscreen, etc.

[0035] As shown in FIG. 10 , the system 200 further includes a support structure 260 that can be worn on a viewer's head. The image capture module 210, the processing module 220, and the virtual image display module 230 (including the first optical signal generator 10, the first combiner 20, and even the second optical signal generator 30 and the second combiner 40) are carried by the support structure. In one embodiment, the system 200 is a head-wearable device such as virtual reality (VR) goggles and augmented reality (AR) / mixed reality (MR) glasses. In this case, the support structure may be a frame with or without lenses for the glasses. The lenses may be prescription lenses used to correct myopia, hyperopia, etc. In addition, the eye tracking module 240 and the interface module 250 may also be carried by the support structure.

[0036] The image capture module 210 may simply include at least one RGB camera 212 to receive multiple image pixels of a target object, i.e., a target image. In another embodiment, the image capture module 210 may further include at least one depth camera 214 to receive corresponding depths of the multiple image pixels. Alternatively, the image capture module 210 may include positioning components to receive both the multiple image pixels and the corresponding depths of the target object. To measure the depth of the target object and the environment, the depth camera 214 may be a time-of-flight camera (ToF camera), which uses time-of-flight technology to determine the distance between the camera and the object for each point in the image by measuring the round-trip time of an artificial light signal provided by a laser or LED, such as a LiDAR. ToF cameras can measure distances ranging from a few centimeters to several kilometers. Other devices, such as a structured light module, an ultrasound module, or an IR module, can also function as a depth camera used to detect the depth of the target object and the environment.

[0037] A rectification process is performed to incorporate depth information corresponding to multiple image pixels to derive more accurate coordinates of the target object and its shape. The multiple image pixels provide 2D coordinates, such as X and Y coordinates, for each feature point of the target object. However, such 2D coordinates are inaccurate because they do not take depth into account. Therefore, as shown in FIGS. 11A-11B, the image capture module 210 can align or overlay an RGB image containing multiple image pixels with the depth map so that feature points in the RGB image overlap with corresponding feature points in the depth map. The depth of each feature point is then obtained. The RGB image and the depth map may have different resolutions and sizes. Therefore, in one embodiment shown in FIG. 11B, peripheral portions of the depth map that do not overlay the RGB image can be cropped. The depth of the feature point is used to calibrate the X and Y coordinates from the RGB image to derive the actual X and Y coordinates. For example, a feature point has X and Y coordinates (a, c) in the RGB image and a z coordinate (depth) from the depth map. The actual XY coordinates will be (a+b*depth, c+d*depth), where b and d are calibration parameters and the symbol "*" means multiplication. Thus, image capture module 210 utilizes simultaneously captured image pixels and their corresponding depths to adjust the horizontal and vertical coordinates for the target object, respectively.

[0038] The processing module 220 may include a processor and memory for generating virtual image information related to the target object. Additionally, the processing module 220 may serve as a computational power center for other modules of the system 200, such as the image capture module 210 and the virtual image display module 230. To generate the virtual image information, the viewing angle of the target object from the eye of a viewer with a retinal disorder, as well as other 3D-related effects such as shading, as well as the brightness and intensity of red, blue, and green colors, may be considered.

[0039] Similar to the virtual image display module 120 in the portable training system 100, the virtual image display module 230 in the visual aid system 200 includes a first optical signal generator 10 and a first combiner 20 for projecting a virtual image into the eye of a viewer with a retinal disorder. The virtual image display module 230 may further include a second optical signal generator 30 and a second combiner 40 for the other eye of the viewer, which may also have a retinal disorder or remain healthy. The above description of the first optical signal generator 10 and the first combiner 20 also applies to the second optical signal generator 30 and the second combiner 40. Again, for a viewer whose disorder is in the central region of the macula, particularly the fovea and its adjacent areas, such as an AMD patient, the first optical signal generator 10 generates a plurality of first optical signals for the virtual image based on information from the processing module 220. The first combiner 20 redirects the plurality of first optical signals from the first optical signal generator 10 to an alternative retinal location in the viewer's eye, rather than to the impaired fovea and adjacent regions, to display the plurality of first pixels of the virtual image. For a viewer with retinal damage in the peripheral region of vision, such as a glaucoma patient, the first optical signal generator 10 generates the plurality of first optical signals for the virtual image based on information from the process module 220. The first combiner 20 redirects the plurality of first optical signals from the first optical signal generator 10 to the healthy central region of the macula, including the fovea and adjacent regions.

[0040] Again, the virtual image display module 230 can project the light signals forming the virtual image through different light paths to alternate retinal locations or preferred sensing locations, such as the fovea and adjacent regions, for example, by adjusting the combiners 20, 40. Generally, the light signals forming the virtual image can be projected through a generally central portion of the pupil, a right portion of the pupil, or a left portion of the pupil. In one embodiment, because strong ambient light reduces the size of the pupil, the light signals forming the virtual image are projected through a generally central portion of the pupil to avoid blocking any portion of the virtual image.

[0041] To reduce or block natural light from the environment, the transparency of first combiner 20 and second combiner 40 can be adjusted back and forth as needed automatically or by the viewer via interface module 250. In another embodiment, auxiliary system 200 may further include a light blocker to reduce or block natural light from the environment from entering one or both of the viewer's eyes.

[0042] In addition to the red, green, and blue lasers, the light sources 11 and 21 of the first and second optical signal generators 10 and 20 may further include an infrared (IR) laser, such as a micropulse generator, to generate low-power, high-density electromagnetic waves with wavelengths of approximately 532 nm, 577 nm, or 810 nm and irradiate the viewer's retina for massage. In one embodiment, 810 nm infrared light is generated and irradiated onto the viewer's retina. Heat shock proteins (HSPs) are generated under such electromagnetic wave irradiation. HSPs may contribute to cellular reactivation in the retina, thereby slowing the progression of age-related macular degeneration. Furthermore, because infrared light is invisible to the human eye, the red, green, and blue lasers of the light sources 11 and 21 may be irradiated onto the viewer's retina simultaneously to generate a virtual image to be projected onto the viewer's retina. As a result, the infrared light does not interfere with the virtual image composed of red, green, and blue optical signals. Alternatively, the IR light may be projected between two consecutive image frames.

[0043] As shown in Figure 3, the intensity of the IR light used to illuminate the viewer's retina must be monitored and controlled to avoid retinal damage. A lens 310 is used to collect the IR light reflected from the viewer's eye for an IR light sensor 320 to measure its intensity. If the intensity is too low, a photomultiplier tube (PMT) 330 is used to increase the intensity signal. An IR intensity controller 340 is used to determine whether the intensity of the IR laser diode 14 needs to be adjusted. If an adjustment is necessary, the IR intensity controller 340 sends a signal to the first optical signal generator 10 requesting an adjustment.

[0044] In another embodiment, to support optogenetic treatment for people with retinitis pigmentosa (RP), the light sources 11, 31 of the optical signal generators 10, 30 may further include a light generator that provides light of a specific wavelength to activate channelrhodopsin, which functions as a light-gated ion channel. This clinical treatment was first developed by RetroSense Therapeutics, a biotechnology company developing quality-of-life gene therapies designed to restore vision in patients suffering from retinitis pigmentosa (RP). Retinitis pigmentosa (RP) is a group of genetic disorders characterized by progressive peripheral vision loss and impaired night vision, often ultimately leading to central vision loss and blindness. RP is typically diagnosed in adolescents and young adults.

[0045] Any component in either the training system 100 or the auxiliary system 200 may be used exclusively by a module or may be shared by two or more modules to perform the required functions. In addition, two or more modules described in this specification may be implemented by one physical module. A module described in this specification may be implemented by two or more separate modules. An external server, while not part of the auxiliary system 200, may provide additional computing power for more complex calculations. Each of these modules and the external server described above may communicate with each other via wired or wireless modes. Wireless modes may include WiFi, Bluetooth, Near Field Communication (NFC), Internet, telecommunications, radio frequency (RF), etc.

[0046] The above description of the embodiments is provided to enable any person skilled in the art to make and use the particulars. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the novel principles and particulars disclosed herein may be applied to other embodiments without the exercise of innovative faculty. The subject matter set forth in the claims is not intended to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. Additional embodiments are contemplated within the spirit and true scope of the disclosed particulars. Thus, it is intended that the present invention cover modifications and variations that come within the scope of the appended claims and their equivalents.

[0047] Aspect 1: A portable system for training alternative retinal positions in the eyes of a viewer with a retinal disorder, comprising: An eye tracking module that provides information about the viewer's eyes; a virtual image display module that displays a virtual image that is not centered on the fovea but is centered at an alternative retinal location in the viewer's eye; a first optical signal generator that generates a plurality of first optical signals for the virtual image; a first combiner that redirects the plurality of first optical signals from the first optical signal generator to an alternative retinal location of the viewer's eye to display a plurality of first pixels of the virtual image when the pupil of the viewer's eye is located approximately at the center of the viewer's eye based on eye information from the eye tracking module; a virtual image display module including: Portable systems including: Aspect 2: The portable system of aspect 1, wherein the alternative retinal locations in the viewer's eyes are selected to facilitate binocular fusion. Aspect 3: The portable system of aspect 2, wherein the alternative retinal locations in the viewer's eyes are selected based on visual function and height of the alternative retinal locations. Aspect 4: The portable system of Aspect 3, wherein the alternative retinal position in the viewer's eye is selected to have a first height that is close to a second height of the preferred sensing position of the viewer's other eye. Aspect 5: The portable system of aspect 2, wherein the alternative retinal location in the viewer's eye is selected to be outside the fovea. Aspect 6: The portable system of Aspect 1, wherein the alternative retinal location in the viewer's eye has coordinates based on landmarks in the viewer's eye. Aspect 7: The portable system of Aspect 6, wherein the landmark in the viewer's eye is the optic disc of the viewer's eye. Aspect 8: The portable system of aspect 1, further comprising a feedback module configured to provide feedback when the pupil of the viewer's eye is away from the center of the viewer's eye by more than a predetermined amount based on eye information from the eye tracking module. Aspect 9: The portable system of aspect 8, wherein the feedback includes audio guidance or visual guidance. Aspect 10: The portable system of aspect 9, wherein the visual guidance includes a visible indicator that directs the viewer's eye movements. Aspect 11: The portable system of aspect 9, wherein the audio guidance includes audio feedback indicating orientation relative to the viewer's eye movements. Aspect 12: The portable system of aspect 1, wherein the first combiner redirects the plurality of first optical signals through approximately the center of the pupil of the viewer's eye to an alternative retinal location in the viewer's eye. Aspect 13: The portable system of aspect 1, wherein the first optical signal generator includes a laser light source. Aspect 14: The portable system of aspect 1, further comprising a process module that generates virtual image information for the virtual image display module or executes a training program. Aspect 15: The portable system of aspect 14, wherein the training program does not include the time during which the viewer's eyes blink in the predetermined training time. Aspect 16: The portable system of aspect 1, further comprising a height-adjustable chin holder. Aspect 17: The portable system of aspect 1, wherein the virtual image is a green cross. Aspect 18: The portable system of aspect 1, weighing less than 3 kilograms. Aspect 19: A system for improving vision in an eye of a viewer having a retinal disorder, comprising: an image capture module configured to receive a plurality of image pixels of a target object; a process module configured to generate virtual image information related to the target object; a virtual image display module that displays a virtual image based on the virtual image information, the virtual image not being centered on the fovea but being centered at an alternative retinal location in the viewer's eye; a first optical signal generator that generates a plurality of first optical signals for the virtual image; a first combiner that redirects the plurality of first optical signals from the first optical signal generator to alternative retinal locations in the viewer's eye to display a plurality of first pixels of the virtual image; a virtual image display module including: A system including: Aspect 20: The system of aspect 19, wherein the alternative retinal locations in the viewer's eyes are selected to facilitate binocular fusion. Aspect 21: The system of aspect 20, wherein the alternative retinal locations in the viewer's eyes are selected based on visual function and height of the alternative retinal locations. Aspect 22: The system of aspect 21, wherein the alternative retinal position in the viewer's eye is selected to have a first height that is close to a second height of the preferred sensing position of the viewer's other eye. Aspect 23: The system of aspect 20, wherein the alternative retinal location on the viewer's retina is selected to be outside the fovea. Example 24: The system of example 19, further comprising an eye tracking module that provides information about the viewer's eyes. Aspect 25: The system described in aspect 19, wherein the eye tracking module determines the target object based on the fixation of one or both eyes of the viewer. Aspect 26: The system of aspect 19, wherein the first combiner redirects the plurality of first optical signals through approximately the center of the pupil of the viewer's eye to an alternative retinal location in the viewer's eye. Aspect 27: The system of aspect 19, wherein the virtual image received by the viewer's eye and the actual image received by the viewer's other eye are partially fused. Aspect 28: The system of aspect 19, wherein natural light from the environment is reduced or prevented from entering the viewer's eyes. Aspect 29: The system described in aspect 19, wherein the virtual image information is generated at the viewing angle of the eye of a viewer with a retinal disorder. Aspect 30: The system described in aspect 19, further comprising a support structure mountable on a viewer's head, wherein the image capture module, the processing module, and the virtual image display module are carried by the support structure.

Claims

1. 1. A system for improving vision in an eye of a viewer having a retinal disorder, comprising: an image capture module configured to receive a plurality of image pixels of a target object; a process module configured to generate virtual image information related to the target object; a virtual image display module that projects a laser light signal pixel by pixel onto the retina of the viewer's eye based on the virtual image information to form the virtual image that is not centered on the fovea but is centered at an alternative retinal location in the viewer's eye; a first optical signal generator for generating a plurality of first optical signals for the virtual image; a first combiner that redirects the plurality of first optical signals from the first optical signal generator to an alternative retinal location in the viewer's eye to display a plurality of first pixels of the virtual image; a virtual image display module, wherein the alternate retinal location in the viewer's eye is a retinal location trained to substitute for the fovea for visual fixation; an eye tracking module that provides information about the viewer's eyes to the virtual image display module for redirecting the plurality of first optical signals from the first optical signal generator to alternative retinal locations in the viewer's eyes; Including, A system in which the horizontal and vertical coordinates of each of the plurality of first pixels in the impaired retina are adjusted according to the depth perception of each of the plurality of first pixels captured by a depth camera of the image capture module.

2. The system of claim 1 , wherein the alternative retinal locations in the viewer's eyes are selected to facilitate binocular fusion.

3. The system of claim 2 , wherein the alternative retinal locations in the viewer's eyes are selected based on visual function and height of the alternative retinal locations.

4. 4. The system of claim 3, wherein the alternate retinal position in the viewer's eye is selected to have a first height that is close to a second height of a preferred sensing position of the viewer's other eye.

5. The system of claim 2 , wherein the alternative retinal location on the viewer's retina is selected to be outside the fovea.

6. The system of claim 1 , wherein the eye tracking module determines the target object based on a fixation of one or both eyes of the viewer.

7. 10. The system of claim 1, wherein the first combiner is a reflector configured to redirect the plurality of first optical signals through approximately a center of a pupil of the viewer's eye to an alternative retinal location in the viewer's eye.

8. 10. The system of claim 1, wherein the virtual image display module is configured to form the virtual image such that the virtual image received by the viewer's eye is partially blended with an actual image received by the viewer's other eye.

9. 10. The system of claim 1, wherein the transmittance of the first combiner is adjusted to reduce or prevent natural light from the environment from entering the viewer's eyes, or the system further includes a light blocker.

10. The system of claim 1 , wherein the virtual image information is generated at a viewing angle of the eye of a viewer with the retinal disorder.

11. 10. The system of claim 1, further comprising a support structure mountable on a viewer's head, the image capture module, the processing module, the virtual image display module, and the eye tracking module being carried by the support structure.

12. The system of claim 1 , wherein the image capture module captures both a plurality of image pixels of the target object and their corresponding depths.

Images