Systems and methods for improving vision in patients with retinal damage

Abstract

A portable system and method for training a substitute retinal location of an eye of a retinal impaired viewer, and an auxiliary system for improving vision of such a viewer are disclosed. The portable system for training includes an eye tracking module for providing information of the viewer's eye and a virtual image display module for displaying a virtual image at a center of a substitute retinal location on the viewer's impaired retina instead of a center of a fovea. The virtual image display module further includes a first light signal generator for generating a plurality of first light signals and a first combiner for redirecting the plurality of first light signals to the substitute retinal location when a pupil of the viewer's eye is substantially located at a center of the eye.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 209,405, filed June 11, 2021, entitled “VISION-ASSISTED DEVICE FORUSERS WITH IMPAIRED RETINA”, the entire contents of which are incorporated herein by reference.

[0003] In addition, the entire contents of PCT international patent application No. PCT / US20 / 59317, filed on November 6, 2020, entitled “System and Method for Displaying an Object with Depth,” are incorporated herein by reference. Technical Field

[0004] This invention relates to a system for training the eyes of viewers with retinal impairment and improving the vision of such viewers; more specifically, this invention relates to a system for training an alternative retinal position on the eyes of viewers with retinal impairment to improve the vision of the viewer's eyes. Background Technology

[0005] People with retinal damage lose vision in the central or peripheral areas of their visual field. For example, patients with age-related macular degeneration (AMD) lose vision in the central area of ​​their visual field, while patients with glaucoma lose vision in the peripheral area. The eyes of people with retinal damage typically have a damaged macula (or macula), which is an oval-shaped pigmented area near the center of the retina. The human macula is typically about 5.5 mm (0.22 inches) in diameter and can be subdivided into the central fovea, the central fovea, the avascular area of ​​the central fovea, the central fovea, the parafovea, and the peripheral fovea. The macula is responsible for visual acuity and sharpness, providing central high-resolution color vision in good lighting conditions. The fovea is responsible for central vision (also known as foveal vision), which is essential for activities where visual detail is critical, such as reading and driving. The fovea is surrounded by the parafoveal band and the outer region of the peripheral fovea.

[0006] When a person's eye focuses on an object, they typically align their fovea with the object to achieve better image clarity. Therefore, the visual axis is defined as an imaginary line between the object and the fovea. As mentioned earlier, retinal damage can be caused by age-related macular degeneration (AMD), glaucoma, or other diseases. This can lead to blurred vision in the center or periphery of the visual field or complete vision loss. A person's vision can be improved by training the optimal retinal lobe (PRL) of a still-healthy eye to respond to received light signals. Therefore, we need a portable system that can train the optimal retinal lobe (PRL) in the eyes of people with retinal damage, as well as assistive systems to improve the viewer's vision. Summary of the Invention

[0007] This invention relates to a portable system and method for training an alternative retinal position on the eye of a person with retinal damage, thereby improving the viewer's vision. The viewer's retinal damage may be due to age-related macular degeneration (AMD), glaucoma, or other diseases. Macular degeneration in patients with AMD can lead to blurred or complete loss of vision in the central visual field. Patients with glaucoma may lose vision in the peripheral areas of their visual field rather than the central area. Vision in the central or peripheral areas of these patients can be improved by training an alternative retinal position on the patient's still-healthy eye in response to received light signals. Sometimes, the alternative retinal position is also referred to as the preferred retinal position (PRL). One portable system of the present invention for training an alternative retinal position on the eye of a person with retinal damage includes an eye-tracking module and a virtual image display module. The eye-tracking module provides eye information of the viewer's eye. Based on the eye information provided by the eye-tracking module, when the viewer's pupil is approximately in the center of the eye, the virtual image display module displays a virtual image at the center of the alternative retinal position of the viewer's eye, rather than at the center of the fovea. The virtual image display module includes a first light signal generator and a first combiner. A first light signal generator generates multiple first light signals for a virtual image. A first combiner redirects the multiple first light signals from the first light signal generator to an alternative retinal location on the viewer's eye to display multiple first pixels of the virtual image.

[0008] After training a viewer to fixate on an alternative retinal position instead of the fovea, the assistive system and method can project a virtual image of a target object onto an area adjacent to the fovea (for glaucoma patients) or onto an alternative retinal position in the eye of a trained retinal-damaged individual (for age-related macular degeneration patients) to improve the vision of the retinal-damaged individual. One vision-improving assistive system of the present invention includes an image capturing module, a processing module, and a virtual image display module. The image capturing module is configured to capture a view directly in front of the viewer's eye (a preset target object) or a specific target object being fixed on by the viewer's eye, thereby receiving multiple image pixels. The processing module is configured to generate information about a virtual image associated with the target object. The virtual image display module includes a first light signal generator and a first combiner. The first light signal generator generates multiple first light signals for the virtual image based on the virtual image information provided by the processing module. For individuals with macular degeneration, particularly those with damage to the fovea and its adjacent areas, such as those with age-related macular degeneration, the first combiner redirects the multiple first light signals from the first light signal generator to an alternative retinal position on the viewer's eye, instead of the fovea, to display the multiple first pixels of the virtual image. For patients with peripheral retinal damage (such as glaucoma patients), the first combiner redirects the first light signal to the still healthy central macula, including the fovea and its adjacent areas.

[0009] An alternative retinal position is selected from a portion of the retina that is still healthy. The selection criteria for an alternative retinal position include (1) the height of the alternative retinal position and (2) the relative position of the alternative retinal position to the fovea, so as to allow binocular fixation during eye movements. First, the first height of the alternative retinal position on the eye of a person with retinal damage should be chosen as a second height that is closer to the better sensing position of the other eye of the viewer, whether the retina is damaged or not. Second, the alternative retinal position should be chosen lateral to the fovea of ​​the eye of the person with retinal damage, so that when the viewer's eyes fixate on the peripheral area of ​​his / her visual field, the visual axes of both eyes, from either the alternating retinal position or the better sensing position, can intersect at the target object being fixed on by the viewer's eyes.

[0010] Once an alternative retinal location is selected, its coordinates are generated based on markers on the eye of the person with retinal damage. This coordinates provide an accurate location for the virtual image display module to project the virtual image. These markers can be the optic nerve head of the eye of the person with retinal damage. Attached Figure Description

[0011] Figure 1 This is a block diagram illustrating an embodiment of the system for training an alternative retinal position on the eye of a patient with retinal damage according to the present invention;

[0012] Figure 2This is a schematic diagram illustrating an embodiment of the virtual image display module and eye-tracking module of the present invention;

[0013] Figure 3 This is a schematic diagram illustrating an embodiment of the first optical signal generator and the first combiner of the present invention;

[0014] Figures 4A-4C This is a schematic diagram illustrating an embodiment of the virtual image display module of the present invention, which projects virtual images centered on a position that replaces the retina through different optical paths.

[0015] Figure 5 These are images illustrating an embodiment of the microscopic measurement images of the present invention;

[0016] Figure 6 These are images illustrating an embodiment of the present invention, showing a fundus image displaying the relative positions of the alternative retina, optic nerve head, and fovea;

[0017] Figures 7A-7D This is a schematic diagram illustrating an embodiment of the portable system of the present invention for training an alternative retinal position in the eye of a person with retinal damage;

[0018] Figure 8 This is a block diagram illustrating an embodiment of the assistive system of the present invention for improving vision in the eyes of individuals with damaged retinas;

[0019] Figures 9A-9C These are images illustrating embodiments of the present invention in relation to glaucoma;

[0020] Figure 10 These are schematic diagrams illustrating embodiments of the assistive system of the present invention for improving vision in individuals with damaged retinas; and

[0021] Figures 11A-11B This is a schematic diagram illustrating an embodiment of the present invention that uses depth information to adjust captured images. Detailed Implementation

[0022] The terminology used herein is intended to be interpreted in its broadest and most reasonable manner, even when used in conjunction with the techniques described in detail for certain specific embodiments. The following description may even emphasize certain terms; however, any term interpreted in a limited manner is specifically defined and described in this embodiment.

[0023] This invention relates to a portable system and method for training an alternative retinal position on the eye of a viewer with retinal damage, thereby improving the viewer's vision. The viewer's retinal damage may be caused by age-related macular degeneration (AMD), glaucoma, or other diseases. Macular degeneration in AMD patients can lead to blurred or complete loss of vision in the central visual field, while glaucoma patients lose vision in the peripheral area rather than the central area. Vision in the central or peripheral areas of these patients can be improved by training an alternative retinal position on the viewer's still-healthy eye in response to received light signals. This alternative retinal position is often also referred to as the preferred retinal position (PRL). The portable system for training an alternative retinal position on the eye of a viewer with retinal damage includes an eye-tracking module and a virtual image display module. The eye-tracking module provides eye information about the viewer's eye. When the viewer's pupil is approximately centered on the viewer's eye according to the eye information, the virtual image display module displays the alternative retinal position on the viewer's eye, rather than centered on the fovea. The virtual image display module includes a first light signal generator and a first combiner. In other words, in this scenario, the viewer's eyes are fixed directly forward, and the viewer's visual axis is approximately perpendicular to the viewer's frontal interaction. A first light signal generator produces multiple first light signals for the virtual image. A first combiner redirects the multiple first light signals from the first light signal generator to an alternative retinal location on the viewer's eye to display multiple first pixels of the virtual image.

[0024] After training a viewer's eye to use an alternative retinal position to replace the fovea for fixation, assistive systems and methods can improve the vision of a viewer's retinal-damaged eye and its adjacent areas (for glaucoma patients) or the alternative retinal position of a trained retinal-damaged viewer's eye (for age-related macular degeneration patients) by projecting a virtual image corresponding to the target object onto the fovea. One vision-improving assistive system of the present invention includes an image capturing module, a processing module, and a virtual image display module. The image capturing module is configured to capture a view directly in front of the viewer's eye (a preset target object) or a specific target object being fixed on by the viewer's eye, thereby receiving multiple image pixels. In another embodiment, the image capturing module receives the corresponding depths of the multiple image pixels. The processing module is configured to generate information about a virtual image associated with the target object. The virtual image display module includes a first light signal generator and a first combiner. The first light signal generator generates multiple first light signals for the virtual image based on the virtual image information provided by the processing module. For viewers with macular damage, particularly the fovea and its adjacent areas, such as patients with age-related macular degeneration (AMD), the first combiner redirects multiple first light signals from the first light signal generator to an alternative retinal location on the viewer's eye, rather than to the fovea, from multiple first pixels displaying the virtual image. For those with peripheral retinal damage (e.g., glaucoma patients), the first combiner redirects the first light signals to the still-healthy central area of ​​the macula, including the fovea and its adjacent areas.

[0025] Alternate retinal positions are selected from the still-healthy portion of the retina. Multiple positions on the viewer's retina can be used as alternative retinal positions. The choice of these positions is based on their potential to influence binocular fusion between the viewer's two eyes. Therefore, the alternative retinal position should be chosen in a location that promotes binocular fusion. The selection criteria for the alternative retinal position include (1) the height of the alternative retinal position and (2) the relative position of the alternative retinal position to the fovea, so as to allow binocular fixation during eye movements. First, the first height of the alternative retinal position on the eye of a person with retinal damage should be chosen as a second height that is closer to the better sensing position of the other eye of the viewer, whether the retina is damaged or not. In other words, the first height and the second height are approximately the same. Second, the alternative retinal position should be chosen lateral to the fovea of ​​the eye of a person with retinal damage, so that when the viewer's eyes fixate on the peripheral area of ​​his / her visual field, the binocular visual axes from either the alternating retinal position or the better sensing position can intersect at the target object being fixed on by the viewer's eyes.

[0026] Once an alternative retinal location is selected, its coordinates are generated based on markers on the eye of the person with retinal damage. This coordinates provide an accurate location for the virtual image display module to project the virtual image. These markers may include the optic nerve head of the eye in the person with retinal damage.

[0027] like Figure 1 As shown, the portable system 100 of the present invention for training the eyes of individuals with damaged retinas to establish alternative retinal positions includes an eye-tracking module 110 and a virtual image display module 120. The eye-tracking module 110 is configured to track the viewer's eyes and provide relevant eye information, such as eye movements, pupil position, pupil size, fixation angle (visual angle; visual axis), and the viewer's eye convergence angle. The eye-tracking module 110 may include a first camera 112 for tracking the eye with a damaged retina. The virtual image display module 120 projects a virtual image onto a preset alternative retinal position of the viewer's eye, providing training-appropriate stimulation when the eye information provided by the eye-tracking module 110 indicates that the viewer's pupil is approximately at the center of the eye. At this time, the viewer's eyes are focused on a point directly in front of them, and the viewer's visual axis is also approximately perpendicular to the viewer's frontal view. The virtual image can be preset by a doctor, training expert, or viewer. In one embodiment, the preset virtual image is a red or green cross symbol.

[0028] As described above, the eye-tracking module 110 is configured to track one or both eyes of a viewer and provide relevant eye information (e.g., pupil position, pupil size, fixation angle, and convergence angle for each eye of the viewer). This eye information can be used to determine whether the viewer's pupils are approximately centered in the eye of a person with retinal damage. Figure 2In one embodiment, the eye-tracking module 110 may include a first camera 112 and an eye-tracking reflector 114 to track the eyes of a person with retinal damage. In this embodiment, the infrared (IR) reflectivity of the eye-tracking reflector 114 is approximately 100%. The first camera 112 may further include an IR laser diode and an IR light sensor. The eye-tracking reflector 114 is positioned in the optical path between the first camera 112 and the viewer's eye. The IR light generated by the IR laser diode is reflected by the eye-tracking reflector 114 and then projected onto the viewer's eye. The infrared (IR) light reflected from the viewer's eye can be reflected back through the eye-tracking reflector 114 to the IR light sensor for analysis and determination of eye information (including pupil position). In another embodiment, both retinas of the viewer are damaged. The eye-tracking module 110 may further include a second camera 116 to track the viewer's other eye. In addition to conventional eye-tracking cameras, the first camera 112 and the second camera 116 may also be constructed using microelectromechanical systems (MEMS) technology. The first camera 112 and the second camera 116 can use ultra-infrared light emitters and sensors to detect and export various eye information. The eye-tracking module 110 may further include an integrated inertial measurement unit (IMU), which is an electronic device that uses a combination of accelerometers and gyroscopes to measure and report specific forces, angular rates, body conditions, etc. The integrated inertial measurement unit (IMU) may also further include a magnetometer.

[0029] The eye-tracking module 110 measures the position and size of the viewer's pupils and determines the extent or degree to which the pupils deviate from the center of the viewer's eyes. In one embodiment, the eye-tracking module 110 receives and analyzes 60 frames of reflected IR light per second to determine the pupil position. When the viewer's pupils deviate from the center of the viewer's eyes by more than a preset degree, such as 0.5 degrees, the eye-tracking module 110 notifies the virtual image display module 120 to rest.

[0030] like Figure 3As shown, the virtual image display module 120 includes a first light signal generator 10 and a first combiner 20. The first light signal generator 10 may use a laser, a light-emitting diode (“LED”) including miniature and micro-LEDs, an organic light-emitting diode (“OLED”) or a superluminescent light-emitting diode (“SLD”), a liquid crystal on silicon (LCoS), a liquid crystal display (“LCD”), or any combination thereof as its light source. In one embodiment, the light signal generator 10 is a laser beam scanning projector (LBS projector), which may include a light source 11 of a red laser 15, a green laser 16, and a blue laser 7, a light color modulator such as a dichroic combiner and a polarization combiner, and a two-dimensional (2D) tunable reflector 12, such as a 2D electromechanical system (“MEMS”) mirror. In another embodiment, the light source 11 may further include an infrared (IR) laser 14. The first light signal generator 10 may further include a collimator 13 located between the light source 11 and the 2D tunable reflector 12 to better align (parallelize) the direction of motion of the light signal in a particular direction. Collimator 160 can be a curved lens or a convex lens. 2D adjustable reflector 12 can be replaced by two one-dimensional (1D) reflectors, such as two one-dimensional MEMS mirrors. The LBS projector sequentially generates and scans light signals one by one to form a 2D virtual image with a preset resolution (e.g., 1280x720 pixels per frame). Therefore, the light signal for one pixel is generated and projected onto the first combiner 20 at a time. To allow a user to see such a 2D virtual image with one eye, the LBS projector must sequentially generate light signals for each pixel; for example, a 1280x720 light signal remains for a visual persistence period (e.g., 1 / 18 of a second). Therefore, the duration of each light signal is approximately 60.28 nanoseconds (ns).

[0031] In another embodiment, the first light signal generator 10 is a digital optical processing projector (“DLP projector”) capable of generating a 2D color image in a single operation. Texas Instruments’ DLP technology is one of many technologies that can be used to manufacture DLP projectors. The entire 2D color image frame (e.g., an image frame comprising 1280x720 pixels) is simultaneously projected onto the first combiner 20.

[0032] The first combiner 20 receives multiple light signals generated by the first light signal generator 10 and redirects them to an alternative retinal position in the viewer's eye, excluding the fovea. Here, the first combiner 20 can function as a reflector. The first combiner 20 can be made of glass or plastic materials such as lenses and coated with a specific material such as metal to make it reflective. One advantage of using a reflective combiner instead of a waveguide to guide light signals to the user's eye in the prior art is the elimination of undesirable diffraction effects, such as multiple shadows, color shifts, etc.

[0033] In such Figure 2 In another embodiment shown, the optical path of the virtual image display module 120 may be further designed to include an auxiliary first combiner 25. A light signal generated from the first light signal generator 10 is projected onto the first combiner 20, which redirects the light signal to the auxiliary first combiner 25, which further redirects the light signal to an alternative retinal position of the viewer's eye, excluding the fovea. Furthermore, the virtual image display module 120 may further include a safety reflector 122 disposed between the first combiner 20 and the auxiliary first combiner 25, and a safety sensor 124. In one embodiment, the reflector 122 has a reflectivity of approximately 10%, allowing approximately 90% of the light signal to pass through. The safety sensor 124 receives the reflected light signals from the reflector 122 and measures the intensity of these signals. If the intensity of the light signal exceeds a preset value, the safety sensor 124, for safety reasons, notifies the first light signal generator 10 to turn off the power to the light source or prevent the light signal from being projected into the viewer's eyes to avoid damage to the viewer's eyes.

[0034] In one embodiment, to precisely control the position of the first light signal projected onto the viewer's eye, the first combiner 20 and the auxiliary first combiner 25, each with six degrees of freedom, can independently adjust specific angles of the horizontal axis (or pitch axis, X-axis), vertical axis (or longitudinal axis, Y-axis), and / or depth axis (or vertical axis, Z-axis), for example, by rotation of 5 degrees. The horizontal axis can be set along the direction of the pupil line. The vertical axis can be set to extend along the midline of the face and be perpendicular to the horizontal direction. The depth direction (or vertical axis, Z-axis direction) can be set perpendicular to the frontal plane and perpendicular to both the horizontal and vertical directions. Specifically, the first combiner 20 and the auxiliary first combiner 25 can rotate about the horizontal axis to move the signal projection position above or below the viewer's retina, rotate about the vertical axis to move the light signal projection position to the right or left side of the viewer's retina, and / or move along the depth axis to adjust the interpupillary distance.

[0035] As described above, the virtual image display module 120 projects a virtual image onto a preset alternative retinal position of the viewer's eye, and provides the necessary stimulation at the viewer's pupil when the image is approximately centered on the viewer's eye, based on eye information from the eye-tracking module 120. At this time, the viewer's eye is focused on a point directly in front, and the viewer's visual axis is nearly perpendicular to the plane in front of the viewer. The visual axis is an imaginary line connecting the viewer's fixation point and the fovea through the pupil. This is the point where the viewer most naturally and easily fixates. As a result, the viewer does not need to rotate his / her eyeballs to train the alternative retinal position. Through such fixation training, users with retinal damage, such as patients with macular degeneration, can achieve direct and focused gaze without needing to turn their head to the side of the eye with retinal damage to see the central portion of the image. The eye-tracking module 120 can detect the position and size of the pupil of the retinal-damaged eye and then determine whether the pupil is centered on the viewer's eye. When the pupil is centered in the viewer's eye, the virtual image display module 120 projects a light signal onto a preset alternative retina position. The virtual image display module 120 can pause projection when the pupil deviates from the center of the viewer's eye by a preset amount (e.g., 1 degree), because in that case, the light signal will be projected onto an independent position other than the alternative retina position used for training. When the pupil deviates from the center of the viewer's eye to a certain extent, the light signal may even fail to pass through the pupil, because the system is calibrated to allow the viewer to project the light signal onto a fixed position.

[0036] like Figures 4A-4C As shown, the virtual image display module 120 can project the light signals forming the virtual image 440 onto alternating retinal positions 420 via different optical paths. Specifically, the virtual image is projected onto the viewer's retina, centered on the alternating retinal positions 420 rather than the fovea 410. In one embodiment, the virtual image contains 921,600 pixels in a 1280x720 array. The light signals forming the virtual image can be considered as light beams. Based on the optical path at the center of the light beam, the optical path projection of the light signals can be divided into three categories. Figure 4A In the process, the light signal forming the virtual image 440 is projected through approximately the central portion of the pupil 430; Figure 4B In the process, the light signal forming the virtual image 440 is projected through the upper part of the pupil 430; Figure 4CIn this process, the light signal forming the virtual image 440 is projected through the lower portion of the pupil 430. Alternatively, the light signal forming the virtual image 440 can be projected through the right or left portion of the pupil 430. Projecting the light signal forming the virtual image through approximately the center of the pupil may have the following advantages. First, even with strong ambient light and a smaller pupil size, the virtual image is less likely to be partially obscured. Second, the angle of incidence of the light signal projected onto the virtual image at the surrogate retina position is generally smaller. The first combiner 20 and / or the auxiliary first combiner 25 can be adjusted to perform the projection of the light signal along a selected optical path.

[0037] The system 100 of the present invention may further include a fundus visual field measuring instrument 130 to generate a "retinal sensitivity map" of the amount of light perceived in a specific portion of the retina in the viewer's eye for visual field testing. To avoid repetition, the fundus visual field measuring instrument 130 may share the light source 11 and certain optical elements with the virtual image display module 120. Figure 3 In one embodiment shown, the fundus visual field measuring device 130 includes a light source 11, a set of optical elements 131, a light intensity sensor 136, and a visual field controller 138. The optical element set 131 may include three reflectors 132, 133, and 134 to guide light reflected from the viewer's eye onto the light intensity sensor 136, which may be a charge-coupled device (CCD). The visual field controller 138 may receive electrical signals from the light intensity sensor 136 to generate... Figure 5 The retinal sensitivity map shown provides information to the physician to select an alternative retinal position. An alternative retinal position can be selected according to several guidelines to facilitate fixation. In one embodiment, the fundus visual field meter 130 may be a microsurgical fundus visual field meter or a laser scanning ophthalmoscope (SLO).

[0038] As mentioned earlier, for individuals with retinal damage, an alternative retinal site can be selected from the still-healthy portion of the retina. Multiple locations on the viewer's retina can be used as alternative retinal sites. Figure 5 As shown, micro-field maps typically use color to indicate the health of the viewer's retina. For example, green indicates healthy (fully functional), yellow indicates partially damaged but still functional to some extent (partially functional), and red indicates damaged (no function). Therefore, Figure 5The color of each small square in the diagram represents the degree of retinal function at each specific location. Generally, green indicates full function; yellow indicates partial function; and red indicates no function. The selection of an alternative retinal position from these multiple available healthy positions for training will affect the likelihood of binocular fusion between the viewer's two eyes (e.g., one age-related macular degeneration (AMD) eye and one normal eye, or both AMD eyes). Therefore, it is necessary to select an alternative retinal position to promote binocular fusion. The selection criteria for an alternative retinal position include (1) the height of the alternative retinal position and (2) the relative position of the alternative retinal position to the fovea to allow binocular fixation during eye movements. First, the first height of the alternative retinal position for the retinal-damaged eye should be selected as a second height that is closer to the optimal sensing position of the other eye of the viewer, whether the retina is damaged or not. If the alternative retinal position for the retinal-damaged eye is approximately at the same height as the optimal sensing position of the other eye of the viewer (e.g., the fovea of ​​a normal eye), in other words, the first height and the second height are approximately the same, binocular fixation is more likely to occur. Secondly, the alternative retinal position should be chosen on the outer side of the fovea of ​​the eye of the person with retinal damage, so that when the viewer's eyes are focused on the peripheral area of ​​his / her field of vision, the visual axes of both eyes can cross each other at the target object being viewed by the viewer, whether from the alternating retinal position or the better sensing position.

[0039] Once the location of the alternative retina 630 is determined, 2D coordinates can be generated and the location of the alternative retina can be accurately indicated based on the markings. In such cases... Figure 6 In one embodiment shown, the optic nerve head 610 of the viewer's eye is used as a marker to determine the position of the fovea 620. Then, assuming the fovea 620 is the origin with coordinates (0,0), the coordinates of the retinal position 630 can also be determined.

[0040] The system 100 of the present invention may further include a processing module 140 for executing a training program for a viewer. The processing module 140 may include a processor and memory, serving as the computing center for other modules of the system 100 (e.g., eye-tracking module 110 and virtual image display module 120). A training application / software may be installed in the processing module 140 to perform training for the viewer. Training courses can be customized for each individual. Furthermore, since the system 100 of the present invention is portable, viewers can easily train at home. In one embodiment, a training session lasts approximately 15 minutes. The time a viewer blinks may not be counted in the training session time. An artificial intelligence (AI) model can be used to determine whether blinking occurs. During the training session, the viewer can choose the shape, size, and color of the virtual image used for training, such as a red or green cross or a red or green circle. At the start of training, the viewer's pupils may frequently deviate from the center of their eyes; therefore, a larger virtual image can be used for training. When the viewer's pupils are fixed forward for extended periods, a smaller virtual image can be used for training. The training program can record all relevant data detected during training and generate a training report. All relevant training data and reports can be remotely uploaded to the clinic's or hospital's information system for doctors to use in diagnosis.

[0041] The system 100 of the present invention may further include a feedback module 150, which is configured to provide feedback to the viewer based on eye information when the viewer's pupil is more than a preset distance (e.g., 0.5 degrees) from the center of the viewer's eye. In other words, when the viewer's eyes are no longer directly looking forward and the viewer's visual axis is not perpendicular to the viewer's frontal plane, the feedback module 150 can provide sound and / or visual feedback and guide the viewer's pupil back to the center of the eye. Visual guidance includes a visual indicator indicating the direction of the viewer's eye movement, such as a flashing arrow showing the direction the viewer's pupil should move. Such visual guidance can be displayed by a virtual image display module 120. Sound guidance includes sound feedback to indicate the direction of the viewer's eye movement, which can be performed by a speaker.

[0042] The system 100 of the present invention may further include an interface module 160, which allows the viewer to control various functions of the system 100. The interface module 160 can be operated by voice, gestures, finger / foot movements, and via pedals, keyboard, mouse, knobs, switches, styluses, buttons, joysticks, touchscreens, etc.

[0043] like Figures 7A-7DAs shown, in addition to the light engine 175 including the eye-tracking module 110, the virtual image display module 120, the fundus visual field measurement module 130, and the processing module 140, the portable system 100 may further include a frame 170, which includes a base 171, a chin support 172, a forehead support 173, and a tablet connector 174. The height of the chin support 172 is adjustable. The relative position of the forehead support 173 can be adjusted towards or away from the viewer. In one embodiment, the system 100 with the frame 170 has dimensions of approximately 50-65 cm (height), 30 cm (width), and 30 cm (depth). Furthermore, in one embodiment, the system 100 with the frame 170 weighs approximately 3 kg.

[0044] After the portable system 100 of the present invention trains the viewer's eye to adopt an alternative retinal position for fixation, the viewer can use the assistive system 200 to project a virtual image corresponding to the target object onto their eye to improve the vision of the viewer's retinal-damaged eye and to train the alternative retinal position of the eye in retinal-damaged individuals. Figure 8 As shown, the assistive system 200 for improving vision includes an image capturing module 210, a processing module 220, and a virtual image display module 230. The image capturing module 210 is configured to receive a plurality of image pixels and the corresponding depth of a target object 205. In one embodiment, the image capturing module 210 captures the direct field of vision in front of the viewer's eyes as the target object. In other words, the viewing angle of the image capturing module 210 is perpendicular to the front of the viewer wearing the assistive system 200. The processing module 220 generates information about a virtual image associated with the target object. The virtual image display module 230 displays the virtual image on the eye of a person with retinal damage based on the information from the virtual image. For viewers with macular damage, particularly the fovea and its adjacent areas, such as patients with age-related macular degeneration (AMD), the virtual image display module 230 can project the virtual image at the center of an alternative retinal location in the viewer's eye, rather than at the center of the fovea. For viewers with damage in the peripheral visual field of the retina, such as glaucoma patients, the virtual image display module 230 projects a virtual image at the center of the still-healthy central macula, including the fovea and its adjacent areas. In this case, as... Figures 9A-9C As shown, the virtual image can be scaled down because the still healthy portion of the retina in the central region, which can receive and respond to light signals, is smaller. Thus, the scaled-down virtual image with the same field of view, although smaller in size, is sensed as the target object originally captured by the image capturing module 210. Figure 9A It is an illustration of the view as perceived by a viewer's healthy eye. Figure 9B This is a diagram illustrating how the eyes of a glaucoma patient perceive things. Figure 9CThe illustration shows the view perceived by the eye of a glaucoma patient when the virtual image display module 230 projects a scaled-down virtual image of a target object onto the fovea of ​​the eye of a retinal-damaged patient. To avoid interruption of natural light received by the viewer's eye from the environment, the assistive system 200 can reduce or block natural light from entering the eye of the retinal-damaged patient. Thus, the eye of the retinal-damaged patient will primarily or almost exclusively perceive the virtual image projected by the virtual image display module 230. The virtual image perceived by the retinal-damaged patient's eye and the real image perceived by the viewer's other healthy eye may at least partially merge into a single image. Binocular fusion can also occur when each of the viewer's eyes has a damaged retina and receives a virtual image separately from the virtual image display module 230.

[0045] The vision-improving assistive system 200 of the present invention may further include an eye-tracking module 240 and an interface module 250. Similar to the eye-tracking module 110 in the training system 100, the eye-tracking module 240 in the assistive system 200 may be configured to track one or both eyes of a viewer and provide relevant eye information, such as eye movements, pupil position, pupil size, fixation angle (viewing angle; visual axis), and the convergence angle of the viewer's eyes. The eye-tracking module 240 may further include cameras 242 and 244 to determine target objects based on the fixation of one or both eyes of the viewer. The interface module 250 allows the viewer to control various functions of the assistive system 200. The interface module 250 can be operated via voice, gestures, or finger movements, and via pedals, keyboard, mouse, knobs, switches, styluses, buttons, joysticks, touchscreens, etc.

[0046] like Figure 10 As shown, the assistive system 200 also includes a support structure 260 that can be worn on the viewer's head. The image capturing module 210, processing module 220, virtual image display module 230 (including a first light signal generator 10, a first combiner 20, and even a second light signal generator 30 and a second combiner 40) are supported by the support structure. In one embodiment, the assistive system 200 is a head-mounted device, such as virtual reality (VR) goggles and augmented reality (AR) / mixed reality (MR) glasses. In this case, the support structure can be a frame with or without lenses for the glasses. The lenses can be prescription lenses for correcting myopia, hyperopia, etc. In addition, the eye-tracking module 240 and the interface module 250 can also be supported by the support structure.

[0047] Image capturing module 210 includes at least one RGB camera 212 to receive multiple image pixels of a target object, i.e., a target image. In another embodiment, image capturing module 210 may further include at least one depth camera 214 to receive the corresponding depth of the multiple image pixels. Additionally, image capturing module 210 may include positioning elements to receive the corresponding depth of the multiple image pixels and the target object. To measure the depth of the target object and the environment, depth camera 214 may be a time-of-flight (ToF) camera, which uses time-of-flight technology to determine the distance between each point between the camera and the object. Images are acquired by measuring the round-trip time of an artificial light signal provided by a laser or LED (such as LiDAR). ToF cameras can measure distances from a few centimeters to several kilometers. Other devices, such as structured light modules, ultrasonic modules, or infrared modules, can also be used as depth cameras to detect the depth of target objects and the environment.

[0048] To merge depth information into multiple image pixels for a more accurate determination of the target object's coordinates and shape, an adjustment process was performed. Multiple image pixels provide two-dimensional coordinates, such as XY coordinates, for each feature point of the target object. However, such two-dimensional coordinates are inaccurate because they do not take depth into account. Therefore, as... Figures 11A-11B As shown, the image capturing module 210 can align or overlap an RGB image containing multiple image pixels with a depth map, so that feature points in the RGB image are superimposed on corresponding feature points in the depth map, thereby obtaining the depth of each feature point. The RGB image and the depth map may have different sharpness and size. Therefore, in situations such as... Figure 11B In the illustrated embodiment, the outer portion of the depth map that does not overlap with the RGB image can be clipped. The depth of the feature point is used to calibrate the XY coordinates from the RGB image to obtain the true XY coordinates. For example, a feature point has XY coordinates (a, c) in the RGB image and z-coordinate (depth) in the depth map. The true XY coordinates are (a + b * depth, c + d * depth), where b and d are calibration parameters, and the symbol "*" represents multiplication. Therefore, the image capturing module 210 uses multiple simultaneously captured image pixels and their corresponding depths to adjust the x and y coordinates of the target object respectively.

[0049] Processing module 220 may include a processor and memory to generate information about a virtual image associated with a target object. Furthermore, processing module 220 may serve as the computational center for other modules of auxiliary system 200 (e.g., image capturing module 210 and virtual image display module 230). To generate information about the virtual image, the viewpoint of the target object from the viewer's damaged retina and other 3D-related effects, such as the intensity and brightness of red, blue, and green hues, as well as shadows, may be taken into consideration.

[0050] Similar to the virtual image display module 120 in the portable training system 100 of the present invention, the virtual image display module 230 in the visual assistance system 200 of the present invention includes a first light signal generator 10 and a first combiner 20 for projecting virtual images onto the eye of a person with retinal damage. The virtual image display module 230 may further include a second light signal generator 30 and a second combiner 40 for the viewer's other eye, which may also have a damaged retina or a healthy retina. The above description of the first light signal generator 10 and the first combiner 20 applies to the second light signal generator 30 and the second combiner 40. Similarly, for individuals with damage to the central macula, particularly the fovea and its adjacent areas, such as patients with age-related macular degeneration (AMD), the first light signal generator 10 generates multiple first light signals for the virtual image based on information from the processing module 220. The first combiner 20 redirects the multiple first light signals from the first light signal generator 10 to a location that replaces the retina in the viewer's eye, rather than the damaged fovea and its adjacent areas, to display multiple first pixels of the virtual image. For viewers with damaged retinas in the peripheral regions of their field of vision, such as glaucoma patients, the first light signal generator 10 generates multiple first light signals for the virtual image based on information from the processing module 220. The first combiner 20 redirects the multiple first light signals from the first light signal generator 10 to the still healthy central region of the macula, including the fovea and its adjacent areas.

[0051] As described above, the virtual image display module 230, for example by adjusting the combiners 20 and 40, can project the light signal forming the virtual image onto an alternative retinal position or a preferred sensing position, such as the fovea and its adjacent area, through different optical paths. Typically, the light signal forming the virtual image can be projected through approximately the central portion of the pupil, the right portion of the pupil, and the left portion of the pupil. In one embodiment, the light signal forming the virtual image is projected through approximately the central portion of the pupil to avoid any part of the virtual image being obstructed, since the pupil size would be reduced due to strong ambient light.

[0052] To reduce or block natural light from the environment, the transparency of the first combiner 20 and the second combiner 40 can be adjusted automatically or by the viewer via the interface module 250 when needed. In another embodiment, the assistive system 200 of the present invention may further include a light shield to reduce or prevent natural light from the environment from entering the viewer's eyes.

[0053] In addition to red, green, and blue lasers, the light sources 11 and 21 of the first and second light signal generators 10 and 20 may further include IR (infrared) lasers, such as micropulse generators, to generate low-power, high-density electromagnetic waves with wavelengths of approximately 532 nm, 577 nm, or 810 nm, which are radiated onto the viewer's retina to achieve a massage effect. In one embodiment, 810 nm infrared light is radiated onto the viewer's retina. This electromagnetic radiation generates heat shock proteins (HSPs). HSPs can help reactivate cells in the retina, thereby slowing age-related macular degeneration. Furthermore, since infrared light is invisible to the human eye, it can simultaneously radiate onto the viewer's retina when the red, green, and blue lasers of the light sources 11 and 21 generate virtual images to be projected onto the viewer's retina. Therefore, the infrared light does not interfere with the virtual image composed of red, green, and blue light signals. Alternatively, the infrared light can be projected between two consecutive image frames.

[0054] like Figure 3 As shown, the intensity of the infrared light radiating to the viewer's retina must be monitored and controlled to avoid retinal damage. Lens 310 is used to collect infrared light reflected from the viewer's eye for the IR light sensor 320 to measure its intensity. When the intensity is too low, a photomultiplier tube (PMT) 330 is used to amplify the intensity signal. IR intensity controller 340 is used to determine whether the intensity of the IR laser diode 14 needs adjustment. If adjustment is required, IR intensity controller 340 sends a signal to the first light signal generator 10 requesting adjustment.

[0055] In another embodiment, the light sources 11, 31 of the light signal generators 10, 30 may further include a light generator that provides light of a specific wavelength to activate channelrhodopsins, thereby providing adjunctive therapy for patients with retinitis pigmentosa (RP). This clinical treatment was first developed by RetroSense Therapeutics, a biotechnology company that develops gene therapies to enhance vision in patients with retinitis pigmentosa (RP) that cause blindness. Retinitis pigmentosa (RP) is a group of inherited diseases characterized by peripheral vision loss and difficulty at night, which in many cases eventually leads to central vision loss and blindness. Retinitis pigmentosa (RP) typically occurs in adolescents and young adults.

[0056] All components in the training system 100 or auxiliary system 200 of this invention can be used exclusively by a single module or shared by multiple modules to perform the required functions. Furthermore, two or more modules described herein can be implemented by a single physical module. A module described herein can be implemented by two or more separate modules. An external server is not part of the auxiliary system 200 but can provide additional computing power for more complex calculations. Each of the above modules can communicate with the external server via wired or wireless means. Wireless means can include WiFi, Bluetooth, Near Field Communication (NFC), the Internet, telecommunications, radio frequency (RF), etc.

[0057] The embodiments described in this specification provide various possible and non-limiting embodiments of the present technology. After reading this specification, those skilled in the art will recognize that changes can be made to the embodiments described herein without departing from the scope of the present technology. It must be emphasized that the above detailed description is a specific illustration of feasible embodiments of the present invention; however, these embodiments are not intended to limit the scope of the present invention. All equivalent implementations or modifications that do not depart from the spirit of the present invention should be included within the scope of protection of this application.

Claims

1. A system for improving vision in viewers with retinal damage, comprising: An image capturing module is used to receive an image pixel of each of multiple feature points of a target object and a corresponding depth of the image pixel; A processing module is used to generate information about a virtual image related to the target object; An eye-tracking module is used to provide information about one of the viewer's eyes; and A virtual image display module, based on information from the virtual image, displays the virtual image at the center of a substitute retina location on the viewer's eye, rather than at the center of the fovea, comprising: A first optical signal generator, which is a laser beam scanning projector, includes a light source, a two-dimensional adjustable reflector, and a collimator located between the light source and the two-dimensional adjustable reflector. The first optical signal generator generates a plurality of first optical signals for the virtual image; and A first combiner is configured to redirect the plurality of first light signals from the first light signal generator to the alternative retinal position on the viewer's eye, based on information from the eye-tracking module, when the viewer's pupil is approximately located at the center of the viewer's eye, in order to display the virtual image; The virtual image perceived by the viewer's eye and an image perceived by the viewer's other eye are at least partially merged into a single visual image; The image pixel includes a horizontal coordinate and a vertical coordinate, and The image capturing module uses the corresponding depth to adjust the horizontal and vertical coordinates of the image pixels.

2. The system of claim 1, wherein the alternative retinal position on the viewer's eye is selected to obtain a first height at a second height that is closer to a better sensing position of the viewer's other eye.

3. The system of claim 1, wherein the alternative retinal position on the viewer's retina is selected such that it is located outside the fovea.

4. The system of claim 1, wherein the eye-tracking module determines the target object being gazed at based on the gaze position of one or both of the viewer's eyes.

5. The system of claim 1, wherein the first combiner redirects the plurality of first light signals to the alternative retinal location on the viewer's eye via approximately the center of the viewer's pupil.

6. The system of claim 1, wherein natural light from the environment is reduced or blocked from entering the viewer's eyes.

7. The system of claim 1, wherein the information of the virtual image is generated based on a viewpoint of the viewer's eye.

8. The system of claim 1, further comprising: A support structure that can be worn on the viewer's head; wherein the support structure carries the image capturing module, the processing module and the virtual image display module.

Images