Optoelectronic binocular instrument for hypermetropia correction and method for binocular hypermetropia correction

Abstract

The invention relates to an opto-electronic binocular instrument for hypermetropia correction and a method for binocular hypermetropia correction. The instrument comprises two opto-electronic lenses (103, 110; 203, 204) and an imaging subsystem that captures images of the eyes. Using pupil tracking performed by processing the images of the eyes, the system is able to determine the depth at which the subject is gazing. Pupil tracking is performed at very high speed, using a high-performance graphics card and a highly parallelized pupil detection algorithm. The method comprises two steps. In a first step, a calibration is performed, in which the subject is asked to gaze at objects at different distances and the size and position of the pupils are measured. In a second step of the method, a correction is performed using the instrument. The system continuously captures and processes images to calculate the correction to be applied and finally corrects hypermetropia by applying said correction.

Description

Technical Field

[0001] This invention relates to an optoelectronic binocular device for automatically correcting hyperopia in the eye in real time without subject intervention, and a method for correcting hyperopia using such a device. Background Technology

[0002] Farsightedness affects 100% of the population over 50. It's a natural aging process that causes the eye to lose its accommodative ability as we age. Therefore, automatic farsightedness correction is a hot topic globally with a huge potential market.

[0003] As a large problem, several solutions have been proposed. However, most solutions used to date are localized and based on static approaches. The simplest solution currently available is to use monofocal glasses to see near objects. Typically, these glasses have moderate correction, allowing focus on near objects, but defocus on distant objects. Another solution that utilizes the same glasses to provide near and far vision is bifocal or progressive lenses. However, these lenses also present some challenges in terms of stereoscopic vision and the training required to view each distance within the optimal area for using the lenses.

[0004] Another widely used method is called monocular vision. This technique uses a different focus for each eye, one always focusing on distant objects and the other always focusing on near objects. The brain selects the appropriate image based on the viewing distance. However, this method has significant drawbacks. Only a narrowed range of distances can achieve perfect focusing for each eye. Furthermore, summing the images from both eyes is impossible, thus affecting stereoscopic vision.

[0005] There are other systems based on the use of photoelectric lenses, such as the photoelectric lens described in document US20120133891A1, in which the photoelectric lens is proposed to use an improved CMOS sensor to perform corneal reflection tracking to correct myopia (not hyperopia).

[0006] In document US8690321B2, a photoelectric lens was used to correct farsightedness using an external camera that tracks the subject's face and eyes. The acquired information was then sent to the eyeglasses. This invention is quite limited because it relies on an external camera to make the system work.

[0007] In document WO2014179857A1, a system with an adjustable lens was proposed. This method uses gaze to control the applied optical power. However, only a limited range of optical power values ​​can be applied to the lens. These discontinuous optical power values ​​may lead to an uncomfortable experience for the subject.

[0008] In the paper US20120194781A1, a system using a dynamic focusing Alvarez lens and corneal reflection was proposed to correct farsightedness.

[0009] Furthermore, pupil tracking is a widely used technology in many fields, from marketing to medicine and video games. As a popular technology, there are many alternative methods to achieve it.

[0010] For example, Mompeán, J., Aragón, JL, Prieto P., and Artal P have proposed a highly parallelized implementation of pupil tracking that performs at high speed and high precision using a graphics processing unit (GPU). (“GPU-Accelerated High-Speed ​​Eye Pupil Tracking System”, 27th International Symposium on Computer Architecture and High-Performance Computing, SBAC-PAD, pp. 17-24, October 2015).

[0011] Emmanuel Chirre, Pedro Prieto, and Pablo Artal have proposed an alternative approach to understanding the relationship between pupillary dynamics and eye accommodation (“Dynamics of the near response under natural viewing conditions with an open-view sensor”, Biomed. Opt. Express 6, 4200-4211, 2015). This information is used to train and control the responses of subjects. Summary of the Invention

[0012] The purpose of the present invention presented in this patent is to provide a method and apparatus for real-time automatic binocular hyperopia correction without any action from the subject.

[0013] This invention provides a photoelectric binocular device for hyperopia correction, comprising:

[0014] A high-speed pupil tracking system, comprising:

[0015] An image capture subsystem for the eyes of a subject undergoing farsightedness correction includes at least one camera and several light sources oriented towards the eyes, and

[0016] An image processing subsystem, used to calculate pupil size and interpupillary distance, is capable of processing images of both eyes using a high-performance implementation based on a highly parallel pupil tracking algorithm utilizing a graphics processing unit (GPU).

[0017] Two photoelectric lenses, which provide variable optical power and can be positioned in front of the eyes of the subject undergoing farsightedness correction, one in front of each eye.

[0018] The camera is used to perform pupil tracking, and the lens is responsible for applying the desired optical correction at each moment.

[0019] The present invention also provides a method for correcting binocular hyperopia, which uses an optoelectronic binocular device for correcting binocular hyperopia according to the preceding claims, and includes a first calibration stage and a second hyperopia correction stage having the following steps:

[0020] a) Calibration phase:

[0021] Subjects undergoing farsightedness correction look towards objects placed at a distance d. 远 distant goals,

[0022] Images are captured using the image capture subsystem.

[0023] The captured image is processed by the image processing subsystem, and the pupil size S is calculated. 远 And interpupillary distance,

[0024] The subject being corrected looks at the object placed at a distance d. 近 The nearest target,

[0025] Images are captured via the image capture subsystem.

[0026] The captured image is processed by the image processing subsystem, and the pupil size S is calculated. 近 And interpupillary distance,

[0027] b) Hyperopia correction stage:

[0028] Images are captured via the image capture subsystem.

[0029] The captured image is processed by the image processing subsystem, and the pupil size S is calculated. 当前 And interpupillary distance,

[0030] The following formulas are used to calculate the distance D that the subject sees:

[0031] D = d 近 +[(d 远 -d 近 ) / (s 远 -s 近 )]*(s 当前 -s 近 )

[0032] D=(H / 2)*tan(90-α), where α=atan(d / r)

[0033] Where d is the distance the pupil moves, α is the angle of eye rotation, r is the radius of the eye, and H is the interpupillary distance when the subject is looking at infinity.

[0034] The optical power to be applied to the photoelectric lens is calculated based on the distance D, and

[0035] The calculated optical power is applied to the photoelectric lens.

[0036] In the hyperopia correction stage, each step is performed several times consecutively.

[0037] Correction is guided by information received from a proprietary system that tracks the size, shape, and position of the eye's pupils.

[0038] This invention performs continuous tracking of the size and position of the pupil in each eye. These parameters are used to calculate the corrections that must be applied to the lens, photoelectric system, or variable-focus optical engine system at each moment. Pupil tracking is performed by capturing images of both eyes with a camera and providing appropriate illumination. The images are then processed using a high-performance and parallel implementation of a pupil detection algorithm in a graphics processing unit (GPU). Attached Figure Description

[0039] Figure 1 A diagram of a binocular photoelectric instrument for hyperopia correction according to the present invention is shown.

[0040] Figure 2 A simplified version of the binocular photoelectric instrument for hyperopia correction according to the present invention is shown in the figure.

[0041] Figure 3 This shows the changes in pupil size when the human eye looks at distant and near objects.

[0042] Figure 4 This illustrates the changes in pupil position when the human eye looks at distant and near targets.

[0043] Figure 5 A diagram is shown of a method for real-time binocular hyperopia correction, which includes a calibration phase and a correction phase with their respective steps.

[0044] Figure 6 A diagram illustrating the operation of the instrument of the present invention is shown.

[0045] Figure 7 This diagram illustrates the angle of the eye when looking at a nearby target. Detailed Implementation

[0046] This invention includes a method and associated instrument for automatic and binocular correction of farsightedness in real time. Furthermore, the method has three different modes for controlling the application of correction.

[0047] exist Figure 1 and Figure 2 The diagram schematically illustrates the most important part of the actual implementation of the instrument capable of carrying out the method proposed in this invention. Figure 2 It is a simplified version of the instrument. Figure 1 The system includes two LEDs 102 and 109 for infrared illumination; illumination is necessary so that the camera 107 can capture high-quality images. Additionally, the system has two photoelectric lenses 103 and 110 placed in front of the subject's eyes 101 and 108. These are used to apply calculated optical power so that the subject can focus on a target. On the other hand, the system has multiple lenses 104, 105, 111, 112, and lens 106 to correctly focus the image of the pupil in the camera. The system continuously captures images processed by the pupil tracking subsystem. High-performance parallel implementation is used for pupil tracking using a graphics processing unit (GPU). Using the obtained parameters regarding the size and position of the two pupils, the optical power that lenses 103 and 110 must apply to correct the subject's farsightedness is calculated. Finally, the photoelectric lenses 103 and 110 apply correction to the object.

[0048] Figure 2 A simplified version of the optical instrument proposed in this invention is shown. The system has two infrared LEDs 207 and 208 for illumination; this illumination is necessary for cameras 202 and 205 so they can capture high-quality images. Additionally, two photoelectric lenses 203 and 204 are placed in front of the eyes of subjects 201 and 206. As in the unsimplified version, the system continuously captures images, which are then processed by a pupil tracking subsystem. The obtained information is then used to calculate the optical power that lenses 203 and 204 must apply to correct the subject's hyperopia. Finally, the correction is applied using photoelectric lenses 203 and 204.

[0049] When the pupil of the human eye focuses on a stimulus based on its distance, it decreases and increases in size. Figure 3 The diagram illustrates the pupillary constriction when viewing a near target 302 and the pupillary dilation when viewing a distant target 301. This effect is used in the described method to calculate the distance the subject is viewing, thereby determining the optical power that should be applied to the lens; this is one of three methods presented here for calculating this value. During calibration, pupil size is calculated and stored while the subject is viewing both near and distant targets. This information is later used in the correction phase to interpolate the distance the subject is viewing. The formula used is:

[0050] D = d 近 +[(d 远 -d 近 ) / (S远 -S 近 )]*(S 当前 -S 近 )

[0051] Furthermore, the human eye can converge and diverge when focusing on a target based on its distance. Figure 4 The diagram illustrates the convergence of the eye when viewing a near target 402 and the divergence of the pupil when viewing a distant target 401. This behavior is used in the described method to calculate the distance the subject is looking at. This information is used to calculate the optical power that should be applied to the lens, which is the second of three methods for calculating this value. To calculate the distance the subject is looking at using interpupillary distance, the following formula is used:

[0052] D=(H / 2)*tan(90-α), where a=atan(d / r)

[0053] In this formula, d is the distance the pupil moves, α is the angle of eye rotation, r is the radius of the eye, and H is the interpupillary distance when the subject is looking at infinity.

[0054] First, calculate the angle α of eye rotation when looking at a near target. This is done using the distance d of pupil movement and the radius r of the eye. After calculating the eye rotation angle α, the distance D the subject is looking at can be calculated using the previous formula; however, the interpupillary distance H of the subject when looking at a distant target must first be measured.

[0055] A third, more accurate method for calculating the distance perceived by a subject involves combining pupil size and pupil convergence / divergence. Additionally, temporal dynamics of the pupil relative to the object's distance can be obtained to determine the required optical power to be applied. This is a "learning" method.

[0056] exist Figure 5 The method described in this invention is illustrated schematically. First, calibration 501 is completed for the subject. The subject is asked to look at a distant object 502, and a series of images 503 are captured. Finally, these images are processed using an image processing subsystem to calculate the pupil size and interpupillary distance 504. Later, the subject is asked to look at a near target 505, and another series of images 506 are captured. These images are processed using the image processing subsystem to calculate the pupil size and interpupillary distance. Information from the calibration phase can be used in the learning phase to perform automatic optimization. The method then moves 508 to the correction phase 509. The correction phase 509 operates continuously, always starting with capturing an image of the subject's pupil 510. The image processing subsystem then processes this image to obtain the pupil size and interpupillary distance 511. Afterward, correction 512 is calculated using one of the three methods described above. Finally, the newly calculated optical power is applied to the photoelectric lens 513 to allow the subject to properly focus on the object.

[0057] Figure 6 This includes an example illustrating system operation. In the figure, the X-axis represents time, the left Y-axis displays a scale of pupil size and interpupillary distance in millimeters, and the right Y-axis displays a scale of the refractive power applied by the lens. The first row of curve 601 shows the distance between the two pupils over time; the second line 602 shows the pupil radius; and the third line 603 shows the refractive power applied to the phototransistor. As expected, increasing the interpupillary distance also increases their size and decreases the refractive power applied to the lens. Furthermore, decreasing the interpupillary distance decreases their size and increases the amount of refractive power applied to the phototransistor.

[0058] The pupil tracking system is implemented using a highly parallelized algorithm for a graphics processing unit (GPU), achieving very high speed and thus enabling the processing of a large number of samples per second. This processing speed reduces system latency and enhances its robustness and accuracy, allowing the system to react quickly to changes in the subject's pupil, while also reducing overall system latency and improving the user experience. The pupil tracking algorithm searches for the boundaries of the eye's pupil. First, it performs preprocessing on the captured image to eliminate reflections caused by infrared illumination and reduce noise that may be present in the image. Then, starting from the initial position, it tests the surrounding pixels to search for large gradient changes, as the pupil boundaries typically have large gradient changes. Finally, it randomly selects the found boundary points and performs several ellipse fitting iterations. To select the best-fitting ellipse, the distances between all ellipses and all points are calculated, and the ellipse with the minimum distance is selected.

[0059] As indicated in this document, the following reference numerals are linked to the different elements and steps described as part of this invention.

[0060] The right eye of 101 subjects

[0061] 102 Infrared LED Lighting

[0062] 103 Photoelectric Lens

[0063] 104 plane mirrors

[0064] 105 Dichroic Mirror

[0065] 106 Lenses

[0066] 107 Cameras for pupil tracking

[0067] Left eye of 108 subjects

[0068] 109 Infrared LED Lighting

[0069] 110 Photoelectric Lens

[0070] 111 Plane Mirror

[0071] 112 Prisms

[0072] Right eye of 201 subjects

[0073] 202 Cameras used for pupil tracking

[0074] 203 Photoelectric Lens

[0075] 204 photoelectric lens

[0076] 205 Cameras for pupil tracking

[0077] Left eye of 206 subjects

[0078] 207 Infrared LED Lighting

[0079] 208 Infrared LED Lighting

[0080] 301. Pupils of subjects when looking at distant targets.

[0081] 302 Pupils of subjects when looking at a near target

[0082] 401 Pupils of subjects when looking at distant targets

[0083] 402 Pupils of subjects when looking at a near target

[0084] 501 Calibration Process

[0085] 502 Appearance from a distance

[0086] 503 Image capture status

[0087] 504. Measurement of pupil status

[0088] 505 Close-up view

[0089] 506 Image capture status

[0090] 507. Measurement of Pupil Status

[0091] 508 Transition from Calibration to Processing

[0092] 509 Corrective Treatment

[0093] 510 Get Image Status

[0094] 511 Measuring Pupil Status

[0095] 512 Calculate the correction status

[0096] 513 Apply correction to the lens state

[0097] 601 shows the line indicating the interpupillary distance.

[0098] 602 shows the line representing the pupil radius.

[0099] 603 shows the line representing the refractive power applied to the lens.

[0100] 701 Schematic diagram of human eye movements during convergent gaze.

Claims

1. A photoelectric binocular device for correcting farsightedness, characterized in that, The photoelectric binoculars include: A high-speed pupil tracking system, comprising: A binocular subsystem for capturing images of the eyes of a subject undergoing farsightedness correction includes at least one camera (107; 202, 205) and several light sources (102, 109; 207, 208) oriented toward the eyes, and A binocular image processing subsystem, used to calculate pupil size and interpupillary distance in each eye, is capable of processing binocular images using a high-performance implementation based on a graphics processing unit (GPU) and a highly parallelized pupil tracking algorithm. Two photoelectric lenses (103, 110; 203, 204) provide variable optical power and can be positioned in front of the eyes of the subject undergoing hyperopia correction, one in front of each eye. The photoelectric binoculars are configured to perform calibration and hyperopia correction processes. In the calibration process, When the subject looks at the object placed at a distance d 远 When targeting a distant object, the image capture binocular subsystem is configured to capture an image, and the image processing binocular subsystem is configured to process the captured image and calculate the pupil size S. 远 And interpupillary distance, When the subject looks at the object placed at a distance d 近 When the target is near, the image capture binocular subsystem is configured to capture an image, and the image processing binocular subsystem is configured to process the captured image and calculate the pupil size S. 近 and interpupillary distance; and In the aforementioned hyperopia correction process, The image capture binocular subsystem is configured to capture images, and The image processing binocular subsystem is configured as follows: Process the captured image and calculate the pupil size S 当前 And interpupillary distance, The distance D seen by the subject was calculated using the following formulas: ,in, Where d is the distance the pupil moves, α is the angle of eye rotation, r is the radius of the eye, and H is the interpupillary distance when the subject is looking at infinity. The optical power to be applied to the photoelectric lenses (103, 110; 203, 204) is calculated based on the distance D. The calculated optical power is applied to the photoelectric lenses (103, 110; 203, 204). The hyperopia correction process is performed continuously several times.

2. The photoelectric binocular device for hyperopia correction according to claim 1, wherein, The light sources (102, 109; 207, 208) are infrared LEDs.

3. The photoelectric binocular device for hyperopia correction according to claim 1 or 2, further comprising a plurality of mirrors (104, 105, 111, 112) and at least one lens (106) for focusing an image of the pupil of the eye of the subject being corrected for hyperopia onto at least one camera (107; 202, 205).

4. The photoelectric binocular device for hyperopia correction according to claim 3, wherein, The mirrors are plane mirrors (104, 111), dichroic mirrors (105), and prisms (112).

Images