An eye tracking and photographing system for a strabismus measuring instrument and a strabismus measuring instrument

Abstract

The present invention provides an eye tracking and photographing system for a strabismus measuring instrument, comprising: a camera group, the camera group includes a plurality of cameras and a moving component, and the plurality of cameras are used to capture eye images from different angles; a slide rail, the slide rail is controllably slidably connected to the plurality of cameras through the moving component; a controller, the controller is electrically connected to the camera group and the moving component, and is used to obtain eye images from the plurality of cameras and compare them, so as to control the position and / or the angle relative to the slide rail of one or more cameras according to the comparison result. The present invention designs a set of eye tracking system, by arranging a plurality of cameras on the slide rail, so as to automatically select the cameras in the corresponding directions for photographing and uploading images for accurately calculating the strabismus degree after the eyeball rotates during the strabismus measurement process.

Description

Technical Field

[0001] The present invention relates to the field of ophthalmic medical devices, and particularly to a strabismus measuring instrument eye tracking and photographing system and a strabismus measuring instrument. Background Art

[0002] Strabismus is one of the common clinical eye diseases that affect 2%-5% of the population. When strabismus patients view with both eyes, their binoculars cannot coordinate and cooperate properly. Its common manifestations include esotropia, exotropia, and vertical strabismus.

[0003] Currently, the main clinical measurement methods for strabismus are corneal light reflection test and prism cover test (PCT). The former measures the strabismus type and deviation angle (i.e., strabismus degree) based on the position of light reflection to the center of the pupil, and can achieve accurate measurement of the strabismus degree. The latter requires the optometrist to alternately cover the eyes, observe the eye movement, and then estimate the degree of deviation to quickly determine the strabismus type. Among them, the corneal light reflection test needs to be calculated based on the eye image, so it is greatly affected by the image shooting angle and quality. In order to obtain accurate calculation results, an image of the front of the pupil should be obtained as much as possible.

[0004] However, since the camera group needs to collect the facial and eye features of the patient at an upward viewing angle, and the eyes of strabismus patients will involuntarily deflect before and after the eyeballs are blocked, and the deflection amplitude and angle vary according to different strabismus types. When the strabismus of the observer is greater than 40 degrees, the gaze direction of the patient's eyes deviates too much to the outside, causing distortion of the captured image of the center of the eyeball by the camera group, thereby reducing the image accuracy. At this time, taking pictures with a fixed camera position cannot obtain the best frontal image of the pupil, affecting the accuracy of strabismus degree detection. Summary of the Invention

[0005] In order to be able to capture the frontal image of the pupil in real time to achieve accurate measurement of the strabismus degree, in the first aspect of the present invention, a strabismus measuring instrument eye tracking and photographing system is proposed, including: a camera group, the camera group includes a plurality of cameras, and the plurality of cameras are used to capture eye images from different angles; a slide rail, the slide rail is connected to the plurality of cameras through a moving component provided thereon; a controller, the controller is electrically connected to the camera group and the moving component, and is used to obtain eye images from the plurality of cameras and compare them, so as to control the position and / or horizontal angle of one or more of the cameras on the slide rail according to the comparison result.

[0006] In one or more embodiments, the camera group includes 2n + 1 cameras, where n is a natural number and n ≥ 1. The first camera in the camera group is fixedly arranged at the middle position of the slide rail, and the remaining second cameras are symmetrically arranged on both sides of the first camera and can slide relative to the slide rail.

[0007] In one or more embodiments, the slide rail is an arc-shaped slide rail, and its bending direction faces the side of the person to be detected. The central angle α of the arc-shaped slide rail satisfies α ∈ [15°, 25°].

[0008] In one or more embodiments, the included angle β between the perpendicular line of the concave surface of the arc-shaped slide rail and the horizontal plane satisfies β ∈ [10°, 20°].

[0009] In one or more embodiments, the moving component is slidably connected to the slide rail, and the horizontal angle of the moving component is adjustable.

[0010] In one or more embodiments, the controller is further configured to: obtain a first eye image captured by the first camera, and determine the eye orientation in the first eye image; selectively obtain a second eye image captured by a second camera in the corresponding unilateral according to the eye orientation; compare the distortion rates of the second eye images, and upload the second eye image with the minimum distortion rate in the unilateral and the first eye image.

[0011] In one or more embodiments, the controller is further configured to: determine whether the distortion rate of the eye image with the minimum distortion is greater than or equal to a distortion threshold; in response to the distortion rate of the eye image with the minimum distortion being greater than or equal to the distortion threshold, determine the distortion direction of the eye image; control the corresponding second camera to perform position and / or angle fine-tuning according to the distortion direction.

[0012] In one or more embodiments, the controller is further configured to: determine the type of strabismus based on the first eye image; correspondingly adjust the position and / or angle of the second camera on the other side of the first camera on the slide rail according to the type of strabismus and the position adjustment result of the second camera on one side of the first camera.

[0013] In one or more embodiments, the controller is further configured to: divide a moving range on the slide rail for each second camera; determine an adjustment range of the upward viewing angle of the corresponding second camera according to the included angle β of the arc-shaped slide rail within the moving range; control the corresponding second camera to take a picture according to the height of the human eye and the adjustment range of the upward viewing angle of the second camera, and upload the obtained second eye image.

[0014] In a second aspect of the present invention, a strabismus measuring instrument is proposed, which includes the strabismus measuring instrument eye tracking and photographing system in any of the above embodiments.

[0015] The beneficial effects of the present invention include: The present invention designs a set of eye tracking systems. By arranging multiple cameras on a slide rail, after the eyeball rotates during the strabismus measurement, the camera in the corresponding direction is automatically selected for shooting, and the position and angle of the corresponding camera on the slide rail are adjusted based on the distortion direction, so as to obtain a high-precision frontal image of the eye, reduce the distortion of the detection area in the image, improve the image acquisition quality, thereby reducing the difficulty of image acquisition for algorithm processing, and improving the detection accuracy; and the camera group includes an infrared camera, which can capture the change of the eyeball state after being blocked, and can further improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other embodiments can be obtained based on these drawings.

[0017] Figure 1 It is a front view structural schematic diagram of an eye tracking and shooting system of a strabismus measuring instrument according to an embodiment of the present invention;

[0018] Figure 2 It is a structural schematic diagram of an arc-shaped slide rail and a moving component according to an embodiment of the present invention;

[0019] Figure 3 It is a structural schematic diagram of a strabismus of an eye tracking and shooting system of a strabismus measuring instrument according to an embodiment of the present invention;

[0020] Figure 4 It is a side view structural schematic diagram of an eye tracking and shooting system of a strabismus measuring instrument according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clear, the following will further describe the embodiments of the present invention in detail with reference to specific embodiments and the accompanying drawings.

[0022] It should be noted that all the expressions using "first" and "second" in the embodiments of the present invention are used to distinguish two entities or non-identical parameters with the same name. It can be seen that "first" and "second" are only for the convenience of expression and should not be construed as a limitation on the embodiments of the present invention. This will not be repeated in the subsequent embodiments.

[0023] In order to be able to capture the frontal image of the pupil in real time to accurately measure the strabismus degree, in an embodiment of the present invention, an eye tracking and shooting system for a strabismus measuring instrument is proposed, asFigure 1 As shown in the figure, it includes: a camera group 100, which includes a plurality of cameras for capturing eye images from different angles; a slide rail 200, which is connected to the plurality of cameras through a moving component 110 arranged thereon; a controller 300 (not shown), which is electrically connected to the camera group 100 and the moving component 110, and is used to obtain eye images from the plurality of cameras and compare them, so as to control the position and / or horizontal angle of one or more cameras on the slide rail according to the comparison results.

[0024] In one embodiment, as Figure 1 shown, the eye tracking system of the present invention is arranged inside the housing of a strabismus measuring instrument (hereinafter referred to as a strabismus instrument), behind the observation window of the strabismus instrument, and fixed below the display module of the strabismus instrument, so as to capture the user's face image when the user observes the visual target generated by the display module through the observation window, and obtain the eye image through post-processing; the plurality of cameras are arranged in sequence on the slide rail, each camera is aligned with the observation window, and the shooting screen can cover the entire observation window, so that while the plurality of cameras capture eye images from multiple angles, the position and angle of the plurality of cameras can be finely adjusted through the slide rail, so as to ensure that at least one camera can clearly obtain the frontal image of the pupil for calculating the strabismus degree.

[0025] In another embodiment, as Figure 2 shown, the way for the present invention to controllably realize the sliding of the camera group on the slide rail includes: upper and lower racks 201 and 202 are arranged on the back of the slide rail, and a moving component 110 is arranged between the upper and lower racks 201 and 202, including: a gear 111, which is connected to a stepping motor arranged on the front of the slide rail through a rotating shaft and a gap in the middle of the slide rail, and the stepping motor is fixed to the camera as the base of the camera. When the stepping motor is controlled to rotate, the gear 111 will move left and right along the upper and lower racks, thereby driving the stepping motor and the camera to move in sequence; optionally, a stop block 203 is arranged at the corresponding position of the slide rail to limit the moving range of the stepping motor. When the rotation of the stepping motor is blocked, it will stop rotating and wait for the next command. Among them, the present invention will configure the above-mentioned gear 111 and stepping motor structure for each camera to realize the independent control of the movement of each camera. This embodiment can realize precise tooth position adjustment, thereby precisely controlling the position of the camera on the slide rail, and further improving the quality of the captured image.

[0026] In one embodiment, the moving component further includes a ball joint connection structure disposed between the stepper motor and the camera. Among them, the ball head end is in rolling connection with the base of the camera. A hydraulic rod mechanism is also disposed between the base of the camera and the housing of the moving stepper motor to control the horizontal angle of the camera through the telescopic movement of the hydraulic rod. This embodiment can automatically adjust the shooting angle according to the eye height, thereby improving the quality of the captured image.

[0027] In one embodiment, the camera group 100 of the present invention includes 2n + 1 cameras, where n is a natural number and n ≥ 1. The first camera 101 in the camera group 100 is fixedly disposed at the middle position of the slide rail 200, and the remaining second cameras 102 and 103 are symmetrically disposed on both sides of the first camera 101 and can slide relative to each other along the slide rail.

[0028] In one embodiment, multiple cameras can use panchromatic imagers or imagers that operate only within a specific color range such as infrared. For example, the first camera 101 fixedly disposed at the middle position of the slide rail 200 uses a panchromatic imager to be used as a reference image for functions including face recognition, face orientation judgment and auxiliary adjustment, and strabismus type recognition, etc. The remaining second cameras 102 and 103 use imagers that operate within a specific color range such as infrared to collect eye images in the strabismus state after the eyes are blocked for calculating the strabismus degree. Among them, during the strabismus measurement process, the eyes need to be blocked by a filter, and this filter only allows part of the visible light or invisible light to pass through, so as to achieve the effect of blocking the line of sight. However, an imager that operates within a specific color range such as infrared can capture the filtered light of the filter to form an image, so that the eye images captured by such cameras can still see the eyes behind the filter. For strabismus patients, when losing the visual point (i.e., after the line of sight is blocked), the eyeballs cannot remain in the direct vision state and deflect. That is, after the eyes are blocked and the eyeballs deflect, the cameras that mainly play the shooting role are the second cameras located on both sides of the slide rail. Therefore, generally speaking, only the second cameras need to use imagers that operate within a specific color range such as infrared.

[0029] In one embodiment, as Figure 3As shown, the slide rail 200 can optionally be an arc-shaped slide rail 200, whose bending direction faces the side of the person to be detected, and the central angle α of the arc-shaped slide rail belongs to [15°, 25°]. Specifically, the central angle in this embodiment is the central angle determined with the arc length of the arc-shaped slide rail 200 as the arc side. The arc-shaped slide rail 200 enables the cameras on both sides of the arc-shaped slide rail to better capture the frontal image of the pupil after the eyeball deviates, thereby ensuring the calculation accuracy of the strabismus degree. If the central angle is too small, the difference between the eye images collected by the cameras on both sides and the eye images collected by the middle camera is relatively small, which affects face positioning and face modeling, and will further reduce the capture effect of the key features of the eyes of strabismus patients, affecting the accuracy of strabismus detection; if the central angle is too large, it means that the length of the arc-shaped slide rail becomes larger, increasing the installation space, and further increasing the volume of the measuring instrument.

[0030] In one embodiment, in order to adapt to the movement on the arc-shaped slide rail 200, the gear 11 in the moving component 110 can be prepared from a material with certain elasticity and support, such as PVC material, etc.

[0031] In one embodiment, as Figure 4 shown, the included angle β between the perpendicular line of the concave surface of the arc-shaped slide rail 200 and the horizontal plane belongs to [10°, 20°]. If the angle is too large, the distortion of the collected image increases, reducing the detection accuracy; if the angle is too small, the capture of the observer's eyes by the camera is incomplete, and the collected data is not comprehensive, affecting the detection effect. Specifically, the elevation angle β between the arc-shaped slide rail 200 and the horizontal plane in this embodiment is a fixed value, the purpose of which is to ensure both the complete capture of the patient's eyes by the camera, guarantee the comprehensiveness of data collection, and eliminate the generation of image distortion during collection.

[0032] In one embodiment, the controller 300 of the present invention is further configured to: obtain the first eye image captured by the first camera 101, and judge the orientation of the eyeball in the first eye image; selectively obtain the second eye image captured by the second camera in the corresponding unilateral according to the orientation of the eyeball; compare the distortion rates of the second eye images, and upload the second eye image with the minimum distortion rate and the first eye image.

[0033] Specifically, in this embodiment, the first camera 101 provided in the middle of the slide rail 200 is used to determine the deflection direction of the patient's eyeball, such as left or right, and then the corresponding second camera is called to capture an image according to the deflection direction. If there are multiple cameras in this direction, the second eye images captured by each second camera are compared, and the second eye image with the minimum distortion rate and the first eye image are uploaded for more complex calculations in the upper computer to determine the strabismus degree of the patient.

[0034] In one embodiment, only the first camera remains on in the standby state. When it captures a face image and identifies and confirms it, the second camera on one side of the first camera 101 will be selectively turned on. In this way, the power consumption of the device can be reduced and the generation amount of invalid image data can be decreased. Additionally, optionally, when the first camera fails to recognize a face, the captured image data will not be saved or uploaded. Optionally, the power on and off of the entire eye tracking shooting system can be controlled by setting a switch. After powering off, the first camera 101 will also stop working.

[0035] In one embodiment, the controller 300 of the present invention is further configured to: determine whether the distortion rate of the eye image with the smallest distortion is greater than or equal to a distortion threshold; in response to the distortion rate of the eye image with the smallest distortion being greater than or equal to the distortion threshold, determine the distortion direction of the eye image; and control the corresponding second camera to perform position and / or angle fine-tuning according to the distortion direction.

[0036] Specifically, the distortion of an image is mainly caused by radial distortion, which is due to the inherent characteristics of the convex lens of the lens itself. The reason is that light bends more at places far from the center of the lens than at places close to the center, resulting in greater distortion at the edges of the picture than at the center of the image. In other words, when the object being photographed is closer to the center of the image, that is, when the lens is directly facing the target for shooting, the amount of distortion is smaller. Additionally, in order to obtain high-precision calculation results, it is also a necessary condition to use an image with less distortion. Therefore, in combination with the above two points, in this embodiment, the second eye image with the smallest distortion is selected for uploading to detect the strabismus degree. Furthermore, in order to further improve the detection accuracy, a distortion threshold is also preset in this embodiment. When the minimum distortion rate of the selected second eye image is greater than the distortion threshold, the moving component can be adjusted according to the direction in which the distortion deviates from the center of the image. When the distortion is to the left, the corresponding moving component is controlled to move left by a preset step length. Similarly, when the distortion is to the right, the corresponding moving component is controlled to move right by a preset step length. By adjusting the position and angle of the corresponding second camera on the slide rail, the distortion rate of the image can be further reduced and a frontal image can be ensured to be obtained.

[0037] In one embodiment, the method of controlling the corresponding second camera to perform position and / or angle fine-tuning includes: based on the moving component in the above embodiment, sending an instruction to the stepper motor to control the gear to rotate, thereby driving the corresponding second camera to move left and right on the slide rail. Then, the image captured by the second camera is re-acquired, and the distortion rate of the second eye image is recalculated and compared with the preset distortion threshold until the adjustment and movement of the second camera stop when it is less than the preset distortion threshold, and the corresponding second eye image is uploaded.

[0038] In one embodiment, facial recognition can be performed on a face through the first camera 101, and the positional relationship between the recognized facial features and the corresponding camera is saved, so that when the next detection is performed, the position of the camera can be quickly adjusted according to the facial recognition result.

[0039] In one embodiment, facial recognition can be performed on a face through the first camera 101, and the recognized facial features and the corresponding strabismus degree measurement results are saved, so that when the next detection is performed, the historical strabismus degree measurement results of the user can be quickly retrieved according to the facial feature recognition result.

[0040] In one embodiment, in addition to comparing the distortion rate of the eye images, the present invention also calculates the area of the pupil region in the eye images, and uploads the second eye image with the largest pupil region area and the smallest distortion rate for calculating the strabismus degree.

[0041] In one embodiment, the controller 300 of the present invention is further configured to: determine the strabismus type based on the first eye image; and correspondingly adjust the position and / or angle of the second camera on the other side of the first camera 101 on the slide rail according to the strabismus type and the position adjustment result of the second camera inside one side of the first camera. Specifically, since most patients' strabismus is synchronous (i.e., both eyes squint in the same direction) or symmetric (i.e., both eyes squint in opposite directions), this embodiment proposes that the strabismus type can be determined according to the first image, and then the position of the second camera on the other side can be adjusted in advance according to the strabismus type and the adjustment result of the unilateral second camera, so as to shorten the adjustment time and improve the strabismus measurement efficiency.

[0042] In one embodiment, the controller 300 of the present invention is further configured to: divide a moving range on the slide rail 200 for each second camera; determine the elevation angle adjustment range corresponding to the second camera according to the included angle β of the arc-shaped slide rail 200 within the moving range; control the corresponding second camera to take a picture according to the human eye height and the elevation angle adjustment range of the second camera, and upload the obtained second eye image.

[0043] Specifically, in order to avoid interference between the individually adjustable second cameras, this embodiment assigns respective moving ranges to the second cameras, and respectively determines the elevation angle adjustment ranges within the moving ranges, so as to select the second camera within the appropriate elevation angle adjustment range according to the human eye height for taking pictures.

[0044] Through the above embodiments, the present invention converts the problem of tracking the eyes into the problem of calculating the distortion rate of the eye images, and adjusts the position of the corresponding second camera according to the distortion direction, so as to obtain a high-precision frontal image of the eyes for uploading to accurately calculate the strabismus degree by the strabismus meter.

[0045] In a second aspect of the present invention, a strabismus measuring instrument is provided, which includes the strabismus measuring instrument eye tracking and photographing system in any of the above embodiments.

[0046] The above are exemplary embodiments disclosed by the present invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments disclosed by the present invention as defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments herein do not need to be performed in any specific order. The serial numbers of the disclosed embodiments of the present invention above are only for description and do not represent the superiority or inferiority of the embodiments.

[0047] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the embodiments disclosed by the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features between the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the embodiments of the present invention as above, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included within the protection scope of the embodiments of the present invention.

Claims

1. An eye tracking and shooting system for a strabismus measuring instrument, characterized in that: include: A camera group, the camera group comprising a plurality of cameras, the plurality of cameras being used to capture eye images from different angles; A slide rail, wherein the slide rail is connected to the plurality of cameras via a moving component disposed thereon; A controller, the controller being electrically connected to the camera group and the moving assembly, and being used to obtain eye images from the multiple cameras and compare them, so as to control the position and / or horizontal angle of one or more of the cameras on the slide rail according to the comparison result; The camera group includes 2n+1 cameras, where n is a natural number and is greater than or equal to 1, the first camera in the camera group is fixedly arranged at the middle position of the slide rail, and the remaining second cameras are symmetrically arranged on both sides of the first camera and can slide relatively along the slide rail; Acquire a first eye image captured by the first camera, and determine the orientation of the eyeball in the first eye image; selectively acquire a second eye image captured by the second camera on the corresponding side according to the orientation of the eyeball; determine whether the distortion rate of the second eye image with the smallest distortion is greater than or equal to a distortion threshold; in response to the distortion rate of the second eye image with the smallest distortion being greater than or equal to the distortion threshold, determine the distortion direction of the eye image; and control the corresponding second camera to perform position and / or angle fine-tuning according to the distortion direction.

2. The eye tracking and shooting system of the strabismus measuring instrument according to claim 1, characterized in that: The slide rail is an arc-shaped slide rail, the bending direction of which is toward the side of the person being detected, and the central angle α∈[15°, 25°] of the arc-shaped slide rail.

3. The eye tracking and shooting system of the strabismus measuring instrument according to claim 2, characterized in that: The included angle β∈[10°, 20°] between the concave surface perpendicular line of the arc-shaped slide rail and the horizontal plane.

4. The eye tracking and shooting system of the strabismus measuring instrument according to claim 2, characterized in that: The moving component is slidably connected to the slide rail, and the horizontal angle of the moving component is adjustable.

5. The eye tracking and photographing system of the strabismus measuring instrument according to claim 1, characterized in that: The controller is also used for: In response to the distortion rate of the second eye image with the smallest distortion being less than the distortion threshold, the second eye image with the smallest distortion rate and the first eye image are uploaded.

6. The eye tracking and photographing system of the strabismus measuring instrument according to claim 1, characterized in that: The controller is also used for: determining a strabismus type based on the first eye image; According to the squint type and the position adjustment result of the second camera on one side of the first camera, the position and / or angle of the second camera on the other side of the first camera on the slide rail is correspondingly adjusted.

7. The eye tracking and photographing system of the strabismus measuring instrument according to claim 4, characterized in that: The controller is also used for: Allocate a moving range on the slide rail for each of the second cameras; Determine the upward viewing angle adjustment range corresponding to the second camera according to the included angle β of the arc slide rail within the moving range; According to the height of the human eye and the upward angle adjustment range of the second camera, the corresponding second camera is controlled to shoot and the acquired second eye image is uploaded.

8. A strabismus measuring instrument, characterized in that: It comprises the eye tracking and shooting system of the strabismus measuring instrument as described in any one of claims 1 to 7.

Images