Eye tracking shooting system of strabismus measuring instrument and strabismus measuring instrument

Abstract

The invention provides an eye tracking shooting system of a strabismus measuring instrument, comprising a camera group which comprises a plurality of cameras and a moving assembly, and the plurality of cameras are used for capturing eye images from different angles; the sliding rail is in controllable sliding connection with the plurality of cameras through a moving assembly; and the controller is in electric control connection with the camera group and the moving assembly and is used for obtaining eye images from the plurality of cameras and comparing the eye images so as to control the positions of the one or more cameras on the sliding rail and / or the angles of the one or more cameras relative to the sliding rail according to a comparison result. According to the invention, a set of eye tracking system is designed, and a plurality of cameras are arranged on the slide rail, so that the camera in the corresponding direction is automatically selected to shoot and upload an image for accurately calculating the strabismus degree after eyeballs rotate in the strabismus measurement process.

Description

Technical Field

[0001] The present invention relates to the field of ophthalmic medical equipment, and in particular to an eye tracking and shooting system of a strabismus measuring instrument and a strabismus measuring instrument. Background Art

[0002] Strabismus is one of the common clinical eye diseases that affects 2%-5% of the population. When patients with strabismus use their binocular vision, their eyes cannot be properly coordinated and cooperated. Common manifestations include esotropia, exotropia, and up and down strabismus.

[0003] At present, the clinical measurement methods for strabismus mainly use two test methods: corneal light reflex test and prism cover test (PCT). The former measures the type of strabismus and the angle of deviation (i.e., the degree of strabismus) based on the position of light reflection to the center of the pupil, which can achieve accurate measurement of the degree of strabismus, while the latter requires the optometrist to cover the eyes alternately, observe eye movements, and then estimate the degree of deviation to quickly determine the type of strabismus. Among them, the corneal light reflex test needs to be calculated based on the eye image, so it is greatly affected by the image shooting angle and quality. In order to obtain accurate calculation results, the image of the front of the pupil should be obtained as much as possible.

[0004] However, since the camera group needs to capture the patient's facial and eye features at an upward angle, and strabismus patients will experience involuntary deflection before and after the eyeballs are blocked, and the amplitude and angle of the deflection vary depending on the type of strabismus, when the observer's strabismus is greater than 40 degrees, the patient's eye gaze direction shifts too far outward, causing the camera group to capture the image of the center of the eyeball distorted, thereby reducing the image accuracy. At this time, using a fixed camera position to shoot cannot obtain the best frontal image of the pupil, affecting the accuracy of strabismus degree detection. Summary of the invention

[0005] In order to be able to capture the frontal image of the pupil in real time to achieve accurate measurement of the degree of strabismus, a strabismus measuring instrument eye tracking and shooting system is proposed in a first aspect of the present invention, comprising: a camera group, the camera group includes a plurality of cameras, and the plurality of cameras are used to capture eye images from different angles; a slide rail, the slide rail is connected to the plurality of cameras via a moving component arranged thereon; a controller, the controller is electrically connected to the camera group and the moving component, and is used to obtain eye images from the plurality of cameras and compare them, so as to control the position and / or horizontal angle of one or more of the cameras on the slide rail according to the comparison results.

[0006] In one or more embodiments, the camera group includes 2n+1 cameras, where n is a natural number and is greater than or equal to 1. The first camera in the camera group is fixedly arranged at the middle position of the slide rail, and the remaining second cameras are symmetrically arranged on both sides of the first camera and can slide relatively along the slide rail.

[0007] In one or more embodiments, the slide rail is an arc-shaped slide rail, the bending direction of which is toward the side of the person being detected, and the central angle α∈[15°, 25°] of the arc-shaped slide rail.

[0008] In one or more embodiments, an angle β∈[10°, 20°] is formed between a vertical line of the concave surface of the arc-shaped slide rail and a horizontal plane.

[0009] In one or more embodiments, the moving component is slidably connected to the slide rail, and the horizontal angle of the moving component is adjustable.

[0010] In one or more embodiments, the controller is also used to: obtain a first eye image captured by the first camera, and determine the direction of the eye in the first eye image; selectively obtain a second eye image captured by the second camera on the corresponding side according to the direction of the eye; compare the distortion rates of the second eye images, and upload the second eye image with the smallest distortion rate on the side together with the first eye image.

[0011] In one or more embodiments, the controller is also used to: determine whether the distortion rate of the eye image with the least distortion is greater than or equal to a distortion threshold; in response to the distortion rate of the eye image with the least distortion being greater than or equal to the distortion threshold, determine the distortion direction of the eye image; and control the corresponding second camera to perform fine-tuning of the position and / or angle according to the distortion direction.

[0012] In one or more embodiments, the controller is also used to: determine the type of strabismus based on the first eye image; and adjust the position and / or angle of the second camera on the other side of the first camera on the slide rail according to the type of strabismus and the position adjustment result of the second camera on one side of the first camera.

[0013] In one or more embodiments, the controller is also used to: divide the moving range on the slide rail for each second camera; determine the upward angle adjustment range of the corresponding second camera according to the angle β of the arc slide rail within the moving range; control the corresponding second camera to shoot according to the human eye height and the upward angle adjustment range of the second camera and upload the acquired second eye image.

[0014] In a second aspect of the present invention, a strabismus measuring instrument is provided, comprising an eye tracking and photographing system of the strabismus measuring instrument as in any of the above embodiments.

[0015] The beneficial effects of the present invention include: the present invention designs an eye tracking system, which arranges multiple cameras on a slide rail, so that when the eyeball rotates during the strabismus measurement process, the camera in the corresponding direction is automatically selected for shooting, and the position and angle of the corresponding camera on the slide rail are adjusted based on the distortion direction to obtain a high-precision frontal image of the eye, so as to reduce the distortion of the detection area in the image, improve the image acquisition quality, thereby reducing the difficulty of algorithm processing for image acquisition, so as to improve the detection accuracy; and the camera group includes an infrared head camera, which can capture the changes in the state of the eyeball after being blocked, and can further improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a schematic diagram of the front view structure of an eye tracking and shooting system of a strabismus measuring instrument according to an embodiment of the present invention; Figure 2 It is a structural schematic diagram of a curved slide rail and a moving assembly according to an embodiment of the present invention; Figure 3 A schematic diagram of a strabismus structure of an eye tracking and photographing system of a strabismus measuring instrument according to an embodiment of the present invention; Figure 4 The present invention is a schematic diagram of the side view structure of an eye tracking and shooting system of a strabismus measuring instrument according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the embodiments of the present invention are further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0019] It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are for distinguishing two non-identical entities with the same name or non-identical parameters. It can be seen that "first" and "second" are only for the convenience of expression and should not be understood as limitations on the embodiments of the present invention. The subsequent embodiments will not explain this one by one.

[0020] In order to capture the frontal image of the pupil in real time to achieve accurate measurement of the degree of strabismus, in one embodiment of the present invention, a strabismus measuring instrument eye tracking shooting system is proposed, such as Figure 1As shown, it includes: a camera group 100, which includes multiple cameras, and the multiple cameras are used to capture eye images from different angles; a slide rail 200, which is connected to the multiple cameras via a moving component 110 arranged thereon; a controller 300 (not shown), which is electrically connected to the camera group 100 and the moving component 110, and is used to obtain eye images from the multiple cameras and compare them, so as to control the position and / or horizontal angle of one or more cameras on the slide rail according to the comparison results.

[0021] In one embodiment, if Figure 1 As shown, the eye tracking system of the present invention is arranged inside the shell of a strabismus measuring instrument (hereinafter referred to as the strabismus instrument), located behind the observation window of the strabismus instrument, and fixed below the display module of the strabismus instrument, so as to realize capturing the user's facial image when the user observes the sight mark generated by the display module through the observation window, and obtaining the eye image through post-processing; multiple cameras are arranged in sequence on the slide rail, each camera is aimed at the observation window, and the shooting picture can cover the entire observation window, so that the multiple cameras can capture the eye image from multiple angles and the positions and angles of the multiple cameras can be fine-tuned through the slide rail, so as to ensure that at least one camera can clearly obtain the front image of the pupil for calculating the strabismus degree.

[0022] In another embodiment, Figure 2 As shown, the controllable method of realizing the sliding of the camera group on the slide rail of the present invention includes: setting upper and lower racks 201 and 202 on the back of the slide rail, and setting a moving assembly 110 between the upper and lower racks 201 and 202, including: a gear 111, the gear 111 is connected to the stepper motor set on the front of the slide rail through the rotating shaft and the gap in the middle of the slide rail, and the stepper motor is fixedly connected to the camera as the base of the camera. When the stepper motor rotates under control, the gear 111 will move left and right along the upper and lower racks, thereby driving the stepper motor and the camera to move in turn; optionally, a stopper 203 is set at the corresponding position of the slide rail to limit the moving range of the stepper motor, and when the rotation of the stepper motor is hindered, it will stop rotating and wait for the next command to be received. Among them, the present invention will configure the above-mentioned gear 111 and stepper motor structure for each camera to realize the individual control of the movement of each camera. This embodiment can realize accurate tooth position adjustment, thereby accurately controlling the position of the camera on the slide rail, thereby improving the quality of the captured image.

[0023] In one embodiment, the moving assembly further includes a ball head connection structure disposed between the stepper motor and the camera, wherein the ball head end is rollingly connected to the camera base, and a hydraulic rod mechanism is further disposed between the camera base and the housing of the moving stepper motor to control the horizontal angle of the camera by telescoping the hydraulic rod. This embodiment can automatically adjust the shooting angle to suit the eye height, thereby improving the quality of the captured image.

[0024] In one embodiment, the camera group 100 of the present invention includes 2n+1 cameras, where n is a natural number and is greater than or equal to 1. The first camera 101 in the camera group 100 is fixedly arranged in the middle position of the slide rail 200, and the remaining second cameras 102 and 103 are symmetrically arranged on both sides of the first camera 101 and can slide relative to each other along the slide rail.

[0025] In one embodiment, the plurality of cameras may use full-color imagers or imagers that work only within a specific color range such as infrared. For example, the first camera 101 fixed at the middle position of the slide rail 200 uses a full-color imager to serve as a reference image for face recognition, face orientation judgment and auxiliary adjustment, and strabismus type recognition, and the remaining second cameras 102 and 103 use imagers such as infrared that work within a specific color range to collect eye images in a strabismus state after the eyes are blocked for calculating the degree of strabismus. In the process of strabismus measurement, the eyes need to be blocked by a filter, and the filter only allows part of the visible light or invisible light to pass through, thereby blocking the line of sight. However, an imager working in a specific color range such as infrared can capture the filtered light of the filter to form an image, so that the eye image captured by this type of camera can still see the eye behind the filter. For strabismus patients, when the viewpoint is lost (that is, the line of sight is blocked), the eyeball cannot remain in a straight-looking state and deflects, that is, after the eye is blocked and the eyeball is deflected, the camera that mainly plays a shooting role is the second camera located on both sides of the slide rail. Therefore, generally speaking, only the second camera needs to use an imager working in a specific color range such as infrared.

[0026] In one embodiment, Figure 3As shown, the slide rail 200 may be an arc-shaped slide rail 200, the bending direction of which is toward the side of the person being tested, and the center angle α∈[15°, 25°]. Specifically, the center angle in this embodiment is the center angle determined by the arc length of the arc-shaped slide rail 200 as the arc side. The arc-shaped slide rail 200 enables the cameras located on both sides of the arc-shaped slide rail to better capture the frontal image of the pupil after the eyeball is deflected, thereby ensuring the calculation accuracy of the strabismus degree. If the center angle is too small, the difference between the eye images captured by the cameras on both sides and the eye images captured by the middle camera is small, which affects the positioning and modeling of the face, and will also reduce the effect of capturing the key features of the eyes of strabismus patients, affecting the accuracy of strabismus detection; the larger the center angle, the longer the length of the arc-shaped slide rail, which increases the installation space and thus increases the volume of the measuring instrument.

[0027] In one embodiment, in order to adapt to the movement on the arc-shaped slide rail 200, the gear 11 in the moving assembly 110 can be made of a material with certain elasticity and support, such as PVC material.

[0028] In one embodiment, Figure 4 As shown, the angle β∈[10°, 20°] between the concave vertical line of the arc-shaped slide rail 200 and the horizontal plane is larger than 10°. If the angle is too large, the distortion of the captured image will increase, reducing the detection accuracy; if the angle is too small, the camera will not fully capture the observer's eyes, and the collected data will not be comprehensive, affecting the detection effect. Specifically, in this embodiment, the elevation angle β between the arc-shaped slide rail 200 and the horizontal plane is a fixed value, the purpose of which is to ensure that the camera can fully capture the patient's eyes, ensure the comprehensiveness of data collection, and prevent the generation of collected image distortion.

[0029] In one embodiment, the controller 300 of the present invention is also used to: obtain a first eye image captured by the first camera 101, and determine the direction of the eye in the first eye image; selectively obtain a second eye image captured by the second camera on the corresponding side according to the direction of the eye; compare the distortion rates of the second eye images, and upload the second eye image with the smallest distortion rate together with the first eye image.

[0030] Specifically, this embodiment determines the direction of eye deflection of the patient, such as left or right, by means of a first camera 101 disposed in the middle of the slide rail 200, and then retrieves the image captured by the second camera in the corresponding direction according to the deflection direction. If there are multiple cameras in the direction, the second eye images captured by each second camera are compared, and the second eye image with the smallest distortion rate and the first eye image are uploaded, so as to perform more complex calculations in the host computer to determine the degree of strabismus of the patient.

[0031] In one embodiment, in the standby state, only the first camera remains turned on, and when it captures a face image and recognizes and confirms it, it will selectively turn on the second camera on one side of the first camera 101. In this way, the power consumption of the device can be reduced and the amount of invalid image data generated can be reduced. In addition, optionally, when the first camera does not recognize a face, the image data captured by it is not saved or uploaded; optionally, the power on and off of the entire eye tracking shooting system can be controlled by setting a switch, and the first camera 101 will also be in a stopped working state after powering off.

[0032] In one embodiment, the controller 300 of the present invention is also used to: determine whether the distortion rate of the eye image with the least distortion is greater than or equal to the distortion threshold; in response to the distortion rate of the eye image with the least distortion being greater than or equal to the distortion threshold, determine the distortion direction of the eye image; and control the corresponding second camera to perform position and / or angle fine-tuning according to the distortion direction.

[0033] Specifically, the distortion of the image is mainly caused by radial distortion, which is caused by the inherent characteristics of the convex lens of the lens itself. The reason for this is that the light is more bent away from the center of the lens than near the center, so that the distortion at the edge of the picture is greater than that at the center of the image. In other words, the closer the photographed object is to the center of the image, that is, when the lens is facing the target, the smaller the distortion; in addition, in order to obtain high-precision calculation results, using a less distorted image is also a necessary condition. Therefore, in combination with the above two points, this embodiment chooses to upload the second eye image with the smallest distortion for detecting the degree of strabismus. In addition, in order to further improve the detection accuracy, a distortion threshold is also preset in this implementation. When the minimum distortion rate of the screened second eye image is greater than the distortion threshold, the moving component can be adjusted according to the direction in which the distortion deviates from the center of the image. When the distortion is biased to the left, the corresponding moving component is controlled to move to the left by a preset step length. Similarly, when the distortion is biased to the right, the corresponding moving component is controlled to move to the right by a preset step length. By adjusting the position and angle of the corresponding second camera on the slide rail, the distortion rate of the image is further reduced and a frontal image is obtained.

[0034] In one embodiment, the method of controlling the corresponding second camera to fine-tune the position and / or angle includes: based on the moving component in the above embodiment, sending instructions to the stepper motor to control the rotation of the gear, thereby driving the corresponding second camera to move left and right on the slide rail, and then re-acquiring the image captured by the second camera, and recalculating the distortion rate of the second eye image and comparing it with a preset distortion threshold, until it is less than the preset distortion threshold, stop adjusting and moving the second camera, and upload the corresponding second eye image.

[0035] In one embodiment, the first camera 101 can be used to perform facial recognition on a human face, and the positional relationship between the recognized facial features and the corresponding camera can be saved, so that the position of the camera can be quickly adjusted according to the facial recognition result during the next detection.

[0036] In one embodiment, the first camera 101 can be used to perform facial recognition on a human face, and the recognized facial features and the corresponding strabismus degree measurement results can be saved, so that when the next detection is performed, the historical strabismus degree measurement results of the user can be quickly retrieved according to the facial feature recognition results.

[0037] In one embodiment, in addition to comparing the distortion rates of eye images, the present invention also calculates the area of ​​the pupil region in the eye image, and uploads the second eye image with the largest pupil area and the smallest distortion rate for calculating the degree of strabismus.

[0038] In one embodiment, the controller 300 of the present invention is also used to: determine the type of strabismus based on the first eye image; and adjust the position and / or angle of the second camera on the other side of the first camera 101 on the slide rail according to the type of strabismus and the position adjustment result of the second camera on one side of the first camera. Specifically, since most patients have synchronous strabismus (i.e., both eyes strabismus in the same direction) or symmetry (i.e., both eyes strabismus in two directions), this embodiment proposes that the type of strabismus can be determined based on the first image, and then the position of the second camera on the other side can be adjusted in advance according to the type of strabismus and the adjustment result of the second camera on one side, thereby shortening the adjustment time and improving the efficiency of strabismus measurement.

[0039] In one embodiment, the controller 300 of the present invention is also used to: divide the moving range on the slide rail 200 for each second camera; determine the upward angle adjustment range of the corresponding second camera according to the angle β of the arc slide rail 200 within the moving range; control the corresponding second camera to shoot based on the height of the human eye and the upward angle adjustment range of the second camera and upload the acquired second eye image.

[0040] Specifically, in order to avoid interference between the second cameras that can be adjusted individually, this embodiment allocates a respective movement range to each second camera, and determines the upward angle adjustment range within the movement range respectively, so as to select the second camera within the appropriate upward angle adjustment range for shooting according to the height of the human eye.

[0041] Through the above embodiments, the present invention converts the eye tracking problem into the problem of calculating the eye image distortion rate, thereby adjusting the position of the corresponding second camera according to the distortion direction, so as to obtain a high-precision frontal image of the eye for uploading for the strabismus instrument to accurately calculate the degree of strabismus.

[0042] In a second aspect of the present invention, a strabismus measuring instrument is provided, which includes an eye tracking and photographing system of the strabismus measuring instrument as described in any of the above embodiments.

[0043] The above are exemplary embodiments disclosed in the present invention, but it should be noted that various changes and modifications may be made without departing from the scope of the embodiments disclosed in the claims. The functions, steps and / or actions of the method claims according to the disclosed embodiments described herein do not need to be performed in any particular order. The serial numbers of the embodiments disclosed in the above embodiments of the present invention are for description only and do not represent the advantages and disadvantages of the embodiments.

[0044] A person skilled in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the disclosure of the embodiments of the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features in the above embodiments or different embodiments can also be combined, and there are many other changes in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of simplicity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the embodiments of the present invention.

Claims

1. An eye tracking and shooting system for a strabismus measuring instrument, characterized in that: include: A camera group, the camera group comprising a plurality of cameras, the plurality of cameras being used to capture eye images from different angles; A slide rail, wherein the slide rail is connected to the plurality of cameras via a moving component disposed thereon; A controller is electrically connected to the camera group and the moving component, and is used to obtain eye images from the multiple cameras and compare them, so as to control the position and / or horizontal angle of one or more of the cameras on the slide rail according to the comparison results.

2. The eye tracking and shooting system of the strabismus measuring instrument according to claim 1, characterized in that: The camera group includes 2n+1 cameras, where n is a natural number and is greater than or equal to 1. The first camera in the camera group is fixedly set in the middle position of the slide rail, and the remaining second cameras are symmetrically arranged on both sides of the first camera and can slide relatively along the slide rail.

3. The eye tracking and shooting system of the strabismus measuring instrument according to claim 1 or 2, characterized in that: The slide rail is an arc-shaped slide rail, the bending direction of which is toward the side of the person being detected, and the central angle α∈[15°, 25°] of the arc-shaped slide rail.

4. The eye tracking and shooting system of the strabismus measuring instrument according to claim 3, characterized in that: The included angle β∈[10°, 20°] between the concave perpendicular line of the arc-shaped slide rail and the horizontal plane.

5. The eye tracking and photographing system of the strabismus measuring instrument according to claim 1, characterized in that: The moving component is slidably connected to the slide rail, and the horizontal angle of the moving component is adjustable.

6. The eye tracking and photographing system of the strabismus measuring instrument according to claim 2, characterized in that: The controller is also used for: Acquire a first eye image captured by the first camera, and determine the orientation of an eyeball in the first eye image; selectively acquiring a second eye image captured by a second camera on the corresponding side according to the eyeball orientation; The distortion rates of the second eye images are compared, and the second eye image with the smallest unilateral internal distortion rate is uploaded together with the first eye image.

7. The eye tracking and photographing system of the strabismus measuring instrument according to claim 6, characterized in that: The controller is also used for: Determine whether the distortion rate of the eye image with the smallest distortion is greater than or equal to a distortion threshold; In response to the distortion rate of the eye image with the smallest distortion being greater than or equal to a distortion threshold, determining a distortion direction of the eye image; The corresponding second camera is controlled to perform position and / or angle fine-tuning according to the distortion direction.

8. The eye tracking and photographing system of the strabismus measuring instrument according to claim 7, characterized in that: The controller is also used for: determining a strabismus type based on the first eye image; According to the squint type and the position adjustment result of the second camera on one side of the first camera, the position and / or angle of the second camera on the other side of the first camera on the slide rail is correspondingly adjusted.

9. The eye tracking and photographing system of the strabismus measuring instrument according to claim 5, characterized in that: The controller is also used for: Allocate a moving range on the slide rail for each of the second cameras; Determine the upward viewing angle adjustment range corresponding to the second camera according to the included angle β of the arc slide rail within the moving range; According to the height of the human eye and the upward angle adjustment range of the second camera, the corresponding second camera is controlled to shoot and the acquired second eye image is uploaded.

10. A strabismus measuring instrument, characterized in that: It comprises the eye tracking and shooting system of the strabismus measuring instrument as described in any one of claims 1 to 9.

Images