A strabismus degree detection system based on cover test

Abstract

The application discloses a strabismus degree detection system based on a covering test, which comprises a head support, a calibration point, a display, a camera, a light source, a voice broadcaster and a control device; the head support is arranged opposite to the display, and the planes where the head support and the display are located are parallel to each other; the calibration point is installed above the head support through a support composed of two symmetrical vertical columns and a crossbeam on the vertical columns, and the calibration point is installed at the middle of the crossbeam; the camera and the light source are respectively installed on the display and are located in the same plane, and the plane is parallel to the plane where the display is located; the field of view of the camera covers the head support; the display, the camera and the voice broadcaster are all connected with the controller. The application is not affected by the kappa angle, can accurately measure the strabismus degree, and solves the defects that the traditional strabismus degree detection technology is relatively subjective and needs professional doctors to participate in measurement.

Description

Technical Field

[0001] This invention belongs to the field of ophthalmic disease detection technology, specifically relating to a strabismus degree detection system based on occlusion testing. Background Technology

[0002] When a person with normal vision focuses on a target, both eyes fixate simultaneously. However, in patients with strabismus (crossed eyes), usually only one eye can focus on the target, while the other eye deviates. This inability of both eyes to focus on a target simultaneously is medically termed strabismus. Strabismus can severely impact a patient's vision and hinder their daily life, work, and mental well-being. If strabismus is not detected and treated promptly in childhood, it can easily lead to loss of orientation ability or amblyopia in adulthood.

[0003] Currently, common clinical methods for examining strabismus include corneal reflex, synoptophore, and cover test. The corneal reflex requires the examiner to sit opposite the patient. The examiner holds a pen-and-light at eye level with the patient, instructing the patient to focus on the light source. The examiner then observes the position of the reflection point on the cornea to determine if there is any eye deviation. The corneal reflex cannot accurately determine the degree of strabismus; it can only be used for a rough assessment. If the reflection point is at the edge of the pupil, the strabismus is approximately 10°–15°; if it is between the corneal and pupillary edges, the strabismus is approximately 25°–30°; and if it is at the corneal edge, it is approximately 45°. This method also cannot eliminate the influence of the kappa angle. The synoptophore is a large optoelectronic instrument combining opto-mechanical-electrical components. When measuring the strabismus angle with the synoptophore, the lens is first fixed at 0°, and the patient focuses on a picture through the lens. The doctor alternately turns off the light source on the picture and observes the eye movements. Adjust one side of the lens barrel until neither eye moves when focusing on the picture independently. The reading indicated by the lens barrel arm at this point is the degree of strabismus. However, synoptophores are relatively expensive, and measuring strabismus using a synoptophore requires the involvement of a professional physician. Occlusion tests can measure the manifestness or latentness of strabismus and are simple, easy to perform, and highly repeatable. Occlusion tests are divided into unilateral occlusion tests, occlusion-unocclusion tests, and alternating occlusion tests. Unilateral occlusion tests are used to examine manifest strabismus, occlusion-unocclusion tests are used to examine latent strabismus, and alternating occlusion tests can examine both types of strabismus. However, occlusion tests are subject to the subjective factors of the physician and cannot quantify the degree of strabismus.

[0004] Therefore, there is an urgent need to research an intelligent detection technology that can achieve objective and accurate strabismus degree. Summary of the Invention

[0005] The purpose of this invention is to provide a strabismus degree detection system and method based on occlusion testing, so as to solve the problem that traditional strabismus degree detection technology is relatively subjective and requires the participation of professional physicians in measurement.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] A strabismus degree detection system based on occlusion testing includes a headrest, a calibration point, a display, a camera, a light source, a voice announcer, and a control device. The headrest and display are positioned opposite each other, and their planes are parallel. The calibration point is mounted directly above the headrest via a bracket consisting of two symmetrically arranged columns and a crossbeam on each column, with the calibration point located in the middle of the crossbeam. The camera and light source are mounted on the display, both lying in the same plane parallel to the display's plane, and the camera's field of view covers the headrest. The display, camera, and voice announcer are all connected to the controller.

[0008] Furthermore, the calibration point is a black circular dot.

[0009] Furthermore, the controller includes a system calibration module, a pupil and reflection point positioning module, a calibration module, and an occlusion detection module, wherein:

[0010] The system calibration module is used to implement the following process: Step 11, establish a world coordinate system with the lower left corner of the display as the origin, the height of the display is SH, the width is SW, and the resolution is RWxRH; Step 12, control the camera to acquire images; Step 13, based on the acquired images, obtain the conversion ratio from planar pixels to distance of the headrest, and manually measure the coordinates (X, φ, φ) of the center of the calibration point in the world coordinate system. b ,Y b Z b );

[0011] The pupil and reflective point localization module is used to acquire human eye images and obtain the pixel coordinates of the pupil center, the coordinates of the pupil center in the world coordinate system, and the pixel coordinates of the reflective point center in the pupil based on the acquired human eye images.

[0012] The calibration module is used to call the pupil and reflection point positioning module to calibrate the subject and calculate the Hersberg ratio.

[0013] The occlusion detection module is used to call the pupil and reflective point positioning module and control the voice broadcaster to prompt the test subject to use the occlusion test method to measure the degree of strabismus.

[0014] Furthermore, step 13 includes the following operations:

[0015] The acquired image is converted to grayscale and denoised using Gaussian filtering or other methods. Adaptive thresholding or other methods are then used to perform thresholding on the denoised image. Morphological opening is used to remove isolated points and spurs from the thresholded image, resulting in a morphologically processed image. Canny edge detection or other methods are used to detect the edges of the morphologically processed image. Hough transform or other circular detectors are used to detect calibration points in the processed image, obtaining the pixel coordinates (x, y, y) of these calibration points. b ,y b And the pixel radius r of the calibration point on the image. b The formula for calculating the conversion ratio from planar pixels to distance of the headrest is ratio = R / r. b Where R is the true radius of the calibration point; and the coordinates of the center of the calibration point in the world coordinate system (X, R, R) are... b ,Y b Z b (Obtained using a measuring tape or other distance measuring tools.)

[0016] Furthermore, the implementation process of the pupil and reflective point positioning module is as follows:

[0017] Step 21: Control the camera to continuously capture images of the human eyes at a rate of 60fps, directly capture images containing both eyes, and then flip the images horizontally.

[0018] Step 22: Based on the image obtained in Step 21, first locate the positions of both eyes using the human eye detection method, then use pupil localization technology in the two regions to obtain the pixel coordinates of the two pupil centers; and calculate the coordinates of the pupil centers in the world coordinate system established in the system calibration module.

[0019] Step 23, using the center of the left pupil (x) L ,y L With ) as the center and the radius R of the left pupil as the radius, L Draw a circular region with a radius approximately 1.5 times the radius of the left pupil, and within this region, find the pixel coordinates (x, y) of the center of the reflection point in the left pupil. g ,y g Similarly, the pixel coordinates of the center of the reflective point in the right pupil are obtained.

[0020] Furthermore, the specific process of step 22 is as follows:

[0021] (i) The human eye detection method adopts any of the following methods: (1) Based on the collected multiple human eye images, (2) manually determine the position of the eyes; (3) collect and label multiple positive and negative samples, train the Haar-cascade classifier, and obtain the position of the eyes in the image through the trained classifier; (4) perform human eye detection by training through deep learning methods.

[0022] (II) Pupil position detection employs a segmentation network model from deep learning. After inputting the image, a segmentation probability map is obtained. Then, a binary image is obtained through threshold segmentation. Connectivity analysis is performed on the binary image, and the centroid of the largest blob is used as the pixel coordinate (x) of the left pupil center. L ,y L ), take the longer side of the bounding rectangle of the largest spot as the diameter of the left eye pupil, 2*R. L R L The radius of the pupil;

[0023] (III) Based on the world coordinate system established in step 11 and the coordinates (X, Y, F) of the calibration point measured in step 12 in the world coordinate system. b ,Y b Z b ) Calculate the coordinates of the center of the left pupil in the world coordinate system as (X... L ,Y L Z L );in, Z L =Z b , where r b Let R be the pixel radius of calibration point 3 on the image, and R be the true radius (cm) of the calibration point. b y b () represents the pixel coordinates of the center of the calibration point;

[0024] (iv) Similarly, perform steps (ii) and (iii) on the right eye to obtain the pixel coordinates (x, y, y) of the center of the right pupil. R ,y R Pupil radius R R The coordinates of the center of the right pupil in the world coordinate system (X) R ,Y R Z R ).

[0025] Furthermore, the implementation process of the calibration module is as follows:

[0026] Step 31: Initialize 9 fixation points with known coordinates: P1(X1,Y1,0), P2(X2,Y2,0)...P9(X9,Y9,0); the pixel coordinates of fixation point P0 are (x0,y0), and its world coordinates are (X0,Y0,0), so X0=x0*SW / RW, Y0=(RH-y0)*SW / RW; calculate the offset fixation angle from fixation point P0 to fixation points P1, P2...P9 respectively. RH is the monitor height (cm), and RW is the monitor width (cm).

[0027] Step 32: Display fixation point P0 (X0, Y0, 0) on the monitor, control the voice announcer to prompt the subject to place their head on the headrest, and remind the subject to fixate on coordinate point P0; then, display at least two known fixation points as calibration points on the monitor in sequence, control the voice announcer to remind the subject to fixate on these calibration points in sequence, during which the system calls the pupil and reflective point localization module to locate the pupil and the reflective point within it; each calibration point is displayed for 5 seconds, and the current calibration point is considered stable when the subject's fixation duration is greater than or equal to 2 seconds; calculate the mean and standard deviation of the pupil center position within 2 seconds, and if the pupil center position coordinates are all less than 1.5 times the standard deviation within 2 seconds, the fixation is considered stable; when the current fixation point is stable, record the pixel coordinates of the pupil center and the pixel coordinates of the reflective point in all image frames in the last 2 seconds of each fixation point display;

[0028] Step 33: For each fixation point, the pixel coordinates of the pupil center and the pixel coordinates of the reflection point recorded in the last 2 seconds are discarded, excluding the data for cases where the eyes are closed.

[0029] Calculate the mean x-coordinate of the remaining pupil center pixel after removing data. x-axis variance Dx p Mean of the ordinate ordinate variance Dy p If the x-coordinate of the center of a pupil is greater than 1 / 2... Or the ordinate is greater than If the calibration fails, it fails; otherwise, the calibration succeeds. Similarly, calculate the mean of the x-coordinate of the center pixel of the reflective point. x-axis variance Dx g Mean of the ordinate ordinate variance Dy g If the x-coordinate of the center of a pupil is greater than 1 / 2... Or the ordinate is greater than The calibration will fail if the calibration fails; otherwise, the calibration will succeed.

[0030] If calibration fails, return to step 31, i.e., re-enter the calibration module; if all nine fixation points are calibrated successfully, proceed to step 34.

[0031] Step 34: Calculate the horizontal pixel distances d0 to d9 from the pupil center to the center of the Pulcyn spot at observation points P0 to P9 respectively. Calculate the horizontal pixel distance from the pupil center to the center of the Pulcyn spot at observation points P1–P9, and the offset t1–t9 at point P0, where t i =d i -d0(i = 1, 2, ..., 9);

[0032] Step 35: After obtaining (t1,A1), (t2,A2)……(t9,A9), perform linear fitting on them using the least squares method, A=H0t+b0, and calculate the Hessberg ratio H0 and the intercept b0 of the fitted line.

[0033] Furthermore, in the occlusion detection module, the occlusion test method can be selected as a single-sided occlusion test method, an occlusion-unocclusion test method, or an alternating occlusion method.

[0034] Furthermore, the occlusion detection module employs an occlusion removal method, the specific implementation process of which is as follows:

[0035] Step 41: Control the voice broadcaster to prompt the subject to start the occlusion test. After the test starts, the system calls the pupil and reflective point positioning module to continuously acquire images and obtain the position of both eyes and the pixel coordinates of the center of the pupil and reflective point. If both eyes are not detected at this time, the system will re-enter step 41 and control the voice broadcaster to prompt the subject to adjust the position until both eyes are detected. If both eyes and the center of the pupil and reflective point are detected, the system will proceed to step 42.

[0036] Step 42, monitor the eye state of both eyes: First, the eyes are in the initial state s1 at the start of monitoring, then in the ready state s2, and the voice announcer prompts the subject to cover the right eye; the system calls the pupil and reflective point positioning module to detect the right eye state. If the number of frames in which the pupil is not detected is less than fifty, the right eye is in the blinking state s0; when the right eye is in the blinking state and the right eye pupil is detected again, the right eye state returns to the ready state s2; when the number of frames in which the pupil is not detected reaches fifty or more, the right eye enters the occlusion state s3, at which point step 43 is entered;

[0037] Step 43: The system prompts the subject to remove the shield from the right eye and cover the left eye via a voice broadcast. The system records the first frame at the moment the shield is removed from the right eye, when both the reflective point and the pupil are simultaneously detected, as the keyframe. At this time, the right eye is in instantaneous state s4. The system calculates and saves the difference p0 between the pupil pixel coordinates and the reflective point pixel coordinates in this frame. If the time for the right eye to enter instantaneous state s4 exceeds 2 seconds, the system calculates the mean and variance of the pupil pixel coordinate components within 60 frames. If the variance of the coordinate components is less than the set variance threshold T, the system will determine the keyframe.d Then the eye enters a steady state s5; calculate the difference p1 between the center pixel coordinates of the pupil and the center pixel coordinates of the reflective point within 60 frames when entering the steady state; control the voice broadcaster to prompt the subject that the occlusion test is over and proceed to step 44;

[0038] Step 44, calculate the strabismus degree: calculate the offset of the right eye t = p1 - p0, and calculate the strabismus degree of the right eye A = H0t + b0; similarly, first cover the left eye and then cover the right eye, and calculate the offset and strabismus degree of the left eye.

[0039] Compared with the prior art, the present invention has the following technical effects:

[0040] 1. This invention is unaffected by the kappa angle and can accurately measure strabismus. The Hersberg ratio H is obtained through calibration of the subject. Then, an image algorithm is used to measure the offsets T1 and T2 between the pupil center and the center of the reflection point during the occlusion test. The strabismus degree A0 is calculated using H = H(T1-T2). The process of measuring the offset eliminates the error of the kappa angle. Furthermore, this invention calculates the Hersberg ratio H for each subject during the calibration phase, instead of using an average. The average value of H is 12.5, and the variability among subjects is within ±20% of the mean. Therefore, measuring the Hersberg ratio for each subject makes the final result more accurate.

[0041] 2. The device used in this method includes a camera, an infrared light source, a computer host, and a computer monitor, without the need for professional optical instruments.

[0042] 3. This invention defines eye states, including initial state, ready state, blinking state, occlusion state, momentary state, and stable state. Image information is automatically saved in the momentary and stable states for calculating strabismus degree. Therefore, this invention does not require the involvement of a professional physician and can intelligently measure strabismus degree. Simultaneously, this invention uses a voice broadcaster to interact with the user, providing voice prompts for the subject to perform an occlusion test, and can automatically detect the occlusion state of the subject's eyes, obtaining the strabismus detection result after the occlusion test.

[0043] 4. This invention combines a unilateral occlusion test to measure whether the subject has manifest strabismus. If so, the degree of strabismus is given; otherwise, an alternating occlusion test is performed to determine whether the patient has latent strabismus. If so, the degree of strabismus is given; otherwise, the subject is normal. Therefore, this invention can measure both the manifestness and latentness of strabismus and accurately measure the degree of strabismus. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the strabismus detection system of the present invention.

[0045] Figure 2This is an image showing the eye state during the operation of the strabismus monitoring system of the present invention.

[0046] Figure 3 This is a schematic diagram of the rectangular frame used for human eye detection.

[0047] Figure 4 This is a schematic diagram of pupil and reflective point detection.

[0048] The meanings of the labels in the diagram are as follows:

[0049] 1. Headrest; 2. Eye plane; 3. Calibration point; 4. Eye; 5. World coordinate system; 6. Gaze point; 7. Monitor; 8. Camera; 9. Infrared light source. Detailed Implementation

[0050] The basic principle of this invention is that near-infrared light emitted by a near-infrared source forms a high-brightness reflection point on the cornea of ​​the user's eye, known as the Pulcyn spot. When the head is fixed, the horizontal offset (mm) of the Pulcyn spot from the center of the pupil is linearly related to the gaze offset angle (°), and this slope is called the Hirschberg ratio. First, the camera is calibrated, then the Hirschberg ratio H is adjusted, and then an occlusion / unocclusion experiment is performed on the subject. During the experiment, the camera acquires images of the eye, and the algorithm determines the image frame 1 at the moment of unocclusion and the image frame 2 after the eye stabilizes. The horizontal offsets T1 and T2 of the two spots are obtained using an image processing algorithm. The strabismus degree A0 is calculated using H = H(T1-T2).

[0051] The principle of this invention is that the displacement of the Purkinje spot from the center of the pupil is directly proportional to the shift in the gaze angle. Intelligent detection of the degree of strabismus is achieved through image processing algorithms and assessment of eye condition.

[0052] The strabismus degree detection system based on occlusion testing of the present invention includes a headrest 1, a calibration point 3, a display 7, a camera 8, a light source 9, a voice announcer, and a control device. The headrest 1 and the display 7 are positioned opposite each other, and their planes are parallel. The distance between the display 7 and the plane of the headrest 1 is 30cm to 60cm. The calibration point 3 is mounted directly above the headrest 1 via a bracket consisting of two symmetrically arranged columns and a crossbeam on the columns. The calibration point 3 is located in the middle of the crossbeam, and the distance between the two columns is 18-25cm, allowing an adult to place their head inside while facing the display 7. The camera 8 and the light source 9 are respectively mounted on the display and are located in the same plane, parallel to the plane of the display 7. The field of view of the camera 8 covers the headrest 1, ensuring that the eye image of the person being tested can be captured. The distance between the camera 8 and the infrared light source 9 is no more than 15cm. The display 7, camera 8, and voice announcer are all connected to the controller.

[0053] Preferably, calibration point 3 is a black circular dot.

[0054] Specifically, the controller includes a system calibration module, a pupil and reflective point positioning module, a calibration module, and an occlusion detection module. Wherein:

[0055] The system calibration module is used to implement the following process:

[0056] Step 11: Establish a world coordinate system with the bottom left corner of monitor 7 as the origin. The height of the monitor is SH, the width is SW, and the resolution is RWxRH. For example: a 24-inch monitor has a height of SH = 29.9cm, a width of SW = 53.15cm, and a resolution of 1920x1080.

[0057] Step 12: Control camera 8 to acquire images (set image acquisition format to YUY2, image acquisition rate to 60fps);

[0058] Step 13: Based on the acquired image, obtain the conversion ratio (mm / pixel) from planar pixels to distance for headrest 1, and manually measure the coordinates (X, Y, F) of the calibration point center in the world coordinate system. b ,Y b Z b ).

[0059] Specifically, step 13 includes the following operations: converting the acquired image to a grayscale image, and performing image denoising using Gaussian filtering or other methods; performing thresholding on the denoised image using adaptive thresholding or other methods; using morphological opening to remove isolated points, spurs, etc., from the thresholded image to obtain the morphologically processed image; detecting the edges of the morphologically processed image using Canny edge detection or other methods; and detecting calibration point 3 in the image obtained above using Hough transform or other circular detectors to obtain the pixel coordinates (x, y, x) of calibration point 3. b ,y b And the pixel radius r of calibration point 3 on the image. b The formula for calculating the conversion ratio from planar pixels to distance for head support 1 is ratio = R / r. b Where R is the true radius of the calibration point. The coordinates of the center of the calibration point in the world coordinate system (X, R, R) are... b ,Y b Z b (This information can be obtained using a measuring tape or other distance measuring tools.)

[0060] The pupil and reflective point positioning module is used to implement the following process:

[0061] Step 21: Control camera 8 to continuously acquire images of the human eyes at a rate of 60fps, directly acquiring images containing both eyes, and then horizontally flip the images. Horizontal flipping is done so that the left eye in the image corresponds to the patient's left eye, and the right eye in the image corresponds to the patient's right eye, facilitating observation.

[0062] Step 22: Based on the image obtained in Step 21, first locate the positions of both eyes using a human eye detection method (marked by two rectangular boxes, such as...). Figure 3 , Figure 4 (As shown), then the pixel coordinates of the two pupil centers are obtained using pupil localization technology in the two regions respectively. The coordinates of the pupil centers in the world coordinate system established in the system calibration module are also calculated.

[0063] (I) The human eye detection method can adopt any of the following methods: Based on the collected human eye images, (1) manually determine the position of the eyes. Since the heads of different subjects are fixed on the headrest after the system is fixed, the position of the two eyes in the collected images is roughly the same as the position of the whole image. Therefore, two rectangular frames with fixed length, width and position can be manually selected to frame the area of ​​the two eyes. (2) Collect and label multiple positive samples (images containing human eyes) and negative samples (images without human eyes), train the Haar-cascade classifier, and obtain the position of the eyes in the image through the trained classifier. (3) Train human eye detection through deep learning. For example, use the YOLO model, collect and label several positive and negative samples through labelimg software, and then generate a YOLO format txt sample file. Download the pre-trained model to accelerate the model training. After training, save the model parameters. The position of the eyes in the image can be obtained through the trained classifier.

[0064] (II) Pupil position detection employs a deep learning segmentation network model, such as UNet. Acquired images containing pupils are labeled, with the pupil region designated as the foreground and other regions as the background. Binary cross-entropy loss and ellipse fitting error loss are used as loss terms for network training, and the model parameters are saved after training. When needed, the model parameters are loaded, and a segmentation probability map is obtained after inputting the image. Then, a binary image is obtained through threshold segmentation. Connectivity analysis is performed on the binary image, and the centroid of the largest blob is used as the pixel coordinate (x, y) of the left pupil center. L ,y L ), take the longer side of the bounding rectangle of the largest spot as the diameter of the left eye pupil, 2*R. L (R L (where the pupil radius is 1).

[0065] (III) Based on the world coordinate system established in step 11 and the coordinates (X, Y, F) of the calibration point measured in step 12 in the world coordinate system. b ,Y b Z b ) Calculate the coordinates of the center of the left pupil in the world coordinate system as (X... L ,Y L Z L ).in, Z L =Z b Where r b Let R be the pixel radius of calibration point 3 on the image, and R be the true radius (cm) of the calibration point. b y b ) represents the pixel coordinates for calibration.

[0066] (iv) Similarly, perform steps (ii) and (iii) on the right eye to obtain the pixel coordinates (x, y, y) of the center of the right pupil. R ,y R Pupil radius R R The coordinates of the center of the right pupil in the world coordinate system (X) R ,Y R Z R ).

[0067] Step 23, using the center of the left pupil (x) L ,y L With ) as the center and the radius R of the left pupil as the radius... L A circular region with a radius approximately 1.5 times the size of the image is drawn, and the center of the reflective point in the left pupil is located within this region. First, thresholding is performed on this region; due to the high brightness of the reflective point, the threshold is set to approximately 220 (image pixel values ​​range from 0 to 255). Connectivity analysis is then performed on the resulting binary image, and the centroid of the largest spot is used as the pixel coordinate (x, y) of the center of the reflective point in the left pupil. g ,y g Similarly, the pixel coordinates of the center of the reflective point in the right pupil are obtained.

[0068] The calibration module is used to calibrate the personnel being tested and calculate the Hirschberg ratio.

[0069] Step 31: Initialize 9 fixation points with known coordinates: P1(X1,Y1,0), P2(X2,Y2,0)...P9(X9,Y9,0). The pixel coordinates of fixation point P0 are (x0,y0), and its world coordinates are (X0,Y0,0). Therefore, X0 = x0*SW / RW, Y0 = (RH-y0)*SW / RW. Calculate the offset fixation angles from fixation point P0 to fixation points P1, P2...P9 respectively. RH represents the monitor height, in cm.

[0070] Step 32: Display fixation point P0 (X0, Y0, 0) on the monitor. Control the voice announcer to prompt the subject to place their head on headrest 1 and remind them to fixate on coordinate point P0. Next, display at least two known fixation points as calibration points on the monitor sequentially. Control the voice announcer to remind the subject to fixate on these calibration points sequentially. During this process, the system calls the pupil and reflective point localization module to locate the pupil and its reflective point (i.e., repeat step 2). Each calibration point is displayed for 5 seconds. The current calibration point is considered stable when the subject's fixation duration is greater than or equal to 2 seconds. Assuming the subject is observing the calibration point, calculate the mean and standard deviation of the pupil center position within 2 seconds. If the pupil center position coordinates are all less than 1.5 times the standard deviation within 2 seconds, fixation is considered stable. When the current fixation point is stable, record the pixel coordinates of the pupil center and the reflective point in all image frames during the last 2 seconds of each fixation point display. If the center of the pupil is not detected in the image of the human eye's gaze, it is considered a closed eye condition, and the pixel coordinates of the pupil are recorded as (-1, -1) for easy removal. Similarly, for reflective points, if no reflective point is detected, the coordinates of the reflective point are recorded as (-1, -1) for easy removal.

[0071] Step 33: For the last 2 seconds of each fixation point, the pixel coordinates of the pupil center and the pixel coordinates of the reflection point are recorded, and the data of the closed eyes (i.e. the case with coordinates (-1, -1)) are removed.

[0072] Calculate the mean x-coordinate of the remaining pupil center pixel after removing data. x-axis variance Dx p Mean of the ordinate ordinate variance Dy p If the x-coordinate of the center of a pupil is greater than... Or the ordinate is greater than If the result is negative, the calibration fails; otherwise, the calibration succeeds. Similarly, calculate the mean of the x-coordinate of the center pixel of the reflective point. x-axis variance Dx g Mean of the ordinate ordinate variance Dy g If the x-coordinate of the center of a pupil is greater than... Or the ordinate is greater than If the result is negative, the calibration will fail; otherwise, the calibration will succeed.

[0073] If calibration fails, return to step 31, i.e., re-enter the calibration module. If all nine fixation points are calibrated successfully, proceed to step 34.

[0074] Step 34: Calculate the horizontal pixel distances d0 to d9 from the pupil center to the center of the Pulcyn spot at observation points P0 to P9 respectively. Calculate the horizontal pixel distance from the pupil center to the center of the Pulcyn spot at observation points P1–P9, and the offset t1–t9 at point P0. Where t i =d i -d0(i = 1, 2, ..., 9).

[0075] Step 35: After obtaining (t1,A1), (t2,A2)……(t9,A9), perform linear fitting on them using the least squares method, A=H0t+b0, and calculate the Hessberg ratio H0 and the intercept b0 of the fitted line.

[0076] The occlusion detection module is used to implement the following process:

[0077] The control voice announcer prompts the subject to use the occlusion test method to measure the degree of strabismus. The occlusion test method can be selected as unilateral occlusion test, occlusion-removal test, or alternating occlusion method. The following explanation uses the occlusion-removal method as an example.

[0078] Step 41: The voice announcer prompts the subject to begin the occlusion test. After the test begins, the system calls the pupil and reflective point localization module to continuously acquire images and obtain the pixel coordinates of the center of the reflective point of both pupils. If both eyes are not detected at this time, the system re-enters step 41 and the voice announcer prompts the subject to adjust the position until both eyes are detected. If the center of the pupils and reflective points of both eyes is detected, the system proceeds to step 42.

[0079] Step 42, monitor the condition of both eyes. For example... Figure 2 As shown, in this invention, the eye state is designed to include an initial state (s1), a ready state (s2), an occluded state (s3), a momentary state (s4), a stable state (s5), and a blinking state (s0). First, at the start of monitoring, the eye is in the initial state (s1), and then (generally 2 seconds after the initial state) the eye is in the ready state (s2), and the voice announcer prompts the subject to cover their right eye. The system calls the pupil and reflective point positioning module to detect the right eye state. If the number of consecutive frames without pupil detection is less than fifty, the right eye is in the blinking state (s0). When the right eye is in the blinking state and the right pupil is detected again, the right eye state returns to the ready state (s2). When the number of consecutive frames without pupil detection reaches fifty or more, the right eye enters the occluded state (s3), at which point step 43 is initiated.

[0080] Step 43: The system uses a voice announcer to prompt the subject to remove the shield from their right eye and cover their left eye. The system records the first frame at the instant the shield is removed from the right eye, when both the reflective point and the pupil are simultaneously detected, as the keyframe. At this moment, the right eye is in the transient state (s4). The system calculates and saves the difference p0 between the pupil pixel coordinates and the reflective point pixel coordinates in this frame. If the time it takes for the right eye to enter the transient state (s4) exceeds 2 seconds, the mean and variance of the pupil pixel coordinate components within 60 frames are calculated. If the variance of all coordinate components is less than the set variance threshold T, the system will consider the result. d The eye then enters a steady state (s5). Calculate the difference p1 between the pupil center pixel coordinates and the reflective point center pixel coordinates within 60 frames when entering the steady state. The voice announcer then prompts the subject that the occlusion test is over and proceeds to step 44.

[0081] Step 44, calculate the strabismus degree: calculate the offset of the right eye t = p1 - p0, and calculate the strabismus degree of the right eye A = H0t + b0.

[0082] Similarly, you can cover the left eye first and then the right eye to calculate the offset and strabismus degree of the left eye.

Claims

1. A strabismus degree detection system based on occlusion testing, characterized in that, The system includes a headrest (1), a calibration point (3), a display (7), a camera (8), a light source (9), a voice announcer, and a control device. The headrest (1) and the display (7) are positioned opposite each other, and their planes are parallel. The calibration point (3) is mounted directly above the headrest (1) via a bracket consisting of two symmetrically positioned columns and a crossbeam on each column, with the calibration point (3) positioned in the middle of the crossbeam. The camera (8) and the light source (9) are mounted on the display and are located in the same plane, which is parallel to the plane of the display (7). The field of view of the camera (8) covers the headrest (1). The display (7), camera (8), and voice announcer are all connected to a controller. The controller includes a system calibration module, a pupil and reflection point positioning module, a calibration module, and an occlusion detection module. The system calibration module is used to implement the following process: Step 11, establish a world coordinate system with the lower left corner of the display (7) as the origin, the height of the display (7) is SH, the width is SW, and the resolution is RWxRH; Step 12, control the camera (8) to acquire human eye images; Step 13, based on the acquired human eye images, obtain the conversion ratio from planar pixels to distance of the headrest (1), and manually measure the coordinates of the center of the calibration point in the world coordinate system. ; The pupil and reflective point localization module obtains the pixel coordinates of the center of the pupils of both eyes, the coordinates of the center of the pupils in the world coordinate system, and the pixel coordinates of the center of the reflective point in the pupils based on the acquired human eye images. The calibration module is used to call the pupil and reflection point positioning module to calibrate the subject and calculate the Hersberg ratio. The occlusion detection module is used to call the pupil and reflective point positioning module and control the voice broadcaster to prompt the test subject to use the occlusion test method to measure the degree of strabismus.

2. The strabismus degree detection system based on occlusion testing as described in claim 1, characterized in that, The calibration point (3) is a black circular dot.

3. The strabismus degree detection system based on occlusion testing as described in claim 1, characterized in that, Step 13 includes the following operations: The acquired human eye image is converted into a grayscale image, and Gaussian filtering is used to denoise the human eye image; adaptive thresholding or other methods are used to perform thresholding on the denoised human eye image; The isolated points and spurs in the thresholded human eye image are removed using morphological opening operation to obtain the morphologically processed human eye image; the edges of the morphologically processed human eye image are detected using Canny edge detection or other methods; the calibration points (3) in the processed human eye image are detected using Hough transform to obtain the pixel coordinates of the calibration points (3). , ) and the pixel radius of the calibration point (3) on the human eye image. ; The formula for calculating the conversion ratio from planar pixels to distance of the head support (1) is ratio = R / , where R is the true radius of the calibration point; Coordinates of the calibration point center in the world coordinate system Obtained using a measuring tape.

4. The strabismus degree detection system based on occlusion test as described in claim 3, characterized in that, The implementation process of the pupil and reflective point positioning module is as follows: Step 21: Control the camera (8) to continuously acquire human eye images at a rate of 60fps, directly acquire images containing human eyes, and then horizontally flip the human eye images. Step 22: Based on the human eye image obtained in step 21, first locate the positions of both eyes using the human eye detection method, and then use pupil localization technology in the two regions to obtain the pixel coordinates of the centers of the two pupils. and Calculate the coordinates of the pupil center in the world coordinate system established in the system calibration module; Step 23, centering on the left pupil Centered on the left eye, with the radius of the left pupil... Draw a circular region with a radius of 1.5 times the radius of the left eye pupil, and find the pixel coordinates of the center of the reflection point within the circular region. ; Similarly, the pixel coordinates of the center of the reflective point in the right pupil are obtained.

5. The strabismus degree detection system based on occlusion test as described in claim 4, characterized in that, The specific process of step 22 is as follows: (a) The human eye detection method adopts any of the following methods: (1) Based on the collected multiple human eye images, (2) manually determine the position of the eyes; (3) collect and label multiple positive and negative samples, train the Haar-cascade classifier, and obtain the position of the eyes in the human eye image through the trained classifier. (3) Human eye detection is trained using deep learning methods; (ii) Pupil position detection uses a segmentation network model in deep learning. After inputting the human eye image, a segmentation probability map is obtained. Then, a binary image is obtained through threshold segmentation. Connectivity analysis is performed on the binary image, and the centroid of the largest spot is used as the pixel coordinate of the center of the left pupil. The longer side of the bounding rectangle of the largest spot is taken as the diameter of the left eye pupil, which is 2. , The radius of the pupil; (iii) Based on the world coordinate system established in step 11 and the coordinates of the calibration points measured in step 12 in the world coordinate system Calculate the coordinates of the center of the left pupil in the world coordinate system. ;in, , , ,in Let R be the pixel radius of the calibration point (3) on the human eye image, and let R be the true radius of the calibration point. The pixel coordinates of the center of the calibration point; (iv) Similarly, perform steps (ii) and (iii) on the right eye to obtain the pixel coordinates of the center of the right pupil. Pupil radius The coordinates of the center of the right pupil in the world coordinate system .

6. The strabismus degree detection system based on occlusion test as described in claim 5, characterized in that, The implementation process of the calibration module is as follows: Step 31, initialize 9 gaze points with known coordinates: , ... fixation point The pixel coordinates are Its world coordinates Then there is , ; respectively calculate from the fixation point To fixation point , ... offset gaze angle ... RH is the monitor height, and RW is the monitor width; Step 32: Display the gaze point on the monitor. The voice broadcast system prompts the test subject to place their head on the headrest (1) and reminds the test subject of their gaze point. Then, at least two known gaze points are displayed on the display (7) as calibration points. The voice broadcaster is controlled to remind the test subject to gaze at these calibration points in turn. The system calls the pupil and reflective point positioning module to locate the pupil and reflective point. Each calibration point is displayed for 5 seconds. When the test subject's gaze duration is greater than or equal to 2 seconds, the current calibration point is considered stable. Calculate the mean and standard deviation of the pupil center position within 2 seconds. If the pupil center position coordinates are all less than 1.5 times the standard deviation within 2 seconds, the fixation is considered stable. When the current fixation point is stable, record the pixel coordinates of the pupil center and the pixel coordinates of the reflection point in the human eye image of each fixation point in the last 2 seconds. Step 33: For each fixation point, the pixel coordinates of the pupil center and the pixel coordinates of the reflection point recorded in the last 2 seconds are discarded, excluding the data for cases where the eyes are closed. Calculate the mean x-coordinate of the remaining pupil center pixel after removing data. variance of the x-axis and mean of the y-axis ordinate variance ; If the x-coordinate of the center of a pupil is greater than Or the ordinate is greater than If the value is not found, the calibration fails; otherwise, the calibration succeeds. Similarly, calculate the mean of the x-coordinate of the center pixel of the reflective point. variance of the x-axis Mean of the ordinate ordinate variance ; If the x-coordinate of the center of a pupil is greater than Or the ordinate is greater than If the result is negative, the calibration fails; otherwise, the calibration is successful. If calibration fails, return to step 31, i.e., re-enter the calibration module; if all nine fixation points are calibrated successfully, proceed to step 34. Step 34: Calculate the observation points respectively Horizontal pixel distance from the center of the pupil to the center of the Pulchin spot ,in ; Calculate observation points The horizontal pixel distance from the center of the pupil to the center of the Pulcyn spot is related to the point offset at time ,in (i=1, 2...9); Step 35, we get ( , ), ( , )……( , After that, the least squares method is used to fit straight lines to them. The Hessberg ratio was calculated. The intercept b0 of the fitted line.

7. The strabismus degree detection system based on occlusion test as described in claim 1, characterized in that, In the occlusion detection module, the occlusion test method can be selected from the single-sided occlusion test method, the occlusion-unocclusion test method, or the alternating occlusion method.

8. The strabismus degree detection system based on occlusion test as described in claim 6, characterized in that, The occlusion detection module employs an occlusion-de-occlusion testing method, the specific implementation process of which is as follows: Step 41: Control the voice broadcaster to prompt the test subject to start the occlusion test. After the test starts, the system calls the pupil and reflective point positioning module to continuously collect human eye images and obtain the position of both eyes and the pixel coordinates of the center of the pupil and reflective point. If both eyes are not detected at this time, return to step 41 and control the voice broadcaster to prompt the test subject to adjust the position until both eyes are detected. If both eyes and the center of the pupil and reflective point are detected, proceed to step 42. Step 42, monitor the eye state of both eyes: First, the eyes are in the initial state s1 at the start of monitoring, then in the ready state s2, and the voice broadcaster prompts the subject to cover the right eye; the system calls the pupil and reflective point positioning module to detect the right eye state. If the number of consecutive frames in which the pupil is not detected is less than fifty, the right eye is in the blinking state s0; when the right eye is in the blinking state and the right eye pupil is detected again, the right eye state returns to the ready state s2; when the number of consecutive frames in which the pupil is not detected reaches fifty or more, the right eye enters the occlusion state s3, at which point step 43 is entered; Step 43: The voice announcer prompts the subject to remove the shield from the right eye and cover the left eye. The system records the first frame at the instant the shield is removed from the right eye, when both the reflective point and the pupil are simultaneously detected, as the keyframe. At this time, the right eye is in instantaneous state s4. The system calculates and saves the difference between the pupil pixel coordinates and the reflective point pixel coordinates in the keyframe. After the right eye enters instantaneous state s4 for more than 2 seconds, calculate the mean and variance of the pupil pixel coordinate components within 60 frames. If the variance of the coordinate components is less than the set variance threshold... The eye then enters a steady state s5; calculate the difference between the pupil center pixel coordinates and the reflection point center pixel coordinates within 60 frames when entering the steady state. ; The voice announcer will notify the subject that the occlusion test is over and proceed to step 44; Step 44, Calculate the degree of strabismus: Calculate the deviation of the right eye. The degree of strabismus in the right eye was calculated. = ; Similarly, cover the left eye first, then the right eye, and calculate the offset and strabismus degree of the left eye.

Images