Eyeball motion anomaly detection method and system

Abstract

The invention discloses an eyeball movement anomaly detection method and system, and belongs to the technical field of intelligent medical treatment. Comprising the following steps: collecting a calibration video, and processing the calibration video to obtain a detection threshold value; collecting a detection video, and processing the detection video to obtain detection data; and processing and analyzing the detection data and the detection threshold to obtain an eyeball movement anomaly detection result. Compared with the prior art, the method has the beneficial effects that the abnormal movement of the eyeball is detected by collecting and analyzing the video data, so that the subjectivity and error of manual judgment are reduced, and the movement of the eye position is accurately quantified through a video or image analysis technology, so that the accuracy and reliability of the detection result data are ensured.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical technology, and more specifically, to a method and system for detecting abnormal eye movement. Background Art

[0002] Abnormal eye movement detection is a crucial part of ophthalmology. It provides an important basis for assessing the health of the visual system through real-time monitoring and analysis of eye movements. During the detection process, a specific abnormality, namely unstable or abnormal eye movement deviation, has become the focus of attention. This abnormality is reflected in a variety of ophthalmic diseases. Among them, a transitional type of strabismus between exophoria and constant exotropia is particularly noteworthy. Here we broadly describe it as "abnormal eye movement deviation". Abnormal eye movement deviation has a higher probability of occurrence in children, and the average age of this abnormality is concentrated between 4 and 5 years old. This abnormal deviation is manifested as an involuntary outward deviation of the eyeball when looking at a distance, fatigue, lack of concentration or poor physical condition. In addition, some people with abnormal conditions also show symptoms of monocular photophobia in bright environments and tend to reduce discomfort by closing one eye. Although most people with abnormal eye movement still retain near-stereoscopic vision, if not intervened and treated in time, abnormal eye movement deviation will not only affect the appearance of the abnormal people, but more importantly, it will have a profound negative impact on the visual function of both eyes and overall physical and mental health. Many people with abnormal eye movement deviation find it difficult to discover the existence of abnormal eye movement deviation by themselves in the early stage, and by the time they discover it, they have often missed the best time for intervention.

[0003] However, the current detection of eye movement abnormalities mainly relies on professional means in hospitals, such as the cover-uncover test and the nine-square grid eye position assessment. The implementation of these detection methods is not only highly dependent on the degree of cooperation of the population, but also affected by the doctor's clinical experience and subjective judgment. Therefore, it is difficult to be effectively applied in a home environment. Especially for young children with low cooperation, daily detection of eye movement abnormalities is full of challenges. It is often difficult to accurately capture the subtle changes of eye movement abnormalities in daily life. Therefore, in view of the many limitations of the current methods for detecting eye movement abnormalities, it is urgent to develop a more efficient and accurate method for detecting eye movement abnormalities to overcome the shortcomings of existing technologies, improve detection efficiency, and at the same time improve early recognition rates, so as to provide more timely and effective intervention for the masses. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the problem of how to detect abnormal eye movements in the prior art, the present invention provides a method and system for detecting abnormal eye movements, which can detect abnormal eye movements without being restricted by environmental conditions and can ensure the accuracy of the detection results.

[0006] 2. Technical solution

[0007] The purpose of the present invention is achieved through the following technical solutions.

[0008] The content of this application is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this application is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0009] Some embodiments of the present application propose an eye movement abnormality detection method and system to solve the technical problems mentioned in the above background technology section.

[0010] As the first aspect of the present application, some embodiments of the present application provide a method for detecting abnormal eye movements, comprising the following steps: acquiring a calibration video, and obtaining a detection threshold by processing the calibration video; acquiring a detection video, and obtaining detection data by processing the detection video; and obtaining an abnormal eye movement detection result by processing and analyzing the detection data and the detection threshold.

[0011] Furthermore, the detection data includes detecting the inner angle distance between both eyes in all frame images of the video.

[0012] Furthermore, the process of acquiring the calibration video is specifically as follows: synchronously acquiring the first left-eye video data and the first right-eye video data; and merging the first left-eye video data and the first right-eye video data into the calibration video by using video fusion;

[0013] Furthermore, the duration of capturing the calibration video is greater than or equal to 10 seconds; the clarity of the calibration video is consistent with the clarity of the first left-eye video data and the first right-eye video data.

[0014] Furthermore, the process of processing the calibration video is specifically as follows: identifying and locating the pupil position and the inner and outer eye corner positions of the eye in the calibration video, and obtaining the pupil coordinate data and the inner and outer eye corner coordinate data of the calibration video;

[0015] Calculate the inner angle distance of the left eye, the inner angle distance of the right eye, and the inner angle distance of both eyes in the calibration video according to the pupil coordinate data and the inner and outer eye corner coordinate data of the calibration video;

[0016] The detection threshold is calculated based on the interocular angle distance of the calibration video.

[0017] Furthermore, the process of identifying and locating the pupil position of the eye in the calibration video is specifically as follows: reading images frame by frame from the calibration video and preprocessing each frame of the image;

[0018] The connected domain is used to find the pupil contour in the image and the connected domain that meets the conditions is screened; the screened connected domain is subjected to ellipse fitting to obtain the position and shape of the pupil; and the pupil coordinates of each frame image in the calibration video are obtained and stored through template matching.

[0019] Furthermore, the process of identifying and locating the inner and outer corners of the eye in the calibration video is as follows: reading images frame by frame from the calibration video and scaling and graying each frame; removing noise in the image using Gaussian blur;

[0020] The trained eye inside and outside corner recognition model is used to extract and classify the image features, and the inside and outside eye corner coordinates of each frame in the calibration video are obtained and stored.

[0021] Furthermore, the process of calculating the inner angle distance of the left eye and the inner angle distance of the right eye in the calibration video is as follows:

[0022] According to the pupil coordinates and inner and outer eye corner coordinates obtained in the calibration video, the projection value of the left eye pupil center on the line connecting the inner and outer eye corners in each frame image in the calibration video is calculated as the left eye inner corner distance;

[0023] Calculate the projection value of the right eye pupil center on the line connecting the inner and outer eye corners in each frame of the calibration video as the right eye inner corner distance.

[0024] Furthermore, the analysis and judgment process is specifically as follows: according to the detection data of all the frame images in the detection video, the difference between the detection data of each frame image in the detection video and the detection threshold is calculated and judged;

[0025] If the difference is greater than 0, it is marked as an anomaly;

[0026] If the difference is less than or equal to 0, it is marked as normal;

[0027] Traverse all the frames in the detection video and count the number of consecutive image frames marked as abnormal;

[0028] If the number of consecutive image frames marked as abnormal is greater than 30, it is determined that an abnormal eye movement event has occurred;

[0029] If the number of image frames continuously marked as abnormal is less than or equal to 30, it is determined that no abnormal eye movement event has occurred.

[0030] As the second aspect of the present application, some embodiments of the present application provide an eye movement abnormality detection system, including: a detection threshold acquisition module: collecting a calibration video, and obtaining a detection threshold by processing the calibration video; a detection data acquisition module: collecting a detection video, and obtaining detection data by processing the detection video; a detection analysis module: obtaining an eye movement abnormality detection result by processing and analyzing the detection data and the detection threshold.

[0031] 3. Beneficial effects

[0032] Compared with the prior art, the advantages of the present invention are: this solution makes up for the defects and shortcomings of traditional manual detection methods, and detects abnormal eye movements by collecting and analyzing video data, which not only reduces the subjectivity and errors of human judgment, but also accurately quantifies the movement of eye position through video or image analysis technology, thereby ensuring the accuracy and reliability of the test result data; this solution is suitable for people of all ages, especially children and adolescents, and can be used for daily home monitoring, so that users can regularly perform eye position examinations without going to hospitals or clinics, greatly improving the convenience and accessibility of detection; the current Newcastle eye position control ability scoring standard has a gap in home control ability scoring, and this solution can fill this gap through the collection and analysis of objective data, which not only provides an accurate judgment on whether the eye movement is abnormal, but also can evaluate the patient's eye position control ability in different situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of pupil identification and positioning in one embodiment of the present invention;

[0034] Figure 2 A flowchart of eye corner recognition and positioning in one embodiment of the present invention;

[0035] Figure 3 A schematic diagram of the projection of the pupil center on the line connecting the inner and outer corners of the eye in one embodiment of the present invention;

[0036] Figure 4 A schematic diagram of human eye distortion captured by a camera when the camera distance is equal to the pupil distance in one embodiment of the present invention;

[0037] Figure 5 A schematic diagram of human eye distortion captured by a camera when the camera distance is greater than the pupil distance in one embodiment of the present invention;

[0038] Figure 6 A schematic diagram of human eye distortion captured by a camera when the camera distance is less than the pupil distance in one embodiment of the present invention;

[0039] Figure 7 FIG. 4 is a graph of the inner eye angle distance of both eyes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0041] according to Figures 1 to 7 As shown, a method for detecting abnormal eye movement of the present invention comprises the following steps:

[0042] Collect calibration video and obtain detection threshold by processing the calibration video;

[0043] Collect detection videos and obtain detection data by processing the detection videos;

[0044] By processing and analyzing the detection data and the detection threshold, the eye movement abnormality detection result is obtained.

[0045] In a specific embodiment, the specific process of the eye movement abnormality detection method includes:

[0046] S1. Get the calibration video and get the detection threshold:

[0047] The first left-eye video data and the first right-eye video data are collected by the collection device, and the first left-eye video data and the first right-eye video data are fused into a calibration video; the calibration video is processed to obtain a detection threshold used for judgment in the detection process.

[0048] The acquisition device is used to acquire eye video data or image data of the detected object. The acquisition device is electrically connected to the control device. The control device obtains the data acquired by the acquisition device and processes and stores it.

[0049] Specifically, the acquisition device can be a glasses device for detecting eye movement, and the glasses device is electrically connected to the control device through a Bluetooth or WIFI module. The glasses device includes a video recording system for collecting images and / or video information of the eyes and transmitting them to the control device. The video recording system uses dual cameras, which are responsible for collecting left eye video data and right eye video data respectively; the resolution of the dual cameras is not less than 1080P, and the frame rate is at least 30fps to ensure the clarity and smoothness of the video data.

[0050] Before the acquisition begins, a position check is first performed to ensure that the left eye and right eye of the object being detected are both in the detection area and that complete eye video data can be acquired; after the position check is completed, the left eye video data and the right eye video data are acquired.

[0051] Specifically, the control device verifies the position by judging the camera preview image. It ensures that the left and right eyes of the detected object are located in the center area of ​​the camera field of view, and the eye features are clearly visible. The control device uses facial recognition to assist in the verification, ensuring that the face of the detected object is facing the camera and both eyes are on the same horizontal line.

[0052] During the acquisition process, the first left-eye video data and the first right-eye video data are synchronously acquired, and the acquisition time is no less than 10 seconds to ensure that the dynamic changes of the eyes of the detected object are fully captured.

[0053] After the acquisition is completed, the first left-eye video data and the first right-eye video data are merged into a calibration video of no less than 10 seconds using video fusion. During the video fusion process, the synchronization and consistency of the first left-eye video data and the first right-eye video data are ensured. The clarity of the fused calibration video is consistent with the acquired first left-eye video data and the first right-eye video data, ensuring that the eye features of the detected object can be clearly displayed.

[0054] Specifically, after obtaining the calibration video, the pupil position and the inner and outer corners of the eye in the calibration video are identified and located to obtain the pupil coordinate data and the inner and outer corners of the eye coordinate data of the calibration video; based on the pupil coordinate data and the inner and outer corners of the eye coordinate data of the calibration video, the inner angle distance of both eyes in the calibration video is calculated; based on the inner angle distance of both eyes in the calibration video, calculation is performed to obtain the detection threshold used for judgment during the detection process.

[0055] The interocular distance refers to the horizontal distance between the inner corners of the two eyes, also known as the intraocular distance. The interocular distance varies between individuals and is affected by a variety of factors such as genetics, race, and individual differences. Based on the interocular distance, the detection threshold used to determine whether the eye movement is abnormal can be calculated during the detection process. If the deviation of the eye movement exceeds this threshold range, it may be regarded as abnormal eye movement.

[0056] Specifically, the pupil coordinate data of the calibration video includes the left eye pupil coordinate data and the right eye pupil coordinate data of the person being tested in the calibration video; the inner and outer corner of the eye coordinate data of the calibration video includes the left eye inner corner of the eye coordinate data, the left eye outer corner of the eye coordinate data, the right eye inner corner of the eye coordinate data and the right eye outer corner of the eye coordinate data of the person being tested in the calibration video.

[0057] More specifically, the steps of identifying and locating the pupil position of the eye in the calibration video include: reading images frame by frame from the calibration video, preprocessing each frame of the image, including reducing the scale, binarization, and expansion and corrosion processing, etc., to reduce the amount of calculation and highlight the features of the eye; using connected domain analysis to find the pupil contour in the image, setting an area threshold, a circularity threshold, and a position threshold to screen out connected domains that meet the conditions; performing ellipse fitting on the screened connected domains to obtain the position and shape of the pupil in the calibration video; further verifying the accuracy of the pupil position through template matching, and obtaining and storing the coordinates of the center position of the pupil in each frame of the calibration video.

[0058] Connected Component refers to the area composed of pixels with the same pixel value and adjacent positions in the image. In the present embodiment, connected component analysis is used to find the pupil contour in the image. After preprocessing each frame of image and highlighting the eye features, connected component analysis can be used to screen out qualified connected components, which are likely to be candidate areas of the pupil. Further processing, such as ellipse fitting and template matching, is used to verify the accuracy of these candidate areas and finally determine the position and shape of the pupil. Therefore, connected component analysis can assist in accurately finding the position of the pupil from a complex image background during pupil position identification and positioning.

[0059] In a specific embodiment, the specific process of obtaining pupil coordinate data of the calibration video is as follows:

[0060] Read the first frame of image from the calibration video; pre-process the read image, reduce the image ratio and reduce the amount of calculation; binarize the image and perform expansion and corrosion processing on the image;

[0061] After the preprocessing is completed, the connected domain of the image is found, the pupil contour in the image is found, and written into a list; the area threshold, circularity threshold, and position threshold are set to filter the connected domain that meets the conditions; polygonal approximation is performed on the connected domain, such as ellipse fitting; and template matching is performed to obtain the pupil position of the located calibration video;

[0062] The coordinate data of the pupil center position in each frame image in the calibration video is obtained and stored, and the pupil position is continuously identified and located for the next frame image until all frame images in the calibration video are processed.

[0063] Specifically, the steps of identifying and locating the inner and outer corners of the eye in the calibration video include: reading images frame by frame from the calibration video and scaling and grayscale each frame to reduce the amount of calculation and increase the processing speed; using Gaussian blur technology to remove noise in the image and improve the image quality; loading a trained model for identifying inner and outer corners of the eye, extracting features and classifying the image to obtain the positions of the inner and outer corners of the eye in the calibration video; obtaining and storing the coordinate positions of the inner and outer corners of the eye in each frame of the calibration video.

[0064] In a specific embodiment, the specific process of obtaining the inner and outer eye corner coordinate data of the calibration video is as follows:

[0065] Read the first frame of image from the calibration video; scale and grayscale the read image to reduce the amount of calculation; further perform Gaussian blur processing on the image to remove noise;

[0066] Load the inner and outer corners of the eye training model to obtain the inner and outer corners of the eye position of the current frame image;

[0067] The coordinate data of the inner and outer corners of the eye in each frame image are obtained and stored, and the inner and outer corners of the eye positions are continuously identified and located for the next frame image until all frame images in the calibration video are processed.

[0068] Specifically, after obtaining the pupil center coordinates of the left eye, the inner corner coordinates of the left eye, the outer corner coordinates of the left eye, the pupil center coordinates of the right eye, the inner corner coordinates of the right eye and the outer corner coordinates of the right eye of the person being tested in the calibration video, it is necessary to further calculate the inner angle distance of the left eye, the inner angle distance of the right eye and the inner angle distance of both eyes of the person being tested in the calibration video.

[0069] Considering that in the scenario of hospital testing or home testing, when the distance between the left and right cameras of the acquisition device does not match the pupil distance, the collected video data or image data will be distorted, which will seriously affect the accuracy of the calculation. Therefore, the projection value of the left eye's pupil center on the line connecting the inner and outer corners of the eye is used as the left eye's inner angle distance, and similarly, the projection value of the right eye's pupil center on the line connecting the inner and outer corners of the eye is used as the right eye's inner angle distance, and then the inner eye distance of both eyes is calculated.

[0070] More specifically, the steps of calculating the inner angle distance of the left eye, the inner angle distance of the right eye, and the inner angle distance of both eyes of the subject in the calibration video include:

[0071] According to the pupil coordinate data and the inner and outer eye corner coordinate data of the calibration video, the projection value of the left eye pupil center on the line connecting the inner and outer eye corners in each frame image of the calibration video is calculated as the left eye inner angle distance of the calibration video; the projection value of the right eye pupil center on the line connecting the inner and outer eye corners in each frame image is calculated as the right eye inner angle distance of the calibration video; according to the obtained data of the left eye inner angle distance and the right eye inner angle distance of the calibration video, the binocular inner angle distance of the calibration video is calculated.

[0072] In a specific embodiment, the process of obtaining the detection threshold also includes calculating the detection threshold based on the interocular angle distance of the calibration video, which is used for subsequent analysis and judgment of abnormal eye movement results.

[0073] Since the camera spacing and the pupil distance do not match, this may cause video data distortion. At the same time, the different positions of the pupil in the eye will cause the mean value of the inner eye angle distance to fluctuate up and down, that is, the variance. Therefore, in this embodiment, the method of calculating the inner eye angle distance using the projection value is used to improve the accuracy of the calculation result. The average value of the inner eye angle distance of all frame images in the calibration video is calculated and multiplied by the coefficient sens as the detection threshold. Among them, the coefficient sens is a coefficient used to correct the average value of the inner eye angle distance of the two eyes.

[0074] The average value of the binocular inner angle distance refers to the sum of the left eye inner angle distance and the right eye inner angle distance (in this embodiment, it refers to the projection value of the pupil center on the line connecting the inner and outer eye corners) calculated after processing all the frame images in the calibration video, or a statistical value of their respective average values. This average value reflects a stable state of the binocular inner angle distance of the detected object during the detection process.

[0075] In a specific embodiment, the specific method for calculating the projection of the pupil on the line connecting the inner and outer corners of the eye is as follows:

[0076] First, establish an image coordinate system with the upper left corner of the image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis; in this coordinate system, the coordinates of the inner and outer corners of the eye and the center of the pupil can be accurately located;

[0077] Using image processing to identify and locate the coordinate point of the inner corner of the eye and the coordinate point of the outer corner of the eye; in the image coordinate system, these two coordinate points are connected to form a straight line, which represents the connection line between the inner and outer corners of the eye;

[0078] Using image processing, identify and locate the coordinates of the pupil center point; project the coordinates of the pupil center point vertically onto the line connecting the inner and outer corners of the eye;

[0079] Calculate the distance from the projection point to the inner corner of the eye. This distance reflects the offset of the pupil relative to the inner corner of the eye.

[0080] Calculate the length of the line from the projection point to the inner and outer corners of the eye.

[0081] Specifically, image processing includes technologies such as edge detection and contour extraction. Projecting the coordinates of the center of the pupil vertically onto the line connecting the inner and outer corners of the eye means finding the point on the line closest to the center of the pupil. This point is the projection point of the center of the pupil on the line. If the camera spacing is equal to the pupil distance, the distance from the projection point to the midpoint of the line connecting the inner and outer corners of the eye is 0.5, that is, the distance from the projection point to the inner corner of the eye divided by the total length of the line multiplied by 0.5. The 0.5 here means that the projection point is at the midpoint of the line.

[0082] In this embodiment, the coefficient sens is a coefficient used to correct the average value of the distance between the inner corners of the eyes. In the actual detection process, various factors (such as the mismatch between the camera spacing and the pupil distance, eye movement, and image distortion) may cause a certain error in the calculated distance between the inner corners of the eyes. Therefore, this coefficient is introduced to fine-tune the average value to improve the accuracy of the detection.

[0083] Specifically, the default value of the coefficient sens is 1.05, and the maximum value of the coefficient sens is 1.10. By default, the coefficient sens is 1.07, which means that in the absence of other additional information or special circumstances, the average value of the inner eye angle distance of the eyes is slightly amplified (i.e., multiplied by 1.07); at the same time, in order to cope with extreme situations or special needs, the upper limit of the value of sens is set to 1.05, that is, when it is most necessary to improve the detection sensitivity or deal with larger errors, the coefficient sens can be adjusted to the lowest value. In order to reduce specificity, that is, misjudgment, the value of sense can be enlarged to a maximum of 1.10. By calculating the average value of the inner eye angle distance of the eyes of all frame images in the detection video and correcting it with the coefficient sens, an accurate and reliable detection threshold is obtained for use in the subsequent steps of judging eye movement abnormalities, thereby improving the accuracy and reliability of detection.

[0084] S2. Obtaining test data:

[0085] A detection video is obtained, and the data of the inner angle distance of the left eye and the inner angle distance of the right eye in each frame of the detection video are calculated; based on the data of the inner angle distance of the left eye and the inner angle distance of the right eye in each frame of the detection video, the inner angle distance of both eyes in each frame of the detection video is calculated, and a data set of the inner angle distance of both eyes of all frame images in the detection video is obtained as the detection data.

[0086] Specifically, the second left-eye video data and the second right-eye video data of the detected object are collected by the collection device, and the second left-eye video data and the second right-eye video data are merged into a detection video.

[0087] In step S1, the position of the acquisition device is verified to ensure that the left eye and the right eye of the detected object are both in the detection area and that complete eye video data can be acquired. After obtaining the detection threshold according to the calibration video, the detection video is acquired. During the acquisition process, the second left eye video data and the second right eye video data are synchronously acquired, and the acquisition time is not less than 4 hours to ensure that the dynamic changes of the eyes of the detected object are fully captured.

[0088] In a specific embodiment, the duration of collecting the detection video is 12 hours / day, and the day cycle is 7 to 14 days.

[0089] After the second left-eye video data and the second right-eye video data are collected, the second left-eye video data and the second right-eye video data are merged into a detection video using video fusion. During the video fusion process, the synchronization and consistency of the second left-eye video data and the second right-eye video data are ensured. The clarity of the fused detection video is consistent with the collected second left-eye video data and the second right-eye video data, ensuring that the eye features of the detected object can be clearly displayed.

[0090] Specifically, the pupil position and the inner and outer corners of the eye in the detection video are identified and located to obtain pupil coordinate data and inner and outer corners of the eye coordinate data; based on the pupil coordinate data and the inner and outer corners of the eye coordinate data, the inner angle distance of the two eyes is calculated to form detection data, and the detection data is stored by the control device.

[0091] In a specific embodiment, the specific process of obtaining pupil coordinate data of the detection video is as follows:

[0092] Read the first frame of image from the detection video; pre-process the read image, reduce the image ratio and reduce the amount of calculation; perform binarization processing on the image and perform expansion and corrosion processing on the image;

[0093] After the preprocessing is completed, the connected domain of the image is found, the pupil contour in the image is found, and written into a list; the area threshold, circularity threshold, and position threshold are set to filter the connected domain that meets the conditions; polygonal approximation is performed on the connected domain, such as ellipse fitting; and template matching is performed to obtain the located pupil position;

[0094] The coordinate data of the pupil center position in each frame image is obtained and stored, and the pupil position is continuously identified and located for the next frame image until all frame images in the detection video are processed.

[0095] In a specific embodiment, the specific process of obtaining the inner and outer eye corner coordinate data of the detection video is as follows:

[0096] Read the first frame of image from the detection video; scale and grayscale the read image to reduce the amount of calculation; further perform Gaussian blur processing on the image to remove noise;

[0097] Load the inner and outer corners of the eye training model to obtain the inner and outer corners of the image;

[0098] The coordinate data of the inner and outer corners of the eye in each frame image are obtained and stored, and the inner and outer corners of the eye positions are continuously identified and located for the next frame image until all frame images in the detection video are processed.

[0099] In a specific embodiment, the specific process of calculating the inter-eye angle distance in the detection video is as follows:

[0100] According to the pupil coordinate data and the inner and outer eye corner coordinate data of the detection video, the projection value of the left eye pupil center on the line connecting the inner and outer eye corners in each frame of the detection video is calculated as the left eye inner corner distance in each frame of the detection video; the projection value of the right eye pupil center on the line connecting the inner and outer eye corners in each frame of the detection video is calculated as the right eye inner corner distance in each frame of the detection video;

[0101] Based on the data of the inner angular distance of the left eye in each frame of the detection video and the inner angular distance of the right eye in each frame of the detection video, the inner angular distance of both eyes in each frame of the detection video is calculated, and the data set of the inner angular distance of both eyes for all frames in the detection video is obtained as detection data and input into the data analysis module of the control device for analysis and judgment.

[0102] S3. Detection and analysis:

[0103] According to the detection threshold obtained in step S1 and the detection data obtained in step S2, analysis and judgment are performed to obtain the detection result of eye movement abnormality.

[0104] Specifically, according to the obtained detection threshold and the obtained data stream, it is analyzed to determine whether an abnormal eye movement event occurs, analyze the frequency of abnormal eye movement events, and analyze the duration of abnormal eye movement events. The specific process is as follows:

[0105] Calculate the difference between the inter-eye angle distance of each frame of the detection video and the detection threshold, and make a judgment;

[0106] If the difference is greater than 0, it indicates that the eye position displayed in the current frame image is skewed relative to the normal state, and is marked as 0 (abnormal);

[0107] If the difference is less than or equal to 0, it indicates that the eye position displayed in the current frame image is within the normal range, which is marked as 1 (normal);

[0108] Traverse all frame images, count the number of image frames that are continuously marked as 0 (abnormal), that is, the number of consecutive abnormal frames, and make a judgment;

[0109] If the number of consecutive abnormal frames is greater than 30, it is considered that an abnormal eye movement event has occurred;

[0110] If the number of consecutive abnormal frames is less than or equal to 30, it is determined that no abnormal eye movement event has occurred;

[0111] Record the start frame and end frame of each abnormal eye movement event, as well as the corresponding timestamp; count the number of abnormal eye movement events during the detection video recording period, and calculate their frequency;

[0112] The duration of each abnormal eye movement event is calculated based on the start timestamp and end timestamp of each abnormal eye movement event.

[0113] Specifically, the value of the number of abnormal frames used for judgment was selected based on clinical experience and statistical analysis, aiming to distinguish transient eye deviations from persistent strabismus states.

[0114] In a specific embodiment, taking a detected subject as an example, the eye movement abnormality detection method of this solution is as follows:

[0115] The left-eye video data and the right-eye video data are synchronously collected for 10 seconds; the left-eye and right-eye video data are merged into a calibration video.

[0116] Read images frame by frame from the calibration video, and perform preprocessing and feature extraction on each frame. Use connected domain analysis technology and ellipse fitting method to identify the pupil position, and use the trained eye inner and outer corner recognition model to identify the inner and outer eye corner positions. Calculate the inner corner distance of the left eye and the inner corner distance of the right eye for each frame, and calculate the average of the inner corner distance of both eyes, and multiply it by the correction coefficient sens to get the threshold. Here, the average of the inner corner distance of both eyes is 1.10, and the sense defaults to 1.07, so the detection threshold is 1.177;

[0117] Collect left-eye video and right-eye video data for 8 hours, and merge the left-eye and right-eye video data into a detection video.

[0118] Read images frame by frame from the detection video, perform preprocessing and feature extraction on each frame. Use connected domain analysis technology and ellipse fitting method to identify pupil position, and use the trained eye inner and outer corner recognition model to identify the inner and outer corner positions. Store the inner and outer corner distances of the left eye and right eye for each frame.

[0119] Calculate the difference between the inter-eye angle distance value of each frame and the threshold, and mark it. Count the number of consecutive abnormal frames to determine the occurrence of abnormal eye movement events. Record the start frame and end frame of each event, as well as the corresponding timestamp. Calculate the frequency and duration of abnormal eye movement events. Here, the occurrence of abnormal eye movement events was detected, which lasted 1 time and lasted for 34 seconds. Figure 7 , which is a curve diagram of the inner eye angle distance of both eyes during the detection process.

[0120] This solution collects data and detects eye movement abnormalities. Different from the current manual detection, this solution can meet the needs of all age groups and is applicable to daily home monitoring, which makes up for the need for home control ability scoring in the Newcastle eye control ability scoring standard; this solution can accurately quantify eye movement through video or image analysis, making the test result data more accurate and reliable. In addition, the data of eye movement abnormality events obtained by the eye movement abnormality detection method of this solution can be used as a basis for judging the occurrence and frequency of intermittent exotropia.

[0121] In a specific embodiment, an eye movement abnormality detection system includes a detection threshold acquisition module: acquiring a calibration video, and obtaining a detection threshold by processing the calibration video;

[0122] Acquisition test data module: collects test videos and obtains test data by processing the test videos;

[0123] Detection and analysis module: obtains eye movement abnormality detection results by processing and analyzing detection data and detection thresholds.

[0124] The device designed for abnormal eye movement detection may be a hardware device, including one or more processors and a memory, wherein the memory stores a program and is configured such that the one or more processors perform the following steps:

[0125] Collect calibration video and obtain detection threshold by processing the calibration video;

[0126] Collect detection videos and obtain detection data by processing the detection videos;

[0127] By processing and analyzing the detection data and the detection threshold, the eye movement abnormality detection result is obtained.

[0128] The above schematically describes the invention and its implementation methods, which is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this, and any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and an embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of this patent. In addition, the word "including" does not exclude other elements or steps, and the word "one" before the element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A method for detecting abnormal eye movement, comprising the following steps: Collect calibration video and obtain detection threshold by processing the calibration video; Collect detection videos and obtain detection data by processing the detection videos; By processing and analyzing the detection data and the detection threshold, the eye movement abnormality detection result is obtained.

2. The method for detecting abnormal eye movement according to claim 1, characterized in that: The detection data includes detecting the inner angle distance of both eyes in all frame images of the video.

3. The method for detecting abnormal eye movement according to claim 1, characterized in that: The process of acquiring the calibration video is specifically as follows: synchronously acquiring the first left-eye video data and the first right-eye video data; and using video fusion to merge the first left-eye video data and the first right-eye video data into the calibration video.

4. The method for detecting abnormal eye movement according to claim 3, characterized in that: The duration of capturing the calibration video is greater than or equal to 10 seconds; the clarity of the calibration video is consistent with the clarity of the first left-eye video data and the first right-eye video data.

5. The method for detecting abnormal eye movement according to claim 1, characterized in that: The process of processing the calibration video is specifically as follows: identifying and locating the pupil position and the inner and outer eye corner positions of the eye in the calibration video, and obtaining the pupil coordinate data and the inner and outer eye corner coordinate data of the calibration video; Calculate the inner angle distance of the left eye, the inner angle distance of the right eye, and the inner angle distance of both eyes in the calibration video according to the pupil coordinate data and the inner and outer eye corner coordinate data of the calibration video; The detection threshold is calculated based on the interocular angle distance of the calibration video.

6. The method for detecting abnormal eye movement according to claim 5, characterized in that: The process of identifying and locating the pupil position of the eye in the calibration video is specifically as follows: reading images frame by frame from the calibration video and preprocessing each frame of the image; Use connected domains to find pupil contours in the image and filter connected domains that meet the conditions; Perform ellipse fitting on the selected connected domain to obtain the position and shape of the pupil; Through template matching, the pupil coordinates of each frame in the calibration video are obtained and stored.

7. The method for detecting abnormal eye movement according to claim 5, characterized in that: The process of identifying and locating the inner and outer corners of the eye in the calibration video is specifically as follows: reading images frame by frame from the calibration video and scaling and graying each frame of the image; Use Gaussian blur to remove noise from the image; The trained eye inside and outside corner recognition model is used to extract and classify the image features, and the inside and outside eye corner coordinates of each frame in the calibration video are obtained and stored.

8. The method for detecting abnormal eye movement according to claim 5, characterized in that: The process of calculating the inner angle distance of the left eye and the inner angle distance of the right eye in the calibration video is as follows: According to the pupil coordinates and inner and outer eye corner coordinates obtained in the calibration video, the projection value of the left eye pupil center on the line connecting the inner and outer eye corners in each frame image in the calibration video is calculated as the left eye inner corner distance; Calculate the projection value of the right eye pupil center on the line connecting the inner and outer eye corners in each frame of the calibration video as the right eye inner corner distance.

9. The method for detecting abnormal eye movement according to claim 1, characterized in that: The specific process of analysis and judgment is as follows: according to the detection data of all frame images in the detection video, the difference between the detection data of each frame image in the detection video and the detection threshold is calculated and judged; If the difference is greater than 0, it is marked as an anomaly; If the difference is less than or equal to 0, it is marked as normal; Traverse all the frames in the detection video and count the number of consecutive image frames marked as abnormal; If the number of consecutive image frames marked as abnormal is greater than 30, it is determined that an abnormal eye movement event has occurred; If the number of image frames continuously marked as abnormal is less than or equal to 30, it is determined that no abnormal eye movement event has occurred.

10. An eye movement abnormality detection system, characterized in that: include: Detection threshold acquisition module: collects calibration video and obtains detection threshold by processing the calibration video; Acquisition test data module: collects test videos and obtains test data by processing the test videos; Detection and analysis module: obtains eye movement abnormality detection results by processing and analyzing detection data and detection thresholds.

Images