Abnormal refraction state recognition method and device based on eye abnormal posture

Abstract

The application discloses an abnormal refraction state recognition method and device based on an eye abnormal posture, and the method comprises the following steps: acquiring video data corresponding to a vision test process of a subject, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device; performing image frame processing on the video data, determining a face region in each frame of image, and performing face feature point extraction on the face region to determine eye socket feature points and eyebrow feature points in the face feature points; recognizing an eye abnormal posture based on the eye socket feature points and the eyebrow feature points, and determining an image frame corresponding to the eye abnormal posture, so as to determine vision test data according to the image frame, wherein the vision test data comprises the shape of a target, the distance between the subject and the target, and the blurring degree of the target. The application can accurately judge whether the subject has an abnormal refraction state by extracting the eye abnormal posture.

Description

Technical Field

[0001] The present invention relates to the technical field of eye abnormality analysis, and in particular to a method and device for identifying abnormal refractive states based on abnormal eye postures. Background Art

[0002] In recent years, the incidence of physiological refractive errors, such as myopia and astigmatism, has been increasing year by year, while the age of onset has also been decreasing. According to statistics, the myopia rate among adolescents has reached 52.7%. Furthermore, if these refractive errors are not promptly corrected, they can exacerbate poor eye habits and worsen the degree of myopia. In severe cases, they can lead to pathological myopia, amblyopia, or eye diseases, and even blindness, significantly impacting daily life. Furthermore, according to relevant research, the decreased vision caused by physiological refractive errors often leads to abnormal eye behaviors such as squinting, squinting, tilting the head, and frowning when focusing on an object, in an attempt to temporarily improve vision and see more clearly.

[0003] Existing automated methods for identifying abnormal refractive states of the eye primarily rely on images of the human eye's appearance, using simple artificial intelligence methods based on deep learning neural networks to classify the images. Because people with physiological refractive errors not only differ in appearance from normal eyes, they can also exhibit abnormal eye behavior when fixating on a specific target, as the target moves or changes in size. Some abnormal eye movements, such as squinting and gaze shift, are dynamic processes, and the random nature of a single image can reduce accuracy due to the inability to capture these dynamic processes. Furthermore, existing methods that require subjects to perform tasks such as identifying the direction of letters and shapes on a mobile device can be considered self-assessments of vision using an electronic eye chart. These tests are inherently subject to subjective influences, easily leading to unconscious abnormal eye behavior and deviations from actual results. Therefore, analyzing refractive state based on existing vision testing procedures is inaccurate.

[0004] Therefore, the existing technology still needs to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that, in response to the above-mentioned defects of the prior art, a method and device for identifying abnormal refractive states based on abnormal eye posture is provided, aiming to solve the problem that the analysis of the refractive state of the eye in the existing vision testing process in the prior art is inaccurate.

[0006] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0007] In a first aspect, the present invention provides a method for identifying an abnormal refractive state based on abnormal eye posture, wherein the method comprises:

[0008] Obtaining video data corresponding to a subject's vision test process, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device;

[0009] Performing image frame processing on the video data to determine a face area in each frame image, and extracting facial feature points from the face area to determine orbital feature points and eyebrow feature points among the facial feature points;

[0010] Based on the orbital feature points and the eyebrow feature points, abnormal eye posture is identified, and the image frame corresponding to the abnormal eye posture is determined, so as to determine vision test data based on the image frame, and the vision test data includes the shape of the target, the distance between the subject and the target, and the blur degree of the target.

[0011] In one implementation, obtaining video data corresponding to the subject's vision test process includes:

[0012] adjusting the height of a preset image acquisition device to be parallel to the height of the subject's head, wherein the image acquisition device is located above the target changing device;

[0013] controlling the target change device to play the target change video, wherein the target change video reflects the change in size of the target, the change in the distance between the target and the subject, and the change in the blur level of the target;

[0014] Collect video data corresponding to the subject's vision test based on the target change video.

[0015] In one implementation, determining the face region in each frame of image includes:

[0016] Control a sub-window to shift and slide in each frame of the image, and record the regional features corresponding to each position the sub-window moves to;

[0017] The regional features are screened using a Haar cascade face classifier to obtain the face region.

[0018] In one implementation, extracting facial feature points from the face region and determining eye socket feature points and eyebrow feature points among the facial feature points includes:

[0019] Using the dlib face recognition library, 68 feature points of the face area are detected;

[0020] Based on the 68 feature points, the eye socket feature points and the eyebrow feature points are screened out.

[0021] In one implementation, the identifying abnormal eye posture based on the eye socket feature points and the eyebrow feature points includes:

[0022] Obtaining coordinate data corresponding to the eye socket feature points and the eyebrow feature points respectively, and segmenting the human eye area;

[0023] retaining the feature points of the left and right eye sockets from the human eye region, obtaining the coordinate points of the upper and lower eyelids of the left and right eye sockets, respectively, and calculating the change in the distance between the upper and lower eyelids of the two eyes;

[0024] retaining the coordinates of the eyebrow feature points from the human eye region and obtaining the coordinate changes of the eyebrow feature points;

[0025] The abnormal eye posture is determined based on the change in the distance between the upper and lower eyelids of the two eyes and the change in the coordinates of the eyebrow feature points.

[0026] In one implementation, the identifying abnormal eye posture based on the eye socket feature points and the eyebrow feature points further includes:

[0027] Based on the 68 feature points, feature points of the facial edge are screened out, and the facial size is calculated based on the feature points of the facial edge;

[0028] Based on the facial dimensions, a change in distance between the human face and the image acquisition device is determined.

[0029] In one implementation, the identifying abnormal eye posture based on the eye socket feature points and the eyebrow feature points further includes:

[0030] Based on the 68 feature points, feature points of the facial edge are screened out, and based on the feature points of the facial edge, a position change of the nose tip relative to the entire face is determined;

[0031] Based on the change in position of the nose tip relative to the entire face, a head posture change is determined.

[0032] In a second aspect, an embodiment of the present invention further provides a device for identifying abnormal refractive states based on abnormal eye posture, wherein the device comprises:

[0033] a video data acquisition module, configured to acquire video data corresponding to a subject's vision test process, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device;

[0034] An eye feature extraction module is used to perform image frame processing on the video data, determine the face area in each frame image, and extract facial feature points from the face area to determine the eye socket feature points and eyebrow feature points among the facial feature points;

[0035] An abnormal posture recognition module is used to identify abnormal eye postures based on the eye socket feature points and the eyebrow feature points, and determine the image frame corresponding to the abnormal eye posture, so as to determine vision test data based on the image frame, and the vision test data includes the shape of the target, the distance between the subject and the target, and the degree of blur of the target.

[0036] In a third aspect, an embodiment of the present invention further provides a terminal device, which includes a memory, a processor, and a program for recognizing abnormal eye postures stored in the memory and runnable on the processor. When the processor executes the program for recognizing abnormal eye postures, it implements the steps of the method for recognizing abnormal eye postures as described in any one of the above-mentioned schemes.

[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a program for recognizing abnormal eye postures is stored. When the program for recognizing abnormal eye postures is executed by a processor, the steps of the method for recognizing abnormal eye postures as in any one of the above-mentioned schemes are implemented.

[0038] Beneficial effects: Compared with the prior art, the present invention provides a method for identifying abnormal refractive states based on abnormal eye postures. The present invention first obtains video data corresponding to a subject's vision test process, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device. Then, the video data is subjected to image frame processing to determine the facial area in each frame image, and facial feature points are extracted from the facial area to determine the orbital feature points and eyebrow feature points among the facial feature points. Finally, based on the orbital feature points and the eyebrow feature points, the abnormal eye posture is identified, and the image frame corresponding to the abnormal eye posture is determined to determine the vision test data based on the image frame. The vision test data includes the shape of the target, the distance between the subject and the target, and the blur level of the target. The present invention performs a vision test based on the target change video played by the target change device, induces abnormal eye postures in human eyes with refractive errors, and accurately extracts abnormal eye postures based on the video data, which is conducive to better dynamic analysis of the refractive state of the eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flowchart of a specific implementation of the method for identifying abnormal refractive states based on abnormal eye posture provided by an embodiment of the present invention.

[0040] Figure 2 A schematic diagram of an application scenario for performing vision testing in the abnormal refractive state recognition method based on abnormal eye posture provided in an embodiment of the present invention.

[0041] Figure 3 Schematic diagram of 68 facial feature points obtained using the dlib face recognition library in the abnormal refractive state recognition method based on abnormal eye posture provided by an embodiment of the present invention.

[0042] Figure 4 This is a schematic diagram of segmenting the human eye area in the abnormal refractive state recognition method based on abnormal eye posture provided by an embodiment of the present invention.

[0043] Figure 5 This is a functional block diagram of a device for identifying abnormal eye postures provided by an embodiment of the present invention.

[0044] Figure 6 This is a functional block diagram of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] Currently, mainstream vision testing methods include the E-shaped eye chart and optometry. The eye chart uses a question-and-answer format, requiring participants to identify the orientation of letters of varying sizes. During this process, participants may adopt unusual eye postures, such as squinting or frowning, to better see the target. Frequently engaging in these unusual eye postures, which temporarily improve vision, is a poor eye habit that can alter eye structure and lead to worsening vision. Therefore, identifying unusual eye postures is crucial.

[0047] To this end, this embodiment provides a method for identifying abnormal refractive states based on abnormal eye posture. By continuously changing the target morphology, this method can more precisely identify vision conditions compared to discrete eye charts. Based on this method, the present invention can accurately extract abnormal eye postures, facilitating better dynamic analysis of eye refractive states. In specific implementation, this embodiment first obtains video data corresponding to a subject's vision test process, where the subject undergoes a vision test based on a target change video played by a preset target change device. Then, the video data is subjected to image frame processing to determine the facial region within each frame. Facial feature points are extracted from the facial region, and orbital and eyebrow feature points are determined from the facial feature points. Finally, based on the orbital and eyebrow feature points, the abnormal eye posture is identified, and the image frame corresponding to the abnormal eye posture is determined. Vision test data is then determined based on the image frame. The vision test data includes the target morphology, the distance between the subject and the target, and the blur level of the target. As can be seen, this embodiment, which performs vision testing based on a target-changing video played by a target-changing device, is advantageous for inducing abnormal eye postures in individuals with refractive errors, thereby identifying abnormal eye postures based on facial images. Furthermore, compared to existing methods based on discrete changes in the scaled-down letters of an eye chart, this embodiment allows for continuous and accurate analysis of the subject's vision test data.

[0048] Exemplary Methods

[0049] Based on the above embodiments, the abnormal refractive state recognition method based on abnormal eye posture of the present invention can be applied to terminal devices, which can be intelligent product terminals such as computers and mobile phones. Specifically, Figure 1 As shown in , the method includes:

[0050] Step S100: obtaining video data corresponding to a vision test process of a subject, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device.

[0051] In this embodiment, a new vision testing method is provided, which includes a pre-set target change device. This target change device may include a flat panel display screen that can be used to play and display a target change video. In this embodiment, the target change video reflects the changes in the target's shape, the changes in the distance between the target and the subject, and the changes in the target's blur. For example, the target can take different shapes of the "E", and the subject can perform a vision test based on the target change video. In other words, the subject's vision test in this embodiment is based on a dynamically changing target. Compared with traditional tests that use a fixed target, this facilitates obtaining more accurate vision test data.

[0052] In one implementation, this embodiment includes the following steps when acquiring video data:

[0053] Step S101: adjusting the height of a preset image acquisition device to be parallel to the height of the subject's head, wherein the image acquisition device is located above the target changing device;

[0054] Step S102: Control the target change device to play the target change video, where the target change video reflects the size change of the target, the distance change between the target and the subject, and the blur level change of the target;

[0055] Step S103: collecting video data corresponding to the subject's vision test based on the target change video.

[0056] Specifically, in this embodiment, an image acquisition device is provided above the target change device. The image acquisition device may be a camera. The camera is used to acquire video images during the subject's vision test to obtain video data of the subject's vision test. In order to conveniently acquire a complete facial image of the subject, in this embodiment, the camera needs to be pointed in the direction of the subject, such as Figure 2As shown in , and its height is adjusted to be parallel to the subject's head height. When the subject is taking the vision test, the target change device can be controlled to play the target change video. The target change video reflects the target's morphological changes (such as changes in size), the change in distance between the target and the subject, and the change in the blur of the target. Therefore, the subject can make corresponding judgments based on the target seen, and then the image acquisition device can collect video data of the subject when taking the vision test. In addition, when taking the vision test, this embodiment can set the subject to take the vision test at test points set at 1 meter, 3 meters, and 5 meters away from the target change device. In addition, the subject needs to adjust his sitting posture at each test point to ensure that the camera can capture the entire face. Before the vision test starts at each test point, the subject is required to close his eyes for 5 seconds, and then the target change video starts to play. The subject is required to look at the target with naked eyes throughout the process. After completing the vision test at each test point, the subject is required to move to the next test point and repeat the above steps until the vision test at all test points is completed. Video data will be collected during the entire vision test process.

[0057] Step S200: performing image frame processing on the video data, determining the face area in each frame image, and extracting facial feature points from the face area to determine the eye socket feature points and eyebrow feature points among the facial feature points.

[0058] After obtaining the video data, the terminal device in this embodiment performs image framing processing on the video data to obtain a plurality of frames of images. The terminal device then recognizes each frame of images and determines the facial region in each frame of images. The terminal device then extracts facial feature points from the extracted facial region to determine the eye socket feature points and eyebrow feature points.

[0059] In one implementation, step S200 includes the following steps:

[0060] Step S201: Control a sub-window to shift and slide in each frame of the image, and record the regional features corresponding to each position of the sub-window;

[0061] Step S202: Use a Haar cascade face classifier to screen the region features to obtain the face region.

[0062] Step S203: Detect 68 feature points of the face area using the dlib face recognition library;

[0063] Step S204: Filter out the eye socket feature points and the eyebrow feature points based on the 68 feature points.

[0064] Specifically, this embodiment utilizes the Haar cascade face classifier of the OpenCV library to identify the facial region of each frame. OpenCV is a commonly used open source toolkit in the field of computer vision, which implements the Haar cascade face classifier through pattern matching technology. This embodiment can utilize a subwindow to continuously shift and slide within the image. Each time the subwindow reaches a position, it can obtain the regional features corresponding to that region. The cascade classifier is used to filter the regional features. Once the regional features pass the screening of all classifiers, the region is determined to be a facial region. This embodiment can also reduce all images proportionally to speed up the calculation.

[0065] After the face area is determined, this embodiment can extract the facial feature points of the face area. For example, the dlib face recognition library is used. dlib is a C++ toolkit that includes machine learning algorithms and can detect 68 feature points of the face, such as Figure 3 As shown in . Based on the 68 detected feature points, the eye socket feature points and eyebrow feature points can be filtered out.

[0066] Step S300: Based on the orbital feature points and the eyebrow feature points, identify abnormal eye postures, and determine image frames corresponding to the abnormal eye postures, so as to determine vision test data based on the image frames, wherein the vision test data includes the shape of the target, the distance between the subject and the target, and the blurriness of the target.

[0067] After screening and obtaining the orbital feature points and the eyebrow feature points, this embodiment obtains the human eye region. Therefore, this embodiment can identify abnormal eye postures based on the human eye region and then determine the image frame corresponding to the abnormal eye posture. Because subjects may use abnormal eye postures such as squinting and frowning to see the target clearly during the vision test, after obtaining the image frame corresponding to the abnormal eye posture, the vision test data corresponding to the image frame corresponding to the abnormal eye posture can be further determined.

[0068] In one implementation, step S300 includes the following steps:

[0069] Step S301: obtaining coordinate data corresponding to the eye socket feature points and the eyebrow feature points respectively, and segmenting the eye area;

[0070] Step S302: retaining the feature points of the left and right eye sockets from the eye region, obtaining the coordinate points of the upper and lower eyelids of the left and right eye sockets, respectively, and calculating the change in the distance between the upper and lower eyelids of the two eyes;

[0071] Step S303: retaining the coordinates of the eyebrow feature points in the eye region and obtaining the coordinate changes of the eyebrow feature points;

[0072] Step S304: Determine the abnormal eye posture based on the change in the distance between the upper and lower eyelids of the two eyes and the change in the coordinates of the eyebrow feature points.

[0073] Specifically, if Figure 4 As shown in , this embodiment obtains the coordinate data corresponding to the eye socket feature points and the eyebrow feature points respectively, and segments the human eye region. In addition, this embodiment can retain the feature points of the left and right eye sockets from the human eye region, obtain the coordinate points of the upper and lower eyelids of the left and right eye sockets respectively, and calculate the distance between the upper and lower eyelids of the two eyes and the change in the distance between the upper and lower eyelids. Similarly, this embodiment retains the coordinates of the eyebrow feature points from the human eye region and obtains the coordinate change of the eyebrow feature points; based on the change in the distance between the upper and lower eyelids of the two eyes and the coordinate change of the eyebrow feature points, the abnormal eye posture is determined, for example, whether an action such as frowning occurs. In specific applications, this embodiment can use a deep learning method to input the pre-processed segmented human eye region into a video classification neural network based on a three-dimensional convolution kernel. A variety of deep neural network models that meet the input and output conditions are applicable to this embodiment, such as the more commonly used 3DResNet-34, 3D ResNet-50, 3D ResNet-101 networks, and then automatically identify abnormal eye postures. After obtaining the abnormal eye posture, this embodiment can determine the image frame corresponding to the abnormal eye posture. Since the image frame corresponding to the abnormal eye posture is determined, the distance between the subject and the target, the shape of the target and the degree of blur of the target when the image frame is captured can be obtained, that is, the vision test data corresponding to the image frame can be obtained. Based on these vision test data, the subject's vision condition can be determined.

[0074] Because the subject's head posture changes during the vision test, this embodiment can also determine head posture changes. Specifically, this embodiment screens the facial edge feature points based on 68 feature points and determines the position change of the nose tip relative to the entire face based on these facial edge feature points. Then, based on the position change of the nose tip relative to the entire face, head posture changes are determined.

[0075] It can be seen from this that this embodiment can induce abnormal eye postures in human eyes with refractive errors by changing the distance between the target and the human eye, the shape of the target (such as size), the blurriness of the target, etc., and then determine the subject's vision by identifying the abnormal eye posture, and the vision value can be detected more accurately. This embodiment allows the shape of the target to gradually change while the subject is gazing at the target, and uses parameters such as the distance, size, and clarity between the target and the human eye when the abnormal eye posture occurs as a basis to define whether the subject has vision loss due to refractive error. Different from the more mainstream vision detection methods nowadays, the embodiment does not require the aid of expensive computer optometry instruments, nor does it require experienced ophthalmic medical personnel. It only determines the subject's vision by identifying the abnormal eye posture when the subject's face itself spontaneously changes due to refractive error when gazing at the target with the naked eye, in order to see the target clearly. While adjusting the distance and size of the target, this embodiment collects facial videos of the subject looking at the target. With the help of automated methods such as traditional image processing and neural networks, the appearance features of the human eye structure, movements, etc. during the process of looking at the target are extracted and analyzed. This can help identify the occurrence of these abnormal eye postures and better perform dynamic analysis of the eye's refractive state to improve the recognition accuracy.

[0076] Exemplary devices

[0077] Based on the above embodiments, the present invention also provides an abnormal refractive state recognition device based on abnormal eye posture, such as Figure 5 As shown in , the device includes: a video data acquisition module 10, an eye feature extraction module 20, and an abnormal posture recognition module 30. Specifically, the video data acquisition module 10 is used to obtain video data corresponding to the subject's vision test process, and the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device. The eye feature extraction module 20 is used to perform image frame processing on the video data, determine the face area in each frame image, and extract facial feature points from the face area to determine the orbital feature points and eyebrow feature points among the facial feature points. The abnormal posture recognition module 30 is used to identify abnormal eye postures based on the orbital feature points and the eyebrow feature points, and determine the image frame corresponding to the abnormal eye posture, so as to determine the vision test data based on the image frame, and the vision test data includes the shape and size of the target, the distance between the subject and the target, and the blur level of the target.

[0078] In one implementation, the video data acquisition module 10 includes:

[0079] a device adjustment unit, configured to adjust the height of a preset image acquisition device to be parallel to the height of the subject's head, the image acquisition device being located above the target changing device;

[0080] a video playback unit, configured to control the target change device to play the target change video, wherein the target change video reflects the change in size of the target, the change in the distance between the target and the subject, and the change in the blur level of the target;

[0081] The video acquisition unit is used to acquire video data corresponding to the subject's vision test based on the target change video.

[0082] In one implementation, the eye feature extraction module 20 includes:

[0083] A region feature recording unit, configured to control a sub-window to shift and slide in each frame of the image, and to record the region feature corresponding to each position to which the sub-window moves;

[0084] The face region screening unit is used to screen the region features using a Haar cascade face classifier to obtain the face region.

[0085] In one implementation, the eye feature extraction module 20 further includes:

[0086] A feature recognition unit, configured to detect 68 feature points of the face region using the dlib face recognition library;

[0087] A feature screening unit is used to screen out the eye socket feature points and the eyebrow feature points based on the 68 feature points.

[0088] In one implementation, the abnormal posture recognition module 30 includes:

[0089] The eye region segmentation unit is used to obtain the coordinate data corresponding to the eye socket feature points and the eyebrow feature points respectively, and segment the eye region;

[0090] an eyelid distance calculation unit, configured to retain the feature points of the left and right eye sockets from the eye region, obtain the coordinate points of the upper and lower eyelids of the left and right eye sockets, respectively, and calculate the upper and lower eyelid distance between the two eyes;

[0091] a coordinate change determining unit, configured to retain the coordinates of the eyebrow feature points from the human eye region and obtain the coordinate changes of the eyebrow feature points;

[0092] The abnormal posture determining unit is used to determine the abnormal eye posture based on the distance between the upper and lower eyelids of the two eyes and the coordinate change of the eyebrow feature point.

[0093] In one implementation, the abnormal posture recognition module 30 further includes:

[0094] a facial size determination unit, configured to screen out feature points of a facial edge based on the 68 feature points, and calculate the facial size based on the feature points of the facial edge;

[0095] The distance change determining unit is used to determine the distance change between the human face and the image acquisition device based on the facial size.

[0096] In one implementation, the abnormal posture recognition module 30 further includes:

[0097] a position change determining unit, configured to filter out feature points of a facial edge based on the 68 feature points, and determine a position change of the nose tip relative to the entire face based on the feature points of the facial edge;

[0098] The head posture change determining unit is used to determine the head posture change based on the position change of the nose tip relative to the entire face.

[0099] The working principles of each module in the abnormal eye posture recognition system of this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0100] Based on the above embodiment, the present invention further provides a terminal device, the principle block diagram of the terminal device can be as follows: Figure 6 The terminal device may include one or more processors 100 ( Figure 6 Only one is shown in the figure), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100, for example, a program for identifying abnormal eye postures. When one or more processors 100 execute computer program 102, each step of the method embodiment for identifying abnormal eye postures can be implemented. Alternatively, when one or more processors 100 execute computer program 102, the functions of each module / unit in the apparatus embodiment for identifying abnormal eye postures can be implemented, which is not limited here.

[0101] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0102] In one embodiment, the memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal device. The memory 101 may also be used to temporarily store data that has been output or is about to be output.

[0103] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal device to which the solution of the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0104] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operating database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0105] In summary, the present invention discloses a method and device for identifying abnormal refractive states based on abnormal eye postures. The method includes: obtaining video data corresponding to a subject's vision test process, where the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device; performing image frame processing on the video data, determining the face area in each frame image, and extracting facial feature points from the face area, determining the orbital feature points and eyebrow feature points among the facial feature points; based on the orbital feature points and eyebrow feature points, identifying abnormal eye postures, and determining the image frames corresponding to the abnormal eye postures, so as to determine vision test data based on the image frames, wherein the vision test data includes the shape of the target, the distance between the subject and the target, and the blur degree of the target. The present invention can accurately extract abnormal eye postures, which is conducive to better dynamic analysis of eye abnormalities.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for identifying abnormal refractive state based on abnormal eye posture, characterized in that: The method comprises: Obtaining video data corresponding to a subject's vision test process, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device; Performing image frame processing on the video data to determine a face area in each frame image, and extracting facial feature points from the face area to determine orbital feature points and eyebrow feature points among the facial feature points; Based on the orbital feature points and the eyebrow feature points, identifying abnormal eye postures, and determining image frames corresponding to the abnormal eye postures, so as to determine vision test data based on the image frames, and then determine whether the subject has an abnormal refractive state, the vision test data including the shape of the target, the distance between the subject and the target, and the blur level of the target; The identifying of abnormal eye posture based on the eye socket feature points and the eyebrow feature points includes: respectively obtaining coordinate data corresponding to the eye socket feature points and the eyebrow feature points, and segmenting the human eye area; retaining the feature points of the left eye socket and the right eye socket from the human eye area, respectively obtaining the coordinate points of the upper and lower eyelids of the left eye socket and the right eye socket, and calculating the change in the distance between the upper and lower eyelids of the two eyes; retaining the coordinates of the eyebrow feature points from the human eye area and obtaining the coordinate change of the eyebrow feature points; determining the abnormal eye posture based on the change in the distance between the upper and lower eyelids of the two eyes and the coordinate change of the eyebrow feature points; or, inputting the human eye area into a video classification neural network based on a three-dimensional convolution kernel to automatically identify the abnormal eye posture.

2. The method for identifying abnormal refractive state based on abnormal eye posture according to claim 1, characterized in that: The obtaining of video data corresponding to the subject's vision test process includes: adjusting the height of a preset image acquisition device to be parallel to the height of the subject's head, wherein the image acquisition device is located above the target changing device; controlling the target change device to play the target change video, wherein the target change video reflects the change in size of the target, the change in the distance between the target and the subject, and the change in the blur level of the target; Collect video data corresponding to the subject's vision test based on the target change video.

3. The method for identifying abnormal refractive state based on abnormal eye posture according to claim 1, characterized in that: Determining the face area in each frame of image includes: Control a sub-window to shift and slide in each frame of the image, and record the regional features corresponding to each position the sub-window moves to; The regional features are screened using a Haar cascade face classifier to obtain the face region.

4. The method for identifying abnormal refractive state based on abnormal eye posture according to claim 1, characterized in that: The extracting facial feature points from the facial region to determine the eye socket feature points and eyebrow feature points among the facial feature points includes: Using the dlib face recognition library, 68 feature points of the face area are detected; Based on the 68 feature points, the eye socket feature points and the eyebrow feature points are screened out.

5. The method for identifying abnormal refractive state based on abnormal eye posture according to claim 4, characterized in that: The identifying abnormal eye posture based on the eye socket feature points and the eyebrow feature points further includes: Based on the 68 feature points, feature points of the facial edge are screened out, and the facial size is calculated based on the feature points of the facial edge; Based on the facial dimensions, a change in distance between the human face and the image acquisition device is determined.

6. The method for identifying abnormal refractive state based on abnormal eye posture according to claim 4, characterized in that: The identifying abnormal eye posture based on the eye socket feature points and the eyebrow feature points further includes: Based on the 68 feature points, feature points of the facial edge are screened out, and based on the feature points of the facial edge, a position change of the nose tip relative to the entire face is determined; Based on the change in position of the nose tip relative to the entire face, a head posture change is determined.

7. A device for identifying abnormal refractive states based on abnormal eye posture, characterized in that: The device comprises: a video data acquisition module, configured to acquire video data corresponding to a subject's vision test process, wherein the vision test process is a vision test performed by the subject based on a target change video played by a preset target change device; An eye feature extraction module is used to perform image frame processing on the video data, determine the face area in each frame image, and extract facial feature points from the face area to determine the eye socket feature points and eyebrow feature points among the facial feature points; an abnormal posture recognition module, configured to recognize abnormal eye postures based on the eye socket feature points and the eyebrow feature points, and determine image frames corresponding to the abnormal eye postures, so as to determine vision test data based on the image frames, wherein the vision test data includes the shape of a target, the distance between the subject and the target, and the blur level of the target; The abnormal posture recognition module includes an eye region segmentation unit, an eyelid distance calculation unit, a coordinate change determination unit, and an abnormal posture determination unit. The eye region segmentation unit is used to respectively obtain the coordinate data corresponding to the eye socket feature points and the eyebrow feature points, and segment the eye region; the eyelid distance calculation unit is used to retain the feature points of the left eye socket and the right eye socket from the eye region, obtain the coordinate points of the upper and lower eyelids of the left eye socket and the right eye socket, respectively, and calculate the change in the distance between the upper and lower eyelids of the two eyes; the coordinate change determination unit is used to retain the coordinates of the eyebrow feature points from the eye region and obtain the coordinate change of the eyebrow feature points; the abnormal posture determination unit is used to determine the abnormal eye posture based on the change in the distance between the upper and lower eyelids of the two eyes and the coordinate change of the eyebrow feature points; The abnormal refractive state recognition device based on abnormal eye posture is also used to input the human eye area into a video classification neural network based on a three-dimensional convolution kernel to automatically recognize abnormal eye posture.

8. A terminal device, characterized in that: The terminal device includes a memory, a processor, and an abnormal refractive state recognition program based on abnormal eye posture stored in the memory and runnable on the processor. When the processor executes the abnormal refractive state recognition program based on abnormal eye posture, the steps of the abnormal refractive state recognition method based on abnormal eye posture as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program for identifying an abnormal refractive state based on abnormal eye posture. When the program for identifying an abnormal refractive state based on abnormal eye posture is executed by a processor, the steps of the method for identifying an abnormal refractive state based on abnormal eye posture as described in any one of claims 1 to 6 are implemented.

Images