Eye image map quality control method and device, storage medium and equipment

Abstract

The invention provides an eye image image quality control method and device, a storage medium and equipment, and belongs to the technical field of image recognizing.According to the scheme, after an eye image image is obtained, the eye image image is classified to obtain a visual angle category corresponding to the eye image image, an eye image image judgment condition matched with the recognized visual angle category is obtained, then the eye image image is processed, and the visual angle of the eye image image is obtained. The method comprises the following steps: obtaining a black-and-white-eye semantic segmentation mask graph of an eye image graph, identifying to obtain area information of a black-eye area and area information of a white-eye area in the black-and-white-eye semantic segmentation mask graph, and finally analyzing the area information of the black-eye area and the area information of the white-eye area based on an eye image graph judgment condition. According to the method, whether the eye image graph is qualified or not is judged based on the analysis result, so that the judgment standard for judging whether the eye image graph is qualified or not is unified, subsequent analysis on the eye image graph is facilitated, the judgment process is rapid, the efficiency is high, the manual participation rate is low, and the problem that the quality of the eye image graph is uneven is solved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to an eye image quality control method, device, storage medium and equipment. Background Art

[0002] The theory of traditional Chinese medicine eye diagnosis believes that the sclera area on the human eyeball, that is, the white of the eye area, has corresponding areas of the human body's internal organs. The blood vessels, spots, hillocks, dots and the background color of the white of the eye can reflect the health status of the human body. Therefore, when using traditional Chinese medicine eye diagnosis to diagnose the health status of the human body, it is required to fully expose the white of the eye area. By observing the white of the eye area, the doctor can diagnose the patient's physical condition and give a corresponding diagnosis and treatment plan.

[0003] With the development of computer, optical imaging, machinery and other technologies, an intelligent analysis system that simulates the visual diagnosis of traditional Chinese medicine practitioners has been successfully developed. The system first takes eye images of the patient's left and right eyes from different angles, uses image processing and computer vision technology to extract features from the white area of ​​the eye image, and then combines the theory of traditional Chinese medicine eye diagnosis and artificial intelligence technology to analyze the features of the eye image and the white of the eye, and finally gives the diagnosis and treatment results. Among them, the quality of the eye image played a vital role in the visual diagnosis analysis. The degree of exposure of the white of the eye image directly affects the visual diagnosis system's judgment on the existence of relevant features. For example, the "yellow spot" on the white of the eye indicates that the patient has a wet image, and because the white of the eye is not exposed enough, the "yellow spot" feature is not recognized, then the visual diagnosis system will not diagnose the patient's wet image, thus affecting the accuracy of the diagnosis result; in addition, if the eye image taken is affected by the ambient light source and produces color cast, the white white background becomes blue, or the red blood becomes dark, according to the five-color theory of traditional Chinese medicine, it will inevitably have a great impact on the diagnosis result. Therefore, it is particularly important to control the quality of the eye image during the stage of collecting eye images by the visual diagnosis system.

[0004] Existing traditional Chinese medicine eye diagnosis instruments lack an automatic eye image quality control module, and the control of the eye image quality depends on the operator. Usually, an eye diagnosis instrument needs to be equipped with at least one operator, and the operator must not only be proficient in operating the eye diagnosis instrument, but also be able to judge the quality of the eye image and accurately guide the subject to take pictures in order to obtain accurate diagnosis results. On the one hand, the manual quality control of eye images is easily limited by the operator's personality and subjective judgment, resulting in uneven eye image quality; on the other hand, in the process of guiding the subject to take eye images, the operator is required to accurately express the problem and let the subject actively cooperate, which not only greatly increases the workload of the eye diagnosis instrument operator, but also reduces the subject's photo-taking experience, making the entire photo-taking process time-consuming and labor-intensive, and the qualified rate of the eye image obtained is low, and the standards are not unified. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide an eye image quality control method, apparatus, storage medium and device to unify the qualified standards of eye images and improve the recognition efficiency of eye images.

[0006] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0007] An eye image quality control method, comprising:

[0008] Get an eye image;

[0009] Classifying the eye image based on the viewing direction of the eyeball in the eye image to obtain a viewing angle category corresponding to the eye image;

[0010] Obtaining a black and white eye semantic segmentation mask image of the eye image;

[0011] Obtaining regional information of the black eye region and regional information of the white eye region in the black and white eye semantic segmentation mask image;

[0012] Obtaining eye image judgment conditions that match the viewing angle category;

[0013] Whether the eye image chart is qualified is judged based on the eye image chart judgment condition and the area information of the black eye area and the area information of the white eye area.

[0014] Optionally, in the above-mentioned eye image quality control method, after obtaining the viewing angle category corresponding to the eye image and before obtaining the black and white eye semantic segmentation mask image of the eye image, it also includes:

[0015] Determining whether the viewing angle category is a target viewing angle category;

[0016] When the viewing angle category is not the target viewing angle category, outputting prompt information matching the target viewing angle category;

[0017] When the viewing angle category is the target viewing angle category, continue to execute subsequent steps.

[0018] Optionally, in the above-mentioned eye image quality control method, the region information of the black eye region includes area information of the black eye region, and the region information of the white eye region includes area information of the white eye region; and judging whether the eye image is qualified based on the eye image judgment condition and the region information of the black eye region and the region information of the white eye region includes:

[0019] Whether the eye image chart is qualified is determined based on the area information of the black eye region and the area information of the white eye region.

[0020] Optionally, in the above-mentioned eye image quality control method, judging whether the eye image is qualified based on the area information of the black eye region and the area information of the white eye region includes:

[0021] Determining whether the area information of the black eye region is smaller than a first preset area;

[0022] Determining whether the area information of the white eye region is smaller than a second preset area;

[0023] When the area information of the black eye region is smaller than the first preset area, or the area information of the white eye region is smaller than the second preset area, it is determined that the eye image is unqualified.

[0024] Optionally, in the above-mentioned eye image quality control method, the area information of the black eye region includes the position information of the black eye region, and the area information of the white eye region includes the position information of the white eye region, and judging whether the eye image is qualified based on the eye image judgment condition and the area information of the black eye region and the area information of the white eye region includes:

[0025] Determine whether the position information of the black eye region and the position information of the white eye region meet the eye image judgment condition;

[0026] When the position information of the black eye region and the position information of the white eye region meet the eye image diagram judgment condition, the eye image diagram is determined to be qualified.

[0027] Optionally, in the above-mentioned eye image quality control method, the position information of the black eye area includes the vertex coordinates of the circumscribed rectangle of the black eye area; the position information of the white eye area includes the vertex coordinates of the circumscribed rectangle of the white eye area, and the eye image judgment condition is the constraint condition of the vertex coordinates of the circumscribed rectangles of the black eye area and the white eye area.

[0028] Optionally, in the above-mentioned eye image quality control method, when the position information of the black eye area and the position information of the white eye area meet the eye image judgment conditions, the eye image is judged to be qualified, and feedback information of the eye image is obtained based on the vertex coordinates of the circumscribed rectangle of the black eye area and the vertex coordinates of the circumscribed rectangle of the white eye area.

[0029] An eye image quality control device, comprising:

[0030] An image acquisition unit, used for acquiring an eye image;

[0031] A classification unit, configured to classify the eye image based on the viewing direction of the eyeball in the eye image, and obtain a viewing angle category corresponding to the eye image;

[0032] An image conversion unit, used for obtaining a black and white eye semantic segmentation mask image of the eye image;

[0033] An image processing unit, used for obtaining regional information of a black eye region and regional information of a white eye region in the black and white eye semantic segmentation mask image;

[0034] The judging unit is used to obtain the eye image judging condition that matches the viewing angle category; and judge whether the eye image is qualified based on the eye image judging condition and the area information of the black eye area and the area information of the white eye area.

[0035] A storage medium stores a computer program, which, when executed by a processor, implements each step of any one of the above-mentioned eye image quality control methods.

[0036] An eye image quality control device, comprising: a memory and a processor;

[0037] The memory is used to store programs;

[0038] The processor is used to execute the program to implement each step of any one of the above-mentioned eye image quality control methods.

[0039] Based on the above technical solution, the above solution provided by the embodiment of the present invention, after obtaining the eye image, first classify the eye image to obtain the viewing angle category corresponding to the eye image, obtain the eye image judgment condition matching the identified viewing angle category, and then process the eye image to obtain the black and white eye semantic segmentation mask image of the eye image, and then identify the regional information of the black eye area and the regional information of the white eye area in the black and white eye semantic segmentation mask image, and finally analyze the regional information of the black eye area and the regional information of the white eye area based on the eye image judgment condition, and judge whether the eye image is qualified based on the analysis result, so as to unify the judgment standard of whether the eye image is qualified, so as to facilitate the subsequent analysis of the eye image, and the judgment process is fast, efficient, and low in human participation rate, which solves the problem of uneven quality of eye image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 A schematic diagram of the flow of the eye image quality control method disclosed in the embodiment of the present application;

[0042] Figure 2 It is an eye image diagram of different viewing angle categories;

[0043] Figure 3 The black and white eye semantic segmentation mask map of the eye image;

[0044] Figure 4 A schematic diagram of the training process of the test model disclosed in the embodiment of the present application;

[0045] Figure 5 A schematic diagram of the categories of eye images used to train the test model;

[0046] Figure 6 A schematic flow chart of an eye image quality control method disclosed in another embodiment of the present application;

[0047] Figure 7 This is a coordinate diagram of the eye image diagram;

[0048] Figure 8 It is a structural schematic diagram of the eye image quality control device disclosed in the embodiment of the present application;

[0049] Fig. 9 This is a hardware structure diagram of the eye image quality control device disclosed in the embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] The present invention simulates the way of manually controlling eye image, classifies the process of manual quality control of eye image into the image classification problem in computer vision, and classifies the quality of eye image into five categories through observation, induction and summary of a large number of eye image, namely acceptable, eye image color cast, background image without eyeball area, and closed eye image, wherein the "acceptable" category is the preliminary qualified category, and other categories of eye images are unqualified. However, the eye image quality control method based on image classification is only a qualitative judgment of the quality of the eye image, and it is impossible to identify whether the eye image angle or eye image posture is qualified, so it is impossible to guide the eye diagnosis operator or the subject to accurately adjust the photo posture to obtain a high-quality eye image that is conducive to later analysis, and at the same time shorten the photo shooting time and improve the subject's photo shooting experience. To this end, the present invention further proposes a quantitative eye image quality control method based on image classification and segmentation, which uses the geometric information and relative position relationship of the black and white eyes to further judge the quality of the eye image from a quantitative perspective, and provide accurate photo feedback, thereby improving the quality of the eye image.

[0052] See also Figure 1 The eye image quality control method disclosed in the embodiment of the present application may include:

[0053] Step S101: Obtain an eye image.

[0054] The eye image diagram may be an eye image diagram of a subject obtained by using an eye diagnostic apparatus or other methods, including five pictures of left and right eyes, five pictures of upward, downward, left and right vision, a total of ten eye image diagrams, and the obtained eye image diagram may be as follows: Figure 2 shown.

[0055] Step S102: classifying the eye image based on the viewing direction of the eyeball in the eye image to obtain the viewing angle category corresponding to the eye image.

[0056] In this step, the eye image diagram obtained in step S101 is respectively input into the classification model, and the classification model classifies the eye image diagram based on the direction in which the eyeball is looking in the eye image diagram. For example, when the direction indicated by the eyeball is straight ahead, the eye image diagram is a frontal view image; if the direction indicated by the eyeball is to the left, the eye image diagram is a left-view image; if the direction indicated by the eyeball is to the right, the eye image diagram is a right-view image; if the direction indicated by the eyeball is upward, the eye image diagram is an upward view image; if the direction indicated by the eyeball is downward, the eye image diagram is a downward view image.

[0057] Step S103: Obtaining a black and white eye semantic segmentation mask image of the eye image.

[0058] In this step, after the eye image is classified, an image processing tool is used to process the eye image to obtain a black and white eye semantic segmentation mask image corresponding to the eye image.

[0059] See also Figure 3 , Figure 3 The left side of the figure is the obtained eye image, and the right side is the black and white eye semantic segmentation mask image corresponding to the eye image.

[0060] Step S104: obtaining the region information of the black eye region and the region information of the white eye region in the black and white eye semantic segmentation mask image.

[0061] After obtaining the black-and-white eye semantic segmentation mask image corresponding to the eye image, the black eye area and the white eye area in the black-and-white eye semantic segmentation mask image are identified to obtain regional information of the black eye area and regional information of the white eye area in the black-and-white eye semantic segmentation mask image, wherein the regional information includes but is not limited to area information, position information, and vertex information of the corresponding circumscribed rectangle.

[0062] Step S105: obtaining eye image judgment conditions that match the viewing angle category.

[0063] In this solution, different eye image judgment conditions are pre-configured based on different viewing angle categories of the eye image. Eye image judgment conditions corresponding to eye image images of different viewing angle categories are different. The eye image judgment conditions are used to judge whether the eye image is qualified. Users can configure the specific content of the eye image judgment conditions based on experience. In this step, after the viewing angle category of the eye image is determined in step S102, the eye image judgment conditions matching the viewing angle category are obtained.

[0064] Step S106: judging whether the eye image chart is qualified based on the eye image chart judgment condition and the area information of the black eye area and the area information of the white eye area.

[0065] In this embodiment, the eye image diagram judgment condition is a condition configured based on the area information of the black eye region and the area information of the white eye region. Therefore, after the eye image diagram judgment condition and the area information of the black eye region and the area information of the white eye region are acquired, the area information of the black eye region and the area information of the white eye region are analyzed according to the eye image diagram judgment condition to judge whether the eye image diagram is qualified. For example, in this embodiment, when the area information of the black eye region and the area information of the white eye region do not meet the eye image diagram judgment condition, the eye image diagram can be judged to be unqualified, and when the area information of the black eye region and the area information of the white eye region meet the eye image diagram judgment condition, the eye image diagram can be judged to be qualified.

[0066] In the technical solution disclosed in the present embodiment, after the eye image is obtained, the eye image is first classified to obtain the viewing angle category corresponding to the eye image, and the eye image judgment condition matching the identified viewing angle category is obtained. Then, the eye image is processed to obtain the black and white eye semantic segmentation mask image of the eye image, and then the regional information of the black eye area and the regional information of the white eye area in the black and white eye semantic segmentation mask image are identified. Finally, based on the eye image judgment condition, the regional information of the black eye area and the regional information of the white eye area are analyzed, and based on the analysis result, it is judged whether the eye image is qualified, thereby unifying the judgment standard of whether the eye image is qualified, so as to facilitate the subsequent analysis of the eye image, and the judgment process is fast, efficient, and low in human participation rate, which solves the problem of uneven quality of the eye image.

[0067] In addition to using a segmentation algorithm to perform semantic segmentation on the black eye area and the white eye area on the eye image to obtain their position and geometric information, it is also possible to identify other target objects on the eye image, such as the position and geometric information of the pupil, and then design corresponding eye image judgment conditions, and use the eye image judgment conditions to judge whether the eye image is qualified based on the position and set information of the target object.

[0068] In the technical solution disclosed in this embodiment, after the eye image is obtained, before the eye image is classified based on the viewing direction of the eyeball in the eye image, a pre-trained test model can be used to preliminarily determine whether the eye image is qualified.

[0069] See also Figure 4 , the training process of the test model includes:

[0070] Step S401: Obtain a training eye image.

[0071] In this step, a large number of eye images can be randomly sampled from an open eye image database, and these eye images are recorded as training eye images.

[0072] Step S402: grouping the training eye images.

[0073] After the training eye image diagrams are obtained, based on the name identifiers of these eye image diagrams (the name identifiers can be used to identify which of the following image groups these eye image diagrams belong to), the obtained training eye image diagrams are divided into a front view group, a top view group, a bottom view group, a left view group and a right view group.

[0074] Step S403: extracting eye image features of the grouped training eye image graph;

[0075] After the training eye image images are grouped, for each group of training eye image images, a deep learning model obtained through ImageNet pre-training or other training methods can be used as a feature extractor to extract the eye image features of each training eye image image. The deep learning model can be a data analysis model such as VGG, ResNet, Inception, DenseNet, etc.

[0076] In addition to using the above-mentioned scheme to extract eye image features, you can also use traditional algorithms, that is, feature engineering methods, to first extract the features of the eye image, such as brightness, contrast, texture and other features, and then use a classifier for classification to achieve the whole image classification in the present invention.

[0077] Step S404: performing cluster analysis on the eye images in each group of training eye image images.

[0078] In this step, a clustering algorithm can be used to cluster each group of training eye image diagrams into T piles. The value of T can be set according to user needs. In this solution, the value of T can be set to = 10, that is, each group of training eye image diagrams is clustered into T piles. The clustering algorithm can select unsupervised clustering methods such as KMeans or KMeans variants. Of course, other clustering algorithms can also be selected.

[0079] Step S405: classifying the training eye image after cluster analysis based on user operations to obtain a training data set;

[0080] After clustering analysis is performed on the eye images in each group of training eye images, the eye images in each group of training eye images are manually observed, screened, summarized and concluded with reference to the eye image results clustered by the clustering algorithm, and the individual training eye images in each group of training eye images are reclassified into a number of qualified and unqualified groups. The qualified training eye images are classified into an "acceptable" category, and the unqualified eye images are classified into four categories: a "color cast" category, a "background image without an eye image area" category, a "severe occlusion" category and a "closed eye" category. The eye image diagrams of the above five categories are as follows: Figure 5 As shown. So far, the initial version of the training data set for five categories has been obtained. The reason why the training eye image diagram is clustered first and then manually grouped again is to speed up the processing speed of manual grouping.

[0081] In this embodiment, in addition to the above four unqualified categories, the output categories determined based on the eye image may also include categories such as blurred white eye area, light spots on the white eye, and eyelashes blocking the white eye.

[0082] Step S406: Use the training data set to train a deep learning classification model.

[0083] In this step, the deep learning classification model is trained using the training data set in step S405, so that the deep learning classification model can determine whether the acquired eye image belongs to the "acceptable" category, the "color cast" category, the "background image of the area without eye image" category, the "severe occlusion" category or the "closed eyes" category.

[0084] Step S407: Determine whether the analysis result of the deep learning classification model meets the expected requirements.

[0085] When the deep learning classification model meets the expected requirements, stop training the deep learning classification model.

[0086] When the deep learning classification model does not meet the expected requirements, execute step S408.

[0087] Step S408: Using the trained deep learning classification model, a training eye image is obtained by screening the open eye image database.

[0088] The training eye image diagrams include eye image diagrams of "acceptable" category, "color cast" category, "background image without eye image area" category, "severe occlusion" category and "closed eyes" category, and the number of test eye image diagrams of each category is the same.

[0089] In this step, the deep learning classification model is used to filter training eye image diagrams from the open eye image diagram database. When the screened eye image diagrams reach a preset number or the deep learning classification model cannot filter more training eye image diagrams from the open eye image diagram database, the screening of training eye image diagrams is stopped and subsequent steps are executed.

[0090] Step S409: Use the training eye image that has been reviewed and approved by the user as a training data set, and execute step S406 again.

[0091] In this step, the user reviews the training eye image images screened by the deep learning classification model from the open eye image image database, deletes the training eye image images with incorrect recognition, and uses the remaining training eye image images with correct recognition as the training data set, and executes step S406 again: using the training data set to train the deep learning classification model.

[0092] With the above steps, the training of the deep learning classification model can be realized, so that the deep learning classification model can accurately identify the category to which the acquired eye image belongs, and the categories include: "acceptable" category, "color cast" category, "background image without eye image area" category, "severe occlusion" category and "closed eyes" category, wherein the eye image with the recognition result of "acceptable" category is a qualified eye image, and the eye image with the recognition result of "color cast" category, "background image without eye image area" category, "severe occlusion" category and "closed eyes" category is an unqualified eye image.

[0093] In the technical solution disclosed in this embodiment, the above solution disclosed in this embodiment can be integrated into an eye diagnosis instrument. When the eye diagnosis instrument is used to obtain an eye image, it is necessary to collect an eye image of a certain viewing angle category. When the eye diagnosis instrument is used to obtain the eye image for the first time, the collected user may not be able to make a correct eye posture. Therefore, the eye image obtained by the eye diagnosis instrument is not an eye image of the required viewing angle category. At this time, the eye diagnosis instrument can output corresponding prompt information to the collected user. Therefore, see Figure 6 In this solution, after obtaining the viewing angle category corresponding to the eye image and before obtaining the black and white eye semantic segmentation mask image of the eye image, it also includes:

[0094] Step S501: Determine whether the viewing angle category is a target viewing angle category.

[0095] In this embodiment, the operator of the eye diagnosis instrument can independently select the target viewing angle category, or the eye diagnosis instrument can also use the emmetropia eye image chart, the upward viewing eye image chart, the downward viewing eye image chart, the left viewing eye image chart and the right viewing eye image chart in sequence or in other sequences based on a set order as the target viewing angle category.

[0096] Step S502: when the viewing angle category is not the target viewing angle category, outputting prompt information matching the target viewing angle category.

[0097] When it is determined that the collected eye image is not of the target viewing angle category, prompt information matching the target viewing angle category is output to prompt the user to adjust the eyeball angle according to the requirements of the target viewing angle category. When the viewing angle category is the target viewing angle category, the subsequent steps are continued.

[0098] In this embodiment, the black and white eye semantic segmentation mask image includes a black eye region and a white eye region, the region information of the black eye region may include the area information of the black eye region, and the region information of the white eye region may include the area information of the white eye region. At this time, judging whether the eye image image is qualified based on the eye image image judgment condition and the region information of the black eye region and the region information of the white eye region may include: judging whether the eye image image is qualified based on the area information of the black eye region and the area information of the white eye region. Specifically, judging whether the area information of the black eye region is less than a first preset area; judging whether the area information of the white eye region is less than a second preset area; when the area information of the black eye region is less than the first preset area, or the area information of the white eye region is less than the second preset area, the eye image image is judged to be unqualified. The specific values ​​of the first preset area and the second preset area can be set according to user needs. For example, the first preset area is T_iris, and the second preset area is T_sclera. When the area of ​​the black eye region is smaller than T_iris, the eye image is judged as unqualified, and a prompt message "the black eye region is too small or does not exist" can be output; when the area of ​​the white eye region is smaller than T_sclera, the eye image is judged as unqualified, and a prompt message "the white eye region is too small or does not exist" can be output. When the area of ​​the black eye region is larger than T_iris and the area of ​​the white eye region is larger than T_sclera, the eye image is judged as qualified.

[0099] Furthermore, in this embodiment, the region information of the black eye region may include the position information of the black eye region, and the region information of the white eye region may include the position information of the white eye region. In this embodiment, in addition to judging whether the eye image diagram is qualified based on the area information of the black eye region and the area information of the white eye region, it is also possible to further judge whether the eye image diagram meets the eye image diagram judgment condition based on the position information of the black eye region and the position information of the white eye region. When the position information of the black eye region and the position information of the white eye region meet the eye image diagram judgment condition, the eye image diagram is judged to be qualified. Wherein, the position information of the black eye region includes the vertex coordinates of the circumscribed rectangle of the black eye region; the position information of the white eye region includes the vertex coordinates of the circumscribed rectangle of the white eye region. At this time, the eye image diagram judgment condition includes: the constraint condition of the vertex coordinates of the circumscribed rectangles of the black eye region and the white eye region. When the vertex coordinates of the circumscribed rectangle of the white eye region including the position information of the white eye region meet the constraint condition, it indicates that the eye image diagram is qualified.

[0100] In the technical solution disclosed in this embodiment, when the posture of the eye image diagram is judged based on the vertex coordinates of the circumscribed rectangle of the black eye area and the white eye area, whether the eye image diagram is qualified is judged based on the posture of the eye image diagram, and when the eye image diagram is judged to be unqualified based on the posture of the eye image diagram, prompt information is output to the user based on the posture of the eye image diagram, so that the collected user can adjust the eyeball posture to obtain a qualified eye image diagram. For example, in this embodiment, the specific posture of the posture of the eye image diagram can be calculated based on the vertex coordinates of the circumscribed rectangle of the black eye area and the vertex coordinates of the circumscribed rectangle of the white eye area, and feedback information matching the specific posture is further output to enable the subject to adjust the user posture.

[0101] When the position information of the black eye area and the position information of the white eye area do not meet the judgment condition of the eye image diagram, the eye image diagram is qualified, and the problem type of the eye image diagram is calculated according to the vertex coordinates of the circumscribed rectangle of the black eye area and the vertex coordinates of the circumscribed rectangle of the white eye area.

[0102] For example, for an eye image chart whose area information of the black eye region and the white eye region meet the eye image chart judgment conditions, the specific posture of the eye image chart is further judged according to the vertex coordinates of the circumscribed rectangle of the black eye region and the white eye region, and whether the eye image chart is qualified is judged based on the determined posture. When it is unqualified, a judgment feedback matching the posture of the eye image chart is further generated, and the judgment is fed back to the eye diagnosis operator and the subject, prompting the collected user to make corresponding adjustments to retake the eye image chart of this link. Taking the upper view as an example, the posture of the eye image chart may include "upward vision qualified", "upward vision is not in place or the lower eyelid is not opened", "upward vision is biased to the left", "upward vision is biased to the right", "left vision is qualified", "left vision is biased upward", "left vision is biased downward", "upper and lower eyelids are not opened", etc.

[0103] In this embodiment, the coordinates of the upper left corner of the circumscribed rectangle of the black eye region are marked as (iris_xmin, iris_ymin), the coordinates of the lower right corner are marked as (iris_xmax, iris_ymax), and the height and width are recorded as (iris_h, iris_w), wherein iris_h=iris_ymax-iris_ymin, iris_w=iris_xmax-iris_xmin; the coordinates of the upper left corner of the circumscribed rectangle of the white eye are (sclera_xmin, sclera_ymin), the coordinates of the lower right corner are (sclera_xmax, sclera_ymax), and the height and width are recorded as (sclera_h, iris_w), wherein sclera_h=sclera_ymax-sclera_ymin, sclera_w=sclera_xmax-sclera_xmin, and the coordinate axes of the eye image diagram are shown in FIG. Figure 7As shown in the figure. Taking the "left view" as an example of the eye image perspective, if the distance between the rightmost side of the circumscribed rectangle of the black eye region and the leftmost side of the circumscribed rectangle of the white eye region is greater than the preset value T_relative_left, that is, sclera_xmax - iris_xmax > T_relative_left_xmax, and iris_ymin - sclera_ymin > T_relative_left_ymin, sclera_ymax - iris_ymax > T_relative_left_ymax, then the "left view" image is judged to be qualified; otherwise, it is judged that the "left view" image is unqualified. Further, the pose of the eye image is judged. If sclera_xmax - iris_xmax <= T_relative_left_xmax and sclera_h - iris_h > T_relative_left_h, the pose of the eye image is judged to be "not seeing clearly from the left view"; if iris_ymin - sclera_ymin < T_relative_left_ymin, the pose of the eye image is judged to be "tilted upward from the left view"; if sclera_ymax - iris_ymax < T_relative_left_ymax, the pose of the eye image is judged to be "tilted downward from the left view"; if sclera_h - iris_h < T_relative_left_h, the pose of the eye image is judged to be "the upper and lower eyelids of the left view are not opened"; when none of the above conditions are met, a prompt message of "Please manually check the quality of the eye image" can be fed back.

[0104] Among them, in the above embodiment, the values of T_relative_left, T_relative_left_xmax, T_relative_left_ymin, T_relative_left_ymax, and T_relative_left_h are all empirical values, for example, they can be statistically obtained from a large number of eye images that are judged to be qualified by humans and do not cause adverse effects on the subsequent analysis of eye images.

[0105] In this embodiment, an eye image quality control device is disclosed. For the specific working content of each unit in the device, please refer to the content of the above method embodiment.

[0106] Next, the eye image quality control device provided by the embodiment of the present invention will be described. The eye image quality control device described below can be correspondingly referred to the eye image quality control method described above.

[0107] See Figure 8 , the eye image quality control device may include:

[0108] An image acquisition unit 10, used to obtain an eye image;

[0109] A classification unit 20, configured to classify the eye image based on the viewing direction of the eyeball in the eye image, and obtain a viewing angle category corresponding to the eye image;

[0110] An image conversion unit 30, used for obtaining a black and white eye semantic segmentation mask image of the eye image;

[0111] An image processing unit 40 is used to obtain the region information of the black eye region and the region information of the white eye region in the black and white eye semantic segmentation mask image;

[0112] The judging unit 50 is used to obtain the eye image judging condition that matches the viewing angle category; and judge whether the eye image is qualified based on the eye image judging condition and the area information of the black eye area and the area information of the white eye area.

[0113] The eye image quality control device is also used to implement other embodiments of the above-mentioned eye image quality control method, which will not be described here in detail.

[0114] A storage medium stores a computer program, which, when executed by a processor, implements each step of the eye image quality control method as described in any one of the above.

[0115] Fig. 9 The hardware structure diagram of the eye image quality control device provided by the embodiment of the present invention is shown in FIG. Fig. 9 As shown, it may include: at least one processor 100, at least one communication interface 200, at least one memory 300 and at least one communication bus 400;

[0116] In the embodiment of the present invention, the number of the processor 100, the communication interface 200, the memory 300, and the communication bus 400 is at least one, and the processor 100, the communication interface 200, and the memory 300 communicate with each other through the communication bus 400; obviously, Fig. 9 The communication connections shown for the processor 100, the communication interface 200, the memory 300, and the communication bus 400 are merely optional;

[0117] Optionally, the communication interface 200 may be an interface of a communication module, such as an interface of a GSM module;

[0118] The processor 100 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0119] The memory 300 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory, which stores a preset program corresponding to the above method embodiment.

[0120] The processor 100 is specifically configured to:

[0121] Get an eye image;

[0122] Classifying the eye image based on the viewing direction of the eyeball in the eye image to obtain a viewing angle category corresponding to the eye image;

[0123] Obtaining a black and white eye semantic segmentation mask image of the eye image;

[0124] Obtaining regional information of the black eye region and regional information of the white eye region in the black and white eye semantic segmentation mask image;

[0125] Obtaining eye image judgment conditions that match the viewing angle category;

[0126] Whether the eye image chart is qualified is judged based on the eye image chart judgment condition and the area information of the black eye area and the area information of the white eye area.

[0127] The processor is also used to implement each step of any one of the above-mentioned eye image quality control methods of the present application, and the details will not be repeated here.

[0128] The user information (including but not limited to the user's eye image) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0129] For the convenience of description, the above system is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0130] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.

[0131] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0132] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0133] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0134] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for controlling the quality of an eye image, characterized in that: include: Get an eye image; Classifying the eye image based on the viewing direction of the eyeball in the eye image to obtain a viewing angle category corresponding to the eye image; Obtaining a black and white eye semantic segmentation mask image of the eye image; Obtaining regional information of the black eye region and regional information of the white eye region in the black and white eye semantic segmentation mask image; Obtaining eye image judgment conditions that match the viewing angle category; Whether the eye image chart is qualified is judged based on the eye image chart judgment condition and the area information of the black eye area and the area information of the white eye area.

2. The eye image quality control method according to claim 1, characterized in that: After obtaining the viewing angle category corresponding to the eye image and before obtaining the black and white eye semantic segmentation mask image of the eye image, the method further includes: Determining whether the viewing angle category is a target viewing angle category; When the viewing angle category is not the target viewing angle category, outputting prompt information matching the target viewing angle category; When the viewing angle category is the target viewing angle category, continue to execute subsequent steps.

3. The eye image quality control method according to claim 1, characterized in that: The region information of the black eye region includes area information of the black eye region, and the region information of the white eye region includes area information of the white eye region; and judging whether the eye image chart is qualified based on the judgment condition of the eye image chart and the region information of the black eye region and the region information of the white eye region includes: Whether the eye image chart is qualified is determined based on the area information of the black eye region and the area information of the white eye region.

4. The eye image quality control method according to claim 3, characterized in that: Judging whether the eye image diagram is qualified based on the area information of the black eye region and the area information of the white eye region includes: Determining whether the area information of the black eye region is smaller than a first preset area; Determining whether the area information of the white eye region is smaller than a second preset area; When the area information of the black eye region is smaller than the first preset area, or the area information of the white eye region is smaller than the second preset area, it is determined that the eye image is unqualified.

5. The eye image quality control method according to claim 1, characterized in that: The area information of the black eye region includes the position information of the black eye region, the area information of the white eye region includes the position information of the white eye region, and judging whether the eye image chart is qualified based on the eye image chart judgment condition and the area information of the black eye region and the area information of the white eye region includes: Determine whether the position information of the black eye region and the position information of the white eye region meet the eye image judgment condition; When the position information of the black eye region and the position information of the white eye region meet the eye image diagram judgment condition, the eye image diagram is determined to be qualified.

6. The eye image quality control method according to claim 5, characterized in that: The position information of the black eye area includes the vertex coordinates of the circumscribed rectangle of the black eye area; the position information of the white eye area includes the vertex coordinates of the circumscribed rectangle of the white eye area, and the eye image judgment condition is the constraint condition of the vertex coordinates of the circumscribed rectangles of the black eye area and the white eye area.

7. The eye image quality control method according to claim 6, characterized in that: When the position information of the black eye region and the position information of the white eye region meet the eye image diagram judgment condition, the eye image diagram is determined to be qualified, and feedback information of the eye image diagram is obtained according to the vertex coordinates of the circumscribed rectangle of the black eye region and the vertex coordinates of the circumscribed rectangle of the white eye region.

8. An eye image quality control device, characterized in that: include: An image acquisition unit, used for acquiring an eye image; A classification unit, configured to classify the eye image based on the viewing direction of the eyeball in the eye image, and obtain a viewing angle category corresponding to the eye image; An image conversion unit, used for obtaining a black and white eye semantic segmentation mask image of the eye image; An image processing unit, used for obtaining regional information of a black eye region and regional information of a white eye region in the black and white eye semantic segmentation mask image; The judging unit is used to obtain the eye image judging condition that matches the viewing angle category; and judge whether the eye image is qualified based on the eye image judging condition and the area information of the black eye area and the area information of the white eye area.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the eye image quality control method as described in any one of claims 1 to 7 is implemented.

10. An eye image quality control device, characterized in that: include: including memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the eye image quality control method as described in any one of claims 1-7.

Images