Pupil recognition method and device, equipment and storage medium

Abstract

The invention provides a pupil recognition method and device, equipment and a storage medium. The method comprises the following steps: acquiring a pupil graph of a to-be-detected eye; performing pupil state recognition on the pupil image to determine the shielding degree of the pupil of the to-be-detected eye in the pupil image; and performing pupil information identification based on the pupil image in response to the pupil state identification result representing that the occlusion degree of the pupil of the to-be-detected eye in the pupil image represents that the pupil state is normal.

Description

Technical Field

[0001] The present disclosure relates to the field of ophthalmic technologies, and in particular, to a pupil recognition method, apparatus, device, and storage medium. Background Art

[0002] When taking a pupil image through an ophthalmic biometer or an anterior segment OCT (Optical Coherence Tomography) device, accurately identifying pupil information such as the pupil center and pupil diameter in the pupil image is a very important step.

[0003] In related technologies, pupil information is usually directly recognized from the captured pupil image. However, in the process of implementing the inventive concept of the present disclosure, the inventors found that there are at least the following problems in related technologies:

[0004] There are often situations where the pupil in the captured pupil image is occluded. For example, the pupil is occluded more or even completely due to the subject blinking (see Figure 1A ), or the reflected light point of the fill light on the pupil image falls on or near the pupil contour due to poor shooting position or too small pupil (see Figure 1B and Figure 1C ), or there are irregular light spots on the pupil image due to an intraocular lens implanted in the subject's eye (see Figure 1D ). Therefore, directly recognizing pupil information from the captured pupil image may not be highly accurate. Summary of the Invention

[0005] The present disclosure provides a pupil recognition method, apparatus, device, and storage medium to improve the accuracy and robustness of pupil recognition.

[0006] In a first aspect, an embodiment of the present disclosure provides a pupil recognition method, including:

[0007] Obtaining a pupil image of an eye to be measured;

[0008] Performing pupil state recognition on the pupil image to determine the degree of occlusion of the pupil of the eye to be measured in the pupil image; and

[0009] In response to the pupil state recognition result indicating that the degree of occlusion of the pupil of the eye to be measured in the pupil image represents a normal pupil state, performing pupil information recognition based on the pupil image.

[0010] In a second aspect, an embodiment of the present disclosure further provides a pupil recognition apparatus, including:

[0011] A pupil image acquisition module configured to obtain a pupil image of an eye to be measured;

[0012] A pupil state recognition module, configured to recognize the pupil state of the pupil image to determine the occlusion degree of the pupil of the eye to be measured in the pupil image; and

[0013] A pupil information recognition module, configured to, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, perform pupil information recognition based on the pupil image.

[0014] In a third aspect, an embodiment of the present disclosure further provides an electronic device, which includes:

[0015] One or more processors;

[0016] A storage device, configured to store one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the pupil recognition method as described in the embodiments of the present disclosure.

[0018] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute the pupil recognition method as described in the embodiments of the present disclosure when executed by a computer processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Combined with the drawings and referring to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original elements and elements are not necessarily drawn to scale.

[0020] Figure 1A 、 Figure 1B 、 Figure 1C and Figure 1D Exemplarily show pupil images in 4 different states;

[0021] Figure 2A Exemplarily show a flowchart of the pupil recognition method according to the embodiments of the present disclosure;

[0022] Figure 2B Exemplarily show a pupil image shooting guidance diagram according to the embodiments of the present disclosure;

[0023] Figure 2C and Figure 2D Exemplarily show another pupil image shooting guidance diagram according to the embodiments of the present disclosure;

[0024] Figure 3 Exemplarily show a flowchart of the pupil recognition method according to another embodiment of the present disclosure;

[0025] Figure 4A Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure;

[0026] Figure 4B Exemplarily shows a grayscale histogram according to an embodiment of the present disclosure;

[0027] Figure 5 Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure;

[0028] Figure 6 Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure;

[0029] Figure 7 Exemplarily shows a schematic diagram of a pupil recognition device according to an embodiment of the present disclosure;

[0030] Figure 8 Exemplarily shows a schematic diagram of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0031] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0032] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0033] As used herein, the term "including" and its variations are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0034] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0035] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly specified otherwise in the context, it should be understood as "one or more".

[0036] Figure 2A The flowchart of the pupil recognition method according to an embodiment of the present disclosure is exemplarily shown.

[0037] This method can be executed by a pupil recognition device, which can be implemented in the form of software and / or hardware. Optionally, this device can be implemented by an electronic device. The electronic device can include, but is not limited to, a terminal device connected to an ophthalmic OCT device. The ophthalmic OCT device can include, but is not limited to, a biometer, an anterior segment OCT, etc.

[0038] As Figure 2A shown, the pupil recognition method can include steps S210 - S230.

[0039] In step S210, a pupil image of the eye to be measured is acquired.

[0040] In the embodiments of the present disclosure, an ophthalmic OCT device can be used to collect a pupil image of a patient's eye, and then an associated terminal device can read the collected pupil image from the ophthalmic OCT device, so as to perform pupil information recognition based on the read pupil image.

[0041] In an embodiment of the present disclosure, when using an ophthalmic OCT device to take a pupil image of a patient's eye, the patient's eye will image the optical path of the ophthalmic OCT device, and this imaging appears as a reflection point in the taken pupil image, and this reflection point is defined as the central reflection point. In another embodiment of the present disclosure, when using an ophthalmic OCT device to take a pupil image of a patient's eye, the patient's eye will image the fill lights (such as 6 fill lights can be set), and the imaging of each fill light appears as a reflection point in the taken pupil image. By performing circle fitting on all the reflection points formed by the imaging of the fill lights, a reflection point can be generated at the center of the fitted circle, and this reflection point is defined as the central reflection point. In the process of implementing the embodiments of the present disclosure, the inventors found that when the central reflection point appears at the center of the pupil in the pupil image preview, the probability of taking a pupil image with a normal pupil state (such as no pupil occlusion or less occlusion) is relatively high. And performing pupil information recognition on a pupil image with a normal pupil state can avoid the problem of low pupil recognition accuracy caused by the pupil occlusion problem.

[0042] In addition, in the process of implementing the embodiments of the present disclosure, the inventors also found that when using an ophthalmic OCT device to capture OCT images of a patient's eye, if the optical path of the ophthalmic OCT device is aligned with the corneal apex of the patient's eye during capture and the corneal apex is not blocked, light columns will appear in both the transverse OCT image and the longitudinal OCT image captured. Therefore, when a pupil image is captured when light columns appear in both the captured transverse OCT image and the longitudinal OCT image, the probability of capturing a pupil image with a normal pupil state is relatively high. Because if the corneal apex is not blocked, it indicates that the pupil is likely to be unblocked or less blocked.

[0043] Thus, in the embodiments of the present disclosure, in order to obtain a pupil image with a normal pupil state, it is possible to first observe whether a central reflection point appears at the pupil center in the pupil image preview of the eye to be measured. When a central reflection point appears at the pupil center in the pupil image preview of the eye to be measured, the pupil image is then captured. Alternatively, in other embodiments of the present disclosure, in order to obtain a pupil image with a normal pupil state, it is also possible to first observe whether light columns appear in the captured transverse OCT image and the longitudinal OCT image. When light columns appear in both the captured transverse OCT image and the longitudinal OCT image, the pupil image is then captured.

[0044] That is to say, in the embodiments of the present disclosure, the pupil image is captured when a central reflection point appears at the pupil center in the pupil preview image of the eye to be measured. Alternatively, the pupil image is captured when light columns appear in both the transverse OCT image and the longitudinal OCT image of the eye to be measured.

[0045] As an implementation manner, the pupil image mentioned in step S210 is captured when the image center is aligned with the corneal apex, so as to ensure the capture quality of the pupil image and avoid the situation that pupil recognition is inaccurate due to inappropriate capture position. For example, referring to Figure 2B , when the corneal apex is aligned with the optical path of the OCT device, the imaging of the OCT device in the eye to be measured will show that a central reflection point appears at the pupil center position in the pupil preview image. Thus, by judging whether a central reflection point appears at the pupil center position in the pupil preview image, it is possible to quickly determine whether the pupil center is aligned with the corneal apex, and then determine whether to capture the pupil image. Alternatively, referring to Figure 2C and Figure 2D , it can be judged by viewing the real-time obtained transverse OCT image and longitudinal OCT image of the eye to be measured. When light columns appear in both the transverse OCT image and the longitudinal OCT image, it indicates that the optical path of the OCT device has been aligned with the corneal apex. At this time, when the pupil image is captured, it can be ensured that the image center of the captured pupil image coincides with the corneal apex, so as to ensure that a high-quality pupil image can be obtained, and further avoid affecting pupil recognition due to poor pupil image quality.

[0046] In step S220, pupil state recognition is performed on the pupil image to determine the degree of occlusion of the pupil of the eye to be measured in the pupil image.

[0047] In the embodiments of the present disclosure, pupil information recognition includes, but is not limited to, recognition of the pupil center and pupil diameter. In the captured pupil image, if the pupil is not occluded or is slightly occluded, it generally does not affect pupil information recognition. If the pupil is heavily or severely occluded, it generally affects pupil information recognition. Therefore, in the embodiments of the present disclosure, in order to reduce or avoid the influence of the pupil occlusion problem on pupil information recognition, before performing pupil information recognition based on the pupil image, step S220 is first executed to perform pupil state recognition on the pupil image to determine the degree of occlusion of the pupil of the eye to be measured in the pupil image.

[0048] The pupil state mentioned in step S220 is used to determine whether the pupil image obtained in step S210 can be accurately recognized for pupil information. The pupil state can include normal or abnormal. If the pupil state in the pupil image obtained in step S210 is normal, it means that the pupil image can be accurately recognized for pupil information. If the pupil state in the pupil image obtained in step S210 is abnormal, it means that the pupil image cannot be accurately recognized for pupil information.

[0049] In this embodiment, a normal pupil state means that the degree of occlusion of the pupil of the eye to be measured in the pupil image is less than the preset degree of occlusion. An abnormal pupil state means that the degree of occlusion of the pupil of the eye to be measured in the pupil image is greater than or equal to the preset degree of occlusion.

[0050] In step S230, in response to the pupil state recognition result indicating that the degree of occlusion of the pupil of the eye to be measured in the pupil image represents a normal pupil state, pupil information recognition is performed based on the pupil image.

[0051] When the pupil state recognition result obtained in step 220 indicates that the pupil state in the pupil image obtained in step S210 is normal, it shows that the quality of the pupil image is qualified and will not affect the recognition of pupil information. Using a pupil image with qualified quality for pupil information recognition can improve the accuracy and robustness of pupil information recognition.

[0052] In addition, in this embodiment, in response to the pupil state recognition result indicating that the degree of occlusion of the pupil of the eye to be measured in the pupil image represents an abnormal pupil state, the currently acquired pupil image is discarded, and at the same time, a reminder is given to re - shoot the pupil image until the re - shot pupil image meets the pupil information recognition conditions.

[0053] When the pupil state recognition result obtained in step 220 indicates that the pupil state in the pupil image obtained in step S210 is abnormal, it indicates that the quality of the pupil image is unqualified, and accurate recognition of pupil information cannot be based on this pupil image. At this time, a prompt message indicating abnormal pupil image capture can be displayed to prompt relevant personnel to retake the pupil image, and step S220 is executed to perform pupil state recognition on the retaken pupil image. If the pupil state of the retaken pupil image is normal, pupil information recognition is performed based on the retaken pupil image. If the pupil state of the retaken pupil image is abnormal, a new pupil image is continuously captured until a pupil image with a normal pupil state is obtained. Thus, inaccurate recognition of pupil information caused by poor quality of the pupil image can be avoided.

[0054] Through the embodiments of the present disclosure, the pupil state of the captured pupil image is first recognized, and then it is further determined whether to perform pupil information recognition based on the pupil image according to the pupil state recognition result. When it is recognized that the pupil state is normal, it means that the pupil of the eye to be measured in the pupil image is not blocked or is blocked to a small extent. In this case, using this pupil image for pupil information recognition will not affect the accuracy and robustness of pupil information recognition. When it is recognized that the pupil state is abnormal, it means that the pupil of the eye to be measured in the pupil image is blocked and is blocked to a large extent. In this case, using this pupil image for pupil information recognition will affect the accuracy and robustness of pupil information recognition. In the embodiments of the present disclosure, by first predicting the degree of occlusion of the pupil of the eye to be measured in the captured pupil image, and then determining whether to perform pupil information recognition based on the prediction result, it is possible to avoid directly using the captured pupil image for pupil information recognition and causing inaccurate pupil information recognition due to the pupil being blocked or being blocked to a large extent during pupil image capture.

[0055] As an optional embodiment, step S220 may include at least one of the following:

[0056] Determine the number of reflection points in the pupil image, and perform pupil state recognition based on the determined number of reflection points;

[0057] Determine the grayscale histogram corresponding to the pupil image, and perform pupil state recognition based on the grayscale histogram.

[0058] That is, the pupil state recognition in step S220 can be implemented by at least the following three methods.

[0059] Method 1, the pupil state can be determined only based on the number of reflection points in the pupil image.

[0060] Method 2, the pupil state can be determined only based on the grayscale histogram corresponding to the pupil image.

[0061] In the third method, the pupil state can be comprehensively determined based on the number of reflection points in the pupil image and the grayscale histogram corresponding to the pupil image.

[0062] Continue to refer to Figure 1A , when the pupil state is abnormal, such as when the pupil is severely blocked during the capture of the pupil image, the number of reflection points corresponding to the fill light in the pupil image will be significantly reduced or even no reflection points of the fill light will appear in the pupil image. Continue to refer to Figure 2B , when the pupil state is normal, such as when the pupil is slightly blocked or unblocked during the capture of the pupil image, the number of reflection points corresponding to the fill light in the pupil image will be reduced less or not reduced at all. Therefore, by using the first method, that is, by determining the number of reflection points in the pupil image, the pupil state can be accurately determined.

[0063] Continue to refer to Figure 1A , when the pupil state is abnormal, such as when the pupil is severely blocked during the capture of the pupil image, the influence of eyelid reflection is relatively large. Continue to refer to Figure 1B , when the pupil state is normal, such as when the pupil is slightly blocked during the capture of the pupil image, the influence of eyelid reflection is relatively small. Continue to refer to Figure 2B , when the pupil state is normal, such as when the pupil is unblocked during the capture of the pupil image, there is almost no eyelid reflection. Therefore, by using the second method, that is, by determining the grayscale histogram corresponding to the pupil image, the pupil state can also be accurately determined.

[0064] Continue to refer to Figure 1B , such as when the pupil is slightly blocked during the capture of the pupil image, although the eyelid reflection has a certain influence, in this case, the number of reflection points corresponding to the fill light in the pupil image has not decreased. Using this pupil image for pupil information recognition can also ensure the accuracy rate. Therefore, in this case, it can be considered that the pupil state is normal. And in this case, by using the third method, that is, by determining the number of reflection points in the pupil image and the grayscale histogram corresponding to the pupil image, the pupil state can be determined more accurately.

[0065] Figure 3 Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure.

[0066] As Figure 3 shown, the pupil recognition method may include the following steps:

[0067] In step S310, obtain a pupil image of the eye to be measured.

[0068] This step is the same or similar to the step S210 shown in Figure 2A , and the embodiments of the present disclosure will not be elaborated herein.

[0069] In step S320, determine the area of each reflection point in the pupil image.

[0070] It should be understood that the reflected light points are the images of the supplementary light or other interfering objects in the pupil image. However, compared with other interfering objects, the shape and area of the reflected light points formed by the imaging of the supplementary light in the pupil image are relatively determined. Therefore, in this embodiment, by determining the area of each reflected light point in the pupil image, the reflected light points that belong to the image formed by the supplementary light, that is, the target reflected light points, can be screened out.

[0071] In step S330, determine the target reflected light points whose area values fall within a preset numerical range, and count the number of target reflected light points, where the target reflected light points are the images of the supplementary light in the pupil image.

[0072] It should be understood that the preset numerical range is the pre-set area change range of the target reflected light points corresponding to the supplementary light in the pupil image. For example, the preset numerical range is a range greater than the first area and less than the second area. Among them, the second area is greater than the first area. Detect whether the area of each reflected light point falls within this preset numerical range, and determine the reflected light points that fall within the preset numerical range as the target reflected light points, and count the number of target reflected light points, so that the number of target reflected light points corresponding to the supplementary light in the pupil image can be accurately obtained.

[0073] In step S340, perform pupil state recognition based on the counted number of target reflected light points.

[0074] Continue to refer to Figure 1A 、 Figure 1B and Figure 2B , when taking the pupil image, the degree of occlusion of the pupil may directly affect the number of reflected light points of the image formed by the supplementary light in the pupil image. Therefore, in the embodiments of the present disclosure, based on the number of target reflected light points, the pupil state of the eye to be measured in the pupil image can be more accurately judged and recognized.

[0075] In an embodiment of the present disclosure, step S340 may include: determining whether the number of target reflected light points is greater than a first preset value; among them, the number of target reflected light points being greater than the first preset value indicates that the pupil state is normal; the number of target reflected light points being less than or equal to the first preset value indicates that the pupil state is abnormal.

[0076] In the embodiments of the present disclosure, the number of target reflected light points being greater than the first preset value indicates that the pupil state is normal, and further indicates that the pupil in the pupil image is not occluded or is occluded to a lesser degree. In this case, using this pupil image for pupil information will not affect the accuracy and robustness of information recognition. On the contrary, the number of target reflected light points being less than or equal to the first preset value indicates that the pupil state is abnormal, and further indicates that the pupil in the pupil image is occluded to a greater degree. In this case, using this pupil image for pupil information will seriously affect the accuracy and robustness of information recognition.

[0077] In step S350, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil map represents a normal pupil state, pupil information recognition is performed based on the pupil map.

[0078] This step corresponds to or is similar to Figure 2A the step S230 shown, and the embodiments of the present disclosure will not repeat it here.

[0079] In the embodiments of the present disclosure, by determining the area of each reflection point in the pupil map and determining the target reflection points whose area values fall within a preset numerical range, pupil state recognition can be accurately performed based on the number of target reflection points, thereby ensuring the accuracy of subsequent pupil information recognition.

[0080] Figure 4A Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure.

[0081] As Figure 4A shown, the pupil recognition method may include the following steps:

[0082] In step S410, obtain the pupil map of the eye to be measured.

[0083] This step corresponds to or is similar to Figure 2A the step S210 shown, and the embodiments of the present disclosure will not repeat it here.

[0084] In step S420, determine the gray level histogram corresponding to the pupil map and determine the target gray level partition in the gray level histogram.

[0085] It should be understood that the gray level histogram is a statistic of the gray level distribution in the pupil map. The abscissa of the gray level histogram represents the gray level partition (i.e., gray level) of each pixel point in the pupil map, and the ordinate represents the number of pixel points having pixel points in this gray level partition. Based on the gray level value of each pixel point in the pupil map, the number of pixel points corresponding to each division partition is statistically calculated, and then the gray level histogram corresponding to the pupil map is determined, as shown in Figure 4B shown. In the embodiments of the present disclosure, the target gray level partition refers to the gray level partition with the largest brightness, that is, the gray level partition with the highest gray level value. The gray level partition with the highest gray level value is generally considered to be the eyelid reflection area.

[0086] In step S430, count the number of target pixels corresponding to the target gray level partition in the gray level histogram and calculate the proportion of the target pixels.

[0087] It should be understood that determining the number of target pixels corresponding to the target gray level partition from the gray level histogram means that in the gray level histogram, the ordinate value when the abscissa is the target gray level partition is determined as the number of target pixels. The target pixels may refer to the brightest pixels. Divide the number of target pixels by the total number of pixels in the pupil map to obtain the proportion of the target pixels.

[0088] In step S440, pupil state recognition is performed according to the proportion of target pixels.

[0089] Exemplarily, based on the proportion of target pixels, it can be determined whether the bright area in the pupil image is too large, and then whether the pupil state is normal or abnormal can be determined.

[0090] Exemplarily, step S440 may include: determining whether the proportion of target pixels is less than a second preset value; wherein, the proportion of target pixels being less than the second preset value indicates a normal pupil state; the proportion of target pixels being greater than or equal to the second preset value indicates an abnormal pupil state.

[0091] It should be understood that when the proportion of target pixels is less than the second preset value, it indicates that the bright area in the pupil image is small, that is, the bright area caused by the strong reflection of the eyelid is small. In this case, it means that the pupil is less or not blocked when the pupil image is captured, so using this pupil image will not affect the accurate recognition of pupil information. When the proportion of target pixels is greater than or equal to the second preset value, it indicates that the bright area in the pupil image is too large, that is, the bright area caused by the strong reflection of the eyelid is large. In this case, it means that the pupil is severely blocked when the pupil image is captured, so using this pupil image will affect the accurate recognition of pupil information.

[0092] In step S450, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, pupil information recognition is performed based on the pupil image.

[0093] This step corresponds to and is the same as or similar to Figure 2A the step S230 shown, and the embodiments of the present disclosure will not elaborate herein.

[0094] In the embodiments of the present disclosure, by determining the target gray level partition in the gray level histogram, counting the number of target pixels corresponding to the target gray level partition in the gray level histogram and calculating the proportion of the target pixels, pupil state recognition can also be accurately performed according to the proportion of the target pixels, and thus the accuracy and robustness of pupil information recognition can be ensured.

[0095] Figure 5 Exemplarily shows a flowchart of a pupil recognition method according to another embodiment of the present disclosure.

[0096] As Figure 5 shown, the pupil recognition method may include the following steps:

[0097] In step S510, a pupil image of the eye to be measured is acquired.

[0098] This step corresponds to and is the same as or similar to Figure 2A the step S210 shown, and the embodiments of the present disclosure will not elaborate herein.

[0099] In step S520, determine the area of each reflected light point in the pupil image.

[0100] This step corresponds to or is similar to step S320 shown in Figure 3 and will not be elaborated herein in the embodiments of the present disclosure.

[0101] In step S530, determine the target reflected light points whose area values fall within a preset numerical range, and count the number of target reflected light points.

[0102] This step corresponds to or is similar to step S330 shown in Figure 3 and will not be elaborated herein in the embodiments of the present disclosure.

[0103] In step S540, determine the grayscale histogram corresponding to the pupil image, and determine the target grayscale partition in the grayscale histogram.

[0104] This step corresponds to or is similar to step S420 shown in Figure 4A and will not be elaborated herein in the embodiments of the present disclosure.

[0105] In step S550, count the number of target pixels corresponding to the target grayscale partition in the grayscale histogram and calculate the proportion of the target pixels.

[0106] This step corresponds to or is similar to step S430 shown in Figure 4A and will not be elaborated herein in the embodiments of the present disclosure.

[0107] In step S560, perform pupil state recognition based on the counted number of target reflected light points and the proportion of target pixels.

[0108] In the embodiments of the present disclosure, by comprehensively determining whether the pupil state is normal based on the number of target reflected light points and the proportion of target pixels, the accuracy of pupil state recognition can be improved.

[0109] Exemplarily, step S560 may include: determining whether the number of target reflected light points is greater than a first preset value, and whether the proportion of target pixels is less than a second preset value; wherein, the number of target reflected light points being greater than the first preset value and the proportion of target pixels being less than the second preset value indicates that the pupil state is normal.

[0110] It should be understood that when the number of target reflection points is greater than the first preset value and the proportion of target pixels is less than the second preset value, it indicates that there are enough target reflection points in the pupil image, and the bright area caused by the strong reflection of the eyelid is small, which further indicates that the pupil is not blocked or slightly blocked when the pupil image is captured. Therefore, using this pupil image will not affect the accurate identification of pupil information. When the number of target reflection points is less than or equal to the first preset value, and / or the proportion of target pixels is greater than or equal to the second preset value, it indicates that the number of target reflection points in the pupil image is small, and / or the bright area in the pupil image is too large, which further indicates that the pupil is severely blocked when the pupil image is captured. Therefore, using this pupil image will affect the accurate identification of pupil information.

[0111] In step S570, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, pupil information is recognized based on the pupil image.

[0112] This step is the same as or similar to Figure 2A the step S230 shown, and the embodiments of the present disclosure will not repeat it here.

[0113] In the embodiments of the present disclosure, by simultaneously performing pupil state recognition based on the number of reflection points and the gray histogram in the pupil image, the accuracy and comprehensiveness of pupil state recognition can be improved, and further the accuracy of pupil information recognition can be improved.

[0114] Figure 6 An exemplary flowchart of a pupil recognition method according to another embodiment of the present disclosure is shown.

[0115] As Figure 6 shown, the pupil recognition method specifically includes the following steps:

[0116] In step S610, a pupil image of the eye to be measured is obtained.

[0117] In step S620, pupil state recognition is performed on the pupil image to determine the occlusion degree of the pupil of the eye to be measured in the pupil image.

[0118] In this embodiment, steps S610 and S620 are respectively the same as or similar to Figure 2A the steps S210 and S220 shown, and the embodiments of the present disclosure will not repeat it here.

[0119] In step S630, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, reflection point contour recognition is performed based on the pupil image to obtain a corresponding set of reflection point contour points.

[0120] In step S630, when the pupil state in the pupil image is normal, identify the contour enclosed by all the reflection points in the pupil image to obtain all the contour point sets on the contour, that is, the reflection point contour point sets.

[0121] Exemplarily, in step S630, perform reflection point contour recognition on the pupil image within a preset angle range.

[0122] It should be understood that the preset angle range can be a pre-set angle range of the reflection point contour to be recognized. For example, since the upper and lower parts of the pupil are easily blocked, the preset angle range can be set to a range of -45~45 or 135~225°. By only recognizing the reflection point contour of the pupil image within the preset angle range, only the reflection point contour point set located within the preset angle range can be obtained, avoiding the reflection point contour of the pupil area that is easily blocked, thereby reducing the calculation amount, improving the recognition efficiency, and also avoiding being affected by the occlusion of the upper and lower parts of the pupil, further improving the accuracy of pupil information recognition.

[0123] In step S640, fill the reflection points in the pupil image to obtain the filled pupil image.

[0124] Since the reflection points in the pupil image may fall on the pupil contour or near the pupil contour, affecting the accuracy of pupil contour recognition and further affecting the accuracy of pupil information recognition, therefore, to ensure the accuracy of pupil information recognition, the reflection points in the pupil image can be filled by means of morphological reconstruction to obtain the filled pupil image.

[0125] In step S650, perform pupil contour recognition based on the filled pupil image to obtain the corresponding preliminary pupil contour point set.

[0126] As an implementation method, the filled pupil image can be subjected to polar coordinate transformation to obtain the transformed pupil image, and based on the shortest path search method, perform pupil contour recognition on the transformed pupil image to obtain the pupil candidate contour point set, and then perform polar coordinate inverse transformation on the pupil candidate contour point set to obtain the transformed preliminary pupil contour point set.

[0127] Exemplarily, in step S650, pupil contour recognition can be performed on the filled pupil image within a preset angle range.

[0128] In the embodiments of the present disclosure, the preset angle range may be an angle range that needs to be set in advance for identifying the pupil contour. For example, since the upper and lower parts of the pupil are easily blocked, the preset angle range may be set to the range of -45~45 and / or 135~225°. By only identifying the pupil contour within the preset angle range of the filled pupil image, only the set of initial pupil contour points located within the preset angle range can be obtained, thus reducing the amount of calculation, improving the recognition efficiency, and also avoiding being affected by the occlusion of the upper and lower parts of the pupil, further improving the accuracy of pupil information recognition.

[0129] In step S660, calculate the difference set between the set of initial pupil contour points and the set of reflected light point contour points, and perform pupil information recognition based on the difference set.

[0130] In this embodiment, the contour points located in the set of reflected light point contour points are removed from the set of initial pupil contour points, so as to obtain a difference set that does not contain the reflected light point contour points, and further interference to pupil information recognition caused by the reflected light points of the fill light or other bright spots located on the pupil edge can be avoided. Based on this difference set, pupil information can be recognized more accurately, thus improving the recognition accuracy of pupil information.

[0131] Exemplarily, performing pupil information recognition based on the difference set in step S660 may include: screening out the discrete points in the difference set to obtain an optimized set of pupil contour points; and performing pupil information recognition based on the optimized set of pupil contour points.

[0132] As an implementation, using an outlier detection method, such as the local outlier factor method, the outlier detection method based on statistics or clustering, etc., the discrete points in the difference set can be determined, and all the determined discrete points are removed and optimized from the difference set to obtain an optimized set of pupil contour points. The set of pupil contour points can be used to represent the set of contour points on the fine edge of the finally recognized pupil. Perform circle fitting based on the set of pupil contour points, such as circle fitting by the least squares method, and determine pupil information based on the obtained fitted circle, that is, the pupil center point position and the pupil diameter.

[0133] Exemplarily, the process of determining the discrete points in the difference set based on the local outlier factor method is: calculate the k-th reachable distance of each point in the k-th distance neighborhood of each point in the difference set, that is , where d(o, p) is the distance between two points, is the k-th distance of point o. Calculate the local k-th reachable density of each point, that is , where is the k-th distance neighborhood of point p. The k-th local outlier factor of each point is: , and based on threshold detection, the discrete points in the difference set can be determined. For example, the points with the k-th local outlier factor greater than a preset value are used as discrete points.

[0134] In an embodiment of the present disclosure, by filling the reflected light points in the pupil image, a filled pupil image is obtained, and pupil contour recognition is performed based on the filled pupil image to obtain a corresponding set of preliminary pupil contour points, thereby achieving a rough recognition of the pupil edge. The difference set between the set of preliminary pupil contour points and the set of reflected light point contours is calculated, and pupil information recognition is performed based on the difference set, thereby achieving a fine recognition of the pupil edge. Through the rough-to-fine recognition of the pupil edge, the accuracy of pupil edge recognition is improved, and the interference of reflected light points or light spots generated by optical devices or abnormal states of the patient's eyes on pupil edge recognition is avoided, further improving the accuracy of pupil information recognition.

[0135] Figure 7 An exemplary schematic diagram of a pupil recognition device according to an embodiment of the present disclosure is shown.

[0136] As shown in Figure 7 FIG. 700, the pupil recognition device 700 includes: a pupil image acquisition module 710, a pupil state recognition module 720, and a pupil information recognition module 730. The pupil image acquisition module 710 is configured to acquire a pupil image of an eye to be measured; the pupil state recognition module 720 is configured to perform pupil state recognition on the pupil image to determine the occlusion degree of the pupil of the eye to be measured in the pupil image; and the pupil information recognition module 730 is configured to, in response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, perform pupil information recognition based on the pupil image.

[0137] As an optional embodiment, the pupil state recognition module 720 may include at least one of the following: a first state recognition sub-module configured to determine the number of reflected light points in the pupil image and perform pupil state recognition based on the determined number of reflected light points; a second state recognition sub-module configured to determine the gray level histogram corresponding to the pupil image and perform pupil state recognition based on the gray level histogram.

[0138] As an optional embodiment, the first state recognition sub-module includes: a reflected light point area determination unit configured to determine the area of each reflected light point in the pupil image; a reflected light point number determination unit configured to determine target reflected light points whose area values fall within a preset numerical range and count the number of the target reflected light points, where the target reflected light points are the images of the fill light in the pupil image; and a first state recognition unit configured to perform pupil state recognition based on the counted number of the target reflected light points.

[0139] As an alternative embodiment, the first state recognition unit is configured to: determine whether the number of the target reflected light points is greater than a first preset value; wherein, that the number of the target reflected light points is greater than the first preset value indicates that the pupil state is normal; and that the number of the target reflected light points is less than or equal to the first preset value indicates that the pupil state is abnormal.

[0140] As an alternative embodiment, the second state recognition sub-module includes: a target gray level partition determination unit configured to determine a gray level histogram corresponding to the pupil image and determine a target gray level partition in the gray level histogram; a target pixel ratio calculation unit configured to count the number of target pixels corresponding to the target gray level partition in the gray level histogram and calculate the ratio of the target pixels; and a second state recognition unit configured to perform pupil state recognition according to the ratio of the target pixels.

[0141] As an alternative embodiment, the second state recognition unit is configured to: determine whether the ratio of the target pixels is less than a second preset value; wherein, that the ratio of the target pixels is less than the second preset value indicates that the pupil state is normal; and that the ratio of the target pixels is greater than or equal to the second preset value indicates that the pupil state is abnormal.

[0142] As an alternative embodiment, the pupil information recognition module 730 includes: a reflected light point contour point set determination sub-module configured to perform reflected light point contour recognition based on the pupil image to obtain a corresponding reflected light point contour point set; a reflected light point filling sub-module configured to fill the reflected light points in the pupil image to obtain a filled pupil image; a pupil preliminary contour point set determination sub-module configured to perform pupil contour recognition based on the filled pupil image to obtain a corresponding pupil preliminary contour point set; and a pupil information recognition sub-module configured to calculate the difference set between the pupil preliminary contour point set and the reflected light point contour point set and perform pupil information recognition based on the difference set.

[0143] As an alternative embodiment, the pupil information recognition sub-module is configured to: screen out the discrete points in the difference set to obtain an optimized pupil contour point set; and perform pupil information recognition based on the optimized pupil contour point set.

[0144] As an alternative embodiment, the reflected light point contour point set determination sub-module is configured to perform reflected light point contour recognition based on the pupil image within a preset angle range; and / or the pupil preliminary contour point set determination sub-module is configured to perform pupil contour recognition based on the filled pupil image within a preset angle range.

[0145] As an alternative embodiment, the pupil image is captured when a central reflection point appears at the pupil center in the pupil preview image of the eye to be measured; alternatively, the pupil image is captured when light columns appear in both the transverse OCT image and the longitudinal OCT image of the eye to be measured.

[0146] The pupil recognition device provided by the embodiments of the present disclosure can execute the pupil recognition method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0147] The technical problems solved and the technical effects achieved by the pupil recognition device provided by the embodiments of the present disclosure are the same or similar to those of the pupil recognition method provided by the foregoing embodiments, and will not be elaborated herein.

[0148] It should be noted that the various units and modules included in the above device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present disclosure.

[0149] Figure 8 FIG. is a schematic diagram of an electronic device according to an embodiment of the present disclosure. The following reference Figure 8 , which shows a schematic structure of an electronic device 500 suitable for implementing the embodiments of the present disclosure (such as Figure 8 The terminal device shown can be used in association with an ophthalmic OCT device). The electronic device in the embodiments of the present disclosure can be a terminal device associated with an ophthalmic OCT device, etc. The electronic device in this embodiment is only an example and should not bring any limitation to the functions and usage scope of the embodiments of the present disclosure.

[0150] As Figure 8 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The editing / output (I / O) interface 505 is also connected to the bus 504.

[0151] The following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and a communication device 509. The communication device 509 can allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. However, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices can be implemented or had alternatively.

[0152] Specifically, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above functions defined in the methods of the embodiments of the present disclosure are executed.

[0153] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0154] The electronic device provided by the embodiments of the present disclosure and the pupil recognition method provided by the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be referred to the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0155] The embodiments of the present disclosure provide a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the pupil recognition method provided by the above embodiments is implemented.

[0156] It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0157] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed network.

[0158] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist separately and not be assembled into the electronic device.

[0159] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The foregoing programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in an order different from that noted in the drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0161] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. Among them, the name of the unit does not constitute a limitation to the unit itself in some cases. For example, the first acquisition unit may also be described as "the unit for acquiring at least two Internet protocol addresses".

[0162] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and so on.

[0163] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present disclosure can be achieved, and no limitation is imposed herein.

[0165] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A pupil recognition method, comprising: Obtaining a pupil image of an eye to be measured; Performing pupil state recognition on the pupil image to determine the occlusion degree of the pupil of the eye to be measured in the pupil image; And In response to the pupil state recognition result indicating that the occlusion degree of the pupil of the eye to be measured in the pupil image represents a normal pupil state, performing pupil information recognition based on the pupil image.

2. The method according to claim 1, wherein Performing pupil state recognition on the pupil image includes at least one of the following: Determining the number of reflection points in the pupil image and performing pupil state recognition based on the determined number of reflection points; Determining the grayscale histogram corresponding to the pupil image and performing pupil state recognition based on the grayscale histogram.

3. The method according to claim 2, wherein Determining the number of reflection points in the pupil image and performing pupil state recognition based on the determined number of reflection points includes: Determining the area of each reflection point in the pupil image; Determining target reflection points whose area values fall within a preset numerical range and counting the number of the target reflection points, where the target reflection points are the images of fill lights in the pupil image; and Performing pupil state recognition based on the counted number of the target reflection points.

4. The method according to claim 3, wherein, Performing pupil state recognition based on the counted number of the target reflection points includes: Determining whether the number of the target reflection points is greater than a first preset value; where The number of the target reflection points being greater than the first preset value indicates a normal pupil state; The number of the target reflection points being less than or equal to the first preset value indicates an abnormal pupil state.

5. The method according to claim 2, wherein Determining the grayscale histogram corresponding to the pupil image and performing pupil state recognition based on the grayscale histogram includes: Determining the grayscale histogram corresponding to the pupil image and determining the target grayscale partition in the grayscale histogram; Counting the number of target pixels corresponding to the target grayscale partition in the grayscale histogram and calculating the proportion of the target pixels; and Performing pupil state recognition according to the proportion of the target pixels.

6. The method according to claim 5, wherein, Performing pupil state recognition according to the proportion of the target pixels includes: Determining whether the proportion of the target pixels is less than a second preset value; where The proportion of the target pixels being less than the second preset value indicates a normal pupil state; The proportion of the target pixels being greater than or equal to the second preset value indicates an abnormal pupil state.

7. The method according to claim 1, wherein, Performing pupil information recognition based on the pupil image includes: Performing reflection point contour recognition based on the pupil image to obtain a corresponding set of reflection point contour points; Filling the reflection points in the pupil image to obtain a filled pupil image; Performing pupil contour recognition based on the filled pupil image to obtain a corresponding set of preliminary pupil contour points; and Calculating the difference set between the set of preliminary pupil contour points and the set of reflection point contour points and performing pupil information recognition based on the difference set.

8. The method according to claim 7, wherein Performing pupil information recognition based on the difference set includes: Filtering out discrete points in the difference set to obtain an optimized set of pupil contour points; and Performing pupil information recognition based on the optimized set of pupil contour points.

9. According to the method of claim 7, wherein: Performing reflection point contour recognition on the pupil image within a preset angular range; and / or Perform pupil contour recognition within a preset angular range based on the filled pupil image.

10. The method according to claim 1, wherein the pupil image is captured when a target reflection point appears at the pupil center in the pupil preview image of the eye to be measured; or the pupil image is captured when light columns appear in both the transverse OCT image and the longitudinal OCT image of the eye to be measured.

11. A pupil recognition device, comprising: a pupil image acquisition module configured to acquire a pupil image of an eye to be measured; a pupil state recognition module configured to perform pupil state recognition on the pupil image to determine the degree of occlusion of the pupil of the eye to be measured in the pupil image; and a pupil information recognition module configured to perform pupil information recognition based on the pupil image in response to the pupil state recognition result indicating that the degree of occlusion of the pupil of the eye to be measured in the pupil image represents a normal pupil state.

12. An electronic device, comprising: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, enabling the one or more processors to implement the pupil recognition method according to any one of claims 1-10.

13. A storage medium containing computer-executable instructions, wherein, The computer-executable instructions are used to execute the pupil recognition method according to any one of claims 1-10 when executed by a computer processor.

Images