An iris segmentation method using purkinje reflections and dynamic 3D models

Abstract

The application protects an iris segmentation method using Purkinje reflection and dynamic three-dimensional model, which comprises the following processes: a plurality of infrared light sources are arranged around a camera, and a plurality of frames of eye images with reflection points are obtained from a video sequence captured by the camera; the eye center is calculated by using the position of the reflection points, and an initial three-dimensional eye model is established; the three-dimensional coordinates and posture of the eye center are adjusted according to the change of the inner and outer canthus positions, and the three-dimensional eye model is updated; in the updated three-dimensional eye model, the iris radius is obtained according to the eye center, and a three-dimensional iris circle equation is established; the three-dimensional iris circle is projected onto a two-dimensional image plane to obtain iris ellipse parameters, and an iris segmentation result is generated. By constructing a dynamic three-dimensional model, combining with deep learning to detect eye model parameters including the eye center, the iris radius and the iris center, calculating the three-dimensional space optical axis and the iris ellipse, the iris segmentation adapting to the head posture change and the eye movement is realized.

Description

Technical Field

[0001] This invention relates to the fields of biometrics and computer vision, and more specifically, to an iris segmentation method utilizing Purkinje reflection and dynamic 3D models. Background Technology

[0002] Iris recognition, as a highly reliable biometric technology, has been widely used in security authentication and identity recognition due to its uniqueness and stability. A key step is iris segmentation, which involves accurately locating the inner and outer boundaries of the iris in an eye image, directly affecting the accuracy of recognition. However, traditional two-dimensional iris segmentation methods (such as edge detection and Hough transform) are easily affected by factors such as changes in eye pose, eyelid occlusion, and uneven lighting, leading to decreased segmentation accuracy and difficulty in achieving satisfactory results.

[0003] Existing methods often neglect the three-dimensional anatomical features of the eyeball and lack effective methods for establishing a three-dimensional model of the eyeball, failing to fully utilize the spatial structure of the eyeball. The most critical technology for iris segmentation is establishing the center of the eyeball. Traditional methods, such as the multi-view optical axis intersection method, have high requirements for camera calibration and synchronization and lack robustness. The method of using the maximum gradient of the iris edge has high requirements for algorithms, large computational resources, high image quality, and is easily affected by environmental interference.

[0004] In addition, the established 3D model is only effective when the relative position of the head and the camera is fixed. It is only applicable when the relative position of the camera and the head remains unchanged. When the user's head moves or rotates, the original model parameters become invalid, resulting in segmentation errors. There is a lack of an effective method for dynamically updating the model.

[0005] Therefore, there is an urgent need for a three-dimensional iris segmentation method that can robustly locate the center of the eyeball and dynamically adapt to changes in head posture. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a technical solution for iris segmentation using Purkinje reflection and dynamic 3D models. By constructing a dynamic 3D model, a robust and accurate 3D eye model can be established and dynamically updated, achieving adaptability to changes in head posture and eye movements, thereby realizing a high-precision iris segmentation method.

[0007] An iris segmentation method utilizing Purkinje reflection and dynamic 3D models includes the following steps:

[0008] St10: Several infrared light sources are set up symmetrically around the camera center to obtain several frames of eye images with reflective points from the video sequence captured by the camera.

[0009] St20, establish an eye coordinate system, extract a reflection point in an eye image, and position the reflection point based on the eye coordinate system; use the position of the reflection point, combine a known infrared light source position and a camera parameter, adopt a geometric optical model, and calculate a three-dimensional coordinate of an eyeball center.

[0010] St30, establish an initial three-dimensional eyeball model based on the three-dimensional coordinate of the eyeball center and a preset iris radius.

[0011] St40, detect positions of an inner corner and an outer corner of the patient's eye, adjust the three-dimensional coordinate and the posture of the eyeball center according to changes in the positions of the inner corner and the outer corner, and update the three-dimensional eyeball model.

[0012] St50, obtain the iris radius based on the eyeball center in the updated three-dimensional eyeball model, and establish a three-dimensional iris circle equation.

[0013] St60, project the three-dimensional iris circle to a two-dimensional image plane to obtain iris ellipse parameters, and generate an iris segmentation result.

[0014] In summary, the above technical solution has the following beneficial effects: the present application proposes an iris segmentation method for estimating eyeball parameters by using Purkinje reflection, realizes adaptability to head posture changes and eyeball movements by constructing a dynamic three-dimensional model and directly detecting the position of the iris center in combination with deep learning, and improves the accuracy and robustness of iris segmentation. The present application has significant innovation while avoiding repetition of the prior art, protects original content, and has a wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 It is a flowchart schematic diagram of an iris segmentation method using Purkinje reflection and a dynamic three-dimensional model.

[0016] Figure 2 It is a schematic diagram of a reflection point and an iris center.

[0017] Figure 3 It is a schematic diagram of a reflection point.

[0018] Figure 4 It is a contrast schematic diagram of the roundest iris method, the roundest pupil method, and the light spot method. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0020] The first problem to be solved is how to determine the eye center. The existing method for determining the eye center generally adopts the intersection method of the optical axis, which lacks robustness. The eye movement has the characteristic that the eye center remains stationary. When the eye is looking at the camera, several infrared light sources (such as a ring structure) around the camera are symmetrically arranged with the center of the camera to form Purkinje reflection points (hereinafter referred to as reflection points), and the center of the overall pattern formed by the reflection points is located at the center of the pupil. The processes St10 and St20 of the present application are to obtain the minimum or minimum value position as the center of the eye through variance analysis of the distance between the reflection points and the pupil center.

[0021] As shown in Figure 1 An iris segmentation method using Purkinje reflection and dynamic three-dimensional model, which comprises the following processes:

[0022] St10, several infrared light sources symmetrically arranged with the center of the camera are arranged around the camera, and several frames of eye images of the patient are obtained from the video sequence captured by the camera. Specifically, a video sequence of the user rotating the eyeball within a certain range is collected, and multiple frames of eye images containing the first Purkinje reflection point are obtained.

[0023] St11, pre-processing is performed on each frame of eye image, which includes grayscale, brightness enhancement and high light removal operations. After pre-processing, it is easier to extract the Purkinje reflection point. The specific extraction process is: calculating the image gradient, using threshold segmentation method to detect the highlight area, extracting the possible Purkinje reflection point, then clustering the detected Purkinje reflection point, and positioning the position of the Purkinje reflection point. The Purkinje reflection point is used to determine the eye center, which requires that the light source be symmetrical, fast and simple, and robust. The Purkinje reflection point image is the reflection image of the optical system of the eye on different surfaces of the infrared light source. There are usually four Purkinje images: the first Purkinje image is the reflection of the corneal anterior surface; the second Purkinje image is the reflection of the corneal posterior surface; the third Purkinje image is the reflection of the anterior surface of the lens; and the fourth Purkinje image is the reflection image of the posterior surface of the lens, which is an inverted image. The relative brightness of the Purkinje image can be calculated by the Fresnel reflection equation. The eye can see P3 and P4. The light reflected from the lens can be reflected by the back of the cornea in turn, and these images are "internal viewing phenomenon", and some eye locators use P1 and P4 to measure the position of the eye. The present application mainly analyzes the Purkinje reflection point in the first Purkinje image.

[0024] The specific image preprocessing process is that the input eye image is grayed, the highlight area is enhanced using top-hat transformation, the uneven illumination effect is removed, then the glint is extracted by luminosity or gradient, the glint is extracted by gradient which is more robust, but the internal cavity needs to be eliminated by glint scaling (Dilation and Erosion).

[0025] St20, an eye coordinate system is established, Purkinje reflection points in the eye image are extracted, and the positions of the Purkinje reflection points are located based on the eye coordinate system, the three-dimensional coordinates of the eyeball center are calculated by using the positions of the Purkinje reflection points, combining the known infrared light source position and camera parameters, and adopting a geometric optical model. For each frame of image in the video sequence, the Purkinje reflection points in the eye image are extracted, and the center of the independent reflection point pattern and the pupil center are detected. Here, the camera coordinates, reflection point coordinates, and eyeball center, iris center, etc. coordinates are involved, so the establishment of the spatial coordinate system is very important. Because the relationship between each point is known, in order to facilitate calculation, a camera-centered coordinate system, an eyeball-centered coordinate system and a head-centered coordinate system can be established respectively, and the coordinates of each important point can be converted between the three coordinate systems.

[0026] As shown in Figure 2 and Figure 3 , the calculation of the three-dimensional coordinates of the eyeball center includes the following processes:

[0027] St21, all reflection points are extracted, and the pupil center is extracted, the distance between the pupil center and each reflection point is calculated, and the distance variance is calculated.

[0028] St22, the smaller the distance variance is, the closer to the eyeball center position, when the distance variance between the reflection point and the pupil center is 0 or less than a preset threshold, the eyeball center position is determined.

[0029] St23, the iris boundary is obtained at the position of the eyeball after the eyeball center is determined, the circle is fitted, and the iris radius is determined.

[0030] Because when the eye is looking at the camera, the center of the figure formed by the center-symmetric reflection points is located at the center of the pupil, the optical axis passes through the center of the eyeball and the center of the pupil, and it is determined that the optical axis does not deviate from the direction of the camera, and the position of the eyeball center is determined. The present application analyzes whether the optical axis deviates from the direction of the camera through the overall morphological characteristics of the reflection points and the distance variance between the reflection points and the center of the pupil, thereby determining the position of the eyeball center. The calculation of the optical axis can refer to the existing patent with publication number CN118736168A, and the vector from the eyeball center to the iris center (also the center of the pupil) is used as the optical axis. The optical axis passes through the center of the eyeball, and thus the position of the eyeball center is determined through the center of the pupil. The Purkinje reflection points are obtained by irradiating the eyeball with infrared light sources arranged in a center-symmetric manner around the camera, and when the eye is looking at the camera, the reflection points formed by the infrared light sources irradiating the eyeball are also center-symmetric, and the center of the figure formed by the reflection points is located at the center of the pupil. Because the optical axis passes through the center of the eyeball, the position of the eyeball center is determined. The multiple light spots formed by the several infrared light sources arranged around the camera on the eyeball form a ring structure or other regular structure, and when the eye is looking at the camera, the center of the light source is located at the center of the pupil, and the optical axis passes through the center of the eyeball, thereby determining the position of the eyeball center. For example, the present application forms a regular quadrilateral structure by four reflection points at fixed positions. The centroid of the 4 reflection points is: [(953, 468), (950, 491), (1004, 469), (1001, 492)], and the overall center coordinates of the four reflection points are: [977 480], Figure 2 and Figure 3 The center of the figure formed by the reflection points in the above formula coincides with the center of the pupil, and the distance variance between the 4 center points and the center of the pupil is the smallest.

[0031] The eyeball radius is determined as 2 times the iris radius according to prior knowledge.

[0032] The three-dimensional model parameters of the eyeball are determined, including the eyeball center, the iris radius, and the eyeball radius.

[0033] According to the position of the iris center, the spatial pose of the eyeball and the three-dimensional space iris circle equation can be determined, thereby establishing an initial three-dimensional eyeball model, and the three-dimensional eyeball model is used to realize iris segmentation based on three-dimensional eyeball model parameters.

[0034] The Purkinje reflection points may be incomplete, and the algorithm for calculating the distance between the center of the Purkinje spot and the center of the pupil lacks robustness. The present application calculates the distance between the center of each frame and the center of the pupil to form a distance set, calculates the variance of the distance set, and the smaller the distance variance, the closer to the position of the eyeball center. According to the consistency of the center of the reflection points and the center of the pupil, the determination of the eyeball center is realized, which is simple, robust and accurate.

[0035] AsFigure 4 As shown, the same video is divided into 5 segments, and the positions of the eyeball centers obtained by different methods are analyzed and compared, including the most circular pupil method, the most circular iris method, and the spot method (spot and pupil center minimum variance) of the present application. Compared with the most circular pupil method and the most circular iris method, the spot method can stably obtain an accurate eyeball center.

[0036] St30, an initial three-dimensional eyeball model is established based on the three-dimensional coordinates of the eyeball center and the preset iris radius.

[0037] St40, the positions of the medial and lateral canthi of the patient are detected, the three-dimensional coordinates and attitude of the eyeball center are adjusted accordingly according to the changes in the positions of the medial and lateral canthi, and the three-dimensional eyeball model is updated. Specifically, for a head-mounted camera device, the relative positions of the camera and the eyeball center remain unchanged, but may change relatively due to sliding. For a non-head-mounted camera device, the relative positions of the camera and the eyeball center will change, but the relative positions can also remain unchanged within a relative time range. The relative positions of the camera and the eyeball center are monitored according to the positions of the medial and lateral canthi, and the eyeball center is dynamically updated, including synchronous spatial rotation or re-determining the eyeball center using Purkinje reflection. For a non-head-mounted device, a video is collected, and a three-dimensional eyeball model can also be established using a segment with relatively unchanged canthus positions. The position of the eyeball center is adjusted according to the changes in the canthus positions. This step realizes the monitoring of the medial and lateral canthus positions and the dynamic adjustment of the model: for a non-head-mounted camera device, the relative positions of the camera and the head generally remain unchanged within a relative time, and the positions of the medial and lateral canthi in the image should be basically stable. The positions of the medial and lateral canthi can be detected using a key point detection algorithm (such as face key point detection of Dlib or OpenCV). When the positions of the medial and lateral canthi are detected to change, it is determined that the head attitude or the camera position has changed. The position of the eyeball center and the attitude are adjusted accordingly according to the changes in the canthus positions, and the three-dimensional eyeball model is updated. The effect of this process is to ensure the accuracy of the three-dimensional model by dynamically adjusting the position of the eyeball center, thereby improving the robustness of the segmentation.

[0038] St41, the positions of the medial and lateral canthi are detected using a key point detection algorithm. Specifically, the positions of the medial and lateral canthi are monitored using a face key point detection algorithm, and the threshold value of the position change is set according to experiments.

[0039] St50, in the updated three-dimensional eyeball model, the iris radius is obtained based on the eyeball center, and a three-dimensional iris circle equation is established.

[0040] St51, sample the edge gradient response with different radius according to the eye center, obtain the gradient response curve, and detect multiple peaks to form the iris radius candidate set. The process of gradient response curve acquisition: sample the edge gradient with different radius along the eye center, and obtain the gradient response curve.

[0041] St52, select the corresponding iris radius to establish the three-dimensional iris circle equation according to the requirement. The gradient response curve usually has multiple peaks, indicating the possible iris boundary position. The specific multi-peak detection is: use a smoothing filter to process the curve, detect multiple peaks, and record the corresponding iris radius. Record the iris radius corresponding to multiple peaks to form the iris radius candidate set. According to the actual application requirement (such as paying more attention to the inner boundary or the outer boundary of the iris), select the appropriate iris radius, such as the radius corresponding to the first peak. The beneficial effect is: this multi-peak selection strategy allows flexible adjustment according to different application scenarios, improving the applicability of the method.

[0042] St60, project the three-dimensional iris circle to the two-dimensional image plane to obtain the iris ellipse parameters and generate the iris segmentation result. The present application constructs a dynamic three-dimensional model, directly detects the position of the iris center by combining deep learning, determines the three-dimensional spatial pose of the eyeball and the three-dimensional spatial model of the iris according to the iris center and the three-dimensional model of the eyeball, and obtains the iris ellipse projection and the ellipse parameters. According to the change of the inner and outer canthus positions, the eyeball center position is dynamically adjusted, and the iris ellipse segmentation result of the three-dimensional circle projection of the iris is obtained. This method improves the accuracy and robustness of iris segmentation, and is especially suitable for iris segmentation under non-ideal conditions.

[0043] St61, use the eyeball center and iris radius of the updated three-dimensional eyeball model to establish the iris circle equation in three-dimensional space.

[0044] St62, project the three-dimensional iris circle to the two-dimensional image plane through the perspective projection model to obtain the parameters of the iris ellipse.

[0045] St63, generate the segmentation result of the iris region in the image according to the iris ellipse parameters obtained by projection. The parameters of the iris ellipse include center coordinates, long and short axis lengths, and rotation angles. The iris ellipse parameters can be obtained by using existing methods or referring to the method in the publication No. CN118736168A, an iris accurate unfolding method based on 3D geometric transformation.

[0046] St70, use the edge gradient information of the image to optimize the edge position along the iris ellipse boundary, thereby improving the segmentation accuracy. The segmentation result is optimized by image gradient refinement.

[0047] St71, detecting the position of the occlusion area, removing the iris part of the occlusion area, and making the segmentation result conform to the actual visible iris area. The occlusion area is an occlusion site such as an eyelid, an eyelash, or the like.

[0048] St72, performing morphological processing on the segmentation result. Through the occlusion processing and morphological operation, the effect of smoothing the boundary and eliminating noise is achieved.

[0049] St80, collecting eye images including various eye poses and illumination conditions, labeling the iris center position, and constructing a training data set. Specifically, a supervised learning method is used for training. This method can construct labels including the eye center, the iris ellipse, and implicit parameters including the iris radius, the iris center, the eye radius, the inner canthus, and the outer canthus. Thus, a deep learning model can be trained to predict the iris center, the eye center, and the iris ellipse according to different requirements. For a head-mounted device, only the iris center needs to be predicted, and the iris segmentation can be realized by combining the eye three-dimensional model.

[0050] For a non-head-mounted device, the iris center and the eye three-dimensional model parameters can be predicted to realize iris segmentation.

[0051] St81, training a deep learning model to enable the deep learning model to directly predict the eye center, the iris radius, and the iris center parameters in the input image.

[0052] St82, establishing a three-dimensional space optical axis and an iris ellipse according to the eye center and the iris center predicted by the deep learning model to realize iris segmentation. The iris center predicted by the deep learning model is compared and fused with the eye center estimated by Purkinje reflection, thereby improving the accuracy of eye center positioning. This process uses data preparation and model training to obtain a deep learning model, so that the eye model parameters can be quickly and directly detected according to the eye image of a patient, a three-dimensional space optical axis and an iris ellipse are established according to the eye center and the iris center, and the efficiency is greatly improved.

[0053] The beneficial effects and core innovations of the present application include: 1. Estimating the eye parameters by Purkinje reflection, which is the core and foundation of the present application. By accurately estimating the eye center, the basic accuracy of iris segmentation is improved.

[0054] 2. Establishment and adjustment of dynamic three-dimensional model: by detecting the change of the inner and outer canthus positions, the eye center position is dynamically adjusted, so that the three-dimensional model can adapt to the change of the head pose and the camera position, realizing the adaptability to the change of the head pose and the eye movement, enhancing the robustness of the method, and avoiding the limitation of the fixed model in the existing patents.

[0055] 3. The present method can construct a three-dimensional optical axis according to the iris center and the eye center to realize three-dimensional gaze tracking.

[0056] 4. Flexible selection of iris radius: The selection strategy of multi-peak iris radius selects the appropriate iris radius according to actual needs, improving the accuracy and applicability of the segmentation results.

[0057] 5. Fusion of deep learning and physical model: Combining deep learning to directly detect the position of the iris center and the estimated eyeball center by Purkinje reflection, the advantages of data-driven and physical model are fully utilized to improve the accuracy of iris segmentation.

[0058] 6. Accurate iris segmentation results: Through the projection based on the three-dimensional model and the optimization of the segmentation results, accurate iris segmentation is obtained, which is suitable for iris recognition applications under non-ideal conditions.

[0059] 7. By training a deep learning model to predict the eyeball center and the iris center, it can be applied to videos collected by non-head-mounted devices and head-mounted devices at the same time.

[0060] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments only, any technical solution falling within the scope of the present application should be considered as falling within the protection scope of the present application. It should be noted that for ordinary skilled persons in the art, some improvements and refinements without departing from the principles of the present application should also be considered as falling within the protection scope of the present application.

Claims

1. An iris segmentation method using Purkinje reflections and dynamic three-dimensional model, characterized by, The method comprises the following steps: St10, a plurality of infrared light sources are arranged around the camera in a center-symmetrical manner with respect to the camera, and a plurality of frames of eye images with reflection points are obtained from a video sequence captured by the camera; St20, an eye coordinate system is established, the reflection points in the eye images are extracted, and the positions of the reflection points are located based on the eye coordinate system; the positions of the reflection points are used, combined with known positions of the infrared light sources and camera parameters, a geometric optical model is used to calculate three-dimensional coordinates of the eyeball center; The calculation of the three-dimensional coordinates of the eyeball center comprises the following steps: St21, all reflection points are extracted, and the center of the pupil is extracted; the distance between the center of the pupil and each reflection point is calculated, and the distance variance is calculated; St22, the smaller the distance variance is, the closer the reflection point is to the position of the eyeball center; when the distance variance between the reflection point and the center of the pupil is 0 or less than a preset threshold, the position of the eyeball center is determined; St23, the iris boundary is obtained at the position of the eyeball center after the eyeball center is determined, a circle is fitted, and the iris radius is determined; St30, an initial three-dimensional eyeball model is established based on the three-dimensional coordinates of the eyeball center and the preset iris radius; St40, the positions of the medial canthus and the lateral canthus of the patient are detected, the three-dimensional coordinates and the posture of the eyeball center are adjusted accordingly according to the changes in the positions of the medial canthus and the lateral canthus, and the three-dimensional eyeball model is updated; St50, in the updated three-dimensional eyeball model, the iris radius is obtained based on the eyeball center, and a three-dimensional iris circle equation is established; St60, the three-dimensional iris circle is projected onto a two-dimensional image plane to obtain iris ellipse parameters, and a segmentation result of the iris is generated.

2. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 1, wherein, The method further comprises the following steps: St11, each frame of eye image is preprocessed, and the preprocessing comprises grayscale, brightness enhancement and high light removal operations.

3. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 1, wherein, The method further comprises the following steps: St41, a key point detection algorithm is used to detect the positions of the medial canthus and the lateral canthus.

4. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 3, wherein, The method further comprises the following steps: St51, the edge gradient response is sampled at different radii based on the eyeball center to obtain a gradient response curve, and a plurality of peak values are detected to form an iris radius candidate set; St52, according to requirements, a corresponding iris radius is selected to establish a three-dimensional iris circle equation.

5. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 4, wherein, The method further comprises the following steps: St61, the three-dimensional eyeball model after the update is used to establish an iris circle equation in a three-dimensional space based on the eyeball center and the iris radius; St62, the three-dimensional iris circle is projected onto a two-dimensional image plane through a perspective projection model to obtain parameters of an iris ellipse; St63, a segmentation result of an iris region is generated in the image according to the parameters of the projected iris ellipse.

6. The iris segmentation method using Purkinje reflections and dynamic 3D model according to any one of claims 1-5, characterized in that, The method further comprises the following steps: St70, the edge position is optimized along the boundary of the iris ellipse using edge gradient information of the image, so as to improve the segmentation accuracy.

7. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 6, wherein, The method further comprises the following steps: St71, a shielding area is detected, and the iris part in the shielding area is removed, so that the segmentation result conforms to the actual visible iris area; St72, the segmentation result is subjected to morphological processing.

8. The iris segmentation method using Purkinje reflections and dynamic 3D model according to claim 6, wherein, The method further comprises the following steps: St80, eye images including a plurality of eyeball postures and illumination conditions are collected, the positions of the iris centers are labeled, and a training data set is constructed; St81、Training a deep learning model, enabling the deep learning model to directly predict the eyeball model parameters in the input image, including the eyeball center, the iris center and the iris radius; St82、According to the eyeball center and the iris center predicted by the deep learning model, establishing a three-dimensional space optical axis and an iris ellipse, and realizing iris segmentation.

Images