An iris or pupil segmentation method based on target detection

Abstract

This invention claims protection for an iris or pupil segmentation method based on object detection, belonging to the field of biometrics. It includes the following steps: St10, acquiring training data, which includes an eye image and corresponding oriented bounding box (OBB) labels, with the labels used to annotate elliptical regions; St20, using the training data to train an object detection network with OBB prediction capabilities to obtain a segmentation network model; St30, using the segmentation network model to perform forward inference on the target eye image, outputting predicted OBBs, and performing inverse solving based on the geometric parameters of the predicted OBBs to directly obtain the geometric parameters of the iris or pupil ellipse, thus achieving segmentation of the target eye image. It innovatively constructs a fully closed-loop, minimalist architecture of "image input—oriented bounding box (OBB) object detection—geometric parameter vector inverse ellipse solving," achieving iris / pupil parameter-level extraction that balances high performance, low latency, and strong robustness.

Description

Technical Field

[0001] This invention relates to the field of biometrics, and more specifically, to a method for iris or pupil segmentation based on target detection. Background Technology

[0002] Iris recognition is characterized by its high security in the field of biometrics, and accurate iris segmentation is crucial. Early iris segmentation methods mainly relied on traditional image processing techniques such as edge detection or pixel classification. Although these traditional methods performed reasonably well when processing high-quality images under controlled conditions, their geometric assumptions were easily invalidated when processing low-quality iris images, such as those with blurriness, uneven lighting, or occlusion, leading to drastic fluctuations in fitting accuracy.

[0003] In recent years, dense pixel-level semantic segmentation networks, represented by U-Net, have made breakthroughs in anti-interference capabilities. However, their large encoder-decoder architecture leads to high computational redundancy and memory overhead. For real-time eye-tracking tasks that only require the extraction of elliptical geometric parameters, predicting pixel-level fine contours constitutes a fundamental resource mismatch, making it extremely difficult to achieve high frame rates (>120fps) in real-time deployment on portable bedside medical terminals or edge chips (such as Kendryte K230) with limited computing power.

[0004] To avoid the overhead of dense prediction, some techniques attempt to use global parameter regression methods, directly outputting elliptic parameters from deep fully connected layers of the network. However, these methods often contain a large number of non-standard operators such as spatial soft attention and differentiable mask rendering, which cannot be effectively compiled and hardware accelerated by general industrial-grade inference frameworks (such as TensorRT, NCNN, etc.).

[0005] In summary, existing iris / pupil segmentation technologies have consistently struggled to achieve an ideal balance across the three core dimensions of high precision, strong robustness, and zero-computing-power-redundancy deployment at edge terminals. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a technical solution for iris or pupil segmentation based on target detection. This solution abandons traditional dense pixel prediction or cumbersome post-processing fitting algorithms, and innovatively constructs an "image input" method. Oriented bounding box (OBB) object detection The fully closed-loop minimalist architecture of "inverse ellipse derivation using geometric parameter vectors" incorporates an optional self-supervised edge gradient snapping mechanism during training, achieving iris / pupil parameter-level extraction that balances high performance, low latency, and strong robustness.

[0007] An iris or pupil segmentation method based on target detection includes the following steps:

[0008] St10. Obtain training data. The training data includes eye images and corresponding directional bounding box labels. The directional bounding box labels are used to label the elliptical regions where the iris or pupil is located in the eye images.

[0009] St20. Using the training data, train the target detection network with the ability to predict directional bounding boxes to obtain the segmentation network model;

[0010] St30. Input the target eye image to be segmented into the segmentation network model, obtain the predicted oriented bounding box output by the segmentation network model, and determine the geometric parameters of the predicted elliptical contour of the iris or pupil in the target eye image based on the geometric parameters of the predicted oriented bounding box to achieve segmentation.

[0011] In summary, the above technical solution has the following beneficial effects: This application first uses the coordinates of the four vertices of the iris ellipse oriented bounding box as model training labels, and trains the model by combining the eye image and its corresponding labels, so that the model can predict the oriented bounding box of the iris ellipse. Then, the iris ellipse parameters are calculated based on the four vertices of the oriented bounding box, and finally the iris is accurately segmented based on the iris ellipse parameters. The beneficial effects of this application include: 1. Abandoning the traditional iris segmentation method that relies on edge detection or pixel classification, it adopts a new technical route based on deep learning object detection, with object detection algorithms (such as the YOLO algorithm) as the core, combined with custom parameter training and mathematical relationship verification functions, effectively overcoming the limitations of traditional methods in processing low-quality iris images, and achieving more accurate and faster iris segmentation. 2. It can not only quickly and accurately locate the iris center, but also accurately extract all key parameters of the iris ellipse, including the major axis, minor axis and rotation angle, which improves the iris recognition speed and significantly improves the accuracy, providing more comprehensive and reliable data support for biometrics and security authentication, and achieving a major breakthrough in data integrity and recognition performance compared with traditional methods. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a process for an iris or pupil segmentation method based on target detection;

[0013] Figure 2 This is a flowchart illustrating an iris or pupil segmentation method based on object detection.

[0014] Figure 3 This is a schematic diagram of label calculation for an iris or pupil segmentation method based on object detection;

[0015] Figure 4This is a schematic diagram of a network model for an iris or pupil segmentation method based on object detection. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figures 1-4 As shown, a fast and accurate iris / pupil segmentation method based on target detection includes the following core processes:

[0018] St10. Obtain training data, which includes eye images and corresponding directional bounding box labels. The directional bounding box labels are used to label the elliptical regions where the iris or pupil is located in the eye image.

[0019] Preferably, the oriented bounding box label in process St10 is obtained by establishing a spatial geometric mapping relationship between the parameters of the two-dimensional planar iris / pupil ellipse and the vertex coordinates of the oriented bounding box. The transformation method includes, but is not limited to: using the translation vector and rotation matrix constructed from the center coordinates, semi-lengths of the major and minor axes and the inclination angle of the principal axis to perform affine transformation, and calculating the vertex coordinate vector of the oriented bounding box.

[0020] St20. Using the training data, train the target detection network with directional bounding box prediction capability to obtain the segmentation network model.

[0021] like Figure 4 As shown, the model training uses the YOLOv8n-OBB model, which includes a backbone network, a neck network, and a head network. Specifically, the YOLOv8n-OBB model comprises a backbone network, a neck network, and a head network. The backbone network extracts features through a series of convolutional and deconvolutional layers and uses residual connections and bottleneck structures to reduce the network size and improve performance. The neck network is responsible for multi-scale feature fusion and enhancement to improve the accuracy and robustness of object detection. The head network further processes the feature maps to generate the specific information required for final object detection, including category, location, and confidence.

[0022] The total loss function used in the training process includes a first loss term and a second loss term. The first loss term is used to constrain the geometric consistency between the predicted oriented bounding box and the oriented bounding box label, and the second loss term is used to drive the predicted elliptical contour to snap to the real physical edge in the eye image.

[0023] The segmentation network model is trained using a two-stage progressive training strategy. After the prediction accuracy of the first loss term of the segmentation network model reaches a preset threshold, a second loss term, namely the directional edge gradient loss function, is introduced into the total loss function to participate in the joint optimization of multiple loss functions.

[0024] The second loss term is constructed based on the following method: St21, obtaining the image edge intensity of multiple sampling points on or in the neighborhood of the predicted ellipse contour;

[0025] St22, calculate a statistical value based on the image edge intensity of multiple sampling points;

[0026] St23. Construct a loss value that is negatively correlated with the statistical value, so that the loss value is smaller when the predicted ellipse profile coincides with the actual physical edge.

[0027] Specifically, the statistical calculations in St22 include calculating the mean, median, or quantile of the image edge intensity at multiple sampling points. Preferably, outliers in the image edge intensity at multiple sampling points are truncated or suppressed before calculating the statistical values.

[0028] St30. Input the target eye image to be segmented into the segmentation network model, obtain the predicted oriented bounding box output by the segmentation network model, and determine the geometric parameters of the predicted elliptical contour of the iris or pupil in the target eye image based on the geometric parameters of the predicted oriented bounding box to achieve segmentation.

[0029] As a preferred method, in St30, the geometric parameters of the predicted elliptical profile are obtained by directly solving in reverse by establishing the orthogonal edge vector relationship or algebraic analytical equation of the predicted oriented bounding box. The specific process of reverse solving is as follows: the geometric centroid of the four vertex coordinate vectors of the predicted oriented bounding box is used to extract the coordinates of the ellipse center, and the L2 norm and arctangent function of the orthogonal edge vectors formed by the vertex coordinate vectors are used to extract the semi-length of the major and minor axes and the rotation tilt angle.

[0030] Compared with the prior art, the present invention has the following significant advantages:

[0031] 1. Breaking down the barriers of "large models," this invention establishes a simplified and efficient parameter regression closed loop. It reduces the complexity of iris / pupil segmentation tasks into OBB regression tasks with rotation awareness. By removing the heavy decoder branch from traditional dense semantic segmentation networks, it significantly reduces the number of parameters and computational overhead.

[0032] 2. A two-way integration of knowledge-guided and data-driven approaches. This approach incorporates an optional directional edge gradient loss function during training, enabling the model to not only regress the macroscopic anatomical relative positions of its components through a large number of labels during backpropagation, but also to generate an "active attraction" effect on edge pixels in the underlying image, thus breaking down the barriers between deep learning networks and traditional computer vision features.

[0033] 3. A unique statistical quantile truncation mechanism achieves natural immunity to infrared reflective spots. In near-infrared eye-tracking images, the strong spots inevitably generated by the light source on the cornea can easily lead to explosive conventional gradient loss. This invention introduces a dynamic quantile upper bound truncation to adaptively remove abnormally high gradient noise, retaining only weak edges that conform to natural anatomical boundaries, significantly improving segmentation stability in harsh clinical environments.

[0034] 4. Two-stage progressive scheduling, balancing global convergence and micro-tuning. Pixel-level gradients contain numerous complex local extrema in eye images (such as eyelashes and eyelid shadows). This approach utilizes the native loss of the detection network for macro-regression in the early stages of training. Once the network enters the "trust zone," edge gradient fine-tuning is introduced, ensuring the stability of model optimization and significantly accelerating the overall convergence of the model.

[0035] 5. Zero inference overhead at the deployment end. The edge gradient loss and differentiable isometric sampling operator exist only in the backpropagation during the training phase, used to reshape the feature weights of the network convolutional kernels. During the final deployment of forward inference, this loss branch is completely stripped away, without adding any additional computing power to the edge device, perfectly adapting to the requirements of real-time low-power deployment.

[0036] Example 1: Data Preparation and Forward Mapping of Oriented Bounding Box Labels

[0037] In process St10, in order to provide high-precision rotational supervision labels to the target detection network, it is necessary to establish a spatial geometric mapping relationship between the two-dimensional planar iris or pupil ellipse parameters and the coordinates of the oriented bounding box (OBB) vertex.

[0038] As a preferred embodiment of the spatial geometric mapping relationship transformation in this application, St11 assumes that five parameters of the iris or pupil ellipse on the two-dimensional plane of the eye have been given or obtained. ,in With the center coordinates, and These are the half-lengths of the major and minor axes, respectively. Let be the rotation angle of the principal axis of the two-dimensional ellipse.

[0039] This embodiment employs an affine transformation that decouples the translation vector from the two-dimensional rotation matrix for transformation calculation. St12: First, based on the ellipse's center and tilt angle, a corresponding two-dimensional center translation vector is constructed. With two-dimensional rotation matrix :

[0040]

[0041] St13. In the local canonical space with the center of the ellipse as the origin, the four tangent vertices of the oriented bounding box are naturally bounded by the semi-lengths of the major and minor axes. The four basic vertex vectors are defined as follows:

[0042]

[0043] St14. Using the spatial affine transformation equation, the vertex vectors in the local normal space are mapped globally to the global image coordinate space, and the two-dimensional coordinate vectors of the four vertices of the global oriented bounding box are rigorously calculated. :

[0044]

[0045] The calculated vertex coordinates are used as the oriented bounding box labels for the object detection network (such as the YOLO-OBB architecture with rotation prediction capabilities). It should be noted that the forward transformation relationship in this embodiment includes, but is not limited to, the matrix multiplication form described above, and any equivalent scalar explicit equation system obtained from the algebraic expansion of the above geometric structure is also applicable.

[0046] Example 2: Calculation of optional directional edge gradient loss function and anti-spotting mechanism

[0047] In process St20, the network model employs a rotated target detector architecture. To improve edge-fitting accuracy, a directional edge gradient loss function can be selectively introduced into the total loss function. Its differentiable forward computation flow within the deep learning framework is as follows:

[0048] Double Ellipse Construction and Differentiable Sampling: Let the ellipse parameters corresponding to the oriented bounding box output of the current prediction iteration of the network model be... To extract the radial edge intensity across anatomical boundaries from discrete pixels, this embodiment sets scaling factors on the inner and outer sides of the predicted elliptical contour, respectively. and The inner sampling ellipse and the outer sampling ellipse are constructed.

[0049] Adaptive Scale Iso-Sampling: To ensure the model's scale invariance when the pupil undergoes drastic scaling (such as light reflection), the model dynamically calculates the perimeter of the currently predicted ellipse profile using the Ramanujan ellipse perimeter approximation formula. Based on a given physical sampling step size Pixels, adaptively determining the total number of sampling points for the current contour Using equidistant angular vectors According to the affine transformation matrix structure in Example 1, corresponding inner sampling grids and outer sampling grids are generated on the input eye grayscale image.

[0050] Differentiable difference computation: Using a bilinear interpolation sampling operator with perfect backpropagation gradient (such as PyTorch's F.grid_sample), the set of pixel gray values ​​corresponding to the inner and outer sampling grids in the image is extracted. and The original edge feature intensity at each sampling point is calculated using signed differencing.

[0051]

[0052] Statistical distribution constraints and spot removal: For this original set of edge intensities Dynamically extract the preset high quantile value from the current single sample image. As a statistical upper bound, an absolute hard upper bound threshold is also set. Together, they determine the current effective cutoff threshold. .

[0053] The set that is higher than Abnormally high gradient points (caused by light spot noise caused by direct infrared light shining on the surface of the eyeball) are forcibly truncated and suppressed to For those below Natural, inaccurate, or blurred edges are fully preserved, thus outputting a robust set of edge feature strengths. .

[0054] Loss function penalty construction: Calculate the arithmetic mean of the robust edge feature strengths across all sampled points. A directional marginal gradient loss function is constructed using a negative logarithmic smoothing mapping function. This function provides a strong penalty gradient when edges do not fit, driving network parameters to snap towards pixel gradient extrema.

[0055] In this embodiment, a two-stage progressive optimization strategy is preferably adopted in the training process: In the first training stage (global geometric regression stage), the weight coefficient of the edge gradient loss function is set to zero, and the network's native bounding box loss function is used to guide the model to converge quickly; In the second training stage (local edge adsorption stage), when the network's predicted intersection-over-union (IoU) reaches a preset threshold (such as 0.75), the edge gradient loss is introduced to participate in the joint optimization of multiple loss functions, which avoids the network getting stuck in local extrema in non-iris / pupil regions.

[0056] Example 3: Inference and Prediction: Backward Solving of Oriented Bounding Box to Geometric Ellipse Parameters

[0057] In process St30, in order to ensure the efficiency of real-time parallel computing on edge SoCs (such as K230) with limited computing power, this invention abandons the traditional cumbersome coordinate discussion in the forward inference stage and realizes the reverse solution entirely based on the centroid and norm mechanism of the feature vector.

[0058] St31. Suppose that the four vertices of the predicted oriented bounding box output by the forward inference of the network model are represented as two-dimensional point vectors. .

[0059] St32, Calculation of the center vector of the predicted ellipse profile: Calculation of the center coordinate vector of the predicted ellipse profile. The geometric centroids corresponding to the four vertex vectors of the predicted oriented bounding box are obtained by summing and averaging:

[0060]

[0061] St33, Orthogonal Edge Vector Extraction: Constructing two mutually orthogonal adjacent edge vectors of the predicted oriented bounding box. and :

[0062]

[0063] St34, Extraction of semi-major and minor axes: Predicting the semi-major axis of the ellipse profile. and short axis half length The maximum and minimum values ​​of the L2 norm (Euclidean length) of the two naturally orthogonally adjacent edge vectors are correlated and extracted through adaptive comparison:

[0064]

[0065] St35, Principal Axis Rotation Tilt Angle Calculation: Select the edge vector with the longer predicted orientation bounding box length as the principal axis vector. Using the two-parameter arctangent function The angle between the principal axis vector and the horizontal coordinate axis of the image is directly calculated, which is the predicted rotation angle of the principal axis of the ellipse contour. :

[0066]

[0067] in and These represent the scalar components of the principal axis vector on the X and Y axes in the global image coordinate system, respectively.

[0068] Based on the 5 parameters calculated above The system directly performs high-precision mathematical fitting and final segmentation of the target eye image region.

[0069] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for iris or pupil segmentation based on target detection, characterized in that, Includes the following processes: St10. Obtain training data, which includes an eye image and a directional bounding box label corresponding to the eye image. The directional bounding box label is used to mark the elliptical region where the iris or pupil is located in the eye image. St20. Using the training data, train the target detection network with directional bounding box prediction capability to obtain a segmentation network model; St30. Input the target eye image to be segmented into the segmentation network model, obtain the predicted directional bounding box output by the segmentation network model, and determine the geometric parameters of the predicted elliptical contour of the iris or pupil in the target eye image based on the geometric parameters of the predicted directional bounding box to achieve segmentation.

2. The iris or pupil segmentation method based on target detection according to claim 1, characterized in that, The total loss function used in the training process includes a first loss term and a second loss term. The first loss term is used to constrain the geometric consistency between the predicted oriented bounding box and the oriented bounding box label, and the second loss term is used to drive the predicted elliptical contour to snap to the real physical edge in the eye image.

3. The method for fast and accurate iris segmentation based on target recognition according to claim 2, characterized in that, The second loss term is constructed based on the following method: St21, obtaining the image edge intensity of multiple sampling points on or in the neighborhood of the predicted ellipse contour; St22. Calculate a statistical value based on the image edge intensity of the multiple sampling points; St23. Construct a loss value that is negatively correlated with the statistical value, such that the higher the degree of overlap between the predicted elliptical contour and the real physical edge, the smaller the loss value.

4. The method for fast and accurate iris segmentation based on target recognition according to claim 3, characterized in that, The statistical values ​​calculated in St41 include: Calculate the mean, median, or quantile of the image edge intensity at the multiple sampling points.

5. The iris or pupil segmentation method based on target detection according to claim 1, characterized in that, The directional bounding box label described in process St10 is obtained through the following process: the directional bounding box label is obtained by establishing a spatial geometric mapping relationship between the parameters of the two-dimensional planar iris or pupil ellipse and the coordinates of the directional bounding box vertex. The spatial geometric mapping relationship is as follows: using the translation vector and rotation matrix constructed from the coordinates of the ellipse center, the semi-lengths of the major and minor axes and the inclination angle of the principal axis, a spatial affine transformation is performed to calculate the coordinate vectors of the four vertices of the oriented bounding box.

6. The iris or pupil segmentation method based on target detection according to claim 5, characterized in that, In process St30, the geometric parameters of the predicted elliptical contour are obtained by directly solving in reverse by establishing the orthogonal edge vector relationship or algebraic analytical equation of the predicted oriented bounding box. The specific process of the reverse calculation is as follows: the geometric centroids of the four vertex coordinate vectors of the predicted oriented bounding box are used to extract the coordinates of the ellipse center, and the L2 norm and arctangent function of the orthogonal edge vectors formed by the vertex coordinate vectors are used to extract the semi-lengths of the major and minor axes and the rotation tilt angle.

7. The iris or pupil segmentation method based on target detection according to claim 1, characterized in that, Process St10 includes: St11, Obtain 5 parameters of the iris or pupil ellipse on a two-dimensional plane of the eye. ,in The coordinates of the iris or pupil center. and These are the semi-lengths of the major and minor axes of the two-dimensional ellipse, respectively. The rotation angle of the principal axis of the two-dimensional ellipse; St12. Based on the center and tilt angle of the ellipse, construct the corresponding two-dimensional center translation vector. With two-dimensional rotation matrix : ； St13. In the local canonical space with the center of the ellipse as the origin, the four tangent vertices of the oriented bounding box are naturally enclosed by the semi-lengths of the major and minor axes, and the four basic vertex vectors are defined as follows: ； St14. Using the spatial affine transformation equation, the vertex vectors in the local normal space are mapped globally to the global image coordinate space, and the coordinate vectors of the four vertices of the global oriented bounding box are calculated. : The calculated coordinates of the four vertices are used as the directional bounding box labels for the object detection network.

8. The iris or pupil segmentation method based on target detection according to claim 7, characterized in that, Process St30 includes: St31. Suppose that the four vertices of the predicted oriented bounding box output by the forward inference of the network model are represented as two-dimensional point vectors. ; St32, Calculation of the center vector of the predicted ellipse profile: Calculation of the center coordinate vector of the predicted ellipse profile. The geometric centroids corresponding to the four vertex vectors of the predicted oriented bounding box are obtained by summing and averaging. St33, Orthogonal Edge Vector Extraction: Constructing two mutually orthogonal adjacent edge vectors of the predicted oriented bounding box. and : 。 9. St34, Extraction of semi-major and minor axes: Predicting the semi-major axis of the ellipse profile. and short axis half length The maximum and minimum values ​​of the L2 norm of the two orthogonally adjacent edge vectors are naturally associated and extracted through adaptive comparison; St35, Principal Axis Rotation Tilt Angle Calculation: Select the edge vector with the longer predicted orientation bounding box length as the principal axis vector. Using the two-parameter arctangent function The angle between the principal axis vector and the horizontal coordinate axis of the image is directly calculated, which is the predicted rotation angle of the principal axis of the ellipse contour. : ； in and These represent the scalar components of the principal axis vector on the X and Y axes in the global image coordinate system, respectively. 10.St36, Based on the 5 parameters calculated above. This enables the segmentation of the target eye image.

Images