An iris localization method based on improved YoloV4 network and calculus operator

Abstract

The invention discloses an iris positioning method based on an improved YoloV4 network and a calculus operator, comprising the following steps: using an improved YoloV4 network based on MobilenetV2 to detect an iris image, and locating the approximate position of the pupil center; wherein, using a MobileNetV2 structure to extract backbone features, combining with a YoloV4 network to predict the position of the iris outer circle, generating a rectangular frame representing the iris outer circle boundary on the iris image after detection; performing denoising on the iris image with the generated rectangular frame representing the iris outer circle boundary, and removing noise interference in the image; using an improved Daugman calculus operator, accurately locating the inner and outer boundaries of the iris on the denoised iris image. The method of the invention is effective and reliable, and is more flexible and convenient to use. Regardless of the iris at close range or at long range, a high positioning accuracy rate can be achieved under complex conditions, and it has good robustness and expansibility.

Description

[0001] Technical Area

[0002] The present invention belongs to the field of iris recognition technology, and specifically relates to an iris positioning method based on an improved YoloV4 network and a calculus operator. Background Art

[0003] Due to some special properties, the human iris has been widely used in many places. It can not only be used as an identity recognition technology, but also for disease diagnosis. Therefore, correct iris positioning is an important part of iris application. If the positioning is inaccurate, the iris texture information will be lost, which will directly affect the effect of identity recognition and disease diagnosis.

[0004] In the process of iris recognition, the accuracy of iris positioning directly affects the accuracy of iris recognition. Therefore, iris positioning is the key link of iris recognition and the premise and guarantee of subsequent work. Therefore, the research on iris positioning is regarded by researchers from all over the world as the key and difficulty of the image preprocessing stage, which has very important theoretical and application value.

[0005] The purpose of iris localization is to separate the iris area from the human eye image, and then remove the non-iris information such as pupil and sclera in the iris image. Locating the iris area from the original image is the premise and necessary link to complete iris recognition. In the process of iris recognition, the accuracy of iris localization will directly affect the accuracy of iris recognition. At the same time, in the process of iris image acquisition, there are inevitably problems such as eyelash interference and uneven light intensity, which increase the difficulty of accurately extracting the iris area. Therefore, the research on iris localization is regarded by researchers from all over the world as the focus and difficulty of the image preprocessing stage, and has very important theoretical and application value.

[0006] There are two main types of iris positioning methods, one is the classic iris positioning method, and the other is the positioning algorithm based on deep learning. There are three main types of classic iris positioning methods. The first is based on differential integration. This view holds that the iris is an approximate circular area, so the iris position is simplified to determine the approximate center and radius of the inner and outer boundaries of the iris. The second is based on Hough transform. The third is the grayscale difference method. This method makes full use of the grayscale changes of the iris image to locate the iris. Generally speaking, these traditional methods are simple to operate and have a fast positioning speed. However, it is easily disturbed by noise such as light spots and image blur, has low accuracy, and has limited application scenarios.

[0007] In recent years, deep learning has made rapid progress in the field of target detection. Currently, there are two main approaches: one-stage detection and two-stage detection. The two-stage approach is based on the idea of ​​target candidate boxes, generating a series of sample candidate boxes in advance, and then classifying samples through convolutional neural networks. Region-based convolutional neural networks (R-CNN) are one of them. Since R-CNN needs to select thousands of proposed regions from a picture, its speed is very slow. One-stage detection is based on the idea of ​​regression and does not have target candidate boxes. It uses the method of extracting image feature information through the backbone network to directly regress the object bounding box, so it is faster, such as the Yolo series. Yolo has attracted the attention of many researchers since its early days. The current version has been iteratively developed to YoloV7.

[0008] Due to the advanced nature of the Yolo algorithm, many researchers use it to locate the iris. The Yolo series-based methods are simple to implement, highly resistant to noise interference, and have high accuracy. However, deep learning models require long-term training and a large amount of labeled data, and the cost of labeled data severely limits their scalability to new categories. Summary of the invention

[0009] The technical problem to be solved by the present invention is to provide an iris localization method based on an improved YoloV4 network and calculus operators in view of the deficiencies in the prior art, so as to achieve accurate localization of iris images collected in complex environments without too much annotation.

[0010] To solve the above technical problems, the present invention includes:

[0011] An iris positioning method based on an improved YoloV4 network and a calculus operator comprises the following steps:

[0012] S1: The improved YoloV4 network based on MobilenetV2 is used to detect the iris image and locate the approximate position of the pupil center. The MobileNetV2 structure is used to extract the backbone features, and the YoloV4 network is combined to predict the position of the iris outer circle. After detection, a rectangular box representing the boundary of the iris outer circle is generated on the iris image.

[0013] S2: performing denoising processing on the iris image with the generated rectangular frame representing the outer circle boundary of the iris to remove noise interference in the image;

[0014] S3: The improved Daugman calculus operator is used to accurately locate the inner and outer boundaries of the iris on the denoised iris image.

[0015] Furthermore, in step S1, the improved YoloV4 network includes a backbone network, a neck network and a head; the backbone network adopts a MobileNetV2 network, including 7 bottleneck structures, wherein the 28×28 and 7×7 feature maps output by the fourth and last bottleneck structures are used to predict small-sized irises and large-sized irises, respectively, and these two feature maps are fused in the neck network; the neck network includes a path aggregation network module and a spatial pyramid pooling module; the 7×7 feature map output by the last bottleneck structure is input into the golden font Tower pooling module, the pyramid pooling module uses kernels of sizes 1×1, 5×5, 9×9 and 13×13 to perform maximum pooling operations, and the stride value is set to 1; the output result of the pyramid pooling module is fused with the 28×28 size feature map output by the fourth bottleneck structure in the path aggregation network module, and the path aggregation network module uses upsampling and downsampling operations to set bottom-up and top-down paths to combine low-level and high-level features; the head has two detection heads, which are responsible for receiving and processing a set of aggregated feature maps output by the path aggregation network module for predicting bounding boxes, object scores and classification scores.

[0016] Furthermore, in step S2, the denoising process of the iris image includes:

[0017] S2-1. Remove reflective points: Use the threshold segmentation method to segment the reflective points of the iris image, set a rectangular area of ​​a certain size at the roughly located center of the pupil, and regard the points with grayscale values ​​greater than the set threshold as reflective points. Generate a mask from the reflective points, and then perform morphological dilation operations on the points in the mask so that the mask completely covers the reflective point part; interpolate and fill the reflective point area in the mask with the iris image to filter out the reflective points in the image;

[0018] S2-2. Use morphological closing operation to fill the iris image after removing the reflective points, and filter out the interference of eyelashes in the image.

[0019] Furthermore, in step S3, the inner boundary of the iris is located, and the improved Daugman calculus operator expression is:

[0020]

[0021] In the formula, the center (x0, y0) and the radius r represent the values ​​of the center and radius of the search starting point respectively. The value of the center (x0, y0) is determined by the outer circle of the iris detected by the improved YoloV4 network. n represents the number of points uniformly taken on the circumference with (x0, y0) as the center and r as the radius.

[0022] g θ,rIt represents the radial gray gradient of the θth point on the circumference, and its value is equal to the difference between the gray value of the corresponding point on the circumference with the same center and radius increased by Δr and the gray value of the point, that is:

[0023] g θ,r =I θ,r+Δr -I θ,r

[0024] C θ,r is the compensation factor, which means the average of the absolute values ​​of the difference between the point and the two adjacent circles, that is:

[0025]

[0026] I θ,r / n represents the grayscale mean;

[0027] When the improved Daugman calculus operator reaches its maximum value, the center of the circle P(x p ,y p ) and radius r p , which is the precise positioning result of the inner boundary of the iris, that is, the pupil boundary.

[0028] Furthermore, when locating the inner boundary of the iris, the number of points n uniformly taken on the circumference is set to 32.

[0029] Furthermore, in step S3, the outer boundary of the iris is located, and the improved calculus operator expression is:

[0030]

[0031] In the formula, the center (x1, y1) and the radius r1 represent the center and radius of the search starting point respectively, and n represents the number of points uniformly taken on the circumference of the circle with (x1, y1) as the center and r1 as the radius;

[0032] g θ,r It represents the radial gray gradient of the θth point on the circumference, and its value is equal to the difference between the gray value of the corresponding point on the circumference with the same center and radius increased by Δr and the gray value of the point, that is:

[0033] g θ,r =I θ,r+Δr -I θ,r

[0034] C θ,r is the compensation factor, which means the average of the absolute values ​​of the difference between the point and the two adjacent circles, that is:

[0035]

[0036] When the improved calculus operator reaches the maximum value, the center of the outer boundary of the iris P(x i ,y i ) and radius r i .

[0037] Furthermore, the search for the center of the outer boundary circle is narrowed to a very small area within the located center of the inner boundary circle, and the radius search range is set based on the rectangular frame obtained by improving the YoloV4 network to detect the outer circle, and is set as:

[0038]

[0039] Where r1 is the search radius of the outer boundary of the iris, and rows and cols represent the length and width of the rectangular box obtained by improving the YoloV4 network to detect the outer circle of the iris.

[0040] Furthermore, of the n points uniformly obtained on the circumference, only the parts located in the fan-shaped areas of -45° to 22.5° and 157.5° to 225° are selected.

[0041] Furthermore, in the precise positioning of the outer boundary, the number of equally spaced points n on the circumference is set to 64.

[0042] The beneficial effects of the present invention are:

[0043] The present invention combines the advantages of traditional machine learning and deep learning in iris detection, and proposes an iris positioning method based on an improved YoloV4 network and calculus operator. The lightweight convolutional neural network MobileNetV2 structure is used to extract the main features, and the outer circle position of the iris is predicted in combination with the YoloV4 detection network, so as to roughly determine the position of the pupil center. The problem that the traditional positioning algorithm is easily affected by uneven lighting, eyelashes, glasses frames, etc., leading to positioning failure, is solved, and the accuracy of pupil center positioning is improved; based on the principle of Daugman calculus operator, an improved calculus operator with better robustness is proposed. After removing the noise in the image, the inner and outer boundaries of the iris can be accurately located using the improved calculus operator. Comparative experiments show that the method of the present invention is effective and reliable, and is more flexible and convenient to use. Regardless of the iris at close range or long distance, it can also achieve a high positioning accuracy under complex conditions, and has good robustness and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the iris positioning algorithm of the present invention;

[0045] Figure 2 It is the overall network structure diagram of Mobile_YoloV4 of the present invention;

[0046] Figure 3This is an example diagram of the data set used to improve the YoloV4 network in the present invention;

[0047] Figure 4 It is a diagram of the DW convolution process;

[0048] Figure 5 It is the PW convolution process diagram;

[0049] Figure 6 This is the bottleneck structure diagram of MobileNetV2;

[0050] Figure 7 This is an example of the effect of YOLOv4 network detecting irises at close range;

[0051] Figure 8 This is an example of the effect of the YOLOv4 network in detecting irises at a distance;

[0052] Fig. 9 This is a schematic diagram of a reflective point detection mask;

[0053] Fig.10 is a schematic diagram of iris image denoising;

[0054] Fig.11 This is a schematic diagram of the CASIA iris database;

[0055] Fig.12 This is an example of an iris image with severe interference;

[0056] Fig.13 It is a schematic diagram of the recall curve of the two networks;

[0057] Fig.14 This is the iris effect diagram with severe detection interference of the YOLOv4 network. DETAILED DESCRIPTION

[0058] To facilitate understanding of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Those skilled in the art should understand that the embodiments are only to help understand the present invention and should not be regarded as specific limitations of the present invention.

[0059] like Figure 1 As shown, the present invention provides an iris positioning method based on an improved YoloV4 network and a calculus operator, comprising the following steps:

[0060] S1: The improved YoloV4 network based on MobilenetV2 is used to detect the iris image and locate the approximate position of the pupil center. Among them, the lightweight convolutional neural network MobileNetV2 is used to replace the traditional YoloV4 network CSPDarkNet-53 as the backbone feature extraction network, combined with the YoloV4 network to predict the position of the iris outer circle. After detection, a rectangular box representing the boundary of the iris outer circle is generated on the iris image.

[0061] The purpose of locating the outer circle of the iris is to locate the approximate position of the pupil center from the iris image, reduce the blindness of the iris search, and effectively narrow the search area of ​​the pupil center, thereby improving the algorithm's operation speed. Generally speaking, since the iris images collected in practice are often affected by uneven lighting, reflective points, and image noise that is difficult to remove in preprocessing, the result of pupil positioning is only required to be inside the pupil.

[0062] S2: performing denoising processing on the iris image with the generated rectangular frame representing the outer circle boundary of the iris to remove noise interference in the image.

[0063] The rectangular frame generated after iris detection can be regarded as the outer circle boundary of the iris. The position of its center is not necessarily the true center of the pupil, but it generally falls inside the pupil. In order to find the exact area of ​​the iris, it is also necessary to accurately locate the inner and outer boundaries of the iris. Under the premise of not destroying the original useful characteristics of the image, further removing the noise interference in the image lays a good foundation for subsequent precise positioning.

[0064] S3: The improved Daugman calculus operator is used to accurately locate the inner and outer boundaries of the iris on the denoised iris image.

[0065] The present invention selects YoloV4 as an iris target detection tool, and uses the lightweight convolutional neural network MobileNetV2 structure to extract the main features and roughly determine the position of the pupil center; then uses the improved calculus operator to accurately locate the inner and outer boundaries of the iris, greatly improving the robustness of iris positioning.

[0066] In step S1, the improved YoloV4 network includes a backbone network (Backbone), a neck network (Neck) and a head (Head). The overall network structure is as follows: Figure 2As shown. The 28×28 and 7×7 feature maps output by the fourth and last bottleneck structures of the backbone network are used to predict small-sized irises and large-sized irises, respectively, and the two feature maps are fused in the neck network; the neck network includes a path aggregation network module (PAN) and a spatial pyramid pooling module (SPP); the 7×7 feature map output by the last bottleneck structure is input into the pyramid pooling module, and the pyramid pooling module uses kernels of sizes 1×1, 5×5, 9×9 and 13×13 for maximum pooling operations, and the stride value is set to 1; the output result of the pyramid pooling module is fused with the 28×28 feature map output by the fourth bottleneck structure in the path aggregation network module, and the path aggregation network module uses upsampling and downsampling operations to set bottom-up and top-down paths to combine low-level and high-level features; the head has two detection heads, which are responsible for receiving and processing a set of aggregated feature maps output by the path aggregation network module for predicting bounding boxes, object scores, and classification scores.

[0067] 1) Dataset Introduction

[0068] When training the YoloV4 network, the present invention selects two datasets of different sizes: CASIA-Iris-Thousand and CASIA-Iris-Distance. The CASIA-Iris-Thousand dataset contains 20,000 iris images from 1,000 subjects, which are collected at close range using an IKEMB-100 camera, and each image has 640×480 pixels. The main sources of intra-class variation in CASIA-Iris-Thousand are changes in pupil size under different lighting conditions, as well as reflections from glasses and mirrors. Figure 3 (a) is a sample in the dataset. Since CASIA-Iris-Thousand is the first publicly available iris dataset with 1,000 subjects, it is more universal and diverse.

[0069] The iris images of the CASIA-Iris-Distance dataset are captured by a high-resolution camera at a long distance. Therefore, both irises and facial patterns are included in the image region of interest. It contains 142 subjects and 2567 images. Each image has 2352×1728 pixels. Figure 3 (c) is a sample in the dataset. The imaging system of the high-resolution camera can actively search for patterns such as irises and faces in the field of view, thereby identifying users at a distance of about 3 meters.

[0070] Due to different data sets, the number and size of manually annotated detection boxes are also different. The image in the CASIA-Iris-Thousand data set has only one eye, and the annotated rectangular box is relatively large relative to the entire image, such as Figure 3 (b) As shown in the figure, the image in the CASIA-Iris-Distance dataset has two eyes, and the annotated rectangular box is relatively small compared to the entire image, as shown in the figure. Figure 3 (d) as shown.

[0071] 2) Backbone

[0072] MobileNet was proposed by Google in 2017. Its main purpose is to effectively utilize the computing resources of mobile devices and embedded devices to improve the accuracy of the model. MobileNet is mainly composed of deep separable convolutions. MobileNetV2 uses a reverse residual structure to further improve the performance of the network. The network structure of MobileNetV2 is shown in Table 1. The operations include ordinary convolution (conv2d), deep separable convolution (bottleneck) with a reverse residual structure, and average pooling (avgpool); t represents the expansion coefficient of the intermediate convolution channel; c represents the number of output channels; n represents how many times the layer is repeated; s represents the stride of the convolution.

[0073] Table 1 MobileNetV2 network layer structure

[0074]

[0075] The design idea of ​​depthwise convolution is to divide a standard convolution into two parts. The first part is depthwise convolution (DW); the second part is pointwise convolution (PW). DW convolution is a convolution kernel corresponding to one channel of the image, and the depth of the convolution kernel is 1. Compared with traditional convolution operations, DW convolution only performs independent convolution operations on each channel.

[0076] Since the feature information in space is not fully utilized, the obtained Feature Map has a weak expressive power, so it needs to go through PW convolution. PW convolution mainly uses the convolution kernel of 1×1×M (the depth of the previous layer of Maps) to convolve the previous layer of Maps point by point, so that the feature information in the spatial position can be well integrated to obtain feature information with strong expressive power. Depth-wise separable convolution greatly reduces the amount of calculation and improves the calculation speed.

[0077] The bottleneck structure of MobileNetV2 adopts the inverted residual network structure. Compared with the residual network structure, it first increases the dimension through 1×1 convolution, then through 3×3 DW convolution, and finally through 1×1 convolution for dimensionality reduction. In the structure, the shortcut operation is only executed when the stride is 1 and the shape of the input feature matrix and the output feature matrix are the same, such as Figure 6 shown.

[0078] 3) Neck network (Neck).

[0079] The neck network connects the backbone network and the detection network. The neck network is also called the enhanced feature extraction network, which consists of a spatial pyramid pooling (SPP) module and a path aggregation network (PAN). Figure 2 There are 7 bottleneck structures in the backbone network of . The feature map outputs of these bottleneck structures are fused in the neck network. The neck network connects the feature maps from different layers of the backbone network and sends them as input to the head. The pyramid pooling module performs the maximum pooling operation using kernels of sizes 1×1, 5×5, 9×9, and 13×13, and the stride value is set to 1. Connecting feature maps increases the receptive field of the extracted features, improves the accuracy of detecting small targets, and effectively prevents overfitting. The cascaded feature maps from the pyramid pooling module are fused with the high-resolution feature maps by using PAN. PAN uses upsampling and downsampling operations to set up bottom-up and top-down paths to combine low-level and high-level features.

[0080] 4) Head.

[0081] The head is responsible for receiving a set of aggregated feature maps output by the processing path aggregation network module for predicting bounding boxes, object scores, and classification scores. The head of the traditional YOLO v4 network has three detection heads. Each detection head is a YOLOv3 network that outputs three feature maps of size 19×19, 38×38, and 76×76 for prediction.

[0082] According to the actual situation of iris detection, that is, Figure 3 (b) and Figure 3 (d) It can be seen that since only the outer circle of the iris is detected, there are at most 2 detection targets in one picture. Therefore, only 2 detection heads are needed for the head. At the same time, since the sizes of the irises in the dataset are different, 2 feature maps of different sizes are needed for prediction. According to these specific circumstances, the present invention extracts 28×28 and 7×7 feature maps from the fourth and last bottlenecks, respectively, to predict small-sized irises and large-sized irises. The results of iris detection using the improved YOLO v4 network of the present invention are shown in Figure 2. Figure 7and Figure 8 shown.

[0083] In step S2, the denoising process of the iris image includes the following steps:

[0084] S2-1. Remove the reflective spots. Figure 7 and Figure 8 It can be seen that due to the photography, there are two reflective points inside the pupil. These two reflective points will affect the accuracy of the subsequent positioning of the inner circle of the iris. Therefore, the present invention adopts a threshold segmentation method to segment the reflective points of the image. A rectangular area of ​​a certain size is set at the center of the roughly located pupil, and points with grayscale values ​​greater than the set threshold are considered to be reflective points. A mask is generated from the reflective points, and then a morphological expansion operation is performed on the points in the mask so that the mask can completely cover the reflective point part. The mask detected by the reflective point is as follows: Fig. 9 As shown in the figure, the iris image is interpolated and filled inwards to the reflective point area in the mask to filter out the reflective points in the image. The result of removing the reflective points is shown in the figure. Fig.10 (b) as shown.

[0085] S2-2. Lighten the eyelash area. Considering the scattered and thin line characteristics of human eyelashes, the eyelashes can be regarded as dark details in the image, and the morphological closing operation happens to connect some narrow gaps in the image, eliminate the dark details in the image that are smaller than the structural elements, and keep the overall grayscale value of the image and the size of each large dark area unchanged. Therefore, the morphological closing operation is used to fill the image, connect the adjacent large areas in the image, smooth the boundary of the iris without changing its radius, and filter out the interference of some eyelashes in the image.

[0086] The processing object of grayscale digital morphology is digital image. The dilation operation and erosion operation in grayscale morphological image processing are defined as:

[0087]

[0088]

[0089] In the formula, represents the dilation operation, represents the erosion operation, f(x,y) represents the grayscale value of a point in the input image, and b(x,y) represents the structural element. The dilation operation uses the maximum grayscale value of each area (x,y) where the image f overlaps with the structural element b. Therefore, the grayscale image after dilation is brighter than the original image, the size of bright features will increase, and the size of dark features will decrease. The erosion operation is the opposite. Single dilation and erosion operations will change the size of part of the image area, which is not particularly effective in grayscale image processing, but when they are combined, the effect will become more powerful.

[0090] The morphological closing operation is to first dilate the image and then erode it.

[0091]

[0092] Create a disk structure element with a radius of 3 and use morphological closing operations to process the image. Fig.10 (c) As shown. By comparing the original image and the image after morphological closing operation, it can be found that the morphological closing operation can effectively fade the eyelash area, smooth the iris boundary, and reduce the interference in subsequent precise positioning. The reason why the morphological closing operation is performed after the reflective points are removed is that the reflective points are generally bright spots or bright spots in a smaller area, and the first step of the closing operation, the expansion operation, will cause the smaller bright area in the image to expand. If the morphological closing operation is performed first, the reflective points in the image will be enlarged, making it more difficult to completely remove them. Sometimes, the reflective points will even expand to the boundary of the iris, thereby destroying the boundary of the iris and affecting the subsequent positioning effect.

[0093] Introduction to Daugman calculus operator: The human eye is divided into three parts: pupil, iris and sclera. The pupil has a darker grayscale, the sclera has a lighter grayscale, and the iris has a grayscale in between. Therefore, there is an obvious grayscale jump at the inner and outer boundaries of the iris. The basic idea of ​​the Daugman calculus positioning algorithm is to continuously change the center of the circle (x p ,y p ) and the radius r, calculate the curve integral along the arc according to the gray value, and then calculate the first-order derivative of the integral result with respect to the radius, so as to find the position of the circle with the largest gray gradient value, which is the boundary of the iris. Its mathematical expression is shown in formula (4).

[0094]

[0095] In the above formula, I(x,y) is the gray value of a point in the iris image. To make a curve integral for a circle with a center of (x0, y0) and a radius of r, for the inner boundary of the iris, the integral path is the entire circumference, and because the outer boundary of the iris is easily blocked by eyelashes and eyelids, the integral path of the outer boundary of the iris takes the 90° area on both sides. The purpose of dividing the curve integral by the circumference of the arc 2πr is to normalize and find the mean gray gradient. is the partial derivative with respect to r, G σ (r) is a Gaussian function with standard deviation σ, and * represents a convolution operation.

[0096] By transforming expression (4), we can get its equivalent form:

[0097]

[0098] Among them, the Gaussian function G σ The expression is:

[0099]

[0100] therefore, It can be discretized and approximated as formula (7):

[0101]

[0102] Here, Δr represents the unit increase step of the radius.

[0103] Then, the differential is used to replace the differential in expression (4), and the curve integral is replaced by the sum of the grayscale values ​​of m points separated by Δθ on the integration circle. The differential form of the Daugman positioning algorithm is obtained as shown in equation (8), which is called the Daugman calculus operator.

[0104]

[0105] In formula (8), x = kΔrcos(mΔθ) + x0, y = kΔrsin(mΔθ) + y0, Δr is the unit increment step of the radius, (n0Δr, nΔr) is the radius search range; m is the number of points on each circle, Δθ is the unit increment step of the angle, and the relationship between m and Δθ satisfies The center and radius of the circle when the calculus detection operator takes its maximum value are the optimal iris boundary parameters.

[0106] When the pupil center is located accurately, the calculus operator proposed by Daugman has a good positioning effect. However, if the image is affected by various interferences and the result of pupil center positioning deviates, the effect is not ideal. The present invention proposes an improved operator with better robustness based on the principle of Daugman calculus operator, so that the accuracy of locating the inner and outer boundaries of the iris is higher.

[0107] In step S3, the inner boundary of the iris is accurately located, and the improved Daugman calculus operator expression is:

[0108]

[0109] In formula (9), the center (x0, y0) and the radius r represent the center and radius of the search starting point respectively. The value of the center (x0, y0) is determined by the outer circle of the iris detected by the improved YoloV4 network. n represents the number of points uniformly taken on the circumference with (x0, y0) as the center and r as the radius. When locating the inner boundary of the iris, the number of points uniformly taken on the circumference n is set to 32.

[0110] g θ,rIt represents the radial gray gradient of the θth point on the circumference, and its value is equal to the difference between the gray value of the corresponding point on the circumference with the same center and radius increased by Δr and the gray value of the point, that is:

[0111] g θ,r =I θ,r+Δr -I θ,r (10)

[0112] C θ,r is the compensation factor, which means the average of the absolute values ​​of the difference between the point and the two adjacent circles, that is:

[0113]

[0114] When the improved Daugman calculus operator reaches its maximum value, the center of the circle P(x p ,y p ) and radius r p , which is the precise positioning result of the inner boundary of the iris, that is, the pupil boundary.

[0115] Since the gap between the two adjacent points on the circumference is too large, it is likely that there are some noise or interference points in the image. This compensation factor C is introduced θ,r , which can effectively reduce the influence of these noise and interference points on the total gradient, thus avoiding positioning errors caused by excessive gray gradient of a certain interference point on the detection circle. This process is similar to numerical filtering, but in the problem of accurate iris positioning, the use of this compensation factor can achieve better results than directly using some commonly used numerical filtering methods.

[0116] The size of a person's pupil varies under different lighting conditions. Therefore, in order to locate the iris of different people under different lighting conditions and improve the robustness of the algorithm, the search radius of the inner boundary of the iris is usually set to be relatively large. In order to prevent the outer boundary of the iris from being located when locating the inner boundary of the iris, an I is set in the improved calculus operator expression (9). θ,r / n represents the grayscale mean. Since the grayscale value of the pupil is smaller than that of the iris, when the grayscale gradient and the larger inner and outer boundaries of the iris are detected at the same time, subtracting the grayscale mean will make the positioning result fall on the inner boundary of the iris.

[0117] The calculus operator still needs to search for the maximum value in the three-dimensional space (x0, y0, r), but since the rough positioning result obtained by detecting the outer circle of the iris through the improved YoloV4 network is very close to the true center of the pupil, a smaller circle center search area can be set to reduce the number of iterations and speed up the positioning speed. The radius search range can be set according to the prior conditions of the iris image.

[0118] In step S3, the outer boundary of the iris is located, and the improved calculus operator expression is:

[0119]

[0120] In formula (12), the center (x1, y1) and the radius r1 represent the center and radius of the search starting point, respectively, and n represents the number of points uniformly taken on the circumference of the circle with (x1, y1) as the center and r1 as the radius.

[0121] g θ,r and C θ,r The expression of is the same as that of equation (10) and equation (11).

[0122] When the improved calculus operator reaches the maximum value, the center of the outer boundary of the iris P(x i ,y i ) and radius r i .

[0123] For the precise positioning of the outer boundary of the iris, the calculus operator is similar to that of the precise positioning of the inner boundary of the iris. The first and second terms in formula (12) are consistent with the situation when locating the inner boundary, representing the grayscale gradient and compensation factor respectively. Compared with the positioning of the inner boundary, the calculus operator for positioning the outer boundary removes the grayscale mean. Another difference is that since the centers of the inner and outer boundaries of the iris are generally very close, the search for the center of the outer boundary can be narrowed to a very small area within the located center of the inner boundary, and the radius search range is set based on the rectangular frame obtained by YoloV4 detecting the outer circle, and is set as:

[0124]

[0125] In formula (13), r p is the radius of the inner boundary of the iris, r1 is the search radius of the outer boundary of the iris, and rows and cols represent the length and width of the rectangular box obtained by YoloV4 to detect the outer circle of the iris.

[0126] Furthermore, considering that the outer boundary of the iris is often blocked by eyelashes and eyelids, the n points obtained at equal intervals on the circumference only select the parts located in the fan-shaped areas of -45° to 22.5° and 157.5° to 225°.

[0127] In addition, as the range of accumulated points becomes smaller, in order to have enough points on the circumference to participate in the calculation, in the precise positioning of the outer boundary, the number of points n taken at equal intervals on the circumference is set to 64.

[0128] At this point, iris positioning has been completed, and the positioning results are as follows: Fig.10 (d) as shown.

[0129] Experimental results and analysis

[0130] 1) Iris dataset and data preprocessing

[0131] The experiment uses the iris dataset released by the Biometrics and Security Technology Research Center of the Institute of Automation, Chinese Academy of Sciences. The iris images in this dataset are near-infrared images acquired under certain lighting conditions using a special iris camera. They have good clarity and high quality, and are suitable for testing iris positioning and authentication algorithms. The Chinese Academy of Sciences has released four versions of the iris database so far. Examples of iris images in each version are as follows: Fig.11 As shown in the figure, due to the different development of iris acquisition equipment in different periods, the images of each version are different.

[0132] The earliest version, CASIA-IrisV1.0, selected subjects with relatively large eyes and short and sparse eyelashes for iris collection, and used a circular area of ​​constant intensity to cover the reflective points caused by mirror reflection, and performed a series of image processing. Therefore, there is little interference noise in the image. However, under actual collection conditions, due to various factors such as lighting, eyelashes, eyelids, and differences in the size of human eyes, it is impossible to obtain iris images with almost no noise interference like this, and it is not realistic to manually process the image after collection. Therefore, in response to the requirements of practical applications, the Chinese Academy of Sciences has successively released new databases that are more in line with actual collection conditions. The iris data in the current version of the Chinese Academy of Sciences, CASIA-IrisV4.0, was selected in the experiment. In the iris positioning simulation experiment, the pupil center positioning quality analysis in version 4.0 was selected. At the same time, in order to enrich the diversity of samples, the experiment used two different data sets, CASIA-Iris-Thousand and CASIA-Iris-Distance, in version 4.0.

[0133] In order to verify the robustness of the method of the present invention, the present invention did not deliberately eliminate some iris images with poor quality, insufficient clarity, eyelash obstruction, and severe glasses reflection when selecting iris images. For the CASIA-Iris-Thousand dataset, 1000 images were selected from the first 50 people, and for the CASIA-Iris-Distance dataset, 1000 images were selected from the first 54 people. The Image Labeler provided by Matlab 2022a was used to annotate these 2000 images respectively. A total of 3000 rectangular boxes were annotated. Among them, 80% of the total 1600 images were randomly selected as training sets, and 20% of the total 400 images were used as test sets. Fig.12 Some examples of heavily disturbed iris images are shown.

[0134] Since the input size of the backbone network MobileNetV2 used in this invention is 224×224×3, the preprocessing process before the experiment needs to uniformly convert the size of these 2000 pictures into 224×224×3. This is not only to meet the input requirements of MobileNetV2, but also to adapt to color iris pictures.

[0135] 2) Experimental operating environment and evaluation indicators

[0136] The hardware and software platform parameters of the experiment are: Windows 10 64-bit operating system. The processor is Intel Core i7-8700 3.20GHz dual-core, 16G running memory, NVIDIA GeForce RTX 3060 graphics card. The software environment is Matlab 2022a. The hyperparameters of the training model are set as follows: training batch BatchSize = 4, model training Epoch = 5, learning rate 0.005, and the optimization algorithm uses the Adam algorithm. All experiments were completed on a single graphics card.

[0137] In order to measure the experimental results, the present invention adopts the following indicators: precision, recall, accuracy and intersection over union (IoU). And it can be formally expressed as:

[0138]

[0139]

[0140]

[0141] In the formula, TP (True Positive) means that the prediction is consistent with the label. Usually, the prediction is considered TP when the IoU between the prediction and the label reaches 0.5. FP (False Positive) means that a negative example is predicted as a positive example, and FN (False Negative) means that a positive example is predicted as a negative example.

[0142] The relationship between precision and recall is one that increases while the other increases. If the precision increases, the recall will decrease, and if the recall increases, the precision will decrease. Usually, the two are combined and presented in the form of a precision and recall curve to give an objective evaluation of the performance of the model quality.

[0143] 3) Comparative experiment with traditional YoloV4

[0144] There are two types of traditional YoloV4 backbone networks, one is csp-darknet53-coco, and the other is tiny-yolov4-coco. These networks are trained on the COCO dataset. csp-darknet53-coco is a YOLO v4 network with three detection heads, while tiny-yolov4-coco is a network with two detection heads. Since iris detection is mainly used in mobile phones and embedded devices. The detection model that needs to be generated is small in size. The volume of the detection model with csp-darknet53-coco as the backbone network is usually greater than 200M. Although the accuracy is very high, it is not suitable for storage in embedded devices. Therefore, the comparative experiment here does not include YoloV4 with csp-darknet53-coco as the backbone network.

[0145] When the model training hyperparameters are consistent, the experimental results show that the YoloV4-tiny model size is 21.42M, while the improved YoloV4 model based on MobileNetV2 proposed in this invention is only 5.8M, which is much smaller than the traditional YoloV4-tiny. By default, it is generally believed that IoU = 0.5 indicates accurate positioning. In this case, the recall rate curves of the two networks are as follows: Fig.13 An example of the positioning result is shown in Fig.14 As shown. The yellow rectangular frame is the positioning result of the complex iris image. It can be seen that the iris detection algorithm based on the improved YoloV4 proposed in the present invention has good anti-interference performance and strong robustness.

[0146] from Fig.13 It can be seen that the average accuracy of the improved method proposed in the present invention is close to 100% when Epoch=5, and the performance is better than the traditional yolov4-tiny network. The comparison of the average accuracy of the iris outer circle under different IoUs is shown in Table 2.

[0147] Table 2 Average positioning accuracy of the outer circle of the iris of the two detection networks at different IoU

[0148]

[0149] As can be seen from Table 2, when IoU is less than or equal to 0.7, the performance of the improved algorithm proposed in the present invention is better than that of the traditional YoloV4-tiny, but when IoU is equal to 0.8, the performance drops significantly. However, considering that the model of YoloV4 after the modification by the method proposed in the present invention is much smaller than YoloV4-tiny, this performance drop is completely acceptable.

[0150] 4) Experiment on precise positioning of inner and outer circles of iris

[0151] In order to verify the scalability and robustness of the method proposed in this invention, the image data used in the experiment is different from the image data used in the iris area positioning experiment in Section 2.3.3. For the CASIA-Iris-Thousand dataset, image data are randomly selected from the last 950 people, and for the CASIA-Iris-Distance dataset, image data are randomly selected from the last 88 people. In other words, the iris image data used in the iris detection experiment in this section does not overlap with the iris image data used to train the YoloV4 network, and it is an open set test. The specific composition of the dataset is shown in Table 3.

[0152] Table 3 Dataset composition

[0153]

[0154] The traditional Daugman positioning algorithm iteratively searches for the three parameters of the center and radius in the entire iris image, which takes too long, even tens of seconds, and has no practical significance. Based on the above considerations, the comparison of precise positioning results is based on the same rough positioning algorithm and the same size of search area. Under the same premise, Daugman's calculus operator is compared with the improved operator proposed in this invention. Fig.14 The area between the inner and outer circles shows the result after precise positioning of the iris.

[0155] from Fig.14 It can be seen that despite the serious interference, the iris precise positioning based on the improved calculus operator still achieves satisfactory results. Fig.14 (c) In this image, not only are the eyes small, but the eyelashes also interfere severely. Also, the boundary of the rough iris location, that is, the boundary of the rectangular box where YoloV4 detects the iris, is not completely tangent to the iris boundary. However, the improved calculus operator still accurately locates the iris area.

[0156] The iris inner and outer circle positioning accuracy of different algorithms at close distance and long distance are shown in Table 4 and Table 5 respectively.

[0157] Table 4 Comparison results of close-range iris positioning accuracy

[0158]

[0159] Table 5 Comparison results of long-distance precise positioning

[0160]

[0161] It can be seen from Table 2 that, whether it is an ordinary image or a complex image, the improved operator proposed in the present invention has a higher positioning accuracy than the Daugman calculus operator, which shows that the improvement of the operator is effective and improves the accuracy of iris precise positioning. For ordinary images at close range, the improved operator improves the positioning accuracy by 2.74% compared with the Daugman operator, and for complex images, the improvement is 4.06%. Since the compensation factor and grayscale mean must be calculated in each iteration, the positioning time is slightly longer. For ordinary images at long distances, the improved operator proposed in the present invention has a higher accuracy in accurately positioning the iris than at close range, and far exceeds the Daugman calculus operator. Especially for complex images with glasses, although the performance of the improved operator of the present invention is lower than that of the close range, it still reaches 84%. However, the positioning success rate using the Daugman calculus operator is quite low, with only 14 irises successfully positioned, and there is no example of successful positioning of both eyes. Therefore, it is meaningless to consider the running time.

[0162] Experimental results show that the improved operator proposed in the present invention improves the success rate of iris localization and has stronger robustness, especially the accuracy of iris localization in complex situations with strong interference.

Claims

1. An iris positioning method based on an improved YoloV4 network and a calculus operator, characterized in that: The method comprises the following steps: S1: The improved YoloV4 network based on MobilenetV2 is used to detect the iris image and locate the approximate position of the pupil center. The MobileNetV2 structure is used to extract the backbone features, and the YoloV4 network is combined to predict the position of the iris outer circle. After detection, a rectangular box representing the boundary of the iris outer circle is generated on the iris image. In this step, the improved YoloV4 network includes a backbone network, a neck network and a head; the backbone network adopts a MobileNetV2 network, including 7 bottleneck structures, wherein the 28×28 and 7×7 feature maps output by the fourth and last bottleneck structures are used to predict small-sized irises and large-sized irises, respectively, and these two feature maps are fused in the neck network; the neck network includes a path aggregation network module and a spatial pyramid pooling module; the 7×7 feature map output by the last bottleneck structure is input into the pyramid pooling module. The pyramid pooling module performs a maximum pooling operation using kernels of sizes 1×1, 5×5, 9×9, and 13×13, and the stride value is set to 1; the output result of the pyramid pooling module is fused with the feature map of size 28×28 output by the fourth bottleneck structure in the path aggregation network module, and the path aggregation network module uses upsampling and downsampling operations to set bottom-up and top-down paths to combine low-level and high-level features; the head has two detection heads, which are responsible for receiving and processing a set of aggregated feature maps output by the path aggregation network module for predicting bounding boxes, object scores, and classification scores; S2: performing denoising processing on the iris image with the generated rectangular frame representing the outer circle boundary of the iris to remove noise interference in the image; S3: Using the improved Daugman calculus operator, the inner and outer boundaries of the iris are accurately located on the denoised iris image; In this step, the inner boundary of the iris is located, and the improved Daugman calculus operator expression is: In the formula, the center (x0, y0) and the radius r0 represent the values ​​of the center and radius of the search starting point respectively. The value of the center (x0, y0) is determined by the outer circle of the iris detected by the improved YoloV4 network. n0 represents the number of points uniformly taken on the circumference with (x0, y0) as the center and r0 as the radius. g θ,r It represents the radial gray gradient of the θth point on the circumference, and its value is equal to the difference between the gray value of the corresponding point on the circumference with the same center and radius increased by Δr and the gray value of the point, that is: g θ,r =I θ,r+Δr -I θ,r C θ,r is the compensation factor, which represents the mean of the absolute value of the difference between the point and its two adjacent circles, that is: I θ,r / n0 represents the grayscale mean; When the improved Daugman calculus operator reaches its maximum value, the center of the circle P(x p ,y p ) and radius r p , which is the precise positioning result of the inner boundary of the iris, that is, the boundary of the pupil; In this step, the outer boundary of the iris is located, and the improved calculus operator expression is: In the formula, the center (x1, y1) and the radius r1 represent the center and radius of the search starting point respectively, and n1 represents the number of points uniformly taken on the circumference of the circle with (x1, y1) as the center and r1 as the radius; When the improved calculus operator reaches the maximum value, the center of the outer boundary of the iris P(x i ,y i ) and radius r i .

2. The iris positioning method based on the improved YoloV4 network and calculus operator according to claim 1 is characterized in that: In step S2, the denoising process of the iris image includes: S2-1. Remove reflective points: Use the threshold segmentation method to segment the reflective points of the iris image, set a rectangular area of ​​a certain size at the roughly located center of the pupil, and regard the points with grayscale values ​​greater than the set threshold as reflective points. Generate a mask from the reflective points, and then perform morphological dilation operations on the points in the mask so that the mask completely covers the reflective point part; interpolate and fill the reflective point area in the mask with the iris image to filter out the reflective points in the image; S2-2. Use morphological closing operation to fill the iris image after removing the reflective points, and filter out the interference of eyelashes in the image.

3. The iris positioning method based on the improved YoloV4 network and calculus operator according to claim 1, characterized in that: When locating the inner boundary of the iris, the number of points n0 uniformly taken on the circumference is set to 32.

4. The iris positioning method based on the improved YoloV4 network and calculus operator according to claim 1, characterized in that: The search for the outer boundary circle center is narrowed to the radius of the located inner boundary circle center of 1.2r p The radius search range is set based on the rectangular frame obtained by improving the outer circle of the YoloV4 network detection, and is set as: In the formula, rows and cols represent the length and width of the rectangular box obtained by improving the YoloV4 network to detect the outer circle of the iris.

5. The iris positioning method based on the improved YoloV4 network and calculus operator according to claim 1, characterized in that: When locating the outer boundary of the iris, n1 points uniformly obtained on the circumference are selected, and only the parts located in the fan-shaped areas of -45° to 22.5° and 157.5° to 225° are selected.

6. The iris positioning method based on the improved YoloV4 network and calculus operator according to claim 1, characterized in that: In the precise positioning of the outer boundary, the number of equally spaced points n1 on the circumference is set to 64.

Images