Iris image preprocessing method and device, medium and equipment

Abstract

The invention discloses an iris image preprocessing method and device, a medium and equipment, and belongs to the field of biological feature recognition. According to the method, the iris image is acquired and is respectively input into a plurality of pre-trained deep learning models, the iris texture region, the iris inner boundary region and / or the iris outer boundary region of the iris image are segmented, and the segmentation results of the plurality of deep learning models are subjected to score fusion; segmenting an iris effective area, an iris inner boundary point and / or an iris outer boundary point from the iris image; and fitting the iris inner and outer boundary points to obtain iris inner and outer boundaries. According to the method, the problems of iris inner and outer boundary positioning and / or iris effective region segmentation in iris recognition are solved through the deep learning model, good generalization and robustness are achieved, and the problems of iris image positioning and iris region segmentation under the conditions of strabismus, squinting, low contrast and the like can be well solved. And score fusion is carried out on segmentation results, so that iris segmentation and positioning results are more accurate.

Description

Technical Field

[0001] The present application relates to the field of biometric identification, and in particular to an iris image preprocessing method, device, medium and equipment. Background Art

[0002] Iris recognition is a widely used biometric recognition technology with the characteristics of accuracy, stability, security and reliability. A complete iris identity recognition system mainly includes iris image acquisition, iris image quality assessment, iris image preprocessing, iris image feature extraction and iris feature comparison. Iris image preprocessing includes the positioning of the inner and outer boundaries of the iris, iris noise detection and iris image normalization. The purpose of iris image preprocessing is to obtain the effective area of ​​the iris and normalize it for iris image feature extraction.

[0003] The method for preprocessing in the prior art is to first locate the inner and outer circles of the iris, then detect the iris noise, and finally segment the iris effective area. The inner circle of the iris is the boundary between the pupil and the iris, and the outer circle of the iris is the boundary between the iris and the sclera. Figure 1 This is a schematic diagram of the results of iris inner and outer circle positioning. Figure 1 It can be seen that the annular area of ​​the iris contains a lot of noise, mainly eyelid cover, eyelash cover, light spot, etc. If these noise interferences are not eliminated and regarded as real iris information, it will inevitably cause deviations in the iris feature extraction results and lead to recognition errors. Iris noise detection mainly includes upper and lower eyelid positioning, light spot detection and eyelash detection, and finally generates a noise template.

[0004] After the iris inner and outer circle positioning, eyelid positioning, light spot detection and eyelash detection are completed, a noise template is generated to extract iris features. The so-called noise template refers to the results of eyelid positioning, eyelash detection and light spot detection represented by a binary [0,1] matrix of the same size as the iris image. The noise part of the iris is 0, and the iris and sclera parts are 1. Figure 2 This is the effect after positioning the inner and outer circles of the iris and the eyelids. Figure 3 This is the effect image of the generated noise template.

[0005] Through the positioning results of the inner and outer circles of the iris and the noise template, the part between the inner and outer circles of the iris and whose value is 1 on the noise template is selected to obtain the effective iris area.

[0006] The iris image preprocessing in the prior art generally performs iris positioning and segmentation based on expert systems, digital image processing and other methods, but has poor generalization and robustness. Summary of the invention

[0007] To address the deficiencies of the prior art, the present application provides an iris image preprocessing method, device, medium and equipment, which have high accuracy, generalization and robustness, and make the iris boundary more neat.

[0008] This application provides the following technical solutions:

[0009] In a first aspect, the present application provides an iris image preprocessing method, the method comprising:

[0010] Get iris image;

[0011] The iris image is respectively input into a plurality of deep learning models pre-trained by different training data sets, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and the segmentation results of the plurality of deep learning models are fractionally fused to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image;

[0012] Wherein, when performing the iris effective area segmentation, the iris texture area is used as the foreground area, and the rest of the area is used as the background area, and the iris image is segmented to segment the iris effective area for iris feature extraction from the iris image, which includes only the iris texture area;

[0013] When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area, the iris image is segmented, and the iris inner boundary point / iris outer boundary point is segmented from the iris image;

[0014] Ellipse fitting is performed according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

[0015] Furthermore, the iris image is respectively input into a plurality of deep learning models pre-trained by different training data sets, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and the segmentation results of the plurality of deep learning models are fractionally fused to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image, including:

[0016] Input the iris image into the multiple deep learning models respectively, encode the iris image, and use deconvolution to expand the encoded feature map to the same size as the iris image, so as to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models;

[0017] The value of each pixel point on the first probability feature map / the second probability feature map / the third probability feature map represents the probability that the corresponding pixel point at the same position on the iris image belongs to the iris effective area / the iris inner boundary point / the iris outer boundary point;

[0018] Score-fusing the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain a first fused feature map / a second fused feature map / a third fused feature map;

[0019] The value of each pixel point on the first fusion feature map / the second fusion feature map / the third fusion feature map is compared with the set segmentation threshold. If the value of the pixel point of the first fusion feature map / the second fusion feature map / the third fusion feature map is greater than the segmentation threshold, the corresponding pixel point at the same position on the iris image is judged to belong to the iris effective area / the iris inner boundary point / the iris outer boundary point.

[0020] Furthermore, the fractional fusion of the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position also includes:

[0021] Performing a transformation operation on the iris image to obtain a transformed image; wherein the transformation operation includes image rotation and / or image flipping;

[0022] The transformed images are respectively input into the multiple deep learning models, the iris image is encoded, the encoded feature map is expanded to the same size as the iris image using deconvolution, and the expanded feature map is subjected to an inverse transformation operation opposite to the transformation operation to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models.

[0023] Further, the fractionally fusing the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map includes:

[0024] According to the set weight of each of the first probability feature map / the second probability feature map / the third probability feature map, weighted sum is performed on the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map;

[0025] Among them, the weight of each of the first probability feature map / the second probability feature map / the third probability feature map is determined according to the distribution of the score set of the deep learning model used by the first probability feature map / the second probability feature map / the third probability feature map on its training data set and the false rejection rate of the deep learning model.

[0026] Furthermore, the weight is determined by the following method:

[0027] Use each deep learning model to judge its training data set respectively, and obtain a score set for each deep learning model;

[0028] The score set includes a foreground score determined as a foreground and a background score determined as a background;

[0029] Normalizing the score set of each deep learning model according to the non-overlapping area of ​​the distribution of foreground scores and the distribution of background scores of each deep learning model;

[0030] The weight corresponding to each deep learning model is determined according to the normalized score set of each deep learning model and the false rejection rate of each deep learning model.

[0031] Furthermore, the step of normalizing the score set of each deep learning model according to the non-overlapping area of ​​the distribution of the foreground score and the distribution of the background score of each deep learning model includes:

[0032] The non-overlapping area is calculated by the following formula as a normalized dimension;

[0033] q(k)=μ a (k)-min a (k)+max r (k)-μ r (k)

[0034] Where q(k) is the normalized dimension of the kth deep learning model, k = 1, 2, ..., m, m is the total number of deep learning models, max r (k) is the maximum foreground score of the kth deep learning model, μ r (k) is the average foreground score of the kth deep learning model, min a (k) is the minimum background score of the k-th deep learning model, μ a (k) is the average background score of the kth deep learning model;

[0035] Normalize the score set of each deep learning model according to the normalized dimension of each deep learning model;

[0036]

[0037] in, and are the i-th score of the score set of the k-th deep learning model before and after normalization, i = 1, 2, 3, ..., N k , N k is the total number of training samples in the training data set of the kth deep learning model, j = 0 means is the background score, j = 1 means is the foreground score.

[0038] Furthermore, determining the weight corresponding to each deep learning model according to the normalized score set of each deep learning model and the false rejection rate of each deep learning model includes:

[0039] The weight corresponding to each deep learning model is calculated by the following formula;

[0040]

[0041] Among them, w(k) is the weight corresponding to the k-th deep learning model;

[0042]

[0043] FRR k is the false rejection rate of the k-th deep learning model.

[0044] Further, the first fused feature map / the second fused feature map / the third fused feature map are calculated by the following formula:

[0045]

[0046] Among them, Seg (x,y) represents the value of the pixel at the xth row and yth column of the first fused feature map / the second fused feature map / the third fused feature map, Represents the value of the pixel at the xth row and yth column position of the kth first probability feature map / the second probability feature map / the third probability feature map.

[0047] Furthermore, in the process of encoding the iris image and using deconvolution to expand the encoded feature map to the same size as the iris image, the intermediate feature layers are connected in the dimension of channels.

[0048] Furthermore, the step of performing ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary may also include:

[0049] Find the maximum connected area for the iris effective area / the iris inner boundary point / the iris outer boundary point, and use the maximum connected area as the final iris effective area / the iris inner boundary point / the iris outer boundary point.

[0050] Furthermore, during training, the deep learning model randomly performs data enhancement on training samples in the training data set, and / or the training samples in the training data set include oblique images and / or blurred images;

[0051] Wherein, the data enhancement includes one or more of flipping, translation, noise transformation and elastic deformation;

[0052] The elastic deformation includes: dividing the training sample into sub-blocks of a set size, and performing shape distortion on each sub-block.

[0053] Furthermore, the step of inputting the iris image into a plurality of pre-trained deep learning models may also include:

[0054] Determine whether the grayscale mean of the iris image is less than a set first grayscale threshold or greater than a set second grayscale threshold, and if so, perform histogram equalization on the iris image;

[0055] The first grayscale threshold is smaller than the second grayscale threshold.

[0056] In a second aspect, the present application provides an iris image preprocessing method, the method comprising:

[0057] An image acquisition module, used for acquiring an iris image;

[0058] a segmentation module, configured to input the iris image into a plurality of deep learning models pre-trained by different training data sets, respectively, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and fractionally fuse the segmentation results of the plurality of deep learning models to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image;

[0059] Wherein, when performing the iris effective area segmentation, the iris texture area is used as the foreground area, and the rest of the area is used as the background area, and the iris image is segmented to segment the iris effective area for iris feature extraction from the iris image, which includes only the iris texture area;

[0060] When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area, the iris image is segmented, and the iris inner boundary point / iris outer boundary point is segmented from the iris image;

[0061] The fitting module is used to perform ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

[0062] Furthermore, the segmentation module includes:

[0063] A first processing unit is used to input the iris image into the multiple deep learning models respectively, encode the iris image, and use deconvolution to expand the encoded feature map to the same size as the iris image, so as to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models;

[0064] The value of each pixel point on the first probability feature map / the second probability feature map / the third probability feature map represents the probability that the corresponding pixel point at the same position on the iris image belongs to the iris effective area / the iris inner boundary point / the iris outer boundary point;

[0065] A score fusion unit, used for performing score fusion on the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain a first fused feature map / a second fused feature map / a third fused feature map;

[0066] a judgment unit, used for comparing the value of each pixel point on the first fusion feature map / the second fusion feature map / the third fusion feature map with a set segmentation threshold; if the value of the pixel point of the first fusion feature map / the second fusion feature map / the third fusion feature map is greater than the segmentation threshold, the pixel point at the corresponding same position on the iris image is judged to belong to the iris effective area / the iris inner boundary point / the iris outer boundary point.

[0067] Furthermore, the segmentation module also includes:

[0068] A transformation unit, configured to perform a transformation operation on the iris image to obtain a transformed image; wherein the transformation operation includes image rotation and / or image flipping;

[0069] The second processing unit is used to input the transformed images into the multiple deep learning models respectively, encode the iris image, use deconvolution to expand the encoded feature map to the same size as the iris image, and perform an inverse transformation operation on the expanded feature map that is opposite to the transformation operation to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models.

[0070] Furthermore, the score fusion unit is used for:

[0071] According to the set weight of each of the first probability feature map / the second probability feature map / the third probability feature map, weighted sum is performed on the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map;

[0072] Among them, the weight of each of the first probability feature map / the second probability feature map / the third probability feature map is determined according to the distribution of the score set of the deep learning model used by the first probability feature map / the second probability feature map / the third probability feature map on its training data set and the false rejection rate of the deep learning model.

[0073] Furthermore, the weight is determined by the following modules:

[0074] A score set calculation module is used to use each deep learning model to judge its training data set and obtain a score set for each deep learning model;

[0075] The score set includes a foreground score determined as a foreground and a background score determined as a background;

[0076] A normalization module, for normalizing the score set of each deep learning model according to a non-overlapping area of ​​a distribution of foreground scores and a distribution of background scores of each deep learning model;

[0077] The weight setting module is used to determine the weight corresponding to each deep learning model according to the normalized score set of each deep learning model and the rejection rate of each deep learning model.

[0078] Furthermore, the normalization module includes:

[0079] A normalized dimension calculation unit, used to calculate the non-overlapping area by the following formula as a normalized dimension;

[0080] q(k)=μ a (k)-min a (k)+max r (k)-μr (k)

[0081] Where q(k) is the normalized dimension of the kth deep learning model, k = 1, 2, ..., m, m is the total number of deep learning models, max r (k) is the maximum foreground score of the kth deep learning model, μ r (k) is the average foreground score of the kth deep learning model, min a (k) is the minimum background score of the k-th deep learning model, μ a (k) is the average background score of the kth deep learning model;

[0082] A normalization unit, used to normalize the score set of each deep learning model according to the normalization dimension of each deep learning model;

[0083]

[0084] in, and are the i-th score of the score set of the k-th deep learning model before and after normalization, i = 1, 2, 3, ..., N k , N k is the total number of training samples in the training data set of the kth deep learning model, j = 0 means is the background score, j = 1 means is the foreground score.

[0085] Furthermore, in the weight setting module, the weight corresponding to each deep learning model is calculated by the following formula;

[0086]

[0087] Among them, w(k) is the weight corresponding to the k-th deep learning model;

[0088]

[0089] FRR k is the false rejection rate of the k-th deep learning model.

[0090] Furthermore, in the score fusion unit, the first fused feature map / the second fused feature map / the third fused feature map are calculated by the following formula:

[0091]

[0092] Among them, Seg (x,y)represents the value of the pixel at the xth row and yth column of the first fused feature map / the second fused feature map / the third fused feature map, Represents the value of the pixel at the xth row and yth column position of the kth first probability feature map / the second probability feature map / the third probability feature map.

[0093] Furthermore, during the execution of the first processing unit and the second processing unit, the intermediate feature layers are connected in the dimension of channels.

[0094] Furthermore, the device also includes:

[0095] A post-processing module is used to find the maximum connected area for the iris effective area / the iris inner boundary point / the iris outer boundary point, and use the maximum connected area as the final iris effective area / the iris inner boundary point / the iris outer boundary point.

[0096] Furthermore, during training, the deep learning model randomly performs data enhancement on training samples in the training data set, and / or the training samples in the training data set include oblique images and / or blurred images;

[0097] Wherein, the data enhancement includes one or more of flipping, translation, noise transformation and elastic deformation;

[0098] The elastic deformation includes: dividing the training sample into sub-blocks of a set size, and performing shape distortion on each sub-block.

[0099] Furthermore, the device also includes:

[0100] A histogram equalization module, used to determine whether the grayscale mean of the iris image is less than a set first grayscale threshold or greater than a set second grayscale threshold, and if so, perform histogram equalization on the iris image;

[0101] The first grayscale threshold is smaller than the second grayscale threshold.

[0102] In a third aspect, the present application provides a computer-readable storage medium for iris image preprocessing, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the iris image preprocessing method described in the first aspect.

[0103] In a fourth aspect, the present application provides a device for iris image preprocessing, characterized in that it includes at least one processor and a memory storing computer-executable instructions, and when the processor executes the instructions, it implements the steps of the iris image preprocessing method described in the first aspect.

[0104] This application has the following beneficial effects:

[0105] The present application uses a deep learning model to solve the problems of iris inner boundary positioning, iris outer boundary positioning and / or iris effective area segmentation in iris recognition. The solution is simple, robust and generalizable. The method of the present application can directly segment the iris effective area, and the segmented iris effective area does not include noise such as eyelids, eyelashes, and light spots. The present application equates the iris segmentation and noise detection problems to a semantic segmentation problem of iris texture, which is a single-instance iris segmentation algorithm. Compared with the multi-stage method of the prior art, the segmentation task becomes simple, clear, more accurate, and more generalizable and robust.

[0106] The present application does not adopt the method of determining the inner and outer boundaries of the iris by calculating the gradient of the image in the prior art, but adopts the method of instance segmentation through a deep learning model, and uses deep learning to simultaneously complete the positioning of the inner and outer boundaries of the iris and the segmentation of the effective area of ​​the iris. The parameters of the deep learning model are obtained through the learning of a large amount of data, and are not the empirical expert system used in the prior art. Due to the diversity of data, the deep learning model learns experience from diverse samples, has good generalization and robustness to different changes, and has good performance under different changes in lighting, angles, etc. It can well solve the problems of locating the inner and outer boundaries of the iris image and segmenting the effective area of ​​the iris in cases of strabismus, squinting, etc. And by locating the inner and outer boundaries through ellipse fitting, the inner and outer boundaries of the iris are made more neat, which is conducive to subsequent boundary correction and iris area expansion.

[0107] This application uses multiple deep learning models to segment iris images separately and performs score fusion on the segmentation results. It has good generalization and robustness, and the results of iris segmentation and positioning are more accurate. It is suitable for iris images of various qualities of different iris recognition systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0108] Figure 1 This is the effect diagram of positioning the inner and outer circles of the iris;

[0109] Figure 2 This is the effect picture after positioning the inner and outer circles of the iris and the eyelids;

[0110] Figure 3 This is the effect diagram of the noise template;

[0111] Figure 4 This is a flow chart of the iris image segmentation method of the present application;

[0112] Figure 5 is a schematic diagram of an iris image before preprocessing;

[0113] Figure 6is a schematic diagram of a preprocessed iris image;

[0114] Figure 7 for Figure 5 Schematic diagram of the effective iris area after iris image segmentation;

[0115] Figure 8 Schematic diagram of iris images of various qualities;

[0116] Fig. 9 To use multiple deep learning models Figure 8 Schematic diagram of the results of locating the inner and outer boundaries of the iris and segmenting the effective area;

[0117] Fig.10 is a schematic diagram of an iris image before histogram equalization;

[0118] Fig.11 is a schematic diagram of an iris image after histogram equalization;

[0119] Fig.12 The schematic diagram is the result of locating the inner and outer boundaries of the iris and segmenting the effective area of ​​the iris image after histogram equalization;

[0120] Fig.13 Schematic diagram of the distribution of foreground scores and background scores;

[0121] Fig.14 This is a schematic diagram of an iris image segmentation device of the present application. DETAILED DESCRIPTION

[0122] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0123] The present application embodiment provides an iris image preprocessing method for obtaining the inner and outer boundaries of the iris and the effective area of ​​the iris from the iris image. Figure 4 As shown, the method includes:

[0124] S100: Acquire an iris image.

[0125] The iris image can be collected in real time by an iris collection device, or a pre-collected and stored iris image can be read from a storage space.

[0126] After obtaining the iris image, it can be preprocessed to transform the scale of the iris image and normalize it to a set size to adapt to the input size of the deep learning model. For example, the set size can be 512 pixels in length and width. A specific implementation process of the preprocessing is as follows:

[0127] Determine whether the width (w) and height (h) of the iris image are equal. If w=h, scale the iris image to a size where both the length and width are 512 pixels.

[0128] If w>h, calculate the row supplementary border height The upper and lower boundaries of the iris image are supplemented with black boundaries with a width of w, a height of pad1, and a pixel value of 0, respectively, and the supplemented iris image has a width of w and a height of h or h-2. Then, the supplemented iris image is scaled to a size of 512 pixels in both length and width.

[0129] If w<h, calculate the row supplementary border width The left and right borders of the iris image are supplemented with black borders with a width of pad1, a height of h, and a pixel value of 0, respectively, and the supplemented iris image has a width of w or w-2 and a height of h. Then, the supplemented iris image is scaled to a size of 512 pixels in both length and width.

[0130] In one example, the iris image before preprocessing is as follows: Figure 5 As shown in Figure 2, the preprocessed iris image is Figure 6 shown.

[0131] S200: Inputting the iris image into a plurality of deep learning models that are pre-trained using different training data sets, respectively, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and performing score fusion on the segmentation results of the plurality of deep learning models to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image.

[0132] When performing the iris effective area segmentation, the iris texture area is used as the foreground area and the rest are used as the background area. The iris image is segmented to obtain an iris effective area for iris feature extraction that only includes the iris texture area.

[0133] When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area. The iris image is segmented to segment the iris inner boundary points / iris outer boundary points from the iris image.

[0134] This application uses a deep learning method to solve the problem of iris effective area positioning in iris recognition. The iris image is segmented based on the iris texture, and the iris texture is regarded as a category as the foreground area, and the rest of the non-iris part is regarded as a category, such as the eyelids, eyelashes, the area of ​​the reflected light spot, and other areas, which are uniformly regarded as the non-iris area as the background area. Figure 5 The effective iris area after iris image segmentation is shown as Figure 7 The segmented iris effective area only includes the iris texture area, which is used for iris feature extraction in iris recognition.

[0135] This application solves the iris boundary positioning problem in iris recognition through a deep learning method. Using an instance segmentation scheme, the inner and outer boundary areas of the iris image are segmented as foreground targets to obtain a series of iris inner and outer boundary points.

[0136] The deep learning model of the present application can perform one or more tasks of iris effective area segmentation, iris inner boundary point segmentation and iris outer boundary point segmentation. When performing multiple tasks, they are performed simultaneously through a multi-task supervision method. The obtained iris effective area can be used as the actual iris effective area, and the obtained iris inner boundary points and iris outer boundary points are used for subsequent fitting of the inner and outer boundaries of the iris.

[0137] In different iris recognition systems, the quality of the collected iris images varies greatly (e.g. Figure 8 As shown in the figure, including iris images of various qualities), a positioning and segmentation algorithm with good generalization and strong robustness is extremely important. This application uses a multi-model integration method to set multiple deep learning models (for example, 5) and perform score fusion on the segmentation results output by multiple deep learning models, so that the results of iris segmentation and positioning are more accurate. Figure 8 The iris images of various qualities are shown in the figure to locate the inner and outer boundaries of the iris and segment the effective area. The results are as follows Fig. 9 As shown, it can be seen that the result is relatively accurate.

[0138] Each of the multiple deep learning models is trained using a different training data set. The different training data sets may be different independent data sets established, or may be a total data set divided into multiple different training data sets.

[0139] S300: performing ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

[0140] The present application assumes that the inner and outer boundaries of the iris are ellipses, and fits them as ellipses. The ellipses also include perfect circles, because perfect circles can also be regarded as a special ellipse in terms of expression.

[0141] The general equation of an ellipse is:

[0142] Ax 2 +Bxy+Cx 2 +Dx+Ey+F=0

[0143] Among them, A, B, C, D, E, and F are parameters to be determined. Substituting the inner boundary point / outer boundary point of the iris into the above equation, the expression of the ellipse can be obtained, and the inner boundary / outer boundary of the iris can be obtained.

[0144] It should be noted that at least 5 points are required to obtain the above equation. Therefore, if the number of iris inner boundary points / iris outer boundary points is less than 5, the result without segmentation is directly returned. If the number of points is greater than or equal to 5 points, the above equation is fitted to obtain five parameters. The ellipse described by the equation is the final iris inner boundary / iris outer boundary.

[0145] The inner boundary and the outer boundary of the iris can be used to locate the iris, estimate the actual size and position of the iris in the image, etc., and can also be used to correct the iris image.

[0146] The existing iris image preprocessing is a multi-stage method, which requires locating the inner and outer boundaries of the iris on the iris image, then performing iris noise template detection, and finally segmenting the iris effective area. The iris effective area cannot be directly segmented in one step. Each step has a great impact on the accuracy of iris recognition, and the steps are relatively cumbersome and complicated.

[0147] The present application uses a deep learning model to solve the problems of iris inner boundary positioning, iris outer boundary positioning and / or iris effective area segmentation in iris recognition. The solution is simple, robust and generalizable. The method of the present application can directly segment the iris effective area that only includes the iris texture area, and the segmented iris effective area does not include noise such as eyelids, eyelashes, and light spots. The present application equates the iris segmentation and noise detection problems to a semantic segmentation problem of iris texture, which is a single-instance iris segmentation algorithm. Compared with the multi-stage method of the prior art, the segmentation task becomes simple, clear, more accurate, and more generalizable and robust.

[0148] Existing iris preprocessing methods require noise detection. The types of iris noise include eyelid occlusion, eyelash occlusion, and light spot occlusion. Each type of noise requires detection and segmentation using digital image processing methods. The methods for detecting eyelids, eyelashes, and light spots are also inconsistent. Not only are the steps cumbersome, but traditional noise detection methods are easily affected by factors such as lighting and angles, and the algorithm generalization is poor. Traditional iris and eyelash detection methods use expert systems, and most of them use methods such as threshold segmentation for image segmentation or positioning. This method has poor generalization, is greatly affected by the environment, and the algorithm performs poorly on large-scale data. The present application directly uses a deep learning method to achieve iris effective area segmentation, iris inner boundary point segmentation, and iris outer boundary point segmentation, without the need for iris noise detection, avoiding the problems of various noise detection methods that are cumbersome, easily affected by lighting and angles, and have poor generalization.

[0149] Traditional methods for locating the inner and outer boundaries of the iris are generally based on the gradient of the image (such as calculus detection operators based on gradient amplitude and Hough transform based on edge gradient binarization, etc.). The inner and outer boundaries of the iris are determined by calculating the gradient of the image, and their generalization is poor. Moreover, for strabismus and blurred iris images, the inner and outer boundaries of the iris cannot be accurately located, and image quality needs to be controlled.

[0150] The present application does not adopt the method of determining the inner and outer boundaries of the iris by calculating the gradient of the image in the prior art, but adopts the method of instance segmentation through a deep learning model, and uses deep learning to simultaneously complete the positioning of the inner and outer boundaries of the iris and the segmentation of the effective area of ​​the iris. The parameters of the deep learning model are obtained through the learning of a large amount of data, and are not the empirical expert system used in the prior art. Due to the diversity of data, the deep learning model learns experience from diverse samples, has good generalization and robustness to different changes, and has good performance under different changes in lighting, angles, etc. It can well solve the problems of locating the inner and outer boundaries of the iris image and segmenting the effective area of ​​the iris in cases of strabismus, squinting, etc. And by locating the inner and outer boundaries through ellipse fitting, the inner and outer boundaries of the iris are made more neat, which is conducive to subsequent boundary correction and iris area expansion.

[0151] This application uses multiple deep learning models to segment iris images separately and performs score fusion on the segmentation results. It has good generalization and robustness, and the results of iris segmentation and positioning are more accurate. It is suitable for iris images of various qualities of different iris recognition systems.

[0152] In the prior art, the inner and outer boundaries of the iris are determined by calculating the gradient of the image. When the contrast of the iris image is low, the gradient of the image is not obvious, so it is difficult to locate. The present application can effectively solve the problem of locating the inner and outer boundaries of the iris in images with low contrast.

[0153] Traditional iris recognition usually has many limitations in terms of user cooperation and image conditions, which seriously affect the widespread application of iris recognition. In recent years, many non-cooperative or low-restriction iris recognition methods have also emerged, such as long-distance iris recognition, iris recognition in motion, iris recognition without perception and cooperation, iris recognition under dynamic image environment changes, and iris recognition on mobile devices. The iris area collected under these environments usually contains a variety of noises, such as occlusion caused by eyelids or eyelashes, mirror reflection, strabismus, blur, etc., which affect the segmentation of the effective iris area. In addition, the irises collected under these environments usually contain more complex and difficult-to-locate images such as strabismus and squinting. The inner and outer boundary positioning results of the iris image are more important for the expansion of the iris area, which will directly affect the final recognition effect.

[0154] In response to the above problems, this application proposes a method for iris boundary location and iris effective area segmentation based on a deep learning model, which improves the generalization ability of iris inner and outer boundary location, and can be used to segment the visible iris boundary in complex and difficult-to-locate iris images such as strabismus and squinting. It also improves the generalization ability of iris segmentation, and can be well applied to the above-mentioned noisy scenes, achieving efficient and robust iris segmentation on noisy images.

[0155] The deep learning model used in the present application may be a convolutional neural network. Accordingly, the aforementioned S200 may include:

[0156] S210: Input the iris image into multiple deep learning models respectively, encode the iris image, and use deconvolution to expand the encoded feature map to the same size as the iris image, so as to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models.

[0157] The value of each pixel point on the first probability feature map / the second probability feature map / the third probability feature map represents the probability that the corresponding pixel point at the same position on the iris image belongs to the iris effective area / iris inner boundary point / iris outer boundary point.

[0158] Assuming that the number of deep learning models is 5, the same iris image is processed by 5 deep learning models respectively, and 5 first probability feature maps / second probability feature maps / third probability feature maps can be obtained.

[0159] In order to better combine semantic features and spatial features, in this step, the intermediate feature layers can be connected in the channel dimension to perform feature enhancement.

[0160] In one example, the specific implementation of S210 of the present application includes:

[0161] The deep learning model is fed with an iris image of size 512x512 and 3 channels.

[0162] For the above image, a convolution operation with a convolution kernel of size 7x7, a stride of 2, and 3 pixels of edge padding is used, and BN and ReLU operations are performed to obtain a 64-channel feature map f0 of size 256x256.

[0163] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 2, and 1 pixel padding on the edge to obtain a feature map A1 of size 128x128 and 64 channels.

[0164] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 64-channel feature map of size 128x128.

[0165] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a 64-channel feature map B1 of size 128x128.

[0166] Add feature map A1 and feature map B1 pixel by pixel on each channel to obtain a 64-channel feature map f1 of size 128x128.

[0167] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 64-channel feature map of size 128x128.

[0168] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a 64-channel feature map B2 of size 128x128.

[0169] The feature map f1 and the feature map B2 are added pixel by pixel on each channel to obtain a 64-channel feature map f2 of size 128x128.

[0170] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 64-channel feature map of size 128x128.

[0171] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a 64-channel feature map B3 of size 128x128.

[0172] The feature map f2 and the feature map B3 are added pixel by pixel on each channel to obtain a 64-channel feature map f3 of size 128x128.

[0173] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a stride of 2, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 64x64 with 128 channels.

[0174] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a feature map B4 of size 64x64 and 128 channels.

[0175] A convolution operation with a convolution kernel of size 1x1, a stride of 1, and 0 pixel padding on the edge is performed on the feature map f3 to obtain a feature map A3 of size 64x64 and 128 channels.

[0176] The feature map A3 and the feature map B4 are added pixel by pixel on each channel to obtain a 128-channel feature map f4 of size 64x64.

[0177] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 64x64 with 128 channels.

[0178] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a feature map B5 of size 64x64 and 128 channels.

[0179] The feature map f4 and the feature map B5 are added pixel by pixel on each channel to obtain a 128-channel feature map f5 of size 64x64.

[0180] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 64x64 with 128 channels.

[0181] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a feature map B6 of size 64x64 and 128 channels.

[0182] The feature map f5 and the feature map B6 are added pixel by pixel on each channel to obtain a 128-channel feature map f7 of size 64x64.

[0183] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 64x64 with 128 channels.

[0184] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a feature map B7 of size 64x64 and 128 channels.

[0185] The feature map f7 and the feature map B7 are added pixel by pixel on each channel to obtain a 128-channel feature map f8 of size 64x64.

[0186] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 2, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0187] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and a BN operation is performed to obtain a feature map B8 of size 32x32 and 256 channels.

[0188] A convolution operation with a convolution kernel of size 1x1, a stride of 1, and 0 pixel padding on the edge is performed on the feature map f8 to obtain a feature map A8 of size 32x32 and 256 channels.

[0189] The feature map A8 and the feature map B8 are added pixel by pixel on each channel to obtain a feature map f9 of 256 channels with a size of 32x32.

[0190] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0191] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B9 of size 32x32 and 256 channels.

[0192] Add feature map f9 and feature map B9 pixel by pixel on each channel to obtain a feature map f with 256 channels of size 32x32. 10 .

[0193] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0194] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 32x32 with 256 channels. 10 .

[0195] The feature map f 10 And feature map B 10 Add pixel by pixel on each channel to get a 256-channel feature map f with a size of 32x32. 11 .

[0196] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0197] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 32x32 with 256 channels. 11 .

[0198] The feature map f 11 And feature map B 11 Add pixel by pixel on each channel to get a 256-channel feature map f with a size of 32x32. 12 .

[0199] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0200] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 32x32 with 256 channels. 12 .

[0201] The feature map f 12 And feature map B 12 Add pixel by pixel on each channel to get a 256-channel feature map f with a size of 32x32. 13 .

[0202] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0203] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 32x32 with 256 channels. 13 .

[0204] The feature map f 13 And feature map B 13 Add pixel by pixel on each channel to get a 256-channel feature map f with a size of 32x32. 14 .

[0205] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 2, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 16x16 with 512 channels.

[0206] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 16x16 with 512 channels. 14 .

[0207] For the feature map f 14 Using a convolution kernel of size 1x1, a stride of 1, and a convolution operation with 0 pixels padded on the edges, we get a feature map A of 512 channels of size 16x16. 14 .

[0208] The feature map A 14 And feature map B 14 Add pixel by pixel on each channel to get a 512-channel feature map f with a size of 16x16. 15 .

[0209] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge, and BN and ReLU operations are performed to obtain a feature map of size 16x16 with 512 channels.

[0210] The above feature map is convolved with a convolution kernel of size 3x3, a step size of 1, and a convolution operation with 1 pixel padding on the edge, and a BN operation is performed to obtain a feature map B of size 16x16 with 512 channels. 15 .

[0211] The feature map f 15 And feature map B 15 Add pixel by pixel on each channel to get a 512-channel feature map f with a size of 16x16. 16 .

[0212] At this point, the process of encoding the iris image is completed. The subsequent process is to use deconvolution to expand the encoded feature map to the same size as the iris image to obtain the first / second / third probability feature map.

[0213] The feature map f 16 Use the nearest neighbor interpolation to perform 2x interpolation to obtain a 512-channel feature map f with a size of 32x32. 17 .

[0214] f 17 and f 14 Connect in the channel dimension to get a feature map f with 768 channels of size 32x32 18 .

[0215] f 18 A convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge is performed, and BN and ReLU operations are performed to obtain a feature map of size 32x32 with 256 channels.

[0216] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 256-channel feature map f of size 32x32. 19 .

[0217] The feature map f 19 Use the nearest neighbor interpolation to perform 2x interpolation to obtain a 256-channel feature map f with a size of 64x64. 20 .

[0218] f 20 Connect f8 with f in the channel dimension to obtain a feature map f with a size of 64x64 and 384 channels. 21 .

[0219] f 21 A convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge is performed, and BN and ReLU operations are performed to obtain a feature map of 128 channels with a size of 64x64.

[0220] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 128-channel feature map f with a size of 64x64. 22 .

[0221] The feature map f 22 Use the nearest neighbor interpolation to perform 2x interpolation to obtain a 128-channel feature map f with a size of 128x128. 23 .

[0222] f 23 Connect f3 with f in the channel dimension to obtain a 192-channel feature map f with a size of 128x128 24 .

[0223] f 24 A convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge is performed, and BN and ReLU operations are performed to obtain a 64-channel feature map of size 128x128.

[0224] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 64-channel feature map f with a size of 128x128. 25 .

[0225] The feature map f 25 Use the nearest neighbor interpolation to perform 2x interpolation to obtain a 128-channel feature map f with a size of 256x256. 26 .

[0226] f 26 Connect f0 with f0 in the channel dimension to obtain a 192-channel feature map f with a size of 256x256 27 .

[0227] f 27 A convolution operation with a convolution kernel of size 3x3, a stride of 1, and 1 pixel padding on the edge is performed, and BN and ReLU operations are performed to obtain a 64-channel feature map of size 256x256.

[0228] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 64-channel feature map f with a size of 256x256. 28 .

[0229] The feature map f 28 Use the nearest neighbor interpolation to perform 2x interpolation to obtain a 128-channel feature map f with a size of 512x512. 29 .

[0230] f 29 Use a convolution kernel of size 3x3, a stride of 1, a convolution operation with 1 pixel padding on the edge, and perform BN and ReLU operations to obtain a 16-channel feature map of size 512x512.

[0231] The above feature map is subjected to a convolution operation with a convolution kernel of size 3x3, a step size of 1, and a 1-pixel padding on the edge, and BN and ReLU operations are performed to obtain a 16-channel feature map f with a size of 512x512.30 .

[0232] f 30 Use a convolution kernel of size 3x3, a step size of 1, a convolution operation with 1 pixel padding on the edge, and perform BN and ReLU operations to obtain a single-channel feature map f of size 512x512. 31 .

[0233] f 30 Use a convolution kernel of size 3x3, a step size of 1, a convolution operation with 1 pixel padding on the edge, and perform BN and ReLU operations to obtain a single-channel feature map f of size 512x512. 32 .

[0234] f 30 Use a convolution kernel of size 3x3, a step size of 1, a convolution operation with 1 pixel padding on the edge, and perform BN and ReLU operations to obtain a single-channel feature map f of size 512x512. 33 .

[0235] Finally, for f 31 Using the softmax operation, a first probability feature map P1 of size 512x512 can be obtained for estimating the iris segmentation result. The value of each position on the first probability feature map P1 represents the probability that the corresponding position of the original iris image is the valid iris area.

[0236] f 32 Using the softmax operation, a third probability feature map P3 of size 512x512 can be obtained for estimating the iris outer boundary result. The value of each position on the third probability feature map P3 represents the probability that the corresponding position of the original iris image is the iris outer boundary.

[0237] f 33 Using the softmax operation, a second probability feature map P2 of size 512x512 can be obtained for estimating the inner boundary of the iris. The value of each position on the second probability feature map P2 represents the probability that the corresponding position of the original iris image is the inner boundary of the iris.

[0238] S220: Score-fuse the values ​​of all the first probability feature maps / second probability feature maps / third probability feature maps at the same position to obtain the first fused feature map / second fused feature map / third fused feature map.

[0239] S230: Compare the value of each pixel point on the first fusion feature map / the second fusion feature map / the third fusion feature map with the set segmentation threshold. If the value of the pixel point of the first fusion feature map / the second fusion feature map / the third fusion feature map is greater than the segmentation threshold, the corresponding pixel point at the same position on the iris image is judged to belong to the iris effective area / iris inner boundary point / iris outer boundary point.

[0240] Taking the first fused feature map as an example, assuming that 0.5 is the segmentation threshold, if the value p(x,y) of the pixel point at any position (x,y) of the first fused feature map P1 is greater than 0.5, it indicates that the pixel point at the position (x,y) on the iris image belongs to the iris effective area. The same is true for the second and third fused feature maps.

[0241] In order to further improve the accuracy of the results of iris segmentation and positioning, the present application adopts data enhancement operation during the processing of iris images by the deep learning model (i.e., during the testing process). Specifically, the S200 further includes:

[0242] S211: Performing a transformation operation on the iris image to obtain a transformed image.

[0243] The transformation operation includes image rotation and / or image flipping. In one example, the transformation operation is upside down flipping.

[0244] S212: Input the transformed images into multiple deep learning models respectively, encode the iris image, use deconvolution to expand the encoded feature map to the same size as the iris image, and perform an inverse transformation operation on the expanded feature map that is opposite to the transformation operation to obtain a first probability feature map, a second probability feature map, and a third probability feature map that are the same number as the deep learning model.

[0245] Assuming that the number of deep learning models is 5, the same transformed image is processed by 5 deep learning models respectively, and 5 first probability feature maps / second probability feature maps / third probability feature maps can be obtained. Since the input is a transformed image, the output feature map needs to be inversely transformed to correspond to the original iris image.

[0246] The five first probability feature maps / second probability feature maps / third probability feature maps after the iris image is processed by the five deep learning models are denoted as Seg K (k=1, 2, 3, 4, 5), the five first probability feature maps / second probability feature maps / third probability feature maps after the transformed image is processed by the five deep learning models are denoted as Seg k (k=6,7,8,9,10).

[0247] Next, score fusion is performed in S220 to obtain the first fused feature map / the second probability feature map Seg / the third probability feature map Seg.

[0248] As an improvement of the embodiment of the present application, the aforementioned S220 includes:

[0249] According to the set weight of each first probability feature map / second probability feature map / third probability feature map, the values ​​of all first probability feature maps / second probability feature maps / third probability feature maps at the same position are weighted summed to obtain the first fused feature map / second fused feature map / third fused feature map.

[0250] Among them, the weight w(k) of each first probability feature map / second probability feature map / third probability feature map is determined according to the distribution of the score set of the deep learning model used by the first probability feature map / second probability feature map / third probability feature map on its training data set and the false rejection rate of the deep learning model.

[0251] Specifically, the weight can be determined by the following method:

[0252] S10: Use each deep learning model to judge its training data set respectively to obtain a score set for each deep learning model.

[0253] In this application, the definition The score of the k-th deep learning model (the k-th deep learning model is the model used by the k-th first probability feature map / second probability feature map / third probability feature map) for judging the i-th training sample in its training data set. i=1,2,3,…,N k ,N k is the total number of training samples involved in training the kth deep learning model (i.e., the total number of training samples in the training data set). k = 1, 2, ..., m, where m is the total number of deep learning models and also the total number of the first probability feature map / the second probability feature map / the third probability feature map.

[0254] N k The scores form a score set, which includes the foreground score judged as the foreground and the background score judged as the background. j = 0 means is the background score, j = 1 means is the foreground score. For the deep learning model corresponding to the first probability feature map, the foreground is determined to be the effective area of ​​the iris, for the deep learning model corresponding to the second probability feature map, the foreground is determined to be the inner boundary point of the iris, and for the deep learning model corresponding to the third probability feature map, the foreground is determined to be the outer boundary point of the iris.

[0255] The score set judged by the kth deep learning model as foreground is recorded as N k1 is the total number of foreground scores.

[0256] The score set judged as background by k deep learning models is recorded as N k2 is the total number of background scores.

[0257] Statistics show that the distribution of foreground scores and background scores of the k-th deep learning model basically satisfies the Gaussian distribution, as shown in Fig.13 As shown. Fig.13 In , the solid line represents the distribution of foreground scores, and the dotted line represents the distribution of background scores.

[0258] in,

[0259] max r (k) is the maximum value of the foreground score of the k-th deep learning model;

[0260] min r (k) is the minimum foreground score of the kth deep learning model;

[0261] μ r (k) is the average of the foreground scores of the kth deep learning model;

[0262] max a (k) is the maximum value of the background score of the kth deep learning model;

[0263] min a (k) is the minimum background score of the k-th deep learning model;

[0264] μ a (k) is the average background score of the kth deep learning model.

[0265] S20: Normalize the score set of each deep learning model according to the non-overlapping area of ​​the distribution of the foreground score of each deep learning model and the distribution of the background score.

[0266] Obviously, the overlapping part of the distribution of the foreground score and the background score predicted by the deep learning model can reflect the performance of the deep learning model. The larger the overlapping part, the less obvious the distinction between the segmented foreground and background scores, and the worse the performance. Conversely, the smaller the overlapping part, the higher the distinction between the segmented foreground and background scores, and the better the performance. Therefore, it is necessary to reduce the overlapping part of the scores and increase the difference between the two through the multimodal fusion method.

[0267] The multimodal method in this application adopts score-level fusion. First, the scores of different deep learning models need to be normalized according to the score fusion requirements. The specific implementation methods of normalization include:

[0268] 1. Calculate the non-overlapping area using the following formula as the normalized dimension.

[0269] q(k)=μ a (k)-min a (k)+max r (k)-μ r (k)

[0270] Among them, q(k) is the normalized dimension of the k-th deep learning model. The larger q(k) is, the greater the difference in the distribution of the foreground and background segmentation scores of the k-th deep learning model is, the higher the discrimination is, and the better the effect is. Using this dimension to normalize the scores can reduce the overlapping area of ​​the distribution and thus obtain better segmentation effects.

[0271] 2. Normalize the score set of each deep learning model according to the normalized dimension of each deep learning model. The normalization formula is as follows:

[0272]

[0273] in, and are the i-th score of the score set of the k-th deep learning model before and after normalization, respectively.

[0274] S30: Determine a weight corresponding to each deep learning model according to the normalized score set of each deep learning model and the false rejection rate of each deep learning model.

[0275] The two performance evaluation indicators for evaluating deep learning models are the false rejection rate (FRR) and the false recognition rate (FAR). In segmentation, FRR represents the proportion of foreground pixels that are mistakenly identified as background pixels, and FAR represents the proportion of background pixels that are identified as foreground pixels in segmentation. In the performance evaluation of segmentation algorithms, both are expected to be as small as possible. In multimodal fusion methods, not only the impact of the score dimension needs to be considered, but also the differences in the performance of different deep learning models may reduce the performance of the fused algorithm.

[0276] Therefore, in the score fusion method, this application introduces FRR into the calculation of weights, and calculates the weight corresponding to each deep learning model through the following formula;

[0277]

[0278] Among them, w(k) is the weight corresponding to the kth deep learning model, and m is the total number of the deep learning models.

[0279]

[0280] FRR k is the rejection rate of the k-th deep learning model, m is the total number of deep learning models, and is also the total number of the first probability feature map / the second probability feature map / the third probability feature map involved in score fusion.

[0281] The above is the training process of score fusion, the purpose of which is to obtain the score fusion weight w(k) through training data. After the weight is obtained through training, in the process of iris image preprocessing (i.e., reasoning process), for an unknown iris image, the first probability feature map / the second probability feature map / the third probability feature map are score fused (i.e., score fusion) through the trained weights to obtain the first fused feature map / the second fused feature map / the third fused feature map, and then determine whether the pixel of the iris image belongs to the foreground or the background.

[0282] Specifically, the calculation formulas of the first fusion feature map / the second fusion feature map / the third fusion feature map are as follows:

[0283]

[0284] Among them, Seg (x,y) Indicates the value of the pixel at the xth row and yth column position (i.e., (x, y) position) of the first fused feature map / the second fused feature map / the third fused feature map. Represents the value of the pixel at the xth row and yth column of the kth first probability feature map / second probability feature map / third probability feature map. w(k) represents the weight of the kth first probability feature map / second probability feature map / third probability feature map, that is, the weight of the kth deep learning model mentioned above.

[0285] Seg (x,y) If > segmentation threshold T, it means that the pixel at the (x, y) position of the iris image is the foreground (the foreground refers to the effective iris area, the inner boundary point of the iris or the outer boundary point of the iris), otherwise it is the background.

[0286] After the iris effective area / iris inner boundary point / iris outer boundary point are determined by the first / second / third probability feature map or the first / second / third fusion feature map, they can be directly used as the final result. Of course, the iris effective area / iris inner boundary point / iris outer boundary point can also be post-processed, and the post-processing method includes:

[0287] S240: Find the maximum connected area for the iris effective area / iris inner boundary point / iris outer boundary point, and use the maximum connected area as the final iris effective area / iris inner boundary point / iris outer boundary point.

[0288] By finding the largest connected area, the disconnected outliers can be removed and the segmentation result with the largest connected area can be retained. In this application, for each pixel point segmented, the 4-neighborhood maximum connected area method can be used to find the largest connected area, and the pixel point of the largest connected area is taken as the final result.

[0289] In the positioning of the inner and outer boundaries of the iris, it is more difficult to segment images that are flipped upside down, squinted, and squint. To address the above problems, the deep learning model of the present application randomly performs data enhancement on the training samples in the training data set during training.

[0290] Specifically, the training samples in the training data set are randomly flipped, translated, and noise transformed to make the data diverse. In addition, the elastic deformation method can be used to divide the training samples into sub-blocks of a set size (for example, 10*10 size), and each sub-block is distorted. The deep learning model trained by the above data enhancement method can well solve the segmentation problem of iris images such as strabismus and squinting, which helps to improve the generalization and accuracy of the deep learning model.

[0291] In the prior art, it is not possible to locate the iris boundary well for strabismus and blurred iris images. To solve this problem, the present application makes the training samples in the training data set include strabismus images and / or blurred images, which can better solve the segmentation problem of strabismus and blurred iris images and have more accurate segmentation results.

[0292] In the process of iris image segmentation, there are cases where the iris image with weak illumination or overexposure cannot be segmented or the segmentation result is incorrect, such as Fig.10 To solve the above problems, the method of the present application further includes:

[0293] S110: Determine whether the grayscale mean of the iris image is less than a set first grayscale threshold or greater than a set second grayscale threshold; if so, perform histogram equalization on the iris image;

[0294] The first grayscale threshold is smaller than the second grayscale threshold.

[0295] The present application determines the brightness of the iris image. For example, the value of the first grayscale threshold may be 50, and the value of the second grayscale threshold may be 200. When the grayscale mean of the iris image is less than 50 or greater than 200, the iris image needs to be histogram equalized. The iris image after histogram equalization can be better segmented.

[0296] The iris image before histogram equalization is as follows Fig.10 As shown in the figure, the iris image after histogram equalization is as follows Fig.11 As shown in the figure, the result of segmentation of the iris image after histogram equalization is as follows Fig.12 As shown, it can be seen that the result is relatively accurate.

[0297] The present application also provides an iris image preprocessing device. Fig.14 As shown, the device comprises:

[0298] The image acquisition module 100 is used to acquire an iris image.

[0299] The segmentation module 200 is used to input the iris image into multiple deep learning models that are pre-trained using different training data sets, respectively. Each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and fractionally fuses the segmentation results of the multiple deep learning models to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image.

[0300] When performing the iris effective area segmentation, the iris texture area is used as the foreground area and the rest are used as the background area. The iris image is segmented to obtain an iris effective area for iris feature extraction that only includes the iris texture area.

[0301] When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area. The iris image is segmented to segment the iris inner boundary points / iris outer boundary points from the iris image.

[0302] The fitting module 300 is used to perform ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

[0303] The present application uses a deep learning model to solve the problems of iris inner boundary positioning, iris outer boundary positioning and / or iris effective area segmentation in iris recognition. The solution is simple, robust and generalizable. The method of the present application can directly segment the iris effective area, and the segmented iris effective area does not include noise such as eyelids, eyelashes, and light spots. The present application equates the iris segmentation and noise detection problems to a semantic segmentation problem of iris texture, which is a single-instance iris segmentation algorithm. Compared with the multi-stage method of the prior art, the segmentation task becomes simple, clear, more accurate, and more generalizable and robust.

[0304] The present application does not adopt the method of determining the inner and outer boundaries of the iris by calculating the gradient of the image in the prior art, but adopts the method of instance segmentation through a deep learning model, and uses deep learning to simultaneously complete the positioning of the inner and outer boundaries of the iris and the segmentation of the effective area of ​​the iris. The parameters of the deep learning model are obtained through the learning of a large amount of data, and are not the empirical expert system used in the prior art. Due to the diversity of data, the deep learning model learns experience from diverse samples, has good generalization and robustness to different changes, and has good performance under different changes in lighting, angles, etc. It can well solve the problems of locating the inner and outer boundaries of the iris image and segmenting the effective area of ​​the iris in cases of strabismus, squinting, etc. And by locating the inner and outer boundaries through ellipse fitting, the inner and outer boundaries of the iris are made more neat, which is conducive to subsequent boundary correction and iris area expansion.

[0305] This application uses multiple deep learning models to segment iris images separately and performs score fusion on the segmentation results. It has good generalization and robustness, and the results of iris segmentation and positioning are more accurate. It is suitable for iris images of various qualities of different iris recognition systems.

[0306] The deep learning model used in this application can be a convolutional neural network. Accordingly, the segmentation module includes:

[0307] The first processing unit is used to input the iris image into multiple deep learning models respectively, encode the iris image, and use deconvolution to expand the encoded feature map to the same size as the iris image, so as to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models.

[0308] The value of each pixel point on the first probability feature map / the second probability feature map / the third probability feature map represents the probability that the corresponding pixel point at the same position on the iris image belongs to the iris effective area / iris inner boundary point / iris outer boundary point.

[0309] The score fusion unit is used to perform score fusion on the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map.

[0310] A judgment unit is used to compare the value of each pixel point on the first fusion feature map / the second fusion feature map / the third fusion feature map with a set segmentation threshold. If the value of the pixel point of the first fusion feature map / the second fusion feature map / the third fusion feature map is greater than the segmentation threshold, the corresponding pixel point at the same position on the iris image is judged to belong to the iris effective area / iris inner boundary point / iris outer boundary point.

[0311] Furthermore, the segmentation module also includes:

[0312] The transformation unit is used to perform a transformation operation on the iris image to obtain a transformed image; wherein the transformation operation includes image rotation and / or image flipping.

[0313] The second processing unit is used to input the transformed image into multiple deep learning models respectively, encode the iris image, use deconvolution to expand the encoded feature map to the same size as the iris image, and perform an inverse transformation operation on the expanded feature map that is opposite to the transformation operation to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning model.

[0314] In one example, the score fusion unit is used to:

[0315] According to the set weight of each first probability feature map / second probability feature map / third probability feature map, the values ​​of all first probability feature maps / second probability feature maps / third probability feature maps at the same position are weighted summed to obtain the first fused feature map / second fused feature map / third fused feature map.

[0316] Among them, the weight of each first probability feature map / second probability feature map / third probability feature map is determined according to the distribution of the score set of the deep learning model used by the first probability feature map / second probability feature map / third probability feature map on its training data set and the false rejection rate of the deep learning model.

[0317] In this application, the weights can be determined by the following modules:

[0318] The score set calculation module is used to use each deep learning model to judge its training data set respectively to obtain the score set of each deep learning model.

[0319] The score set includes foreground scores judged as foreground and background scores judged as background;

[0320] A normalization module is used to normalize the score set of each deep learning model according to the non-overlapping area of ​​the distribution of the foreground score of each deep learning model and the distribution of the background score.

[0321] The weight setting module is used to determine the weight corresponding to each deep learning model according to the normalized score set of each deep learning model and the rejection rate of each deep learning model.

[0322] Specifically, the normalization module may include:

[0323] The normalized dimension calculation unit is used to calculate the non-overlapping area as the normalized dimension using the following formula.

[0324] q(k)=μa (k)-min a (k)+max r (k)-μ r (k)

[0325] Where q(k) is the normalized dimension of the kth deep learning model, k = 1, 2, ..., m, m is the total number of deep learning models, max r (k) is the maximum foreground score of the kth deep learning model, μ r (k) is the average foreground score of the kth deep learning model, min a (k) is the minimum background score of the k-th deep learning model, μ a (k) is the average background score of the kth deep learning model.

[0326] A normalization unit is used to normalize the score set of each deep learning model according to the normalization dimension of each deep learning model.

[0327]

[0328] in, and are the i-th score of the score set of the k-th deep learning model before and after normalization, i = 1, 2, 3, ..., N k , N k is the total number of training samples in the training data set of the kth deep learning model, j = 0 means is the background score, j = 1 means is the foreground score.

[0329] In the aforementioned weight setting module, the weight corresponding to each deep learning model can be calculated by the following formula.

[0330]

[0331] Among them, w(k) is the weight corresponding to the kth deep learning model.

[0332]

[0333] FRR k is the false rejection rate of the k-th deep learning model.

[0334] Correspondingly, in the aforementioned score fusion unit, the first fusion feature map / the second fusion feature map / the third fusion feature map can be calculated by the following formula.

[0335]

[0336] Among them, Seg(x,y) Indicates the value of the pixel at the xth row and yth column of the first fused feature map / the second fused feature map / the third fused feature map. Represents the value of the pixel at the xth row and yth column of the kth first probability feature map / second probability feature map / third probability feature map.

[0337] During the execution of the aforementioned first processing unit, the second processing unit, and the third processing unit, the intermediate feature layers may be connected in the dimension of channels.

[0338] As an improvement of the embodiment of the present application, the device may further include:

[0339] The post-processing module is used to find the maximum connected area of ​​the iris effective area / iris inner boundary point / iris outer boundary point, and use the maximum connected area as the final iris effective area / iris inner boundary point / iris outer boundary point.

[0340] During training, the deep learning model of the present application may randomly perform data augmentation on training samples in a training data set, and / or the training samples in the training data set may include oblique images and / or blurred images.

[0341] The data enhancement includes one or more of flipping, translation, noise transformation and elastic deformation.

[0342] The elastic deformation includes: dividing the training samples into sub-blocks of a set size and performing shape distortion on each sub-block.

[0343] As another improvement of the embodiment of the present application, the device may further include:

[0344] The histogram equalization module is used to determine whether the grayscale mean of the iris image is less than a set first grayscale threshold or greater than a set second grayscale threshold. If so, histogram equalization is performed on the iris image.

[0345] The first grayscale threshold is smaller than the second grayscale threshold.

[0346] The device provided in the above embodiment, its implementation principle and the technical effect produced correspond to the embodiment of the above method one by one. For the sake of brief description, the part not mentioned in the embodiment of the device can refer to the corresponding content in the embodiment of the above method. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the modules and units described in the device can refer to the corresponding process in the embodiment of the above method, and will not be repeated here.

[0347] The iris image preprocessing method described in the above embodiments provided by the present application can implement the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in the method embodiment of this specification. Therefore, the embodiment of the present application also provides a computer-readable storage medium for iris image preprocessing, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the iris image preprocessing method including the above embodiments are implemented.

[0348] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0349] The storage medium described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0350] The embodiment of the present application also provides a device for iris image preprocessing, which may be a separate computer, or may include an actual operating device that uses one or more of the methods or one or more of the embodiments of the present specification. The device for iris image preprocessing may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, it implements any one or more of the steps of the iris image preprocessing method described above.

[0351] The device described above may also include other implementation methods according to the description of the method embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment. For details, please refer to the description of the relevant method embodiment, and no further description will be given here.

[0352] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An iris image preprocessing method, characterized in that: The method comprises: Get iris image; The iris image is respectively input into a plurality of deep learning models pre-trained by different training data sets, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and the segmentation results of the plurality of deep learning models are fractionally fused to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image; Wherein, when performing the iris effective area segmentation, the iris texture area is used as the foreground area, and the rest of the area is used as the background area, and the iris image is segmented to segment the iris effective area for iris feature extraction from the iris image, which includes only the iris texture area; When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area, the iris image is segmented, and the iris inner boundary point / iris outer boundary point is segmented from the iris image; Ellipse fitting is performed according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

2. The iris image preprocessing method according to claim 1, characterized in that: The iris image is input into a plurality of deep learning models pre-trained by different training data sets, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and the segmentation results of the plurality of deep learning models are fractionally fused to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image, including: Input the iris image into the multiple deep learning models respectively, encode the iris image, and use deconvolution to expand the encoded feature map to the same size as the iris image, so as to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models; The value of each pixel point on the first probability feature map / the second probability feature map / the third probability feature map represents the probability that the corresponding pixel point at the same position on the iris image belongs to the iris effective area / the iris inner boundary point / the iris outer boundary point; Score-fusing the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain a first fused feature map / a second fused feature map / a third fused feature map; The value of each pixel point on the first fusion feature map / the second fusion feature map / the third fusion feature map is compared with the set segmentation threshold. If the value of the pixel point of the first fusion feature map / the second fusion feature map / the third fusion feature map is greater than the segmentation threshold, the corresponding pixel point at the same position on the iris image is judged to belong to the iris effective area / the iris inner boundary point / the iris outer boundary point.

3. The iris image preprocessing method according to claim 2, characterized in that: The fractional fusion of the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position also includes: Performing a transformation operation on the iris image to obtain a transformed image; wherein the transformation operation includes image rotation and / or image flipping; The transformed images are respectively input into the multiple deep learning models, the iris image is encoded, the encoded feature map is expanded to the same size as the iris image using deconvolution, and the expanded feature map is subjected to an inverse transformation operation opposite to the transformation operation to obtain the same number of first probability feature maps / second probability feature maps / third probability feature maps as the deep learning models.

4. The iris image preprocessing method according to claim 2 or 3, characterized in that: The fractionally fusing the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map includes: According to the set weight of each of the first probability feature map / the second probability feature map / the third probability feature map, weighted sum is performed on the values ​​of all the first probability feature maps / the second probability feature maps / the third probability feature maps at the same position to obtain the first fused feature map / the second fused feature map / the third fused feature map; Among them, the weight of each of the first probability feature map / the second probability feature map / the third probability feature map is determined according to the distribution of the score set of the deep learning model used by the first probability feature map / the second probability feature map / the third probability feature map on its training data set and the false rejection rate of the deep learning model.

5. The iris image preprocessing method according to claim 4, characterized in that: The weights are determined by the following method: Use each deep learning model to judge its training data set respectively, and obtain a score set for each deep learning model; The score set includes a foreground score determined as a foreground and a background score determined as a background; Normalizing the score set of each deep learning model according to the non-overlapping area of ​​the distribution of foreground scores and the distribution of background scores of each deep learning model; The weight corresponding to each deep learning model is determined according to the normalized score set of each deep learning model and the false rejection rate of each deep learning model.

6. The iris image preprocessing method according to claim 5, characterized in that: The step of normalizing the score set of each deep learning model according to the non-overlapping area of ​​the distribution of the foreground score and the distribution of the background score of each deep learning model comprises: The non-overlapping area is calculated by the following formula as a normalized dimension; q(k)=μ a (k)-min a (k)+max r (k)-μ r (k) Where q(k) is the normalized dimension of the kth deep learning model, k = 1, 2, ..., m, m is the total number of deep learning models, max r (k) is the maximum foreground score of the kth deep learning model, μ r (k) is the average foreground score of the kth deep learning model, min a (k) is the minimum background score of the k-th deep learning model, μ a (k) is the average background score of the kth deep learning model; Normalize the score set of each deep learning model according to the normalized dimension of each deep learning model; in, and are the i-th score of the score set of the k-th deep learning model before and after normalization, i = 1, 2, 3, ..., N k , N k is the total number of training samples in the training data set of the kth deep learning model, j = 0 means is the background score, j = 1 means is the prospect score; The step of determining the weight corresponding to each deep learning model according to the normalized score set of each deep learning model and the false rejection rate of each deep learning model includes: The weight corresponding to each deep learning model is calculated by the following formula; Among them, w(k) is the weight corresponding to the k-th deep learning model; FRR k is the false rejection rate of the k-th deep learning model.

7. The iris image preprocessing method according to claim 6, characterized in that: The first fused feature map / the second fused feature map / the third fused feature map are calculated by the following formula: Among them, Seg (x,y) represents the value of the pixel at the xth row and yth column of the first fused feature map / the second fused feature map / the third fused feature map, Represents the value of the pixel at the xth row and yth column position of the kth first probability feature map / the second probability feature map / the third probability feature map.

8. The iris image preprocessing method according to claim 2 or 3, characterized in that: In the process of encoding the iris image and using deconvolution to expand the encoded feature map to the same size as the iris image, the intermediate feature layer is connected in the dimension of the channel.

9. The iris image preprocessing method according to claim 1, characterized in that: The step of performing ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary may also include: Find the maximum connected area for the iris effective area / the iris inner boundary point / the iris outer boundary point, and use the maximum connected area as the final iris effective area / the iris inner boundary point / the iris outer boundary point.

10. The iris image preprocessing method according to claim 1, characterized in that: When training the deep learning model, data enhancement is randomly performed on training samples in the training data set, and / or the training samples in the training data set include oblique images and / or blurred images; Wherein, the data enhancement includes one or more of flipping, translation, noise transformation and elastic deformation; The elastic deformation includes: dividing the training sample into sub-blocks of a set size, and performing shape distortion on each sub-block.

11. The iris image preprocessing method according to claim 1, characterized in that: The step of inputting the iris images into a plurality of pre-trained deep learning models respectively also includes: Determine whether the grayscale mean of the iris image is less than a set first grayscale threshold or greater than a set second grayscale threshold, and if so, perform histogram equalization on the iris image; The first grayscale threshold is smaller than the second grayscale threshold.

12. An iris image preprocessing method, characterized in that: The method comprises: An image acquisition module, used for acquiring an iris image; a segmentation module, configured to input the iris image into a plurality of deep learning models pre-trained by different training data sets, respectively, each deep learning model performs iris effective area segmentation, iris inner boundary point segmentation and / or iris outer boundary point segmentation on the iris image, and fractionally fuse the segmentation results of the plurality of deep learning models to segment the iris effective area, iris inner boundary points and / or iris outer boundary points from the iris image; Wherein, when performing the iris effective area segmentation, the iris texture area is used as the foreground area, and the rest of the area is used as the background area, and the iris image is segmented to segment the iris effective area for iris feature extraction from the iris image, which includes only the iris texture area; When performing the iris inner boundary point segmentation / iris outer boundary point segmentation, the iris inner boundary area / iris outer boundary area is used as the foreground area, and the rest is used as the background area, the iris image is segmented, and the iris inner boundary point / iris outer boundary point is segmented from the iris image; The fitting module is used to perform ellipse fitting according to the iris inner boundary points / the iris outer boundary points to obtain the iris inner boundary / iris outer boundary.

13. A computer-readable storage medium for iris image preprocessing, characterized in that: It comprises a memory for storing processor executable instructions, wherein when the instructions are executed by the processor, the steps of the iris image preprocessing method according to any one of claims 1 to 11 are implemented.

14. A device for iris image preprocessing, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein the processor implements the steps of the iris image preprocessing method according to any one of claims 1 to 11 when executing the instructions.

Images