Multi-level fusion iris liveness detection method, device, equipment, and storage medium

Abstract

The present disclosure provides a multi-level fusion iris liveness detection method, apparatus, device, and storage medium, belonging to the technical field of iris liveness detection. The method includes: obtaining an iris mask and a fused three-channel image based on an iris sample image. Inputting the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector. Obtaining an eye periocular feature vector based on a first feature map. Obtaining a second feature map based on an attention module, an iris mask, and the first feature map. Obtaining an iris feature vector based on the second feature map. Obtaining a fused feature vector based on the eye periocular feature vector and the iris feature vector. Obtaining an iris liveness detection result based on the eye periocular feature vector, the iris feature vector, and the fused feature vector. The present disclosure also provides a multi-level fusion iris liveness detection apparatus, including an image layer fusion module, a feature layer fusion module, and a score layer fusion module. The present disclosure can improve the accuracy of iris liveness detection.

Description

Technical Field

[0001] The present disclosure belongs to the technical field of iris liveness detection, and more specifically, to a multi-level fusion iris liveness detection method, apparatus, device, and storage medium. Background Art

[0002] Iris recognition has become a key technology in the field of biometrics due to its high stability, uniqueness, and contactlessness. However, during the iris image acquisition phase, attackers often forge or overwrite the texture information of a real iris to deceive the iris recognition system, thereby impersonating others or concealing one's identity. Common prosthetic attack methods include printed irises, cosmetic contact lenses, on-screen irises, artificial eyes, and synthetic irises. Iris liveness detection is designed to detect these prosthetic attacks, thereby ensuring the security of iris recognition systems.

[0003] Existing iris liveness detection methods typically use closed-set evaluation protocols, meaning that the training and test sets share similar acquisition environments and attack types. While these detection methods perform well in this scenario, their detection accuracy significantly decreases and their generalization capabilities are insufficient when dealing with unknown acquisition conditions such as complex lighting conditions, sensor variations, and cross-database scenarios. Furthermore, many models that rely solely on raw periocular images or normalized iris images as input struggle to extract comprehensive and accurate features to detect as many attack types as possible. Using both periocular and normalized iris images as input to extract corresponding features significantly increases the computational complexity and processing time of the entire detection model, hindering its real-time operation.

[0004] In summary, the existing technology still has the problem of low accuracy in iris liveness detection. Summary of the Invention

[0005] The purpose of the present disclosure is to provide a multi-level fusion iris liveness detection method, device, equipment, and storage medium to improve the accuracy of iris liveness detection.

[0006] A first aspect of the embodiments of the present disclosure provides a multi-level fusion iris liveness detection method, comprising:

[0007] Obtain an iris mask and a fused three-channel image based on the iris sample image;

[0008] Inputting the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector; the iris liveness detection model includes an attention module;

[0009] The iris liveness detection model is used to: obtain a first feature map based on the fused three-channel image; obtain an eye periocular feature vector based on the first feature map; obtain a second feature map based on the attention module, the iris mask, and the first feature map; obtain an iris feature vector based on the second feature map; and obtain a fused feature vector based on the eye periocular feature vector and the iris feature vector; the second feature map is the first feature map after feature enhancement;

[0010] A target classification output is obtained based on the eye periphery feature vector, the iris feature vector, and the fusion feature vector, and an iris liveness detection result is obtained based on the target classification output.

[0011] A second aspect of the embodiments of the present disclosure provides a multi-level fusion iris liveness detection device, comprising:

[0012] An image layer fusion module, used to obtain an iris mask and fuse three-channel images based on the iris sample image;

[0013] A feature layer fusion module, configured to input the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector; the iris liveness detection model includes an attention module;

[0014] The iris liveness detection model is used to: obtain a first feature map based on the fused three-channel image; obtain an eye periocular feature vector based on the first feature map; obtain a second feature map based on the attention module, the iris mask, and the first feature map; obtain an iris feature vector based on the second feature map; and obtain a fused feature vector based on the eye periocular feature vector and the iris feature vector; the second feature map is the first feature map after feature enhancement;

[0015] A fractional layer fusion module is configured to obtain a target classification output based on the periocular feature vector, the iris feature vector, and the fused feature vector, and to obtain an iris liveness detection result based on the target classification output. A third aspect of the embodiments of the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the steps of the multi-level fusion iris liveness detection method described above.

[0016] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the multi-level fusion iris liveness detection method are implemented.

[0017] The multi-level fusion iris liveness detection method, apparatus, device, and storage medium provided by the disclosed embodiments benefit by addressing two challenges facing iris liveness detection: the combined detection of global attacks (such as printed irises) and local attacks (such as cosmetic contact lenses); and the performance degradation caused by cross-domain issues (such as different acquisition environments and devices). This embodiment provides an accurate and effective approach, improving the accuracy of iris liveness detection.

[0018] On the one hand, this embodiment uses an iris mask-driven attention module to enhance a pre-trained two-stream CNN network. It extracts periocular features for global attacks and iris features for local attacks, respectively. It then performs self-attention feature fusion based on periocular and iris features, enhancing the expressiveness of the fused features. True / false classification is performed on periocular, iris, and fused features, and then fused at a fractional level, improving the accuracy of iris liveness detection.

[0019] On the other hand, in order to mitigate the impact of cross-domain, the CLAHE enhancement map and HOG edge enhancement feature map are extracted from the input image respectively to highlight the texture and edge information of the iris, and three-channel fusion is performed with the original grayscale image to form a fused input image, which helps the subsequent iris liveness detection to extract more discriminative features.

[0020] On the other hand, this embodiment constructs a lightweight iris liveness detection model based on iris segmentation mask-driven attention-enhanced multi-level fusion, which mines multi-level complementary information from multiple aspects such as images, features, and classification scores for iris liveness detection, and uses a shared parameter approach in the feature extraction part of the model to reduce the model size. Specifically, the present application adopts an iris segmentation mask-driven attention-enhanced pre-trained two-stream CNN network. Through a dual-branch architecture, the model can enhance the sensitivity of the prosthetic features in the iris area while reducing environmental background interference, thereby improving the robustness under different lighting and sensor environments, and enhancing the ability to accurately detect various prosthetic types. Therefore, compared with the method of using periocular images and normalized iris images as input to extract corresponding features, the model is lighter, has lower computational complexity and parameter count, and is more conducive to the application of the model in practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1A schematic diagram of the process of a multi-level fusion iris liveness detection method provided in one embodiment of the present disclosure;

[0023] Figure 2 A segmentation mask and segmentation effect diagram provided by an embodiment of the present disclosure;

[0024] Figure 3 An iris training sample image provided by an embodiment of the present disclosure;

[0025] Figure 4 The CLAHE enhanced image, HOG edge feature map and fused three-channel image provided in one embodiment of the present disclosure;

[0026] Figure 5 A schematic diagram of the structure of an iris liveness detection model provided in one embodiment of the present disclosure;

[0027] Figure 6 This is an image processing effect diagram of the attention module provided in one embodiment of the present disclosure;

[0028] Figure 7 A schematic diagram of the workflow of the fractional layer fusion module provided in one embodiment of the present disclosure;

[0029] Figure 8 An approximately binary mask image provided by an embodiment of the present disclosure;

[0030] Figure 9 A schematic diagram of the pixel-by-pixel supervision and self-distillation weighted joint loss training process for the iris liveness detection model provided in one embodiment of the present disclosure;

[0031] Figure 10 A schematic diagram of the workflow of the attention module provided in one embodiment of the present disclosure;

[0032] Figure 11 A schematic diagram of the workflow of a feature layer fusion module provided in one embodiment of the present disclosure;

[0033] Figure 12 This is a structural block diagram of a multi-level fusion iris liveness detection device provided in one embodiment of the present disclosure;

[0034] Figure 13 A schematic block diagram of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary detail.

[0036] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below with reference to the accompanying drawings.

[0037] Please refer to Figures 1 to 11 , Figure 1 This is a flowchart of a multi-level fusion iris liveness detection method provided by an embodiment of the present disclosure. The method may include S101 to S103.

[0038] S101: Obtain an iris mask and a fused three-channel image based on an iris sample image.

[0039] Figure 2 The segmentation mask and segmentation effect diagram provided by an embodiment of the present disclosure. Figure 2 In this embodiment, the iris sample image is the iris image to be detected. Obtaining an iris mask based on the iris sample image includes:

[0040] The iris sample image is annotated with an iris region, and the annotated iris sample image is segmented to obtain an iris mask.

[0041] For example, an iris sample image to be detected can be obtained by an iris scanner or other acquisition device for live / prosthetic discrimination, and the iris sample image is input into an iris segmentation model for iris segmentation to obtain an iris mask.

[0042] Figure 3 This is an iris training sample image provided by an embodiment of the present disclosure. Figure 3 During training, multiple live and prosthetic iris images can be collected as training sample images. The iris regions are manually annotated to construct a non-matching iris segmentation dataset that includes both live and prosthetic irises. This non-matching iris segmentation dataset is then used to train a highly generalizable iris segmentation model for iris segmentation in sample iris images. The iris segmentation model can be constructed using a convolutional neural network or the deep learning-based IrisSegNet model.

[0043] Figure 4 The CLAHE enhancement image, HOG edge feature map and fused three-channel image provided by an embodiment of the present disclosure. Figure 4In this embodiment, obtaining a fused three-channel image based on the iris sample image includes:

[0044] The CLAHE enhancement image and HOG edge feature map are obtained based on the iris sample image.

[0045] The iris sample image, CLAHE enhanced image and HOG edge feature map are fused to obtain a fused three-channel image.

[0046] In this embodiment, the CLAHE-enhanced image is obtained by applying the CLAHE technique to the iris sample image. CLAHE stands for Contrast-Limited Adaptive Histogram Equalization. The resulting CLAHE-enhanced image enhances local contrast, making iris details clearer and facilitating subsequent analysis and processing. The HOG (Histogram of Oriented Gradients) edge feature map highlights the edge information of the iris sample image.

[0047] In digital image processing, images typically consist of one or more channels, each representing a specific attribute of the image. A three-channel image fusion combines the iris sample image, the CLAHE enhanced image, and the HOG edge feature map, each as a channel, into a new image with three channels. This three-channel fused image simultaneously contains information from the original image, enhanced contrast information, and edge feature information, providing richer features for subsequent classification and recognition.

[0048] For example, a CLAHE-enhanced image and a HOG-enhanced feature map are extracted from the input image to highlight the texture and edge information of the iris. The CLAHE-enhanced image, the HOG-enhanced feature map, and the original grayscale image are then subjected to a three-channel fusion to form a fused input image. This fusion of three channels helps the subsequent iris liveness detection model extract more discriminative features.

[0049] S102: Inputting the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector; the iris liveness detection model includes an attention module;

[0050] The iris liveness detection model is used to: obtain a first feature map based on the fusion of three-channel images; obtain an eye periocular feature vector based on the first feature map; obtain a second feature map based on the attention module, iris mask and the first feature map; obtain an iris feature vector based on the second feature map; obtain a fused feature vector based on the eye periocular feature vector and the iris feature vector; the second feature map is the first feature map after feature enhancement.

[0051] Figure 5 This is a schematic diagram of the structure of the iris liveness detection model provided by an embodiment of the present disclosure. Figure 5 In this embodiment, the first feature map is obtained based on the fusion of the three-channel image, including:

[0052] Based on the common convolution layer and maximum pooling layer in the iris liveness detection model, feature extraction is performed on the fused three-channel image to obtain the first feature map.

[0053] In this embodiment, the iris liveness detection model is an attention-enhanced pre-trained two-stream CNN network driven by an iris segmentation mask. The iris liveness detection model includes a common convolution layer and a maximum pooling layer.

[0054] Exemplarily, the input fused three-channel image first passes through a 7×7 common convolution layer and a 3×3 maximum pooling layer to extract low-level features to obtain a first feature map.

[0055] Figure 6 This is an image processing effect diagram of the attention module provided by an embodiment of the present disclosure. Figure 5 and Figure 6 In this embodiment, the iris liveness detection model also includes ResNet Block1, ResNet Block2, a first attention module, ResNet Block3, ResNet Block4, a second attention module, and a convolutional layer. The first attention module is added after ResNet Block2 and before ResNet Block3, and the second attention module is added after ResNetBlock4.

[0056] In this embodiment, obtaining the periocular feature vector based on the first feature map includes:

[0057] Feature extraction is performed on the first feature map based on ResNet Block1, ResNet Block2, ResNet Block3, ResNet Block4 and the convolutional layer to obtain the periorbital intermediate feature map.

[0058] The periocular intermediate feature map is flattened and normalized to obtain the periocular feature vector.

[0059] In this embodiment, obtaining a second feature map based on the attention module, the iris mask, and the first feature map, and obtaining an iris feature vector based on the second feature map include:

[0060] Based on ResNet Block1, ResNet Block2, the first attention module, ResNet Block3, ResNetBlock4, the second attention module and the convolution layer, feature enhancement and feature extraction are performed on the second feature map to obtain the iris intermediate feature map.

[0061] The iris intermediate feature map is flattened and normalized to obtain the iris feature vector.

[0062] For example, the pre-trained two-stream CNN network uses a residual network, ResNet18. ResNet18 has four ResNetBlock modules (ResNet Block 1, ResNet Block 2, ResNet Block 3, and ResNet Block 4), each containing four 3×3 convolutional layers. The pre-trained two-stream CNN network includes two branches: the periocular branch and the iris branch. The periocular branch extracts global features from the input image and processes the input image similarly to a traditional pre-trained CNN network, ensuring robustness to inaccurate mask information. The iris branch extracts local feature maps from the input image. Iris segmentation mask-driven attention modules (i.e., the first and second attention modules) are added after the two intermediate layers, ResNet Block 2 and ResNet Block 4, respectively. These two iris segmentation mask-driven attention modules enable the CNN network to focus more on the iris region of the input image, thereby enhancing the CNN's ability to extract local features. The feature extraction process of these two branches, based on ResNet Blocks, shares parameters.

[0063] The periocular branch and iris branch undergo feature refinement through a convolutional layer and a Rectified Linear Unit (RELU) layer, respectively, to produce a periocular-specific intermediate feature map (i.e., the periocular intermediate feature map) and an iris-specific intermediate feature map (i.e., the iris intermediate feature map). The Rectified Linear Unit (RELU) layer is an important activation function layer in neural networks.

[0064] The model flattens the obtained periorbital intermediate feature map and iris intermediate feature map and performs L2 norm normalization to obtain the periorbital feature vector and iris feature vector.

[0065] S103: Obtaining a target classification output based on the eye periphery feature vector, the iris feature vector, and the fusion feature vector, and obtaining an iris liveness detection result based on the target classification output.

[0066] Figure 7This is a schematic diagram of the workflow of the fractional layer fusion module provided in one embodiment of the present disclosure. Figure 5 and Figure 7 In this embodiment, the iris liveness detection model also includes a classifier.

[0067] The target classification output includes periocular classification output, iris classification output and fusion classification output.

[0068] The target classification output is obtained based on the periocular feature vector, iris feature vector and fusion feature vector, including:

[0069] The periocular feature vector is input into the classifier to obtain the periocular classification output corresponding to the periocular feature vector.

[0070] The iris feature vector is input into the classifier to obtain the iris classification output corresponding to the iris feature vector.

[0071] The fused feature vector is input into the classifier to obtain the fused classification output corresponding to the fused feature vector.

[0072] In this embodiment, the iris liveness detection result is obtained based on the target classification output, including:

[0073] Substitute the periocular classification output, iris classification output, and fusion classification output into the classification score weighted fusion calculation formula to obtain the iris classification score.

[0074] If the iris classification score is greater than or equal to the classification score threshold, the iris liveness detection result is true.

[0075] If the iris classification score is less than the classification score threshold, the iris liveness detection result is false.

[0076] The calculation formula for weighted fusion of classification scores is:

[0077]

[0078] in, represents the fusion classification output, represents the periocular classification output, represents the iris classification output, 、 and is the weight coefficient, which can be , , . Represents the classification output score.

[0079] In this embodiment, the classification score threshold is a preset critical value for determining whether an iris is real or fake.

[0080] The above demonstrates two challenges facing iris liveness detection: the combined detection of global attacks (such as iris printing) and local attacks (such as cosmetic contact lenses); and performance degradation caused by cross-domain issues (such as different acquisition environments and devices). This embodiment provides an accurate and effective solution, improving the accuracy of iris liveness detection.

[0081] On the one hand, this embodiment uses an iris mask-driven attention module to enhance a pre-trained two-stream CNN network. It extracts periocular features for global attacks and iris features for local attacks, respectively. It then performs self-attention feature fusion based on periocular and iris features, enhancing the expressiveness of the fused features. True / false classification is performed on periocular, iris, and fused features, and then fused at a fractional level, improving the accuracy of iris liveness detection.

[0082] On the other hand, in order to mitigate the impact of cross-domain, the CLAHE enhancement map and HOG edge enhancement feature map are extracted from the input image respectively to highlight the texture and edge information of the iris, and three-channel fusion is performed with the original grayscale image to form a fused input image, which helps the subsequent iris liveness detection to extract more discriminative features.

[0083] On the other hand, this embodiment constructs a lightweight iris liveness detection model based on iris segmentation mask-driven attention-enhanced multi-level fusion, which mines multi-level complementary information from multiple aspects such as images, features, and classification scores for iris liveness detection, and uses a shared parameter approach in the feature extraction part of the model to reduce the model size. Specifically, the present application adopts an iris segmentation mask-driven attention-enhanced pre-trained two-stream CNN network. Through a dual-branch architecture, the model can enhance the sensitivity of the prosthetic features in the iris area while reducing environmental background interference, thereby improving the robustness under different lighting and sensor environments, and enhancing the ability to accurately detect various prosthetic types. Therefore, compared with the method of using periocular images and normalized iris images as input to extract corresponding features, the model is lighter, has lower computational complexity and parameter count, and is more conducive to the application of the model in practice.

[0084] Figure 8 An approximate binary mask image provided by an embodiment of the present disclosure. Figure 9 Schematic diagram of pixel-by-pixel supervision and self-distillation weighted joint loss training process for the iris liveness detection model provided in one embodiment of the present disclosure Figure 8 and Figure 9 In one embodiment of the present disclosure, the multi-level fusion iris liveness detection method further includes:

[0085] Input the iris sample training set into the initial iris liveness detection model to obtain the periocular intermediate feature map, iris intermediate feature map, periocular feature vector, iris feature vector, fusion feature vector and target classification output;

[0086] The self-distillation loss is obtained based on the periocular feature vector, iris feature vector, fusion feature vector, target classification output and self-distillation loss function;

[0087] Pixel-by-pixel supervision loss is obtained based on the periorbital intermediate feature map, the iris intermediate feature map, and the pixel-by-pixel supervision loss function;

[0088] A weighted joint loss is obtained based on self-distillation loss, pixel-wise supervision loss and weighted joint loss function;

[0089] The initial iris liveness detection model is updated based on the weighted joint loss to obtain the iris liveness detection model.

[0090] In this embodiment, the self-distillation loss function is:

[0091]

[0092] in, Represents the self-distillation loss, label represents the true label of the iris sample, label=1 represents the real iris, label=0 represents the prosthetic iris, Represents the target classification output, which includes the periorbital classification output, iris classification output, and fusion classification output. When i=1, Represents the eye periocular classification output; when i=2, Represents the iris classification output; when i=3, represents the fusion classification output; () represents the loss calculation function between the target classification output and the true label; the fusion feature is used as the teacher model, and the periorbital feature and iris feature are used as the student model. Represents the classification output of the teacher model in the self-distillation process, that is, the fusion classification output; () represents the KL divergence loss function, () represents the fully connected layer. When i=1, represents the periocular feature vector; when i=2, represents the iris feature vector; when i=3, represents the fused feature vector; Represents the eigenvector of the teacher model, that is, the fusion eigenvector, Indicates that the self-distillation loss dynamically adjusts the weight.

[0093] In this embodiment, Indicates calculation first The sum of the squares of each element of this vector, and then the square root, we get and The distance between these two vectors in terms of L2 norm. This distance measures the student model feature vector after dimension alignment. and the teacher model feature vector The degree of difference between them is used to optimize the student model by minimizing this distance during the feature distillation process, so that the student model can learn the characteristics of the teacher model and improve the model performance.

[0094] The pixel-by-pixel supervision loss function is:

[0095]

[0096]

[0097] in, Represents pixel-by-pixel supervision loss. When j=1, Represents the periocular intermediate feature map, which is used to represent the visual information around the eye. When j=2, Represents the iris intermediate feature map, which is used to represent the visual information of the iris area. Represents the approximate binary mask image generated based on the iris mask, R represents the pixel coordinate set of the iris mask area, () indicates based on The loss function is Indicates that the pixel-wise supervision loss dynamically adjusts the weights.

[0098] In this embodiment, a pixel-by-pixel supervised loss function is used to measure the pixel-by-pixel difference between the intermediate feature maps for the periocular and iris regions and the expected results. An approximate binary mask is generated based on the iris mask, focusing on the differences between pixels within the iris mask region (R). The SmoothL1 loss function is used to calculate the difference between the intermediate feature maps for the periocular and iris regions. This pixel-by-pixel supervised loss function calculates the loss for each pixel, ensuring that the model can more accurately capture and learn the features of the target region (such as the iris region). This makes the model more robust in processing details and local features. The weights are dynamically adjusted as training progresses, gradually optimizing the intermediate feature maps.

[0099] The weighted joint loss function is:

[0100]

[0101] in, represents the weighted joint loss, 、 is the balance coefficient, .

[0102] For example, This self-distillation loss uses the fused features as the teacher model and the periocular and iris features as the student models. This loss is calculated using the Focal loss function, the KL divergence loss function, and the L2 loss. Using the Focal loss function, the true label of the iris sample and the three classification outputs are calculated. This allows knowledge hidden in the training dataset to be directly transferred from the label to all classifiers.

[0103] By introducing the KL divergence loss, the periocular and iris classification outputs (student model) can learn knowledge from the fusion feature classification output (teacher model), thereby promoting the student model to gradually approach the output distribution of the teacher model, allowing the model to learn more comprehensive features.

[0104] It will and A fully connected layer is used to perform dimension alignment. Through feature distillation, the fused features of the teacher model are transferred to the student model and optimized using L2 loss. The feature map implicitly transfers details and high-level information, enabling the student model to more accurately learn the knowledge of the teacher model.

[0105] Indicates that the self-distillation loss dynamically adjusts the weight, The initial value of can be 0.03, and as the training progresses, it increases by epoch*0.005.

[0106] In this embodiment, the self-distillation loss function is a knowledge transfer and optimization loss function, using the fused features as the teacher model and the periocular and iris features as the student model. The self-distillation loss function uses the Focal loss function to introduce knowledge of the sample's true label into the classifier. It uses the KL divergence loss to enable the student model to learn the output distribution from the teacher model. It then uses a fully connected layer to align the feature vector dimensions, optimizes feature distillation using the L2 loss, and dynamically adjusts weights to encourage the model to learn more comprehensive features, improving overall performance.

[0107] For example, is the pixel-by-pixel supervision loss, and the training error is calculated by the Smooth L1 loss function. and Negative feedback is given to the initial iris liveness detection model for the next training step to optimize and update the iris liveness detection model. Use Smooth L1 loss to optimize the intermediate feature map and The difference between them can ensure that the model can capture and learn the characteristics of the target area more robustly. Indicates that pixel-by-pixel supervision loss dynamically adjusts weights, The initial value of can be 1, and as the training progresses, it can be reduced by epoch*0.001.

[0108] In this embodiment, obtaining a training set of iris samples may include: constructing a training library based on an existing publicly available iris liveness detection dataset after data cleaning. Inputting images from the training library into an initial iris liveness detection model for training in batches, saving the optimal training model during the training process and using it for liveness detection of target samples.

[0109] After obtaining the iris sample training set, the data set can also be expanded, including: obtaining a single iris training sample image from each batch of training sample images, performing iris segmentation masking and three-channel fusion image joint data expansion operations on the iris training sample image, such as randomly cropping and flipping the image to expand the data set.

[0110] The self-distillation loss in this embodiment combines multiple loss functions to introduce training data knowledge into the classifier, allowing the student model to learn from the teacher model and learn comprehensive features. The pixel-by-pixel supervision loss uses Smooth L1 loss to ensure that the model robustly captures and learns target region features. The weighted joint loss balances the two, dynamically adjusting weights for continuous optimization during training. This embodiment allows the updated initial model to more accurately determine whether an iris is real, improving detection accuracy and robustness, effectively handling complex iris samples, and enhancing the overall performance of the iris liveness detection model.

[0111] Figure 10 This is a schematic diagram of the workflow of the attention module provided in one embodiment of the present disclosure. Figure 10 In one embodiment of the present disclosure, obtaining a second feature map based on the attention module, the iris mask, and the first feature map includes:

[0112] Performing feature enhancement on the first feature map based on the convolutional layer in the attention module to obtain an enhanced feature map;

[0113] Scaling the iris mask based on the nearest neighbor interpolation method to obtain a target iris mask with the same size as the first feature map;

[0114] Obtain the attention feature map of the iris area based on the enhanced feature map and the target iris mask;

[0115] A second feature map is obtained based on the attention feature map and the first feature map.

[0116] In this embodiment, the attention feature map of the iris region is obtained based on the enhanced feature map and the target iris mask, including:

[0117] The enhanced feature map and the target iris mask are then aligned and multiplied to obtain the attention feature map of the iris region.

[0118] In this embodiment, the attention module includes a first attention module and a second attention module. The first attention module and the second attention module have the same structure and workflow, but they are in different processing stages, so the input feature maps are also different. The first feature map can represent the feature map output by the middle layer ResNet Block 2 or the feature map output by ResNetBlock 4.

[0119] For example, for the first attention module, the input feature map (i.e., the first feature map) is the feature map output by the intermediate ResNetBlock 2 layer. The model uses the first feature map as input and combines it with the iris mask corresponding to the first feature map. This is processed by the convolutional layer in the first attention module to produce an enhanced feature map. The iris mask is scaled to the same size as the first feature map (i.e., the same height H and width W) using the nearest neighbor interpolation method. A positional product is performed on the enhanced feature map and the scaled target iris mask to produce an attention feature map for the iris region. The attention feature map is then added to the first feature map to produce a second feature map after iris enhancement.

[0120] For the second attention module: the input feature map (i.e., the first feature map) is the feature map output by the intermediate layer ResNet Block 4.

[0121] This embodiment can effectively focus on the iris area, enhance feature representation, and increase the attention of the iris liveness detection model to iris features, which helps to improve the accuracy of iris liveness detection.

[0122] Figure 11 This is a schematic diagram of the workflow of the feature layer fusion module provided in one embodiment of the present disclosure. Figure 11 In one embodiment of the present disclosure, obtaining a fused feature vector based on the periocular feature vector and the iris feature vector includes:

[0123] Perform vector concatenation on the periorbital feature vector and the iris feature vector, and determine the attention weight corresponding to the concatenated feature vector.

[0124] The concatenated feature vectors are fused based on the attention weights to obtain an intermediate fusion vector.

[0125] The intermediate fusion vector is input into the fully connected layer to obtain the fused feature vector.

[0126] In this embodiment, feature fusion is performed on the concatenated feature vectors based on the attention weights to obtain an intermediate fusion vector, including:

[0127] The concatenated feature vectors are fused based on the attention weights to obtain a fused feature vector.

[0128] The fused feature vector is average pooled and layer normalized to obtain the intermediate fusion vector.

[0129] In this embodiment, the periocular feature vector is a feature representation extracted from the area surrounding the eye. It can include various periocular feature information, such as periocular skin texture, wrinkles, shape, and color. The iris feature vector is a feature representation extracted from the iris region. It can include iris-specific features, such as iris texture, color, pattern, and vascularity. The intermediate fusion vector is an intermediate result obtained during the feature fusion process.

[0130] Attention weights are used to emphasize or focus on different parts of the concatenated feature vector. Different attention weights can be assigned based on the importance of different tasks or features. For example, in an iris recognition task, the part of the iris feature vector can be assigned a higher attention weight than the part of the periocular feature vector. Attention weights can be learned or set based on prior knowledge.

[0131] For example, dynamic feature fusion is performed on the periocular and iris feature vectors to improve the model's ability to express different features. The periocular and iris feature vectors are concatenated, and a multi-head self-attention mechanism is used to dynamically assign different attention weights to each concatenated feature, achieving more effective feature fusion. Sequence-wise average pooling and layer normalization are performed on the fused features to further enhance their expressiveness. The processed fused features are then integrated through a fully connected layer to produce a fused feature vector containing both periocular and iris information.

[0132] This embodiment utilizes a multi-head self-attention mechanism to enable the iris liveness detection model to flexibly focus on important features based on task requirements, improving feature representation. Sequence-dimensional average pooling and layer normalization further enhance fused feature representation. The integration of fully connected layers ultimately outputs a fused feature vector that combines periocular and iris information, providing a more comprehensive and effective feature representation for subsequent tasks such as iris recognition, thereby improving the accuracy of iris liveness detection.

[0133] Corresponding to the multi-level fusion iris liveness detection method in the above embodiment, Figure 12 This is a structural block diagram of a multi-level fusion iris liveness detection device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 12 The multi-level fusion iris liveness detection device 20 includes: an image layer fusion module 21, a feature layer fusion module 22 and a score layer fusion module 23.

[0134] The image layer fusion module 21 is used to obtain an iris mask and fuse a three-channel image based on the iris sample image.

[0135] The feature layer fusion module 22 is used to input the fused three-channel image into the iris liveness detection model to obtain the eye periphery feature vector, the iris feature vector and the fused feature vector. The iris liveness detection model includes an attention module.

[0136] The iris liveness detection model is used to: generate a first feature map based on the fused three-channel image; generate a periocular feature vector based on the first feature map; generate a second feature map based on the attention module, the iris mask, and the first feature map; generate an iris feature vector based on the second feature map; and generate a fused feature vector based on the periocular feature vector and the iris feature vector. The second feature map is the first feature map after feature enhancement.

[0137] The score layer fusion module 23 is used to obtain a target classification output based on the eye periphery feature vector, the iris feature vector and the fusion feature vector, and obtain an iris liveness detection result based on the target classification output.

[0138] In one embodiment of the present disclosure, the multi-level fusion iris liveness detection device 20 further includes: a loss training module, which is used to input the iris sample training set into the initial iris liveness detection model to obtain the periorbital intermediate feature map, the iris intermediate feature map, the periorbital feature vector, the iris feature vector, the fusion feature vector and the target classification output.

[0139] The self-distillation loss is obtained based on the periocular feature vector, iris feature vector, fusion feature vector, target classification output and self-distillation loss function.

[0140] The pixel-by-pixel supervision loss is obtained based on the periorbital intermediate feature map, the iris intermediate feature map and the pixel-by-pixel supervision loss function.

[0141] A weighted joint loss is obtained based on self-distillation loss, pixel-wise supervision loss and weighted joint loss function.

[0142] The initial iris liveness detection model is updated based on the weighted joint loss to obtain the iris liveness detection model.

[0143] In one embodiment of the present disclosure, the loss training module is specifically used for the self-distillation loss function:

[0144]

[0145] in, Represents the self-distillation loss, label represents the true label of the iris sample, label=1 represents the real iris, label=0 represents the prosthetic iris, Represents the target classification output, which includes the periorbital classification output, iris classification output, and fusion classification output. When i=1, Represents the eye periocular classification output; when i=2, Represents the iris classification output; when i=3, represents the fusion classification output; () represents the loss calculation function between the target classification output and the true label; the fusion feature is used as the teacher model, and the periorbital feature and iris feature are used as the student model. Represents the classification output of the teacher model in the self-distillation process, that is, the fusion classification output; () represents the KL divergence loss function, () represents the fully connected layer. When i=1, represents the periocular feature vector; when i=2, represents the iris feature vector; when i=3, represents the fused feature vector; Represents the eigenvector of the teacher model, that is, the fusion eigenvector, Indicates that the self-distillation loss dynamically adjusts the weight.

[0146] The pixel-by-pixel supervision loss function is:

[0147]

[0148]

[0149] in, Represents pixel-by-pixel supervision loss. When j=1, represents the middle feature map around the eye, which is used to represent the visual information around the eye; when j=2, Represents the iris intermediate feature map, which is used to represent the visual information of the iris area; Represents the approximate binary mask image generated based on the iris mask, R represents the pixel coordinate set of the iris mask area, () indicates based on The loss function is Indicates that the pixel-wise supervision loss dynamically adjusts the weights.

[0150] The weighted joint loss function is:

[0151]

[0152] in, represents the weighted joint loss, 、 is the balance coefficient, .

[0153] In one embodiment of the present disclosure, the feature layer fusion module 22 is specifically configured to perform feature enhancement on the first feature map based on the convolutional layer in the attention module to obtain an enhanced feature map.

[0154] The iris mask is scaled based on the nearest neighbor interpolation method to obtain a target iris mask with the same size as the first feature map.

[0155] The attention feature map of the iris region is obtained based on the enhanced feature map and the target iris mask.

[0156] A second feature map is obtained based on the attention feature map and the first feature map.

[0157] In one embodiment of the present disclosure, the image layer fusion module 21 is specifically configured to obtain a CLAHE enhancement image and a HOG edge feature image based on the iris sample image.

[0158] The iris sample image, CLAHE enhanced image and HOG edge feature map are fused to obtain a fused three-channel image.

[0159] In one embodiment of the present disclosure, the feature layer fusion module 22 is further configured to perform feature extraction on the fused three-channel image based on the common convolution layer and the maximum pooling layer in the iris liveness detection model to obtain a first feature map.

[0160] In one embodiment of the present disclosure, the feature layer fusion module 22 is specifically configured to perform vector splicing on the periorbital feature vector and the iris feature vector, and determine the attention weight corresponding to the spliced feature vector.

[0161] The concatenated feature vectors are fused based on the attention weights to obtain an intermediate fusion vector.

[0162] The intermediate fusion vector is input into the fully connected layer to obtain the fused feature vector.

[0163] See also Figure 13 , Figure 13 This is a schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 13 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules in the above-mentioned device embodiments, such as Figure 12 The functions of modules 21 to 23 are shown.

[0164] It should be understood that in the embodiments of the present disclosure, the processor 301 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0165] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0166] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0167] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the multi-level fusion iris liveness detection method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device 300 described in the embodiments of the present disclosure, which will not be repeated here.

[0168] In another embodiment of the present disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0169] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0170] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0171] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0173] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of the embodiments of the present disclosure.

[0174] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0175] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or replacements within the technical scope disclosed in this disclosure, and such modifications or replacements should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A multi-level fusion iris liveness detection method, characterized in that: include: Obtain an iris mask and a fused three-channel image based on the iris sample image; Inputting the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector; The iris liveness detection model includes an attention module; The iris liveness detection model is used to: obtain a first feature map based on the fused three-channel image; obtain an eye periocular feature vector based on the first feature map; obtain a second feature map based on the attention module, the iris mask, and the first feature map; obtain an iris feature vector based on the second feature map; and obtain a fused feature vector based on the eye periocular feature vector and the iris feature vector; the second feature map is the first feature map after feature enhancement; Obtaining a target classification output based on the periocular feature vector, the iris feature vector, and the fusion feature vector, and obtaining an iris liveness detection result based on the target classification output; The method further comprises: Input the iris sample training set into the initial iris liveness detection model to obtain the periocular intermediate feature map, iris intermediate feature map, periocular feature vector, iris feature vector, fusion feature vector and target classification output; The self-distillation loss is obtained based on the periocular feature vector, iris feature vector, fusion feature vector, target classification output and self-distillation loss function; Pixel-by-pixel supervision loss is obtained based on the periorbital intermediate feature map, the iris intermediate feature map, and the pixel-by-pixel supervision loss function; The pixel-by-pixel supervision loss function is: in, Represents pixel-by-pixel supervision loss. When j=1, represents the middle feature map around the eye, which is used to represent the visual information around the eye; when j=2, Represents the iris intermediate feature map, which is used to represent the visual information of the iris area; Represents an approximate binary mask image generated based on the iris mask, label represents the true label of the iris sample, label=1 represents a real iris, label=0 represents a prosthetic iris, R represents the pixel coordinate set of the iris mask area, () indicates based on The loss function is Indicates that pixel-by-pixel supervision loss dynamically adjusts weights; Obtaining a weighted joint loss based on the self-distillation loss, the pixel-by-pixel supervision loss, and a weighted joint loss function; The initial iris liveness detection model is updated based on the weighted joint loss to obtain an iris liveness detection model.

2. The multi-level fusion iris liveness detection method according to claim 1, characterized in that: The self-distillation loss function is: in, represents the loss from distillation, Represents the target classification output, which includes the periorbital classification output, iris classification output, and fusion classification output. When i=1, Represents the eye periocular classification output; when i=2, Represents the iris classification output; when i=3, represents the fusion classification output; () represents the loss calculation function between the target classification output and the true label; the fusion feature is used as the teacher model, and the periorbital feature and iris feature are used as the student model. Represents the classification output of the teacher model in the self-distillation process, that is, the fusion classification output; () represents the KL divergence loss function, () represents the fully connected layer. When i=1, represents the periocular feature vector; when i=2, represents the iris feature vector; when i=3, represents the fused feature vector; Represents the eigenvector of the teacher model, that is, the fusion eigenvector, Indicates that the self-distillation loss dynamically adjusts the weight; The weighted joint loss function is: in, represents the weighted joint loss, 、 is the balance coefficient, .

3. The multi-level fusion iris liveness detection method according to claim 1, characterized in that: The obtaining a second feature map based on the attention module, the iris mask, and the first feature map includes: Performing feature enhancement on the first feature map based on the convolutional layer in the attention module to obtain an enhanced feature map; Scaling the iris mask based on a nearest neighbor interpolation method to obtain a target iris mask having the same size as the first feature map; Obtaining an attention feature map of the iris region based on the enhanced feature map and the target iris mask; A second feature map is obtained based on the attention feature map and the first feature map.

4. The multi-level fusion iris liveness detection method according to claim 1, characterized in that: The obtaining of a fused three-channel image based on the iris sample image includes: Obtaining a CLAHE enhancement image and a HOG edge feature image based on the iris sample image; The iris sample image, the CLAHE enhanced image and the HOG edge feature image are fused to obtain a fused three-channel image.

5. The multi-level fusion iris liveness detection method according to claim 1, characterized in that: The obtaining of a first feature map based on the fused three-channel image includes: Feature extraction is performed on the fused three-channel image based on the common convolution layer and the maximum pooling layer in the iris liveness detection model to obtain a first feature map.

6. The multi-level fusion iris liveness detection method according to claim 1, characterized in that: The obtaining of a fused feature vector based on the periorbital feature vector and the iris feature vector includes: Performing vector splicing on the periocular feature vector and the iris feature vector, and determining an attention weight corresponding to the spliced feature vector; Perform feature fusion on the concatenated feature vectors based on the attention weights to obtain an intermediate fusion vector; The intermediate fusion vector is input into the fully connected layer to obtain a fusion feature vector.

7. A multi-level fusion iris liveness detection device, characterized in that: include: An image layer fusion module, used to obtain an iris mask and fuse three-channel images based on the iris sample image; A feature layer fusion module, configured to input the fused three-channel image into an iris liveness detection model to obtain an eye periocular feature vector, an iris feature vector, and a fused feature vector; the iris liveness detection model includes an attention module; The iris liveness detection model is used to: obtain a first feature map based on the fused three-channel image; obtain an eye periocular feature vector based on the first feature map; obtain a second feature map based on the attention module, the iris mask, and the first feature map; obtain an iris feature vector based on the second feature map; and obtain a fused feature vector based on the eye periocular feature vector and the iris feature vector; the second feature map is the first feature map after feature enhancement; a score layer fusion module, configured to obtain a target classification output based on the periocular feature vector, the iris feature vector, and the fusion feature vector, and obtain an iris liveness detection result based on the target classification output; A loss training module is used to input the iris sample training set into the initial iris liveness detection model to obtain the periocular intermediate feature map, the iris intermediate feature map, the periocular feature vector, the iris feature vector, the fusion feature vector and the target classification output; The self-distillation loss is obtained based on the periocular feature vector, iris feature vector, fusion feature vector, target classification output and self-distillation loss function; Pixel-by-pixel supervision loss is obtained based on the periorbital intermediate feature map, the iris intermediate feature map, and the pixel-by-pixel supervision loss function; The pixel-by-pixel supervision loss function is: in, Represents pixel-by-pixel supervision loss. When j=1, Represents the periocular intermediate feature map, which is used to represent the visual information around the eye; When j=2, Represents the iris intermediate feature map, which is used to represent the visual information of the iris area; Represents an approximate binary mask image generated based on the iris mask, label represents the true label of the iris sample, label=1 represents a real iris, label=0 represents a prosthetic iris, R represents the pixel coordinate set of the iris mask area, () indicates based on The loss function is Indicates that pixel-by-pixel supervision loss dynamically adjusts weights; Obtaining a weighted joint loss based on the self-distillation loss, the pixel-by-pixel supervision loss, and a weighted joint loss function; The initial iris liveness detection model is updated based on the weighted joint loss to obtain an iris liveness detection model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images