A deep learning-based fingerprint activity detection method

Abstract

The application provides a kind of deep learning-based fingerprint activity detection method, for FLD (Fingerprint Liveness Detection, fingerprint activity detection) system is recognized by detecting the activity of fingerprint to achieve the purpose of identifying fake fingerprint, to solve the problem of low recognition accuracy and poor generalization of existing FLD system, MSFFNet (Multi-scale Spatial Feature Fusion Network, multi-scale spatial feature fusion network) is designed, which is based on EfficientnetV2-S model, simplifies the original network structure, and optimizes the network layer with less contribution. First, the image is processed by ROI (Region Of Interest); MSFFNet network is designed; the processed image is sent into the network for training; the final recognition result is given according to the training result. The problem of low detection accuracy of the current FLD system for data collected by different sensors and different materials is solved.

Description

Technical Field

[0001] This invention relates to the field of biometric identification applications, and in particular to a fingerprint activity detection method based on deep learning. Background Technology

[0002] Fingerprints are the most widely used biometric feature in smart device security verification schemes. However, Automatic Fingerprint Identification Systems (AFIS) are vulnerable to attacks from fake fingerprints made of various materials. In such cases, the fake fingerprint loses its activity, so fingerprint activity detection is often used to detect fake fingerprints. Fingerprint activity detection systems can be implemented using both hardware and software methods. Hardware-based fingerprint activity detection methods collect inherent information about the finger, such as blood pressure, skin deformation, and odor. However, this approach has high hardware requirements and is difficult to upgrade. Therefore, researchers have focused on software-based fingerprint activity detection schemes, which can distinguish between real and fake fingerprints through image processing and recognition algorithms. Software-based fingerprint activity detection schemes are lower in cost, easier to maintain and upgrade, and can be divided into traditional methods and deep learning methods.

[0003] In 2010, researchers proposed a heuristic algorithm for automatically extracting and locating pores. This algorithm uses two different types of filters to perform high-pass filtering on the spectrum of the fingertip image to obtain pore detection, and then associates the extracted pores with small circles to achieve pore localization. Results showed a very obvious correlation peak at the output corresponding to the pores on the fingertip ridge. In 2008, researchers proposed a texture feature algorithm combining LBP and wavelet transform. The LBP operator is used to extract texture features from the fingerprint image, and wavelet energy is used to describe the frequency and direction information of the fingerprint image and its lines. Experimental results showed that this algorithm can effectively detect fingerprint activity. Both of these methods belong to traditional software fingerprint activity detection methods.

[0004] In 2016, researchers first applied convolutional neural networks to fingerprint activity detection. Experimental results showed that this method improved the average classification accuracy by nearly 16% compared to the best fingerprint activity detection algorithm at the time, and won the fingerprint activity detection competition that year. In 2020, researchers attempted to combine traditional methods with deep learning methods. The team used the LGP operator to extract features from images and then processed these features using a deep learning framework. Experiments showed that this semi-end-to-end network achieved the best results at the time. In 2022, researchers pointed out that live fingerprints and forged fingerprints have significant differences in spatial structure. Live fingerprints typically exhibit more continuous and smooth ridges. Therefore, based on the spatial continuity or ridge continuity of fingerprint images, live fingerprints and forged fingerprints can be effectively distinguished. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a fingerprint activity detection method based on deep learning, comprising the following steps: S1. Input fingerprint image; S2. Divide the image dataset into training and testing sets, and extract the ROI from the data; S3. Design and build an MSFFNet network; S4. Input the training set into the network model for training and save it; S5. Classify and judge the output obtained in step S4.

[0006] In the preferred embodiment, step S2 includes the following steps: S21. Calculate the mean of the entire image using the following formula: (1); S22. Divide the image into small blocks. Taking one small block as an example, calculate the standard deviation of the small block using the following formula: (2); Where w represents the width of the image and h represents the height of the image. Let i be the pixel value of a point with width i and height j. The mean u of the entire image is the threshold for determining whether a region is a mask region. The standard deviation of each small block of the image is compared with the mean. The region with a standard deviation greater than the mean is the mask region. Finally, all mask regions are summarized to form the fingerprint ROI image.

[0007] In the preferred embodiment, the specific structure of the MSFFNet network in step S3 is as follows: The input layer is used to input fingerprint images; The first stage consists of convolutional layers with a kernel size of 3×3. The second stage involves SGE processing; The third stage includes a Fused-MBConv1 layer with a kernel size of 24×3×3. The fourth stage consists of two layers of Fused-MBConv4 with a kernel size of 48×3×3. The fifth stage consists of two layers of Fused-MBConv4 with a kernel size of 64×3×3. The sixth stage consists of three MBConv4 layers with a kernel size of 128×3×3. The seventh stage consists of four MBConv6 layers with a kernel size of 160×3×3. The eighth stage consists of seven MBConv6 layers with a kernel size of 256×3×3. In the ninth stage, EMA treatment is performed. The tenth stage involves SK processing; The eleventh stage includes a 1×1 convolutional layer, an adaptive average pooling layer, an unfolded layer, and a fully connected layer. The interval between the unfolded layer and the fully connected layer is obtained by Dropout, and the output size is 1×1280.

[0008] In the preferred embodiment, the main body of MSFFNet in step S4 consists of stem, block, and head; The Block includes an EfficientnetV2-S model scaled down to 0.5, the SGE module, the EMA module, and the SK module; The SGE module is used for image preprocessing to enhance the learning of semantic features in key regions; The EMA module is used for local feature processing of images, enhancing the network's ability to perceive features; The SK module is used for global feature processing of images.

[0009] In the preferred scheme, when the l-th stage of MSFFNet is the SGE module, the input is divided into G groups, where the value of G must not exceed the number of input channels. After grouping, the feature mapping is... Then the output of the i-th group in the l-th layer The calculation formula is: (3); in, It is the activation function, specifically the Sigmoid function; , , These are the introduced parameters, and their number is the same as that of group G; This is the normalized importance coefficient, and its value is given by the following formula: (4); (5); (6); It is the importance coefficient, derived from the formula. Received, among which It is a vector and The included angle.

[0010] In the preferred embodiment, when the first stage of MSFFNet is set to the EMA module, the input processing includes the following steps: S401. For any given input feature map, the EMA divides it into G sub-features according to the cross-channel dimensional direction X, which are used to learn different semantic features, where This way each group has There are 1 channel, and the value of each G must be less than the number of channels; S402. Use three parallel routes to extract the attention weight descriptors of the grouped feature maps. Two of the routes are 1×1 routes, and one route is a 3×3 route. On the 1×1 path, the input is first subjected to one-dimensional global average pooling in the height and width directions respectively; On the 3×3 path, a convolution operation with a kernel of 3 is used to capture more features; S403. The two processing results on the 1×1 path are concatenated along the height direction and decomposed into two vectors using 1×1 convolution. The two-dimensional binomial distribution after convolution is fitted using the Sigmoid function, and then the information of the two 1×1 channels is aggregated using conventional multiplication and multiplied with the original input. S404. The processing results on the 1×1 path are group normalized, and the row global spatial information is encoded by two-dimensional global average pooling, and then converted into its corresponding dimension shape. The formula for two-dimensional global average pooling is as follows: (8); The output on the 3×3 branch is also encoded using two-dimensional global average pooling to encode the global spatial information; S405. For the input on the 1×1 path and the 3×3 path, the result of the previous step is fitted with a linear transformation using the nonlinear function Softmax of the two-dimensional Gaussian mapping. Then, perform matrix multiplication on the result of the 1×1 path and the result of the convolution on the 3×3 path, perform matrix multiplication on the result of the 3×3 path and the normalized output result on the 1×1 path, and add the outputs of the two matrix multiplications. S406. Multiply the output of step S405 by the original input, and calculate the output using the Sigmoid function.

[0011] In the preferred embodiment, when the first stage of MSFFNet is set to the SK module, the input processing includes the following steps: S411. The fingerprint image data is processed using convolutions with kernel sizes of 1, 3, and 5 to obtain three different mapping feature spaces. , , ,in Representative group convolutions, including normalization layers and activation layers, are used to improve the efficiency of convolutions; S412. Generate multi-dimensional data information using global average pooling; Taking the c-th dimension as an example, by reducing the aggregated spatial information U, as shown in the following equation: (9); S413. To further extract feature information, a simple fully connected layer is needed to adaptively select the feature information. The processing method is given by the following formula: (10); in It is an activation function. It is a batch of normalized layers; S414. Use Softmax to calculate the selector, and obtain... These are the feature spaces. The specific descriptor, where the sum of the feature descriptors in each c-dimensional dimension is 1, results in the final feature space mapping. It is achieved through feature space mapping in each dimension. Composed of various elements, among which, From the formula Given, among which, C is the number of channels. L and r are both constants, r is the attenuation rate, and L is 32; Feature space mapping The result is given by the following formula: (11); (12).

[0012] In a preferred embodiment, the images to be judged include fingerprints made of different materials or fingerprints collected by different sensors.

[0013] This invention offers the following advantages: It utilizes deep learning technology to achieve fingerprint activity detection based on spatial features. This method, to some extent, overcomes the shortcomings of traditional methods in detecting fingerprints across sensors and materials, thus improving the system's robustness. Furthermore, by processing images without using operators based on prior human knowledge, it achieves end-to-end computation, thereby improving classification accuracy. Attached Figure Description

[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0015] Figure 1 This is a schematic diagram of the ROI processing procedure in this invention.

[0016] Figure 2 This is a network framework diagram of MSFFNet in this invention.

[0017] Figure 3 This is a framework diagram of the SGE module in this invention.

[0018] Figure 4 This is a framework diagram of the EMA module in this invention.

[0019] Figure 5 This is a framework diagram of the SK module in this invention.

[0020] Figure 6 A flowchart of the present invention. Detailed Implementation

[0021] like Figure 6 As shown, this invention discloses a fingerprint activity detection method based on deep learning, the specific process of which is as follows: Figure 6 As shown, the LivDet2011, LivDet2013, and LivDet2015 datasets were selected in this embodiment. The experimental data used in this embodiment are shown in Table 1.

[0022] Table 1

[0023] Figure 1 The distribution represents a schematic diagram of the ROI process used.

[0024] like Figure 2 As shown, this example implements fingerprint activity detection by constructing MSFFNet. The information of the improved EfficientNetV2-S is shown in Table 2 below: Table 2 Stage Operator Stride Channels Layers 0 Conv3×3 2 24 1 1 Fused-MBConv1,k3×3 1 24 1 2 Fused-MBConv4,k3×3 2 48 2 3 Fused-MBConv4,k3×3 2 64 2 4 MBConv4,K3×3,SE0.25 2 128 3 5 MBConv6,K3×3,SE0.25 1 160 4 6 MBConv6,K3×3,SE0.25 2 256 7 7 Conv1×1&Pooling&FC - 1280 1 like Figure 3 As shown, the input image is fed into the SGE module for processing to enhance the semantic feature learning of key regions. For the input image, the processed output is obtained by equation (3), and the relevant parameters in equation (3) are derived from equations (4), (5), and (6). Among them, the grouping of dimensions is taken as 8, and the constant term is used in the process of calculating the normalized importance coefficient. It was set to 0.0001.

[0025] like Figure 4 As shown, after the image is input into the EMA module, it is grouped into 8 groups; then divided into two paths, one undergoes one-dimensional global average pooling, and the other undergoes convolution operation using a convolution kernel of size 3; then the two paths that have undergone one-dimensional global average pooling are concatenated, and aggregated by multiplication using the Sigmoid function. After that, both outputs are subjected to global average pooling according to equation (8); the outputs are passed through the Softmax function, and the results are aggregated using matrix multiplication, addition, and ordinary multiplication, and finally the output is obtained by Sigmoid.

[0026] like Figure 5 As shown, in the SK module, the input is convolved with convolution kernels of size 1, 3, and 5 to obtain three different mapping spaces; then, the information is aggregated by global average pooling using equation (9); finally, the feature space mapping is obtained by equation (11), where the parameters are determined by equations (9), (10), etc.

[0027] In step 5, we obtained the discrimination results. Here, we use the average classification error rate (ACE) as the scoring standard to evaluate the system output in both the normal case and the ablation experiment. Good results were obtained on this parameter. Therefore, this system can effectively detect fingerprint activity. The calculation method of ACE is shown in the following equations (13), (14), and (15).

[0028] (13) In the above formula, FRR is the proportion of misclassifications in fake fingerprints, and FAR is the proportion of misclassifications in genuine fingerprints. The calculation formulas are shown in Equations 14 and 15: (14) (15) The system performance evaluation is divided into two parts: direct testing and ablation testing. The direct testing part consists of two phases: one where the test set and training set are on the same dataset and use the same materials; and another where the test set and training set come from the same dataset but use different materials.

[0029] The experimental results are shown in Tables 3 and 4. Table 3 shows the experimental results using the same dataset and materials. Table 4 shows the experimental results using the same dataset but with different materials for the test and training sets. In the tables, E represents EcoFlex, G represents Gelatine, L represents Latex, S represents Silgum, W represents WoodGlue, and M represents Modasil. For example, in the 2011 dataset, E, G, LS, W represent the dataset using LivDet2011, with the training set materials being EcoFlex, Gelatine, and Latex, and the test set materials being Silgum and WoodGlue.

[0030] Table 3 Dataset Sensor LiveDet2011 LiveDet2013 LiveDet2015 Biometrika 2.02 1.42 \ Digital 1.8 \ 2.36 Italdata 3.15 1.37 \ Sagem 0.9 \ \ CrossMatch \ \ 1.73 GreenBit \ \ 1.86 His \ \ 1.8 Table 4 Dataset Sensor 2011(E,G,LS,E) 2013 (M,WE,G,L) Biometrika 10.1 4.9 Italdata 22.1 6.3 For the ablation experiments, the results are given in Tables 5, 6, and 7 below. These represent the training and test sets from the same sensor within the same dataset; the training and test sets from different materials; and the training and test sets from different sensors, respectively. Where I represents the Italdata sensor, B represents the Biometrika sensor, S represents the Sagem sensor, and H represents the GreenBit sensor. For example, I 2011-B 2011 indicates that images acquired by the Italdata sensor in the LiveDet2011 dataset were used as the training set, and images acquired by the Biometrika sensor in the LiveDet2011 dataset were used as the test set.

[0031] Table 5 Sensor Evaluation Standards Biometrika Digital Italdata Sagem ACE 2.02 1.8 3.15 1.97 Table 6 Material Sensors 2011(E,G,LS,W) 2013 (M,WE,G,L) Biometrika 7.14 1.53 Digital 4.81 1.31 Table 7 Data evaluation criteria I2011-B2011 I2011-S2011 I2011-B2013 H2011-G2011 I2011-I2013 ACE 17.72 15.04 14.03 5.09 3.7 The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A fingerprint activity detection method based on deep learning, characterized in that... Includes the following steps: S1. Input fingerprint image; S2. Divide the image dataset into training and testing sets, and extract the ROI from the data; S3. Design and build an MSFFNet network; S4. Input the training set into the network model for training and save it; S5. Classify and judge the output obtained in step S4; The specific structure of the MSFFNet network in step S3 is as follows: The input layer is used to input fingerprint images; The first stage consists of convolutional layers with a kernel size of 3×3. The second stage involves SGE processing; The third stage includes a Fused-MBConv1 layer with a kernel size of 24×3×3. The fourth stage consists of two layers of Fused-MBConv4 with a kernel size of 48×3×3. The fifth stage consists of two layers of Fused-MBConv4 with a kernel size of 64×3×3. The sixth stage consists of three MBConv4 layers with a kernel size of 128×3×3. The seventh stage consists of four MBConv6 layers with a kernel size of 160×3×3. The eighth stage consists of seven MBConv6 layers with a kernel size of 256×3×3. In the ninth stage, EMA treatment is performed. The tenth stage involves SK processing; The eleventh stage includes a 1×1 convolutional layer, an adaptive average pooling layer, an unfolded layer, and a fully connected layer. The interval between the unfolded layer and the fully connected layer is obtained by Dropout, and the output size is 1×1280. The main body of MSFFNet in step S4 consists of stem, block, and head; The Block includes an EfficientnetV2-S model scaled down to 0.5, the SGE module, the EMA module, and the SK module; The SGE module is used for image preprocessing to enhance the learning of semantic features in key regions; The EMA module is used for local feature processing of images, enhancing the network's ability to perceive features; The SK module is used for global feature processing of images.

2. The fingerprint activity detection method based on deep learning according to claim 1, characterized in that: Step S2 includes the following steps: S21. Calculate the mean of the entire image using the following formula: （1）； S22. Divide the image into small blocks. Taking one block as an example, calculate the standard deviation of the block using the following formula: （2）； Where w represents the width of the image and h represents the height of the image. Let i be the pixel value of a point with width i and height j. The mean u of the entire image is the threshold for determining whether a region is a mask region. The standard deviation of each small block of the image is compared with the mean. The region with a standard deviation greater than the mean is the mask region. Finally, all mask regions are summarized to form the fingerprint ROI image.

3. The fingerprint activity detection method based on deep learning according to claim 1, characterized in that: When the l-th stage of MSFFNet is the SGE module, the input is divided into G groups, where the value of G must not exceed the number of input channels. After grouping, the feature mapping is... Then the output of the i-th group in the l-th layer The calculation formula is: (3)； in, It is the activation function, specifically the Sigmoid function; , , These are the introduced parameters, and their number is the same as that of group G; This is the normalized importance coefficient, and its value is given by the following formula: (4)； (5)； (6)； It is the importance coefficient, derived from the formula. Received, among which It is a vector and The included angle.

4. The fingerprint activity detection method based on deep learning according to claim 1, characterized in that: When the first stage of MSFFNet is set to the EMA module, the input processing includes the following steps: S401. For any given input feature map, the EMA divides it into G sub-features according to the cross-channel dimensional direction X, which are used to learn different semantic features, where This way each group has There are 1 channel, and the value of each G must be less than the number of channels; S402. Use three parallel routes to extract the attention weight descriptors of the grouped feature maps. Two of the routes are 1×1 routes, and one route is a 3×3 route. On the 1×1 path, the input is first subjected to one-dimensional global average pooling in the height and width directions respectively; On the 3×3 path, a convolution operation with a kernel of 3 is used to capture more features; S403. The two processing results on the 1×1 path are concatenated along the height direction and decomposed into two vectors using 1×1 convolution. The two-dimensional binomial distribution after convolution is fitted using the Sigmoid function, and then the information of the two 1×1 channels is aggregated using conventional multiplication and multiplied with the original input. S404. The processing results on the 1×1 path are group normalized, and the row global spatial information is encoded by two-dimensional global average pooling, and then converted into its corresponding dimension shape. The formula for two-dimensional global average pooling is as follows: (8)； The output on the 3×3 branch is also encoded using two-dimensional global average pooling to encode the global spatial information; S405. For the input on the 1×1 path and the 3×3 path, the result of the previous step is fitted with a linear transformation using the nonlinear function Softmax of the two-dimensional Gaussian mapping. Then, perform matrix multiplication on the result of the 1×1 path and the result of the convolution on the 3×3 path, perform matrix multiplication on the result of the 3×3 path and the normalized output result on the 1×1 path, and add the outputs of the two matrix multiplications. S406. Multiply the output of step S405 by the original input, and calculate the output using the Sigmoid function.

5. The fingerprint activity detection method based on deep learning according to claim 1, characterized in that: When the first stage of MSFFNet is set to the SK module, the input processing includes the following steps: S411. Convolutional kernels of sizes 1, 3, and 5 are used to process the fingerprint image data, resulting in three different mapping feature spaces. , , ,in Representative group convolutions, including normalization layers and activation layers, are used to improve the efficiency of convolutions; S412. Generate multi-dimensional data information using global average pooling; Taking the c-th dimension as an example, by reducing the aggregated spatial information U, as shown in the following equation: (9)； S413. To further extract feature information, a simple fully connected layer is needed to adaptively select the feature information. The processing method is given by the following formula: (10)； in It is an activation function. It is a batch of normalized layers; S414. Use Softmax to calculate the selector, and obtain... These are the feature spaces. For each specific descriptor, the sum of the feature descriptors in each c-dimensional dimension is 1, resulting in the final feature space mapping. It is achieved through feature space mapping in each dimension. Composed of various elements, among which, From the formula Given, among which, C is the number of channels. L and r are both constants, r is the attenuation rate, and L is 32; Feature space mapping The result is given by the following formula: (11)； (12)。 6. The fingerprint activity detection method based on deep learning according to claim 1, characterized in that: The images to be judged include fingerprints from different materials or fingerprints collected by different sensors.

Images