A method for detecting fingerprint activity by improving a FasterNet network

Abstract

The application discloses a fingerprint activity detection method of an improved FasterNet network, which comprises the following steps: step 1, acquiring a fingerprint image dataset and dividing the fingerprint image dataset into a training set and a test set; step 2, pre-processing the training set and the test set, including ROI extraction, cropping, flipping and normalization; step 3, building a GMF module and a basic block module; step 4, based on the GMF module and the basic block module in step 3, building an improved FasterNet network architecture, including an Embeding layer, a Stage1 layer, a first Merging layer, a Stage2 layer, a second Merging layer, a Stage3 layer, a third Merging layer, a Stage4 layer, a max-pooling layer, a convolution layer and a full connection layer; step 5, feeding the fingerprint images processed in step 2 into the built network architecture, performing feature extraction and classifying the fingerprint images; and step 6, outputting a fingerprint image recognition result. The application aims at solving the technical problems of low recognition accuracy, low artificial image pre-processing efficiency and redundancy caused by excessive network parameters in the existing fingerprint activity detection technology, and provides the fingerprint activity detection method of the improved FasterNet network.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to an improved fingerprint activity detection method for FasterNet networks. Background Technology

[0002] Automated Fingerprint Identification Systems (AFIS) are widely used in personal identification and authentication due to their high reliability, strong adaptability, and low cost. However, the security of AFIS is increasingly concerning due to the challenges posed by fingerprint forgery attacks. In practical applications, artificial fingerprint forgery attacks are quite likely to occur. Artificial fingerprint replicas, also known as fake fingerprints, can be easily manufactured using various inexpensive and common materials, such as gelatin, silicone, wood glue, and clay. These materials are readily available, making the process of forging fingerprints relatively simple. Furthermore, with the rapid development of 3D printing technology, sophisticated 3D printing techniques are also being used in fingerprint forgery attacks.

[0003] Therefore, fingerprint liveness detection is crucial for further improving the security of fingerprint applications. Various anti-forgery methods can be broadly categorized into hardware-based and software-based methods. Hardware-based methods require additional hardware devices to detect vital signs, such as body temperature, humidity, blood pressure, pulse, and blood oxygen saturation. While external hardware devices can accurately distinguish between real and fake fingerprints, they also make fingerprint recognition systems more complex and expensive. Furthermore, it is difficult to update these external hardware devices when attackers update their manufacturing techniques to improve handcrafted fake fingerprints. On the other hand, software-based solutions can extract features from the presented fingerprint image (or a series of frames) acquired by the fingerprint sensor without incurring additional hardware costs. Compared to hardware-based methods, software-based methods make fingerprint recognition systems lower cost and easier to update. Software-based methods are further divided into two types: those based on dynamic features (such as ridge deformation and sweating) and those based on static features (such as ridge frequency, elasticity features, and energy spectrum). Dynamic features are acquired from a series of images, making the fingerprint acquisition process very time-consuming. On the other hand, static features are extracted based on a single fingerprint image or a frame of images, making them more efficient. Deep learning is a software-based fingerprint activity detection algorithm that typically processes a single image frame and can achieve better results than traditional algorithms.

[0004] In the paper "DeFraudNet: End2End Fingerprint Spoof Detection using Patch Level Attention," the authors preprocessed the images using LBP and Gabor filters and extracted fingerprint image features using the DenseNet network to obtain fingerprint image classification results. However, experimental results show that the algorithm in this paper has a high error rate on some datasets and cannot perform well in fingerprint classification tasks. In the paper "Densely Connected Convolutional Network Optimized by Genetic Algorithm for FingerprintLiveness Detection," the authors combined the DenseNet network and a genetic algorithm to complete the fingerprint liveness detection task. The genetic algorithm was used to optimize the parameter selection of the DenseNet network to achieve higher detection efficiency. However, the fingerprint liveness detection task is essentially an image binary classification task. Using multiple algorithms in combination will lead to excessive computational resource requirements and cannot achieve the goal of lightweight design. Therefore, how to ensure the lightweight nature of the network while maintaining accuracy is an unsolved problem. Summary of the Invention

[0005] The purpose of this invention is to solve the technical problems of low recognition accuracy, low efficiency of manual image preprocessing, and redundancy caused by excessive network parameters in existing fingerprint activity detection technologies, and to provide an improved fingerprint activity detection method based on FasterNet network.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] An improved fingerprint activity detection method for FasterNet network includes the following steps:

[0008] Step 1: Obtain the fingerprint image dataset and divide it into training and test sets;

[0009] Step 2: Preprocess the training and test sets, including ROI extraction, cropping, flipping, and normalization;

[0010] Step 3: Build the GMF module and the basic block module;

[0011] Step 4: Based on the GMF module and basic block module in Step 3, build an improved FasterNet network architecture, including the Embedding layer, Stage 1 layer, first Merging layer, Stage 2 layer, second Merging layer, Stage 3 layer, third Merging layer, Stage 4 layer, max pooling layer, convolutional layer, and fully connected layer.

[0012] Step 5: Input the fingerprint images processed in Step 2 into the constructed network architecture to extract features and classify the fingerprint images;

[0013] Step 6: Output the fingerprint image recognition result.

[0014] Step 2 specifically includes the following sub-steps:

[0015] Step 1) Divide the image into n small blocks. Taking one small block as an example, calculate the mean of that small block, and then calculate the standard deviation within that small block. The formula for calculating the mean is as follows, where p is the mean of the small block, and m and n are the length and width of the small block; a ij The value of the target pixel;

[0016]

[0017] The formula for calculating standard deviation is as follows:

[0018]

[0019] Step 2) Compare the mean and standard deviation from Step 1. If the mean of the small area is greater than the standard deviation, it is a masked area. Save the location information of all masked areas in a list and merge them to obtain the ROI region.

[0020] Step 3) Crop the fingerprint image of the ROI region obtained in Step 2;

[0021] Step 4) Randomly select either horizontal or vertical flipping to flip the cropped image from Step 3 to obtain the enhanced image;

[0022] Step 5) Quantize the flipped image from step 4 within the range of [-1,1] to achieve normalization, avoid data overflow caused by excessively large data, and make the data mean 0 and the standard deviation 1, which conforms to the normal distribution standard.

[0023] In step 3, the GMF module built is as follows:

[0024] The data is divided into several data groups. Each data group, Feature_n, is duplicated into two identical copies, a and b. Feature a is fed into a Global Max Pooling layer for global max pooling. The output is then duplicated again into two identical copies, a_1 and a_2. Feature a_1 is fed into the first convolutional layer for convolution, and feature a_2 is fed into the second convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features undergo a softmax operation and are multiplied by Feature_n to obtain the feature after processing the first channel. For feature b, it first enters the Avg... In the pooling layer, global max pooling is performed, and the output is copied into two identical feature sets, b_1 and b_2. Feature b_1 is fed into the third convolutional layer for convolution, and feature b_2 is fed into the fourth convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved again. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the feature after processing the second channel. The features of the first and second channels are concatenated to obtain the new Feature_n. All data sets processed in the above way are concatenated to obtain the output of the GMF module.

[0025] In step 3, the basic block module constructed is as follows:

[0026] First, the input is fed into the PConv layer and undergoes a partial convolution to obtain feature c. Feature c is then copied into two identical copies. The first copy is fed into the Avg pool for global average pooling, and the output is fed into the PConv layer for partial convolution. The second copy is fed into the Glo pool for global max pooling, and the output is fed into the PConv layer for partial convolution. The outputs of the two partial convolutions are concatenated to obtain feature d. Feature d is then divided into two identical copies. The input of the first channel is fed into the first convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The input of the second channel is fed into the second convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The outputs of these two channels are multiplied to obtain feature e. Finally, features c, d, and e are multiplied to obtain the output of the improved basic feature extraction module.

[0027] In step 4, the improved FasterNet network architecture is constructed as follows:

[0028] The input to the Embedding layer is used to input the fingerprint image. The output of the Embedding layer is connected to the input of the GMF module. The output of the GMF module is connected to the input of the Stage 1 layer. The output of the Stage 1 layer is connected to the input of the first Merging layer. The output of the first Merging layer is connected to the input of the Stage 2 layer. The output of the Stage 2 layer is connected to the input of the second Merging layer. The output of the second Merging layer is connected to the input of the Stage 3 layer. The output of the Stage 3 layer is connected to the input of the third Merging layer. The output of the third Merging layer is connected to the input of the Stage 4 layer. The output of the Stage 4 layer is connected to the input of the max pooling layer. The output of the max pooling layer is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the fully connected layer. The fully connected layer outputs the classification result.

[0029] The Embedding layer contains a Batch Normalization (BN) layer and a convolutional layer. Input first enters the BN layer, and its output then enters the convolutional layer. The output of the convolutional layer is the Embedding output, used for initial image contour feature extraction. After inserting the GMF module into the Embedding layer, it perceives local image features and processes receptive field information of different regions in a grouped manner, thus achieving more refined fingerprint image features. This module can work well with the PConv structure in FasterNet to process local features. Stage 1 has one basic block module, and Stage 2 has two basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module, resulting in the Stage 2 output. Stage 3 has four basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module. The output of the second basic block module enters the third basic block module, and its output enters the fourth basic block module, resulting in the Stage 3 output. Stage 4 has input features first entering the first basic block module... The output of the block module goes into the second basic block module, and the output of the Stage 4 layer is obtained. All Merging layers contain Dropout layers and BN layers. The input features first go into the Dropout layer, and the output goes into the BN layer. The Dropout layer is used to randomly drop some neurons to prevent overfitting.

[0030] The fingerprint image input and process are as follows: The processed image in step 2 first enters the Embedding layer. The output of the Embedding layer enters the GMF module. The output of the GMF enters the Stage 1 layer, which is a basic block layer. The output of Stage 1 enters the first Merging layer. The output of the first Merging layer enters the Stage 2 layer, which is the first basic block layer of Stage 2. The output of Stage 2 enters the second basic block layer of Stage 2. The output of Stage 2 enters the second Merging layer. The output of Stage 3 enters the first basic block layer of Stage 3. The output of Stage 3 enters the second basic block layer of Stage 3. The output of Stage 3 enters the third basic block layer of Stage 3. The output of Stage 3 enters the fourth basic block layer of Stage 3. Then it enters the third Merging layer. The output of Stage 4 enters the first basic block layer of Stage 4. The output of Stage 4 enters the second basic block layer of Stage 4. The output of Stage 4 enters the max average pooling layer. Then it enters a convolutional layer. The output of Stage 4 enters a fully connected layer to obtain the classification result.

[0031] An improved FasterNet network includes: an Embedding layer, a Stage 1 layer, a first Merging layer, a Stage 2 layer, a second Merging layer, a Stage 3 layer, a third Merging layer, a Stage 4 layer, a max-average pooling layer, a convolutional layer, and a fully connected layer.

[0032] The input to the Embedding layer is used to input the fingerprint image. The output of the Embedding layer is connected to the input of the GMF module. The output of the GMF module is connected to the input of the Stage 1 layer. The output of the Stage 1 layer is connected to the input of the first Merging layer. The output of the first Merging layer is connected to the input of the Stage 2 layer. The output of the Stage 2 layer is connected to the input of the second Merging layer. The output of the second Merging layer is connected to the input of the Stage 3 layer. The output of the Stage 3 layer is connected to the input of the third Merging layer. The output of the third Merging layer is connected to the input of the Stage 4 layer. The output of the Stage 4 layer is connected to the input of the max pooling layer. The output of the max pooling layer is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the fully connected layer. The fully connected layer outputs the classification result.

[0033] The GMF module that was built is as follows:

[0034] The data is divided into several data groups. Each data group, Feature_n, is duplicated into two identical copies, a and b. Feature a is fed into a Global Max Pooling layer for global max pooling. The output is then duplicated into two identical copies, a_1 and a_2. a_1 is fed into the first convolutional layer for convolution, and a_2 is fed into the second convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the features processed in the first channel. For feature b, it first enters the Avg... In the pooling layer, global max pooling is performed, and the output is copied into two identical copies, b_1 and b_2. b_1 is fed into the third convolutional layer for convolution, and b_2 is fed into the fourth convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the features processed in the second channel. The features of the first and second channels are concatenated to obtain the new Feature_n. All data groups processed in the above way are concatenated to obtain the output of the GMF module.

[0035] The constructed basic block module is as follows: First, the input is fed into the PConv layer and undergoes a partial convolution to obtain feature c. Feature c is copied into two identical copies. The first copy is fed into the Avg pool for global average pooling, and the output is fed into the PConv layer for partial convolution. The second copy is fed into the Glo pool layer for global max pooling, and the output is fed into the PConv layer for partial convolution. The outputs of the two partial convolutions are concatenated to obtain feature d. Feature d is divided into two identical copies. The input of the first channel is fed into the first convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The input of the second channel is fed into the second convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The outputs of these two channels are multiplied to obtain feature e. Features c, d, and e are multiplied together to obtain the output of the improved basic feature extraction module.

[0036] The Embedding layer contains a Batch Normalization (BN) layer and a convolutional layer. Input first enters the BN layer, and its output then enters the convolutional layer. The output of the convolutional layer is the Embedding output, used for initial image contour feature extraction. After inserting the GMF module into the Embedding layer, it perceives local image features and processes receptive field information of different regions in a grouped manner, thus achieving more refined fingerprint image features. This module can work well with the PConv structure in FasterNet to process local features. Stage 1 has one basic block module, and Stage 2 has two basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module, resulting in the Stage 2 output. Stage 3 has four basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module. The output of the second basic block module enters the third basic block module, and its output enters the fourth basic block module, resulting in the Stage 3 output. Stage 4 has input features first entering the first basic block module... The output of the block module goes into the second basic block module, and the output of the Stage 4 layer is obtained. All Merging layers contain Dropout layers and BN layers. The input features first go into the Dropout layer, and the output goes into the BN layer. The Dropout layer is used to randomly drop some neurons to prevent overfitting.

[0037] Compared with the prior art, the present invention has the following technical effects:

[0038] 1) This invention introduces a Grouping Multi-Scale Fusion Module (GMF) into the original Fasternet network to further improve the classification accuracy of fingerprint liveness detection tasks. This module effectively extracts local feature information from fingerprint images and highlights important details through weighted processing. Through the GMF module, the network can dynamically adjust the importance weights of each feature map to increase attention to key regions, thus more accurately capturing subtle differences between live and forged fingerprints. Compared to traditional methods, the module proposed in this invention exhibits better performance in fingerprint liveness detection tasks. Furthermore, this module has fewer parameters, making it suitable for embedding in hardware devices. Since embedded systems typically have resource constraints, such as computing power and storage space, choosing a module with fewer parameters can reduce the hardware burden while ensuring performance. This makes the GMF module proposed in this invention an ideal solution for deployment on embedded devices.

[0039] 2) This invention improves the basic block in the original Fasternet network structure by extracting global features of the image through residual connections. This improved structure can better capture the overall information of the image and further improve the generalization performance of the system. In the original Fasternet network, PConv convolution extracts local information by performing convolution operations on parts of the structure. In this way, the network can focus on important details in the image. However, in some cases, focusing only on local information may lead to the neglect of overall features; to solve this problem, this invention improves the basic block. Through three-level residual connections, combined with the traditional convolution method, the network can more fully perceive image information of different dimensions, thereby extracting image features. This helps to enhance the system's understanding and representation of the overall image structure. Through this improvement, we further improve the generalization performance of the system. The system can better adapt to different detection scenarios and sample variations, and has stronger robustness and adaptability. Attached Figure Description

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments:

[0041] Figure 1 This is a flowchart of the present invention;

[0042] Figure 2 A schematic diagram of the improved Fasternet network structure provided by this invention;

[0043] Figure 3 This is a schematic diagram of the GMF module structure used in this invention;

[0044] Figure 4 This is a schematic diagram of the improved basic block module of the present invention. Detailed Implementation

[0045] This invention proposes an improved fingerprint activity detection method for FasterNet networks, which includes the following steps:

[0046] Step 1: Obtain the fingerprint image dataset and divide it into training and test sets;

[0047] Step 2: Preprocess the training and test sets, including ROI extraction, cropping, flipping, and normalization;

[0048] Step 3: Build the GMF module and the basic block module;

[0049] Step 4: Based on the GMF module and basic block module in Step 3, build an improved FasterNet network architecture, including the Embedding layer, Stage 1 layer, first Merging layer, Stage 2 layer, second Merging layer, Stage 3 layer, third Merging layer, Stage 4 layer, max pooling layer, convolutional layer, and fully connected layer.

[0050] Step 5: Input the fingerprint images processed in Step 2 into the constructed network architecture to extract features and classify the fingerprint images;

[0051] Step 6: Output the fingerprint image recognition result.

[0052] Step 2 specifically includes the following sub-steps:

[0053] Step 1) Divide the image into n small blocks. Taking one small block as an example, calculate the mean of that small block, and then calculate the standard deviation within that small block. The formula for calculating the mean is as follows, where p is the mean of the small block, and m and n are the length and width of the small block; a ij The value of the target pixel;

[0054]

[0055] The formula for calculating standard deviation is as follows:

[0056]

[0057] Step 2) Compare the mean and standard deviation from Step 1. If the mean of the small area is greater than the standard deviation, it is a masked area. Save the location information of all masked areas in a list and merge them to obtain the ROI region.

[0058] Step 3) Crop the fingerprint image of the ROI region obtained in Step 2;

[0059] Step 4) Randomly select either horizontal or vertical flipping to flip the cropped image from Step 3 to obtain the enhanced image;

[0060] Step 5) Quantize the flipped image from step 4 within the range of [-1,1] to achieve normalization, avoid data overflow caused by excessively large data, and make the data mean 0 and the standard deviation 1, which conforms to the normal distribution standard.

[0061] In step 3, the GMF module built is as follows:

[0062] The data is divided into several data groups. Each data group, Feature_n, is duplicated into two identical copies, a and b. Feature a is fed into a Global Max Pooling layer for global max pooling. The output is then duplicated again into two identical copies, a_1 and a_2. Feature a_1 is fed into the first convolutional layer for convolution, and feature a_2 is fed into the second convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features undergo a softmax operation and are multiplied by Feature_n to obtain the feature after processing the first channel. For feature b, it first enters the Avg... In the pooling layer, global max pooling is performed, and the output is copied into two identical feature sets, b_1 and b_2. Feature b_1 is fed into the third convolutional layer for convolution, and feature b_2 is fed into the fourth convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved again. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the feature after processing the second channel. The features of the first and second channels are concatenated to obtain the new Feature_n. All data sets processed in the above way are concatenated to obtain the output of the GMF module.

[0063] In step 3, the basic block module constructed is as follows:

[0064] First, the input is fed into the PConv layer and undergoes a partial convolution to obtain feature c. Feature c is then copied into two identical copies. The first copy is fed into the Avg pool for global average pooling, and the output is fed into the PConv layer for partial convolution. The second copy is fed into the Glo pool for global max pooling, and the output is fed into the PConv layer for partial convolution. The outputs of the two partial convolutions are concatenated to obtain feature d. Feature d is then divided into two identical copies. The input of the first channel is fed into the first convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The input of the second channel is fed into the second convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The outputs of these two channels are multiplied to obtain feature e. Finally, features c, d, and e are multiplied to obtain the output of the improved basic feature extraction module.

[0065] In step 4, the improved FasterNet network architecture is constructed as follows:

[0066] The input to the Embedding layer is used to input the fingerprint image. The output of the Embedding layer is connected to the input of the GMF module. The output of the GMF module is connected to the input of the Stage 1 layer. The output of the Stage 1 layer is connected to the input of the first Merging layer. The output of the first Merging layer is connected to the input of the Stage 2 layer. The output of the Stage 2 layer is connected to the input of the second Merging layer. The output of the second Merging layer is connected to the input of the Stage 3 layer. The output of the Stage 3 layer is connected to the input of the third Merging layer. The output of the third Merging layer is connected to the input of the Stage 4 layer. The output of the Stage 4 layer is connected to the input of the max pooling layer. The output of the max pooling layer is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the fully connected layer. The fully connected layer outputs the classification result.

[0067] The Embedding layer contains a Batch Normalization (BN) layer and a convolutional layer. Input first enters the BN layer, and its output then enters the convolutional layer. The output of the convolutional layer is the Embedding output, used for initial image contour feature extraction. After inserting the GMF module into the Embedding layer, it perceives local image features and processes receptive field information of different regions in a grouped manner, thus achieving more refined fingerprint image features. This module can work well with the PConv structure in FasterNet to process local features. Stage 1 has one basic block module, and Stage 2 has two basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module, resulting in the Stage 2 output. Stage 3 has four basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module. The output of the second basic block module enters the third basic block module, and its output enters the fourth basic block module, resulting in the Stage 3 output. Stage 4 has input features first entering the first basic block module... The output of the block module goes into the second basic block module, and the output of the Stage 4 layer is obtained. All Merging layers contain Dropout layers and BN layers. The input features first go into the Dropout layer, and the output goes into the BN layer. The Dropout layer is used to randomly drop some neurons to prevent overfitting.

[0068] The fingerprint image input and process are as follows: The processed image in step 2 first enters the Embedding layer. The output of the Embedding layer enters the GMF module. The output of the GMF enters the Stage 1 layer, which is a basic block layer. The output of Stage 1 enters the first Merging layer. The output of the first Merging layer enters the Stage 2 layer, which is the first basic block layer of Stage 2. The output of Stage 2 enters the second basic block layer of Stage 2. The output of Stage 2 enters the second Merging layer. The output of Stage 3 enters the first basic block layer of Stage 3. The output of Stage 3 enters the second basic block layer of Stage 3. The output of Stage 3 enters the third basic block layer of Stage 3. The output of Stage 3 enters the fourth basic block layer of Stage 3. Then it enters the third Merging layer. The output of Stage 4 enters the first basic block layer of Stage 4. The output of Stage 4 enters the second basic block layer of Stage 4. The output of Stage 4 enters the max average pooling layer. Then it enters a convolutional layer. The output of Stage 4 enters a fully connected layer to obtain the classification result.

[0069] This invention also proposes an improved FasterNet network, which includes: an Embedding layer, a Stage 1 layer, a first Merging layer, a Stage 2 layer, a second Merging layer, a Stage 3 layer, a third Merging layer, a Stage 4 layer, a max-average pooling layer, a convolutional layer, and a fully connected layer.

[0070] The input to the Embedding layer is used to input the fingerprint image. The output of the Embedding layer is connected to the input of the GMF module. The output of the GMF module is connected to the input of the Stage 1 layer. The output of the Stage 1 layer is connected to the input of the first Merging layer. The output of the first Merging layer is connected to the input of the Stage 2 layer. The output of the Stage 2 layer is connected to the input of the second Merging layer. The output of the second Merging layer is connected to the input of the Stage 3 layer. The output of the Stage 3 layer is connected to the input of the third Merging layer. The output of the third Merging layer is connected to the input of the Stage 4 layer. The output of the Stage 4 layer is connected to the input of the max pooling layer. The output of the max pooling layer is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the fully connected layer. The fully connected layer outputs the classification result.

[0071] The GMF module that was built is as follows:

[0072] The data is divided into several data groups. Each data group, Feature_n, is duplicated into two identical copies, a and b. Feature a is fed into a Global Max Pooling layer for global max pooling. The output is then duplicated into two identical copies, a_1 and a_2. a_1 is fed into the first convolutional layer for convolution, and a_2 is fed into the second convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the features processed in the first channel. For feature b, it first enters the Avg... In the pooling layer, global max pooling is performed, and the output is copied into two identical copies, b_1 and b_2. b_1 is fed into the third convolutional layer for convolution, and b_2 is fed into the fourth convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the features processed in the second channel. The features of the first and second channels are concatenated to obtain the new Feature_n. All data groups processed in the above way are concatenated to obtain the output of the GMF module.

[0073] The constructed basic block module is as follows: First, the input is fed into the PConv layer and undergoes a partial convolution to obtain feature c. Feature c is copied into two identical copies. The first copy is fed into the Avg pool for global average pooling, and the output is fed into the PConv layer for partial convolution. The second copy is fed into the Glo pool layer for global max pooling, and the output is fed into the PConv layer for partial convolution. The outputs of the two partial convolutions are concatenated to obtain feature d. Feature d is divided into two identical copies. The input of the first channel is fed into the first convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The input of the second channel is fed into the second convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The outputs of these two channels are multiplied to obtain feature e. Features c, d, and e are multiplied together to obtain the output of the improved basic feature extraction module.

[0074] The Embedding layer contains a Batch Normalization (BN) layer and a convolutional layer. Input first enters the BN layer, and its output then enters the convolutional layer. The output of the convolutional layer is the Embedding output, used for initial image contour feature extraction. After inserting the GMF module into the Embedding layer, it perceives local image features and processes receptive field information of different regions in a grouped manner, thus achieving more refined fingerprint image features. This module can work well with the PConv structure in FasterNet to process local features. Stage 1 has one basic block module, and Stage 2 has two basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module, resulting in the Stage 2 output. Stage 3 has four basic block modules. Input features first enter the first basic block module, and its output enters the second basic block module. The output of the second basic block module enters the third basic block module, and its output enters the fourth basic block module, resulting in the Stage 3 output. Stage 4 has input features first entering the first basic block module... The output of the block module goes into the second basic block module, and the output of the Stage 4 layer is obtained. All Merging layers contain Dropout layers and BN layers. The input features first go into the Dropout layer, and the output goes into the BN layer. The Dropout layer is used to randomly drop some neurons to prevent overfitting.

[0075] Example 1

[0076] like Figure 1 As shown, an improved fingerprint activity detection method for FasterNet network is presented. This example includes 6 steps.

[0077] 1. First, the data is divided into training and testing sets, containing data from 5 sensors and 6 materials. The data in the training and testing sets must not be duplicated.

[0078] 2. Extract ROIs, crop, flip, normalize, and remove sensor white edges from fingerprint images in the training and test sets to ensure image integrity. Crop the ROI images to 224×224 size and then flip them to enhance the network's ability to process noisy images.

[0079] 3. Build the GMF (Grouping multi-scale fusion module) module and the basic block (basic feature extraction module) module;

[0080] 4. Based on the modules in step 3, build an improved FasterNet network architecture, including Embedding, GMF, Stage 1 layer, first Merging layer, Stage 2 layer, second Merging layer, Stage 3 layer, third Merging layer, Stage 4 layer, max average pooling layer, Conv1×1 layer, and fully connected layer.

[0081] 5. Pack the training set data and feed it into the network to train the network parameters. Finally, use the test set to test the experimental data.

[0082] 6. Obtain the fingerprint image recognition result;

[0083] The network architecture of this invention was tested on CentOS 7 with a single NVIDIA GTX 2080Ti GPU. The training epochs were 400, with image data in batches of 12. The learning rate adjuster used was the ReduceLROnPlateau learning rate decay scheduler with a learning rate decay factor of 0.1. The waiting period for the metric was 15 epochs, and all other settings were default. The optimizer used was the Adam adaptive moment estimation optimizer. When testing the network with the test set, all network parameters remained unchanged, using the parameters obtained in the last training epoch.

[0084] Example 2

[0085] See Figure 2 The improved Fasternet network consists of Embedding, GMF modules, Stage 1, Mering, Stage 2, Mering, Stage 3, Mering, Stage 4, a global pooling layer, a Conv1×1 layer, and a fully connected layer. The basic structure of each Stage is as follows: Figure 4 As shown, it contains a PConv and two sets of convolutions, with a batch normalized (BN) layer and a ReLU activation function in between.

[0086] Batch normalization: Calculate the mean and variance of a batch of image data, and transform the data into a normal distribution with a variance of 1 and a mean of 0 based on the calculated mean and variance. The formula is as follows:

[0087]

[0088] In the formula, ν and β are parameters introduced by the BN layer and can be learned independently. β and σ β It uses random offsets and random scaling to increase the model's learning ability.

[0089] The ReLU activation function is a non-linear activation function. The reciprocal of the ReLU function is always equal to 1 in the positive part. Therefore, using the ReLU activation function in deep networks will not lead to gradient minima explosion. The ReLU formula is as follows:

[0090]

[0091] In the convolutional layer, convolution is used to extract image contour features. This structure can effectively preprocess images. The convolution formula is as follows:

[0092]

[0093] Where D is the channel tensor of input x.

[0094] Dropout is a random dropout layer that randomly discards values ​​from some neurons to reduce the model's fit and prevent overfitting.

[0095] PConv is a novel convolutional method that applies regular Convolution to a portion of the input channels for spatial extraction, while keeping the remaining channels unchanged. For consecutive or regular memory accesses, the first or last consecutive channel is treated as representative of the entire feature map for computation. Without loss of generality, the input and output feature maps are assumed to have the same number of channels. Therefore, the FLOPs of PConv are 1 / 16 of those of regular convolution, calculated as follows:

[0096] h×w×(k 2 ×c p ×c+c×(cc p ))

[0097] Furthermore, PConv requires significantly less memory, only one-quarter of that of conventional convolutions. This structure effectively reduces computational redundancy and memory access, as shown in the following formula:

[0098]

[0099] Where h represents the height of the image, w represents the width of the image, k is the size of the convolution kernel, and c is the total number of channels in the entire image.

[0100] c pThis refers to the number of channels used in the convolution operation. Reducing the computational cost of convolution can significantly save computing resources, improve system speed, and lessen the computational burden on the hardware.

[0101] Example 3

[0102] In traditional network modules, using multiple re-convolutions leads to stacking more deep convolutional objects, which not only loses some original image features but also consumes a large amount of memory and computing resources. To address this issue, this paper reduces the structure of the original FasterNet and uses the GMF module for feature grouping to process the fingerprint feature mapping domain. Feature grouping allows each group of data to focus on different receptive fields, enhancing the network's ability to perceive features. GMF can be described by the following two processes.

[0103] 1. Feature Grouping: Using a parallel subnetting approach, the features of different subnets need to be divided first. For any given input feature map... EMA divides X into G sub-features along the cross-channel dimension to learn different semantics, where X = [X0, X1, X2, ..., X...]. G-1 ], Typically, the value of G should be less than the number of channels; in this paper, G is set to 8.

[0104] 2. Parallel Subnetwork: The large local receptive field enables modules to collect information at multiple scales. Therefore, GMF uses two parallel routes to extract attention weight descriptors for grouped feature maps. The same operation is performed on each group of data. Taking Feature_2 as an example, Feature_2 is copied into two identical copies, a and b. Feature a enters the Global pooling layer for global max pooling. The output is then copied into two identical copies, a_1 and a_2. a_1 enters a Conv 1×1 layer for convolution, and a_2 enters a Conv 3×3 layer for convolution. The two convolution results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features undergo a softmax operation and are multiplied by Feature_2 to obtain the features processed in the first channel. For feature b, it first enters the Avg pooling layer for global max pooling, as shown in the following equation.

[0105]

[0106] Where H and W represent the image height and width, i represents the x-coordinate of a pixel, j represents the y-coordinate of a pixel, and x... c(i,j) represents the pixel value at point (i,j). The output is copied into two identical copies, b_1 and b_2. b_1 is fed into a 5×5 Conv layer for convolution, and b_2 is fed into a 7×7 Conv layer for convolution. The two convolution results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features are subjected to a softmax operation, as shown in the following formula.

[0107]

[0108] Where e refers to the natural base, x i This represents the value of the i-th output, and n represents the total number of outputs. Multiplying this by Feature_2 yields the processed feature for the second channel. Concatenating the features from the first and second channels results in the new Feature_2. Concatenating these eight sets of data again yields the output of the GMF module.

[0109] The improved Fasternet network was experimentally tested on LivDet2011 and LivDet2013. The results compared with existing convolutional neural networks are shown in the following tables. Table 1 shows the experimental results for LivDet2011, Table 2 shows the experimental results for LivDet2013, Table 3 shows the experimental results across materials, and Table 4 shows the experimental results across datasets. In the table, Bio represents Biometrika, Dig represents Digital, Ita represents Itaklata, and Sag represents Sagem. These are all names of sensors.

[0110] Table 1

[0111]

[0112] Table 2

[0113]

[0114] Table 3

[0115]

[0116]

[0117] Table 4

[0118]

[0119] As shown in the table above, the improved Fasternet network proposed in this invention performs excellently in fingerprint activity detection tasks, outperforming traditional deep learning networks. Furthermore, it achieves better results in cross-material and cross-dataset scenarios, demonstrating that this invention improves the generalization of the FLD system.

Claims

1. An improved fingerprint activity detection method for FasterNet network, characterized in that, It includes the following steps: Step 1: Obtain the fingerprint image dataset and divide it into training and test sets; Step 2: Preprocess the training and test sets, including ROI extraction, cropping, flipping, and normalization; Step 3: Build the GMF module and the basic block module; Step 4: Based on the GMF module and basic block module in Step 3, build an improved FasterNet network architecture, including the Embedding layer, Stage 1 layer, first Merging layer, Stage 2 layer, second Merging layer, Stage 3 layer, third Merging layer, Stage 4 layer, max pooling layer, convolutional layer, and fully connected layer. Step 5: Input the fingerprint images processed in Step 2 into the constructed network architecture to extract features and classify the fingerprint images; Step 6: Output the fingerprint image recognition result; In step 3, the GMF module built is as follows: The data is divided into several data groups. Each data group, Feature_n, is duplicated into two identical copies, a and b. Feature a is fed into a Global Max Pooling layer for global max pooling. The output is then duplicated again into two identical copies, a_1 and a_2. Feature a_1 is fed into the first convolutional layer for convolution, and feature a_2 is fed into the second convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved. The output is then convolved again, and the output features undergo a softmax operation and are multiplied by Feature_n to obtain the feature after processing the first channel. For feature b, it first enters the Avg... In the pooling layer, global max pooling is performed, and the output is copied into two identical feature sets, b_1 and b_2. Feature b_1 is fed into the third convolutional layer for convolution, and feature b_2 is fed into the fourth convolutional layer for convolution. The two convolutional results are concatenated, and the concatenated features are convolved again. The output is then convolved again, and the output features are subjected to a softmax operation and multiplied by Feature_n to obtain the feature after processing the second channel. The features of the first and second channels are then concatenated to obtain the new Feature_n. All data sets processed in the above way are concatenated to obtain the output of the GMF module. In step 3, the basic block module constructed is as follows: First, the input is fed into the PConv layer and undergoes a partial convolution to obtain feature c. Feature c is then copied into two identical copies. The first copy is fed into the Avg pool for global average pooling, and the output is fed into the PConv layer for partial convolution. The second copy is fed into the Glo pool layer for global max pooling, and the output is fed into the PConv layer for partial convolution. The outputs of the two partial convolutions are then concatenated to obtain feature d. The feature d is divided into two equal parts. The input of the first channel is fed into the first convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The input of the second channel is fed into the second convolutional layer for convolution, and the output is fed into the Dropout layer for regularization. The outputs of these two channels are multiplied to obtain feature e. The features c, d, and e are multiplied together to obtain the output of the improved basic feature extraction module. In step 4, the improved FasterNet network architecture is constructed as follows: The input to the Embedding layer is used to input the fingerprint image. The output of the Embedding layer is connected to the input of the GMF module. The output of the GMF module is connected to the input of the Stage 1 layer. The output of the Stage 1 layer is connected to the input of the first Merging layer. The output of the first Merging layer is connected to the input of the Stage 2 layer. The output of the Stage 2 layer is connected to the input of the second Merging layer. The output of the second Merging layer is connected to the input of the Stage 3 layer. The output of the Stage 3 layer is connected to the input of the third Merging layer. The output of the third Merging layer is connected to the input of the Stage 4 layer. The output of the Stage 4 layer is connected to the input of the max pooling layer. The output of the max pooling layer is connected to the input of the convolutional layer. The output of the convolutional layer is connected to the input of the fully connected layer. The fully connected layer outputs the classification result.

2. The method according to claim 1, characterized in that, Step 2 specifically includes the following sub-steps: Step 1) Divide the image into n small blocks. Taking one small block as an example, calculate the mean of the image in that small block, and then calculate the standard deviation within that small block. The formula for calculating the mean is as follows, where p is the mean of the small block, and m and n are the length and width of the small block. The value of the target pixel; ； The formula for calculating standard deviation is as follows: ； Step 2) Compare the mean and standard deviation from Step 1. If the mean of the small area is greater than the standard deviation, it is a masked area. Save the location information of all masked areas in a list and merge them to obtain the ROI region. Step 3) Crop the fingerprint image of the ROI region obtained in Step 2; Step 4) Randomly select either horizontal or vertical flipping to flip the cropped image from Step 3 to obtain the enhanced image; Step 5) Quantize the flipped image from step 4 within the range of [-1,1] to achieve normalization, avoid data overflow caused by excessively large data, and make the data mean 0 and the standard deviation 1, which conforms to the normal distribution standard.

3. The method according to claim 1, characterized in that, The Embedding layer contains a Batch Normalization (BN) layer and a convolutional layer. The input first goes into the BN layer, and the output of the BN layer then goes into the convolutional layer. The output of the convolutional layer is the output of the Embedding layer, which is used to initially extract image contour features. After inserting the GMF module into the Embedding layer, it perceives local feature information of the image and processes the receptive field information of different regions in a grouping manner, thereby achieving more refined fingerprint image features. This module can work well with the PConv structure in Fasternet to process local features. Stage 1 has one basic block module, Stage 2 has two basic block modules. Input features first enter the first basic block module, and the output enters the second basic block module, resulting in the Stage 2 output. Stage 3 has four basic block modules. Input features first enter the first basic block module, and the output enters the second basic block module. The output of the second basic block module enters the third basic block module, and the output of the third basic block module enters the fourth basic block module, resulting in the Stage 3 output. Stage 4 has input features first enter the first basic block module, and the output enters the second basic block module, resulting in the Stage 4 output. All Merging layers include Dropout layers and BN layers. Input features first enter the Dropout layer, and the output enters the BN layer. The Dropout layer is used to randomly drop some neurons to prevent overfitting.

4. The method according to claim 1, characterized in that, The fingerprint image input and process are as follows: The processed image in step 2 first enters the Embedding layer. The output of the Embedding layer enters the GMF module. The output of the GMF enters the Stage 1 layer, which is a basic block layer. The output of Stage 1 enters the first Merging layer. The output of the first Merging layer enters the Stage 2 layer, which is the first basic block layer of Stage 2. The output of Stage 2 enters the second basic block layer of Stage 2. The output of Stage 2 enters the second Merging layer. The output of Stage 3 enters the first basic block layer of Stage 3. The output of Stage 3 enters the second basic block layer of Stage 3. The output of Stage 3 enters the third basic block layer of Stage 3. The output of Stage 3 enters the fourth basic block layer of Stage 3. Then it enters the third Merging layer. The output of Stage 4 enters the first basic block layer of Stage 4. The output of Stage 4 enters the second basic block layer of Stage 4. The output of Stage 4 enters the max average pooling layer. Then it enters a convolutional layer. The output of Stage 4 enters a fully connected layer to obtain the classification result.

Images