An Improved Fingerprint Activity Detection Method for Mobilevit Networks

Abstract

An improved fingerprint activity detection method using a Mobilevit network includes the following steps: Step 1: Acquire a fingerprint image dataset and divide it into training and testing sets, then preprocess the fingerprint images; Step 2: Build a multimodal feature extraction (MFE) module and a diffusion analysis (DA) module; Step 3: Build a Mobilevit network model; Step 4: Integrate the new DA module with the Mobilevit network to obtain an improved Mobilevit network; Step 5: Input the fingerprint image into the network and obtain the classification result. The purpose of this invention is to address the technical problems of existing technologies, such as the high error rate caused by the lack of comprehensive feature extraction, insufficient extraction of features, and the overemphasis on deep features while neglecting shallow features, leading to a high classification error rate. This invention provides an improved fingerprint activity detection method using a Mobilevit network.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to fingerprint activity detection technology based on neural networks, specifically to a fingerprint activity detection method using an improved Mobilevit network. Background Technology

[0002] Systems that identify people based on their physical or behavioral characteristics are called biometric systems. Biometric systems are widely used based on personal features such as fingerprints, face, iris, retina, hand geometry, signature, and voice. In the event of a crime, hair, rashes, saliva, bloodstains, fingerprints, or security camera footage can be used as evidence at the scene. Sometimes, there may be no security cameras at the location of the crime, or a clear image may not be available from them. Fingerprints are considered the most desirable feature than DNA mapping because DNA mapping has higher time requirements and is more expensive than fingerprint recognition systems. Fingerprint data has been frequently used in criminal identification systems since the 19th century, and with technological advancements, it has been generated using various materials. Similar fingerprint data can be easily generated using different materials, such as gelatin, wood glue, etc. Distinguishing between genuine and fake fingerprint data can prevent fraudulent activities caused by using fake fingerprints.

[0003] With fingerprint authentication systems becoming a popular target for attackers, security attacks using fake fingerprints made from various materials have garnered significant attention. To mitigate this threat, researchers have proposed several CNN-based Fingerprint Activity Detection (FLD) methods, which can automatically extract key features from fake fingerprints and train a model to distinguish between live and fake fingerprints. However, their detection performance on fingerprints made from "untrained" materials remains limited. To achieve generalization performance across various materials in FLD, several cross-material approaches have been proposed. Existing methods primarily focus on generating fake fingerprints with new material patterns using pattern information from well-known materials. However, these methods support data augmentation during training rather than improving runtime detection on untrained materials. Therefore, a detection method is needed to accurately infer fake fingerprints made from untrained materials and improve the classification accuracy of trained fingerprints while reducing the error rate.

[0004] Defects and shortcomings of existing technology: 1) Existing neural network models do not have the ability to extract multimodal fingerprint features, and cannot effectively extract fingerprint feature information from multiple dimensions, resulting in a weak ability to express fingerprint features.

[0005] 2) In the real world, fingerprints are vulnerable to different types of attacks, including unseen materials and sensors. Therefore, classification results in different scenarios are more effective in detecting the performance of fingerprint activity detection methods. Most existing algorithms cannot achieve high classification results in new scenarios.

[0006] Therefore, the applicant proposes an improved fingerprint activity detection method for Mobilevit networks. Summary of the Invention

[0007] The purpose of this invention is to address the technical problems of existing technologies, such as the lack of comprehensive and sufficient feature extraction for distinguishing between live and forged fingerprints, the overemphasis on deep features while neglecting shallow features, leading to a high classification error rate. This invention provides an improved fingerprint activity detection method based on Mobilevit networks.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An improved fingerprint activity detection method for Mobilevit networks includes the following steps: Step 1: Obtain the fingerprint image dataset and divide it into training and test sets, and preprocess the fingerprint images; Step 2: Build the Multimodal Feature Extraction (MFE) module and the Diffusion Analysis (DA) module; Step 3: Build the Mobilevit network model; Step 4: Integrate the new DA module with the Mobilevit network to obtain the improved Mobilevit network.

[0009] Step 5: Input the fingerprint image into the network and obtain the classification result.

[0010] In step 1, the fingerprint image needs to be preprocessed. First, the fingerprint ROI region is extracted. This invention uses a manually labeled dataset method to extract the ROI region. The fingerprint region of each image is manually selected, the region with the fingerprint image is selected and the remaining white edge is removed. Next, the fingerprint image is cropped to a uniform size. Then the image is normalized to obtain the preprocessed fingerprint image.

[0011] In step 2, the multimodal feature extraction module (MFE) specifically comprises: The input features are processed through two channels: a left channel and a right channel. The features in the left channel first enter the first convolutional layer for convolution. The output of the first convolutional layer enters the second convolutional layer for convolution. The output of the second convolutional layer enters the third convolutional layer for convolution. The output of the third convolutional layer enters the fourth convolutional layer for convolution. The output of the fourth convolutional layer is then subjected to dimensionality reduction, converting the three-dimensional features into two-dimensional features to reduce network computational overhead. This two-dimensional feature is then transformed to extract semantic information from the image, enhancing the inherent representation and expressiveness of the features. The output of this two-dimensional feature is then transformed into a three-dimensional feature, which enters the fifth convolutional layer for convolution. The output of this three-dimensional feature is the output of the left channel. The input features simultaneously enter the right channel. The features in the right channel first enter the SGE module to enhance the preprocessing of the image and distinguish different fingerprint image regions. The output of the SGE module enters the SK module to extract global information of the fingerprint image. It uses a design of double convolution with receptive fields of different sizes to enhance feature sensitivity. The features obtained after processing by the SK module enter the EMA module to enhance the network's grouping ability, thereby improving the ability to extract local features. The features obtained after processing by the EMA module enter the Shuffle module to further extract information from local features and analyze the features from the perspectives of spatial attention and channel attention. The output of the Shuffle module enters the Triplet module to rotate the feature group and analyze its information from three dimensions. The output of the Triplet module is the output of the right channel. The outputs of the left and right channels are merged, and the merged features are then convolved again to obtain the output of the MFE module.

[0012] In step 2, the diffusion analysis module DA specifically comprises: The features input into the DA module enter channel a, where they undergo a convolution process. The resulting features then enter channels a-1, a-2, and a-3 for more detailed image feature extraction. The features entering channel a-3 undergo a feature recombination along the channel direction to disrupt feature continuity and improve network robustness. The recombined features are then the output of channel a-3. The features entering channel a-2 undergo feature recombination along the channel direction and then enter a depthwise separable convolutional layer. This convolution effectively reduces computational cost. The result of the convolution is multiplied by the output of channel a-3, and the product is then subjected to a Tanh operation to obtain the output of channel a-2. The features entering channel a-1 first undergo a transformation to extract semantic information from the image. The output of the second transformation undergoes a second transformation. The output of the second transformation enters a convolutional layer for convolution. The result of the convolution is added to the output of channel a-2 to obtain the output of channel a-1. Features entering the DA module simultaneously enter channel b. The features entering channel b first undergo a convolution operation. The resulting features then enter channels b-1, b-2, and b-3 for more detailed image feature extraction. Features entering channel b-3 undergo a channel-direction recombination, scrambling the original feature order, resulting in the output of channel b-3. Features entering channel b-2 first undergo a channel-direction recombination, scrambling the features. The resulting scrambled features undergo a partial convolution. The resulting features are multiplied with the output of channel b-3. The output of this multiplication is then subjected to a sigmoid operation, using a different activation function than channel a to prevent high feature similarity and to compensate for weaknesses. The features after the sigmoid operation are the output of channel b-2. The features entering channel b-1 first enter a convolutional layer for convolution processing. The processed features then enter a second convolutional layer for convolution processing, and so on. The processed features then enter a third convolutional layer for convolution processing. The processed features are added to the output of channel b-2 to obtain the output of channel b-1. The outputs of channel a-1, channel b-1, the output of channel a after its first convolution, and the output of channel b after its first convolution are then fused to obtain the output of the DA module.

[0013] In step 3, the Mobilevit network model constructed is specifically as follows: The features entering the network first go into a convolutional layer for convolution processing. The resulting features then go into the first Block 1. The output of the first Block 1 goes into the second Block 1. The output of the second Block 1 goes into the third Block 1. The output of the third Block 1 goes into the first MFE. The output of the first MFE goes into the fourth Block 1. The output of the fourth Block 1 goes into the second MFE. The output of the second MFE goes into the fifth Block 1. The output of the fifth Block 1 goes into a convolutional layer for convolution operation. The output of the convolutional layer goes into a fully connected layer to obtain the classification result.

[0014] In step 4, the diffusion analysis module DA from step 2 is fused with the Mobilevit network from step 3 to obtain an improved Mobilevit network; the improved Mobilevit network is as follows: The features entering the network first go into a convolutional layer for convolution processing. After convolution, the features are fed into the first Block 1. The output of the first Block 1 goes into the second Block 1. The output of the second Block 1 goes into the third Block 1. The output of the third Block 1 goes into the MFE. The output of the MFE goes into the DA module. The output of the DA module goes into the second DA module. The output of the second DA module goes into the fourth Block 1. The output of the fourth Block 1 goes into the second MFE. The output of the second MFE goes into the fifth Block 1. The output of the fifth Block 1 goes into a convolutional layer for convolution. The output of the convolutional layer goes into a fully connected layer, finally yielding the classification result.

[0015] Compared with the prior art, the present invention has the following technical effects: 1) This invention introduces an MFE (Multi-modal feature extraction) module into the original Mobilevit network. This module combines five attention modules and designs a dual-channel feature extraction model, which has the advantage of enhancing feature extraction capabilities.

[0016] 2) This invention improves upon the Mobilevit block module in the original Mobilevit network structure by designing a DA module based on the concept of diffusion feature analysis. It processes features through six small channels, fusing information extracted from different channels. Different depth strategies are employed for information extraction and analysis at different channel levels. The outer channels (channels a-3, b-3) use a shallow feature extraction scheme, while the inner channels (channels a-1, b-1) use a deep feature extraction scheme. More advanced processing is applied to the features, including convolution and transform. Finally, residual connections are used to fuse the features with those before they enter the small channels, avoiding the extraction of noisy deep features. In addition to traditional channel and spatial feature extraction, this module also analyzes different expressive features of the image. Convolutional structures are used for feature extraction from an image perspective, while transform structures are used for feature analysis from a semantic perspective. Finally, semantic and convolutional features are fused, providing a more comprehensive analysis of the intrinsic features of the fingerprint image.

[0017] 3) In the experimental section, the experimental results of the network model proposed in this invention are the best, proving that the method has good classification performance (see the Implementation Section for details). Attached Figure Description

[0018] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a flowchart of the present invention; Figure 2 A schematic diagram of the improved Mobilevit network structure provided by this invention; Figure 3 This is a schematic diagram of the MFE module structure used in this invention; Figure 4 This is a schematic diagram of the improved DA module of the present invention. Detailed Implementation

[0019] like Figure 1 As shown, an improved fingerprint activity detection method for Mobilevit networks includes the following steps: Step 1: Obtain the fingerprint image dataset and divide it into training and test sets, and preprocess the fingerprint images; Step 2: Build the Multimodal Feature Extraction (MFE) module and the Diffusion Analysis (DA) module; Step 3: Build the Mobilevit network model; Step 4: Integrate the new DA module with the Mobilevit network to obtain the improved Mobilevit network.

[0020] Step 5: Input the fingerprint image into the network and obtain the classification result.

[0021] In step 1, the fingerprint image needs to be preprocessed. First, the fingerprint ROI region is extracted. This invention uses a manually labeled dataset method to extract the ROI region. The fingerprint region of each image is manually selected, the region with the fingerprint image is selected and the remaining white edge is removed. Next, the fingerprint image is cropped to a uniform size. Then the image is normalized to obtain the preprocessed fingerprint image.

[0022] In step 2, as Figure 3 As shown, the multimodal feature extraction module (MFE) specifically comprises: The input features are processed through two channels: a left channel and a right channel. The features in the left channel first enter the first convolutional layer for convolution. The output of the first convolutional layer enters the second convolutional layer for convolution. The output of the second convolutional layer enters the third convolutional layer for convolution. The output of the third convolutional layer enters the fourth convolutional layer for convolution. The output of the fourth convolutional layer is then subjected to dimensionality reduction, converting the three-dimensional features into two-dimensional features to reduce network computational overhead. This two-dimensional feature is then transformed to extract semantic information from the image, enhancing the inherent representation and expressiveness of the features. The output of this two-dimensional feature is then transformed into a three-dimensional feature, which enters the fifth convolutional layer for convolution. The output of this three-dimensional feature is the output of the left channel. The input features simultaneously enter the right channel. The features in the right channel first enter the SGE module to enhance the preprocessing of the image and distinguish different fingerprint image regions. The output of the SGE module enters the SK module to extract global information of the fingerprint image. It uses a design of double convolution with receptive fields of different sizes to enhance feature sensitivity. The features obtained after processing by the SK module enter the EMA module to enhance the network's grouping ability, thereby improving the ability to extract local features. The features obtained after processing by the EMA module enter the Shuffle module to further extract information from local features and analyze the features from the perspectives of spatial attention and channel attention. The output of the Shuffle module enters the Triplet module to rotate the feature group and analyze its information from three dimensions. The output of the Triplet module is the output of the right channel. The outputs of the left and right channels are merged, and the merged features are then convolved again to obtain the output of the MFE module.

[0023] In step 2, as Figure 4 As shown, the diffusion analysis module DA specifically comprises: The features input into the DA module enter channel a, where they undergo a convolution process. The resulting features then enter channels a-1, a-2, and a-3 for more detailed image feature extraction. The features entering channel a-3 undergo a feature recombination along the channel direction to disrupt feature continuity and improve network robustness. The recombined features are then the output of channel a-3. The features entering channel a-2 undergo feature recombination along the channel direction and then enter a depthwise separable convolutional layer. This convolution effectively reduces computational cost. The result of the convolution is multiplied by the output of channel a-3, and the product is then subjected to a Tanh operation to obtain the output of channel a-2. The features entering channel a-1 first undergo a transformation to extract semantic information from the image. The output of the second transformation undergoes a second transformation. The output of the second transformation enters a convolutional layer for convolution. The result of the convolution is added to the output of channel a-2 to obtain the output of channel a-1. Features entering the DA module simultaneously enter channel b. The features entering channel b first undergo a convolution operation. The resulting features then enter channels b-1, b-2, and b-3 for more detailed image feature extraction. Features entering channel b-3 undergo a channel-direction recombination, scrambling the original feature order, resulting in the output of channel b-3. Features entering channel b-2 first undergo a channel-direction recombination, scrambling the features. The resulting scrambled features undergo a partial convolution. The resulting features are multiplied with the output of channel b-3. The output of this multiplication is then subjected to a sigmoid operation, using a different activation function than channel a to prevent high feature similarity and to compensate for weaknesses. The features after the sigmoid operation are the output of channel b-2. The features entering channel b-1 first enter a convolutional layer for convolution processing. The processed features then enter a second convolutional layer for convolution processing, and so on. The processed features then enter a third convolutional layer for convolution processing. The processed features are added to the output of channel b-2 to obtain the output of channel b-1. The outputs of channel a-1, channel b-1, the output of channel a after its first convolution, and the output of channel b after its first convolution are then fused to obtain the output of the DA module.

[0024] In step 3, the Mobilevit network model constructed is specifically as follows: The features entering the network first go into a convolutional layer for convolution processing. The resulting features then go into the first Block 1. The output of the first Block 1 goes into the second Block 1. The output of the second Block 1 goes into the third Block 1. The output of the third Block 1 goes into the first MFE. The output of the first MFE goes into the fourth Block 1. The output of the fourth Block 1 goes into the second MFE. The output of the second MFE goes into the fifth Block 1. The output of the fifth Block 1 goes into a convolutional layer for convolution operation. The output of the convolutional layer goes into a fully connected layer to obtain the classification result.

[0025] In step 4, the diffusion analysis module DA from step 2 is fused with the Mobilevit network from step 3 to obtain an improved Mobilevit network; as shown Figure 2As shown, the improved Mobilevit network is as follows: The features entering the network first go into a convolutional layer for convolution processing. After convolution, the features are fed into the first Block 1. The output of the first Block 1 goes into the second Block 1. The output of the second Block 1 goes into the third Block 1. The output of the third Block 1 goes into the MFE. The output of the MFE goes into the DA module. The output of the DA module goes into the second DA module. The output of the second DA module goes into the fourth Block 1. The output of the fourth Block 1 goes into the second MFE. The output of the second MFE goes into the fifth Block 1. The output of the fifth Block 1 goes into a convolutional layer for convolution. The output of the convolutional layer goes into a fully connected layer, finally yielding the classification result.

[0026] Example: like Figure 1 As shown, an improved fingerprint activity detection method for Mobilevit networks includes the following steps: Step 1: Obtain the fingerprint image dataset and divide it into training and test sets, and preprocess the fingerprint images.

[0027] Step 2: Build the MFE (Multi-modal feature extraction) module and the DA (Diffusion analysis) module.

[0028] Step 3: Build the improved Mobilevit network model, including Conv3×3, Block1, Block1, Block1, MFE, Block1, MFE, Block1, Conv1×1, Linear.

[0029] Step 4: Integrate the new DA module with the improved Mobilevit network.

[0030] Step 5: Input the fingerprint image into the network and obtain the classification result.

[0031] The network architecture of this invention was tested on CentOS 7 with dual NVIDIA GTX 1060Ti GPUs. The training epochs were 200, with image data in batches of 36. The learning rate adjuster used was the ReduceLROnPlateau learning rate decay scheduler with a learning rate decay factor of 0.1. The waiting period for the metric was 15 epochs, and all other settings were default. The optimizer used was the Adam adaptive moment estimation optimizer. When testing the network with the test set, all network parameters remained unchanged, using the parameters obtained in the last training epoch.

[0032] like Figure 2 As shown, the improved Mobilevit network consists of Conv3×3, Block1, Block1, Block1, MFE, DA, DA, Block1, MFE, Block1, Conv1×1, and Linear. The MFE module is as follows: Figure 3 As shown, the DA module is as follows Figure 4 As shown.

[0033] MFE employs operations such as convolution and transform, and utilizes existing modules such as SGE, SK, EMA, Shuffle, and Triplet.

[0034] Convolution operations are as follows: ; In the above formula, b is the output feature map, a is the input feature map, w is the convolution kernel, i and j are the pixel positions of the new feature map, and u and v are the feature point positions of the convolution kernel.

[0035] The Transform operation is as follows: ; Q, K, and V represent Query, Key, and Value, respectively.

[0036] It is the dimension of the key.

[0037] SGE module: The SGE module needs to group the input image according to the channel dimension. Let's assume the input image has C channels. The feature mapping is first divided into G groups, and the feature mapping after grouping is... Assuming that SGE has learned the semantic part of the fingerprint image in this step, feature vectors at the ridge edges can be obtained, while other locations remain almost unchanged, approximating as zero vectors. To address the unavoidable approximation noise, a spatial averaging function is employed. By statistically analyzing global features, the features of the entire space will not be affected by too much noise, thereby further enhancing the semantic feature learning of key regions, as shown in Equation (1).

[0038] ; The importance coefficient is obtained by the dot product of global semantic features and local features. It can represent the similarity between global features and local features, or it can be represented as... ,in It is a vector and The included angle. This indicates Features and direction The closer the samples are, the larger the initial coefficients are likely to be. To avoid excessively large coefficients between different samples, it is necessary to normalize t in space, as shown in equations (2), (3), and (4).

[0039] ; ; ; in It is a constant term, and to ensure numerical stability, its value is set to 0.0001 in this paper.

[0040] To ensure that the normalization of the insertion can represent the unit transformation, it is necessary to... Introduce a new pair of parameters and This is used to scale and shift the normalization, as shown in equation (5).

[0041] ; These two parameters and The only parameter introduced by SGE is , and its number is the same as that of the group G. In this paper, there are 8 parameters. Therefore, the impact of adding these two parameters on the millions of parameters in the network is negligible and will not increase the computational burden of the network. Finally, the obtained vector needs to be spatially scaled using the Sigmoid function, as shown in Equation (6).

[0042] ; Finally, all feature groups are obtained. .

[0043] EMA module: In traditional network modules, using multiple re-convolutions leads to stacking more deep convolution objects, which not only loses some original image features but also consumes a lot of memory and computing resources. To solve this problem, this paper optimizes the structure of the EMA module [4] to replace the repeated stacked convolution calculations in the original EfficientNetV2-S network. This module processes the fingerprint feature mapping domain. By using parallel subnets, each subnet can focus on different receptive fields, enhancing the network's ability to perceive features.

[0044] Using parallel subnetting requires first dividing the features of different subnets. For any given input feature map... EMA divides the cross-channel dimension X into G sub-features for learning different semantic features, where Typically, the value of G should be less than the number of channels; in this paper, G is set to 8.

[0045] The large local receptive field allows the module to collect information at multiple scales; therefore, EMA employs three parallel paths to extract attention weight descriptors for grouped feature maps. Two paths are... Path, 1 path is Path. In Along the path, instead of convolution, 1D global average pooling is used to process information in both the H and W directions. This pooling method can better consider local information. Along the path, a convolutional operation with a kernel of 3 is used to capture more features. and In convolution, the following is used The original number of channels is replaced by a new form. Similar to CA attention, the results of two 1D global average pooling processes are concatenated along the height direction, and then... The convolution is decomposed into two vectors, and the sigmoid function is used to fit the two-dimensional binomial distribution after convolution. Then, the two vectors are multiplied using conventional multiplication. The information from the channels is aggregated. Channels are used to expand the feature space. This allows the module to encode inter-channel information, adjust the importance of channels, and store spatial structure information within the channels.

[0046] The method of cross-spatial information aggregation allows the module to extract richer local receptive field information, utilizing two-dimensional global average pooling. The output of the branch is encoded with global spatial information and then converted into its corresponding dimensional shape, as shown in Equation (7).

[0047] ; exist In this branch, two-dimensional global average pooling is also used to encode the global spatial information, and its output is fitted with a linear transformation using the nonlinear function Softmax, which is a two-dimensional Gaussian mapping. To capture the pixel-level relationships in the fingerprint image, the Sigmoid function is used to calculate the feature map of the multiplied output, and the final output size is the same as the input.

[0048] SK module: The SK module divides image processing into three steps: segmentation, fusion, and selection. During fusion, it integrates with the shallow features from the segmentation, learning different types of knowledge. Finally, it selects from the learned knowledge to obtain the output feature map.

[0049] The segmentation stage requires the use of two convolutional kernels to transform the image. Assume the original feature domain is... After performing two convolutions with kernels of 3 and 5, a new feature domain is obtained. , Batch normalization and BN layers are added sequentially after convolution.

[0050] The fusion stage requires obtaining feature reduction values, which are feature vectors obtained from the feature map. These vectors can adaptively adjust the main features in the feature domain. First, the feature domains obtained in the previous step are fused.

[0051] ; Global flat pooling is used to generate a single-channel statistical data S, and the embedded global information is obtained. The information of the two dimensions H and W is reduced and calculated as follows.

[0052] ; In the above formula, gp represents global average pooling. To achieve more accurate and better adaptive results in the feature domain, a more compact feature Z is needed. A fully connected layer is used to process S, resulting in a more efficient reduction in dimension Z.

[0053] ; Where fc represents a fully connected layer. It is the ReLU function. It is batch normalization, W is a value belonging to The feature domain of the space. The selection stage requires choosing the final combined vector. For feature Z, it needs to be differentiated into two different sets of feature values, which are then merged with the original feature domain. This paper uses softmax for segmentation.

[0054] ; ; Both A and B belong to Domains, a and b are respectively and Each has its own light attention vector. The final feature domain V is the sum of the light attention features from the two branches multiplied by the original feature domain.

[0055] ; The sum V obtained is all the features. .

[0056] Shuffle module: For a given feature map, the complete feature domain X is divided into G groups according to the channel dimension, i.e. The obtained set of features is then divided into two branches according to the channels, namely... The two branches are used to generate the channel attention map and the spatial attention map, respectively.

[0057] Channel attention maps require global average pooling of the feature domain to embed global information.

[0058] ; The obtained channel statistics S belong to Spatial domain. Then, compact features are generated through a gating mechanism and a sigmoid activation function. The formula is defined as: ; in , used to scale and move parameters.

[0059] The spatial attention map and channel attention map of the Shuffle module complement each other in information extraction. For spatial information processing, group normalization is used to obtain spatial statistical information, and linear computation is employed to enhance spatial information processing. The final output is as follows.

[0060] ; in, Represents the sigmoid activation function operation. Finally, the two branches are connected, resulting in a concatenated output with the same number of channels as the input. .

[0061] After each channel group is processed in both spatial and channel dimensions, it flows along the channel dimension and aggregates into a new feature domain of the same size as the initial feature domain.

[0062] Triplet module: The Triplet module processes images using three branches: two branches capture cross-dimensional interactive features between channel C and spaces H and W; and the last branch is used to construct a spatial attention mechanism to enhance spatial features.

[0063] The pool layer is used to reduce the channel dimension, reducing the original dimension to 2-dimensional, and connecting the average pooling feature and the maximum pooling feature in this dimension, thus preserving rich tensor information while reducing the amount of computation.

[0064] ; In the first branch, the H dimension and the C dimension establish an interaction relationship.

[0065] Input tensor Rotate 90° counterclockwise along the H-axis. The tensor after rotation. Shape After Z-pooling, The shape of the tensor is After convolution and batch normalization layers, the dimension is obtained as follows: The intermediate output is then processed by generating corresponding attention weights using a sigmoid function, which are multiplied by the original feature domain. Finally, the result is rotated 90° clockwise along the H-axis to match the size of the input feature domain.

[0066] In the second branch, an interaction is established between the W and C dimensions. The computation process is the same as in the previous branch, only the input tensor is processed. Rotate 90° counterclockwise around the W-axis. The tensor after rotation. Shape After Z-pooling, The shape of the tensor is After convolution and batch normalization layers, the dimension is obtained as follows: The intermediate output is then processed by generating corresponding attention weights using a sigmoid function, which are multiplied by the original feature domain. Finally, the result is rotated 90° clockwise along the W-axis to match the size of the input feature domain.

[0067] In the third branch, spatial dimensions interact with each other. Input tensor The characteristic shape is obtained by Z-pooling. of The tensor, after passing through convolution, batch normalization layers, and a sigmoid function, outputs a feature shape of... Multiplying this by the original feature domain yields a third feature domain, with the size remaining unchanged.

[0068] This invention proposes a channel diffusion DA module, which extends dual-channel feature processing to six-channel processing and finally outputs its features. The DA module uses the following operations: Reshape: This involves reorganizing the feature vectors of a module, randomly changing the order of the 3D features. H represents height, W represents width, and C represents channels. The features H×W×C before processing will be randomly reorganized into W×H×C and other feature representations.

[0069] Pconv: PConv is a novel convolutional method that applies regular Convolution to a portion of the input channels for spatial extraction, while keeping the remaining channels unchanged. For consecutive or regular memory accesses, the first or last consecutive channel is treated as representative of the entire feature map for computation. Without loss of generality, the input and output feature maps are assumed to have the same number of channels. Therefore, the FLOPs of PConv are 1 / 16 of those of regular convolution, calculated as follows: ; Furthermore, PConv requires significantly less memory, only one-quarter of that of conventional convolutions. This structure effectively reduces computational redundancy and memory access, as shown in the following formula: ; Where h represents the height of the image, w represents the width of the image, k is the size of the convolution kernel, and c is the total number of channels in the entire image. This refers to the number of channels used in the convolution operation. Reducing the computational cost of convolution can significantly save computing resources, improve system speed, and lessen the computational burden on the hardware.

[0070] DSConv: Depthwise separable convolution consists of depthwise convolution and pointwise convolution. Depthwise convolution is used to extract spatial features, while pointwise convolution is used to extract channel features. Depthwise separable convolution groups convolutions along the feature dimension, performing independent depthwise convolution on each channel, and then using a 1x1 pointwise convolution to aggregate all channels before the output. Depthwise convolution uses single-channel kernels, requiring convolution on every channel of the input, resulting in an output feature map with the same number of channels as the input feature map. That is, the number of input feature map channels = the number of convolution kernels = the number of output feature maps. Pointwise convolution (PWConv) essentially uses 1x1 convolution kernels to increase the dimensionality.

[0071] Tanh: In this invention, the left channel uses the Tanh activation function, which is calculated as follows: ; Where X represents the input feature.

[0072] Sigmoid: In this invention, the right channel uses the Tanh activation function, which is calculated as follows: ; Where X represents the input feature.

[0073] The improved Mobilevit network was experimentally tested on LivDet2011, LivDet2013, and LivDet2015. The results compared to existing convolutional neural networks are shown in the tables below. Table 1 presents the experimental results for LivDet2011, Table 2 for LivDet2013, Table 3 for LivDet2015, Table 4 for cross-material experiments, and Table 5 for cross-dataset experiments. In these tables, Bio represents Biometrika, Dig represents Digital, Ita represents Itaklata, Sag represents Sagem, Cro represents Crossmatch, Gre represents Greenbit, and His represents Biometrika; these are all sensor names. The experimental evaluation metric used is ACE, or Average Classification Error Rate; a lower ACE result indicates better experimental results.

[0074] Table 1

[0075] Table 2

[0076] Table 3

[0077] Table 4

[0078] Table 5

[0079] As shown in the table above, the improved Mobilevit network proposed in this invention performs excellently in the fingerprint activity detection task, outperforming traditional deep learning networks, and achieves better results in cross-material and cross-dataset scenarios, proving that this invention improves the generalization of the FLD system. Fingerprint activity detection is a crucial part of fingerprint recognition systems, responsible for protecting system security. This invention proposes an improved Mobilevit network to accomplish this task, addressing the problem of insufficient extraction of image feature dimensionality information in fingerprint activity detection. It employs residual connections to combine deep and shallow information and diffuses a dual-channel model to form a six-channel system for multi-dimensional image information extraction. The proposed fingerprint activity detection method effectively classifies fingerprint images and provides a foundational network design concept for future fingerprint activity detection methods.

Claims

1. An improved fingerprint activity detection method for Mobilevit networks, characterized in that, Includes the following steps: Step 1: Obtain the fingerprint image dataset and divide it into training and test sets, and preprocess the fingerprint images; Step 2: Build the improved Mobilevit network model; Step 3: Input the fingerprint image into the improved Mobilevit network and obtain the classification result; In step 2, the improved Mobilevit network model is as follows: The features entering the network first go through a convolutional layer. After convolution, the features are processed by three Block 1 layers. The outputs of the Block 1 layers go through two DA layers. The outputs of the Block 1 layers go through three Block 1 layers, three Block 1 layers, a convolutional layer, and a fully connected layer to obtain the classification result. The Multimodal Feature Extraction (MFE) module is as follows: The input features are fed into the left and right channels respectively. The features in the left channel are processed by four convolutional layers in sequence. The resulting output is subjected to dimensionality reduction, that is, the three-dimensional features are converted into two-dimensional features, and the two-dimensional features are transformed. Then, the resulting output is subjected to dimensionality increase, that is, the two-dimensional features are converted into three-dimensional features. The three-dimensional features are fed into the convolutional layer for convolution operation. The output obtained is the output of the left channel. The features of the right channel first enter the SGE module, and then the output of the SGE module enters the SK module. The features obtained after processing by the SK module enter the EMA module. The features obtained after processing by the EMA module enter the Shuffle module. The output of the Shuffle module enters the Triplet module. The output of the Triplet module is the output of the right channel. The outputs of the left and right channels are merged, and the merged features are then convolved again to obtain the output of the multimodal feature extraction module (MFE). The diffusion analysis module DA specifically includes: The features input to the diffusion analysis module DA enter channel a, and first undergo a convolution process. The resulting features then enter channels a-1, a-2, and a-3. The features entering channel a-3 undergo a feature recombination along the channel direction to obtain the output of channel a-3. The features entering channel a-2 first undergo feature recombination along the channel direction, and then enter a depthwise separable convolutional layer. The result of the convolution is multiplied with the output of channel a-3, and the product is then subjected to a Tanh operation to obtain the output of channel a-2. The features entering channel a-1 undergo two transform operations in sequence. The outputs are then fed into a convolutional layer. The result of the convolution is added to the output of channel a-2 to obtain the output of channel a. Features entering the diffusion analysis module DA simultaneously enter channel b. The features entering channel b first undergo a convolution operation. The features obtained after convolution are then entered into channels b-1, b-2, and b-3. The features entering channel b-3 undergo a channel-direction recombination, resulting in the output of channel b-3. The features entering channel b-2 first undergo a channel-direction recombination, and the resulting features undergo a partial convolution. The features obtained after partial convolution are multiplied with the features output from channel b-3. The output of the multiplication is then subjected to a sigmoid operation, and the features after the sigmoid operation are the output of channel b-2. The features entering channel b-1 are processed through three convolutional layers in sequence. The processed features are added to the output of channel b-2 to obtain the output of channel b. The outputs of channel a, channel b, the output of channel a after its first convolution, and the output of channel b after its first convolution are fused together to obtain the output of the diffusion analysis module DA.

2. The method according to claim 1, characterized in that, In step 1, the fingerprint image is preprocessed. First, the fingerprint ROI region is extracted. The ROI region is extracted using a manually labeled dataset method. The fingerprint region of each image is manually selected, and the area with the fingerprint image is selected and the remaining white edges are removed. Next, the fingerprint image is cropped to a uniform size. Then, the image is normalized to obtain the preprocessed fingerprint image.

Images