A face presentation attack detection method and device based on ViT and local clue enhancement

Abstract

The application discloses a face presentation attack detection method and device based on ViT and local clue enhancement, and relates to the technical field of computer vision. The application constructs a double-channel feature extraction network, extracts fine artifact features such as printing particles and screen moire through a local high-frequency texture enhancement module, and simultaneously utilizes a global space-time Transformer to model face structure consistency; then, the application fuses the double-channel features through a self-adaptive feature calibration fusion module, outputs a living body probability through multi-task decoding, and adopts adversarial training to improve cross-domain robustness. The application takes into account global semantics and local details, has high detection precision and strong generalization capability, is lightweight, and is suitable for mobile end face living body detection scenes.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and biometric recognition technology, specifically relating to a method and system for face presentation attack detection (PAD) that combines a vision transformer (ViT) with local texture feature enhancement. Background Technology

[0002] Facial recognition technology has been widely used in key areas such as financial payments, smart security, and identity authentication, making its security paramount. However, existing systems are vulnerable to "presentation attacks" such as printed photos, video playback, 3D masks, and even deepfakes, leading to serious economic losses and security risks.

[0003] Existing detection methods are mainly divided into two categories:

[0004] (1) Traditional methods rely heavily on hand-designed textures or motion features (such as LBP and HOG), which have limited generalization capabilities in the face of complex scenes and new attacks.

[0005] (2) CNN-based deep learning methods: Although they improve detection performance, they are limited by the local receptive field characteristics of convolutional neural networks, and have a bottleneck in capturing global inconsistencies (such as subtle contradictions in reflective properties between facial edges and the central region).

[0006] In recent years, Vision Transformer (ViT), with its global attention mechanism, has provided a new approach for simultaneously analyzing the correlations of multiple facial regions. However, directly applying ViT to face liveness detection still presents the following challenges:

[0007] (1) Loss of local details: When processing images, ViT segments them into coarse image patches, which easily ignores high-frequency local forgery traces (such as screen moiré patterns and printing particles).

[0008] (2) Cross-domain generalization problem: After the model is trained on known attack data, its performance drops significantly when faced with different acquisition devices, lighting environments or unknown attack types.

[0009] (3) High computational complexity: The standard ViT model has a large number of parameters and computational load, making it difficult to deploy directly on mobile devices or in scenarios with high real-time requirements.

[0010] Therefore, there is an urgent need for a lightweight face rendering attack detection scheme that can capture facial structural consistency using global attention mechanisms, enhance the perception of local subtle artifacts, and has good cross-domain generalization ability. Summary of the Invention

[0011] 4.1 Technical problems to be solved

[0012] The present invention aims to solve the technical problems of existing face presentation attack detection models, such as insufficient generalization ability in complex scenarios, low detection accuracy for highly realistic attacks (such as high-quality screen replay), and difficulty in taking into account both global semantics and local detailed features.

[0013] 4.2 Technical Solution

[0014] This invention provides a face rendering attack detection method based on ViT and local cue enhancement. This method achieves simultaneous perception of global facial structural consistency and local high-frequency artifacts by constructing a dual-path feature extraction network.

[0015] 4.3 Beneficial Effects

[0016] Compared with the prior art, the present invention has the following significant advantages:

[0017] (1) Balancing local and global aspects: By introducing parallel lightweight convolutional branches in the ViT architecture, the shortcomings of Transformer in capturing local subtle artifacts (such as print textures and screen noise) are effectively compensated, while retaining ViT's reasoning ability for global facial structure consistency.

[0018] (2) Strong generalization ability: The model is initialized with large-scale pre-trained weights and combined with local texture constraints, enabling the model to learn essential live features that are independent of specific acquisition devices, which significantly improves the detection accuracy in unknown scenarios such as cross-device and cross-lighting (reducing the ACER index).

[0019] (3) High efficiency and practicality: By adopting sparse attention or local window attention mechanisms and designing lightweight convolution modules, the complexity of the model is effectively controlled while ensuring high detection accuracy, making it easier to migrate and deploy to mobile or embedded devices. Attached Figure Description

[0020] Figure 1 is a flowchart of the face rendering attack detection method provided in an embodiment of the present invention. Detailed Implementation

[0021] 6.1 System Overall Architecture and Data Flow

[0022] System overall architecture:

[0023] The system consists of an overall system (100) and its six functional modules: (110) image preprocessing and block embedding module, (120) local high-frequency texture enhancement module (LCE), (130) global spatiotemporal Transformer encoding module, (140) adaptive feature calibration fusion module, (150) multi-task classification and regression decoding module and (160) adversarial robust training module.

[0024] Data flow: Raw face image I_raw→(110)→X_patch / F_img→(120) and (130) are processed in parallel→F_local / F_global→(140)→F_fused→(150)→y_cls,y_depth→(160) and the network parameters θ are updated.

[0025] Among them, (120)(130)(140)(150) are the main body of the model; (110) ensures the consistency of the input caliber; and (160) enables the model to have cross-domain robustness under adversarial perturbations.

[0026] 6.2 Data and Symbol Conventions

[0027] B: Batch Size;

[0028] H,W: Input image height and width (default 224×224);

[0029] C=3: Number of RGB channels;

[0030] P: Patch segment size (default 16);

[0031] N = H × W / P^2: Number of patches per image (default 196);

[0032] D: Transformer embedding dimension (default 768);

[0033] L: Number of Encoder stacking layers (default 12).

[0034] Original input tensor: I_raw∈R^(BCH*W).

[0035] 6.3 Detailed Description of Each Functional Module

[0036] (110) Image preprocessing and block embedding module

[0037] Unstructured images are converted into serialized vectors and location indexes are created to eliminate distribution differences caused by different shooting devices and lighting conditions.

[0038] Sub-units: Face alignment unit (111), Patch embedding layer (112), global classification token and location encoding (113).

[0039] Input: I_raw∈R^(BCHW)

[0040] deal with:

[0041] (111) Face alignment: Five key points, namely eyes, nose tip, and mouth corner, are detected by MTCNN, and affine transformation is performed. The face is then cropped and normalized by bilinear interpolation to I_raw∈R^(BC224224).

[0042] (112) Patch embedding: Linear projection is achieved using a two-dimensional convolutional layer with kernel size P and stride P.

[0043] X_patch=Conv2d (I_crop,kernel=P,stride=P)

[0044] The output dimension is B×D×(H / P)×(W / P), which becomes X_patch∈R^(BND) after Flattening and Transpose.

[0045] (113) Cls Token and Pos Embedding:

[0046] X_0=Concat ([x_class],X_patch)+E_pos

[0047] Where x_class∈R^(1*D) are learnable parameters, and E_pos∈R^((N+1)*D) are sinusoidal positional codes.

[0048] (120) Local High Frequency Texture Enhancement Module (LCE)

[0049] Used to explicitly extract high-frequency signals such as moiré patterns and print particles that are easily smoothed out by ViT.

[0050] Sub-units: multi-level depthwise separable convolutional unit (121), spatial attention gating unit (122), feature vectorization unit (123).

[0051] Input: I_crop∈R^(B3224224)

[0052] deal with:

[0053] (121) Multi-level depthwise separable convolutions: Construct a three-level convolution stack, each layer containing 3×3 depthwise separable convolutions and GELU activations:

[0054] F_l (i)=GELU (DWConv_3×3 (F_l (i-1))),i=1,2,3

[0055] Where F_l (0) = I_crop, the number of output channels are [64, 128, 256], and the spatial resolution is kept at 56×56.

[0056] (122) Spatial attention gating: Generate mask M_s to focus artifact regions:

[0057] M_s=σ(Conv_1×1 (F_l (3))))

[0058] F_local=M_s⊙F_l (3)

[0059] Where σ is the Sigmoid function.

[0060] (123) Eigenvectorization:

[0061] F_local_vec=Linear (GAP (F_local))∈R^(BD)

[0062] (130) Global Spatiotemporal Transformer Encoding Module

[0063] Used to model long-range dependencies and lighting consistency between different regions of a face.

[0064] Sub-units: Sparse window attention unit (131), shift window partitioning unit (132), feedforward network unit (133).

[0065] Input: X_0∈R^(B*(N+1)*D)

[0066] deal with:

[0067] (131) Sparse window attention: Divide the N patches into W×W windows (W=7), and calculate self-attention within the window. The complexity is reduced from O(N^2) to O(NW^2).

[0068] (132) Shift window mechanism: Alternate use of rule window and offset window to break boundary isolation and achieve near global modeling.

[0069] (133) Feedforward network: two fully connected layers, with the intermediate dimension extended to 4D.

[0070] (140) Adaptive Feature Calibration Fusion Module (AFC)

[0071] Used to inject local texture cues into global semantic features.

[0072] Sub-units: Channel attention recalibration unit (141), weighted fusion unit (142).

[0073] Input: Global feature F_global=X_L (0), local feature F_local_vec

[0074] deal with:

[0075] (141) Channel attention recalibration:

[0076] g=σ(FC_2 (ReLU (FC_1 (GAP (F_global)))))

[0077] (142) Weighted fusion:

[0078] F_fused=g⊙F_global+(1−g)⊙F_local_vec

[0079] (150) Multi-task classification and regression decoding module

[0080] Used to output liveness detection results and depth monitoring signals.

[0081] Sub-units: Classification head (151), Deep regression head (152).

[0082] Input: F_fused∈R^(BD)

[0083] deal with:

[0084] (151) Classification Header:

[0085] P_cls=Softmax (FC_2 (Dropout (GELU (FC_1 (F_fused)))))∈R^(B2)

[0086] (152) Deep regression head:

[0087] L_Depth=||D_map−D_gt||_1

[0088] (160) Adversarial Robust Training Module

[0089] Used to improve cross-domain generalization capabilities.

[0090] Sub-units: PGD attack unit (161), hybrid loss calculation unit (162).

[0091] deal with:

[0092] (161) PGD adversarial training:

[0093] max||δ||_∞≤ε L_CE (f (I_crop+δ),y)

[0094] (162) Mixed loss function:

[0095] L_total=λ_1 L_CE+λ_2 L_Depth+λ_3 L_Adv

[0096] 6.4 Training and Inference Process

[0097] Training process

[0098] S1 Data Loading: Read RGB images and labels, filter low-quality samples, and divide the training set and validation set in an 8:2 ratio.

[0099] S2 Preprocessing and Embedding: Image enhancement, alignment, and generation of patch sequences.

[0100] S3 Local Feature Extraction: The LCE module outputs local feature vectors.

[0101] S4 Global Encoding: The Swin-Transformer Encoder outputs global features.

[0102] S5 Feature Fusion and Decoding: Generates fused features and outputs classification probabilities and depth loss.

[0103] S6 Adversarial Training: Generate adversarial examples and backpropagate to update parameters.

[0104] S7 Persistence: Saves model weights and configuration files.

[0105] Reasoning process

[0106] I1 Environment Restoration: Load model checkpoints and freeze parameters.

[0107] I2 Image Preprocessing: Normalization, Resizing.

[0108] I3 Forward Calculation: Passes through each module in sequence and outputs the liveness probability.

[0109] I4 Decision Output: Threshold τ=0.5 determines whether the target is alive or an attacker.

Claims

1. A face rendering attack detection method based on ViT and local cue enhancement, characterized in that, Includes the following steps: The face image to be detected is acquired, and it is segmented into a sequence of image patches through a convolutional layer and linearly embedded to generate an initial feature tensor containing positional encoding. The initial feature tensor is processed through parallel processing pathways: the first pathway uses a local high-frequency texture enhancement module to extract high-frequency residual features of the input image through at least two levels of depthwise separable convolution and generate local feature vectors; the second pathway uses a global spatiotemporal Transformer encoding module to encode the initial feature tensor through a multi-head self-attention mechanism and output global semantic features. The adaptive feature calibration fusion module uses a channel attention mechanism to calculate the fusion weights of the local feature vector and the global semantic features, generating a weighted fusion feature. The weighted fusion features are input into the multi-task decoder, which outputs the face liveness classification probability and the deep regression residual, and updates the network parameters based on the adversarial training strategy.

2. The method according to claim 1, characterized in that, The local high-frequency texture enhancement module includes multi-level depth separable convolutional units and spatial attention gating units, used to extract high-frequency artifact features such as printed particles and screen moiré patterns.

3. The method according to claim 1, characterized in that, The global spatiotemporal Transformer encoding module employs sparse window attention and shifted window partitioning mechanisms to reduce computational complexity and model the structural consistency between different regions of the face.

4. The method according to claim 1, characterized in that, The adaptive feature calibration fusion module dynamically adjusts the fusion weights of the local feature vector and the global semantic features through a channel attention mechanism to enhance the feature representation capability of attack samples.

5. A face rendering attack detection device based on ViT and local cue enhancement, characterized in that, include: Image preprocessing and block embedding module, local high-frequency texture enhancement module, global spatiotemporal Transformer coding module, adaptive feature calibration fusion module, multi-task classification and regression decoding module; The apparatus is configured to perform the method according to any one of claims 1 to 4.

Images