A face recognition system based on a face mask detection model

Abstract

The present application relates to the technical field of face detection, and specifically discloses a face recognition system based on a face mask detection model, which is used to solve the problems of face recognition accuracy reduction, forgery risk, difficulty in real-time discrimination and difficulty in dynamic adaptation to complex attack and defense threat distribution in high-security sensitive scenarios such as bank service windows caused by face mask occlusion, forgery attack and multi-source environmental interference, and comprises a data acquisition and enhancement module, a feature extraction and multi-task neural network discrimination module, a forgery attack detection and risk blocking module, and a system deployment and model self-adaptive training module. Through the integration of multi-channel high-resolution visible light and infrared imaging, the enhancement of forgery attack samples based on a generative adversarial network, the adaptive difficult example sampling and the cooperative optimization of a multi-task deep discrimination network, the present application realizes efficient recognition and real-time risk blocking of real and fake mask faces, and improves the security and robustness of the face recognition system in bank scenarios.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a facial recognition system based on a facial mask detection model. Background Technology

[0002] Given the high security requirements of intelligent security and financial services, high-risk industries such as banks deploy automated identity verification systems based on facial recognition. However, due to the complex scenarios during business transactions, such as customers wearing masks, partial facial occlusion, changes in lighting, low-resolution monitoring images, changes in posture, and non-compliant behavior from some customers, traditional facial recognition models are highly susceptible to interference from masks, occlusion, and lighting conditions during feature extraction and discrimination. This leads to weakened representation capabilities of key areas, distorted overall feature space distribution, and reduced cross-domain generalization ability. This not only causes misidentification and false rejection but also leads to a severe imbalance between the number of occluded samples and real samples during peak business periods. The model struggles to effectively capture and adapt to the complexity of real-world data distribution during training and inference. In particular, when malicious attackers use masks or fabricate facial regions, traditional feature processing and detection mechanisms struggle to distinguish between "aggressive substitution inputs" and the actual facial features after removing masks. This creates security vulnerabilities that allow security policies to be bypassed, identities to be impersonated, and business operations to be maliciously tampered with, potentially resulting in customer asset losses and increased risk liabilities for banks. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a facial recognition system based on a face mask detection model. By integrating multi-channel high-resolution visible light and infrared imaging, forged attack sample enhancement based on generative adversarial networks, adaptive hard example sampling and multi-task deep discriminant network collaborative optimization, it achieves efficient identification and real-time risk blocking of real and forged masked faces, thereby improving the security and robustness of facial recognition systems in scenarios such as banks.

[0004] To achieve the above objectives, the present invention provides the following technical solution:

[0005] A facial recognition system based on a face mask detection model includes:

[0006] The data acquisition and enhancement module is used to acquire facial mask images of people in a banking service window environment through multi-channel high-definition cameras. It performs multi-scale preprocessing, sharpness filtering, and automatic facial key point localization on the acquired images, and performs pixel-level segmentation and annotation of mask wearing status and mask type. Simultaneously, it uses a generative adversarial network to generate various types of forged attack images, and dynamically expands the training set using an automatic hard example sampling mechanism. The feature extraction and multi-task neural network discrimination module adopts an improved YOLOv5 network backbone structure and integrates multi-scale feature pyramids and attention mechanisms. It integrates multiple task branches for mask detection, face detection, mask type recognition, and forged region segmentation. A forged recognition auxiliary path is added to the detection head layer to reconstruct the mask area. Face images are restored using an image inpainting network, and the restored images are compared with a standard database of unmasked faces for feature consistency verification. A high-dimensional distribution alignment algorithm is used to identify potential forgery samples. The forgery attack detection and risk blocking module uses frequency domain analysis and a local PatchGAN sub-network to perform multi-scale forgery detection on the input images. For identified aggressive features, it automatically blocks recognition requests and records detailed event logs. The system deployment and model adaptive training module supports automatic export of models in ONNX, OpenVINO, and TensorRT formats and multi-platform inference deployment during the training process. Based on real-time data distribution monitoring and the OHEM weighting mechanism for difficult examples, the training strategy and sample distribution are adjusted to achieve continuous adaptive iterative optimization of the model.

[0007] Preferably, the feature extraction and multi-task neural network discrimination module includes: based on the YOLOv5 backbone structure, integrating a spatial attention module and a channel attention module. The multi-scale features output by the backbone feature extraction layer are fused through FPN and PAN paths and then fed into the mask detection branch, face detection branch, mask type recognition branch, and auxiliary branch for forgery region detection, respectively. Each branch is equipped with an independent loss function and weight normalization parameters. During the training phase, the information collaboration and generalization ability between the multi-task branches are optimized through joint loss.

[0008] Preferably, the spatial attention module introduces channel recalibration and spatial adaptive weighting mechanisms into the feature maps of each layer output by the backbone network, automatically generates spatial weight masks using response statistics within the local receptive field, and combines global average pooling and max pooling features to redistribute spatial information through pointwise convolution.

[0009] Preferably, the forgery attack detection and risk blocking module specifically includes: for the input mask-removed image, firstly, multi-scale frequency domain analysis is performed through wavelet transform and discrete cosine transform to extract local texture features and edge noise features. Then, the analysis results are input into the PatchGAN type adversarial network for secondary forgery discrimination, and a forgery confidence threshold is used for screening. When the detected forgery confidence is greater than the preset threshold, the recognition request is automatically blocked, and an event tracing log is automatically generated and stored in the background, supporting data tracing of each attack behavior at the time, image feature and model output levels.

[0010] Preferably, in the forgery attack detection and risk blocking module, the discriminative structure of the PatchGAN-type adversarial sub-network consists of multiple cascaded convolutional layers. It uses a method of gradually expanding the receptive field to distinguish between real and fake probabilities for each small region of the input face image. In the discriminative output layer, the sigmoid activation function is used to normalize the local forgery score. During the training process, the model's ability to distinguish between deepfake, face-swapping, and image fusion attacks is enhanced by introducing alternating inputs of real samples and highly realistic forgery samples. This also ensures that the model outputs interpretable feature heatmaps to support business security auditing.

[0011] Preferably, the system deployment and model adaptive training module specifically includes: during the model training and inference deployment phase, an automated format conversion and hardware adaptation mechanism is integrated to support the automatic export of PyTorch-trained models to ONNX, OpenVINO, TensorRT, TensorFlow Lite, and PaddlePaddle formats according to business needs, and to automatically select the optimal inference backend according to the actual deployment environment, supporting dynamic loading and updating of the model; at the same time, combined with the real-time data collection distribution, the sampling probability of training samples is dynamically adjusted through adaptive weighting of sample difficulty and Online Hard Example Mining strategy.

[0012] Preferably, in the system deployment and model adaptive training module, the automated format conversion and hardware adaptation mechanism includes: after the model training is completed, automatically analyzing the chip architecture, memory bandwidth and computing power of the target deployment platform, prioritizing the selection of inference engines that support FP16 or INT8 acceleration for model quantization and compression, and realizing model weight export, parameter reordering and inference interface redirection through a fully automated script, while automatically loading and verifying model metadata during the deployment phase.

[0013] Preferably, the data acquisition and enhancement module specifically includes: for face images acquired in banking business window scenarios, using facial key point detection and pose estimation algorithms to automatically select frontal mask images that meet preset quality requirements, and using a forgery attack sample generation strategy based on generative adversarial networks to continuously enhance and expand the original training data in multiple dimensions such as mask wearing deformation, partial occlusion, image blurring, and image synthesis, while using an adaptive sampling mechanism to weight sample difficult-to-identify samples.

[0014] In the process of acquiring facial images for banking service window scenarios, the data acquisition and enhancement module first integrates multi-channel high-definition visible light and near-infrared cameras with a resolution of ≥1920×1080 pixels and a frame rate of ≥60 frames per second to simultaneously capture images under ambient illumination conditions of 10–1000 lux. The data is then processed in real-time via an embedded edge computing terminal, utilizing technologies based on HRNet and FAN (Face Alignment). The algorithm employs a facial landmark detection and 3D pose estimation algorithm (based on a network) to perform frame-by-frame facial feature localization and head pose angle (Pitch, Yaw, Roll) filtering within ±15°. It removes side-view, low-quality, and motion-blurred samples, retaining only high-quality, unobstructed, mask-wearing images as raw data. Then, based on a conditional generative adversarial network (such as cGAN or StyleGAN2), the algorithm automatically generates images with variations in mask type, irregular mask wearing (e.g., exposed nose, unilateral slippage), partial facial occlusion (occlusion area of ​​5%–20% of the total facial area), different lighting and skin tone conditions, image resolution downsampling (128×128 to 1024×1024), and Gaussian blur. The dataset includes multi-dimensional and multi-level forgery attack samples such as (standard deviation range of 2-5 pixels) and JPEG compression artifacts (compression rate of 20%-80%). All original and enhanced samples are automatically labeled according to mask wearing status, image quality level, and distribution of difficult examples. An adaptive sampling mechanism based on loss value and confidence is adopted to dynamically increase the sampling probability of difficult examples with an accuracy of less than 85% or a loss ranking in the top 20% to 2-4 times the base probability. This ensures that difficult examples can participate in the model optimization process at a high frequency in each training epoch, thereby forming a continuously dynamic, balanced, and realistic multimodal, multi-morphological high-quality face mask dataset that closely reflects the distribution of actual attack and defense threats. This provides rich and challenging multi-sample support for subsequent deep model training.

[0015] When acquiring facial images in banking service window scenarios, the data acquisition and enhancement module first integrates at least two CMOS cameras with a resolution of 1920×1080 pixels or higher, a frame rate of 60 frames per second or higher, and supporting dual-spectrum imaging of visible light and near-infrared wavelengths of 780 to 900 nanometers. These cameras acquire color and infrared facial images in parallel under ambient illumination of 10–1000 lux. The module then runs high-performance HRNet and FAN (Face Alignment) based on an embedded ARM architecture edge computing device. The Network algorithm extracts at least 68 facial key point coordinates in real time for each frame. It uses spatial pose regression to perform full detection and filtering of head pose angles (Pitch, Yaw, Roll). High-quality frontal masked face images are automatically selected if Pitch, Yaw, and Roll are all within ±15°, eye distance is at least 60 pixels, key facial areas are unobstructed, and image blur is below a specified threshold (e.g., BRISQUE score less than 35). Low-quality samples caused by motion blur, strong backlighting, side face occlusion, or camera offset are automatically removed. Then, the algorithm is used... Conditional Generative Adversarial Networks (StyleGAN2-cGAN structure) take "original image + attribute vector" as input. The attribute space incorporates mask type (e.g., medical, N95, cloth), wearing position (complete coverage, nose exposed, unilateral offset), local occlusion area (mouth, nose, chin, cheek, occlusion area 5%–20%), illumination color temperature (2700K, 4000K, 6500K), skin color type (Lab values ​​of 5 common races), resolution gradient (128×128, 256×256, 512×512, 1024×1024), and height. The system employs eight main axes, including sparse blur (standard deviation 2 / 3 / 4 / 5 pixels) and JPEG compression (20%–80%), each containing multiple sub-attributes. This enables batch amplification of single samples to generate fake mask and real face attack samples under arbitrary combinations of the aforementioned spatial conditions. Furthermore, a patch-level discriminator is introduced to perform authenticity detection on local fusion and mask edge and skin tone transition details in the generated results, further enhancing the difficulty of forgery detection and improving sample authenticity. After data augmentation, the system integrates image quality grading algorithms based on BRISQUE and NIQE to automatically grade all original and... Forged samples are scored and categorized in multiple dimensions. At the same time, an adaptive sampling scheduling mechanism for difficult examples is established in conjunction with the training process of the detection model. For difficult-to-identify samples with model accuracy below 85%, loss ranking in the top 20%, and confidence below 0.6, their sampling probability in training rounds is increased to 2 to 4 times the base probability in real time. Combined with the sample index table and difficult example category labels, the participation frequency of samples is dynamically adjusted to ensure that multiple types of difficult examples and high-complexity forged attack samples participate in high-frequency mixed participation in each round of training. This significantly improves the robustness, generalization and security protection capabilities of the model against real forgery attacks and defenses in bank window scenarios.

[0016] Preferably, the data acquisition and enhancement module includes: an integrated CMOS image sensor with a frame rate of ≥60 frames per second and a near-infrared illumination component with a wavelength range of 780 to 900 nanometers, used to simultaneously acquire face image data in the color visible light and infrared spectral bands within an ambient illumination range of ≤50 lux to ≥1000 lux, and to ensure real-time alignment of multi-source data acquisition based on a distributed timestamp calibration mechanism with an error of no more than 10 milliseconds; and to achieve consistency through cross-domain feature mapping with a spatial resolution of no less than 640×480 pixels and an error of less than 1 pixel. The alignment algorithm performs pixel-level registration between visible light images and infrared data, then inputs them into the model backbone. Simultaneously, it employs an enhancement strategy for detected forged attack samples with a confidence level greater than or equal to 0.85. This involves performing simulation operations on the samples, including interpolation mixing with an amplitude of 10% to 30%, style transfer perturbation with parameter disturbance range of ±20%, and local occlusion area of ​​5% to 15% of the total image area. The sampling weights are dynamically adjusted to achieve a ratio of attack samples to real samples of 1:3 to 1:1, thus realizing adaptive balanced sampling of the distribution of multi-source data, attack samples, and real samples.

[0017] Preferably, the data acquisition module integrates a dynamic difficult case generation and discrimination feedback linkage mechanism, the process of which includes:

[0018] Step 1, Loss Distribution Tracking and Hard Example Selection: After each training iteration, the individual loss of the batch samples is statistically analyzed, and the samples with the top 20% loss values ​​and confidence scores below 0.7 are automatically marked as hard example candidates.

[0019] Step 2, Dynamic Difficult Example Generation: For the selected difficult examples, adversarial perturbation based on PGD is combined with the principal component perturbation method in the feature space to generate fake samples with complex structure, blurred edges and local texture mutations. At the same time, the perturbation is jointly guided according to the category features of the original sample, the mask wearing area and the fake area.

[0020] Step 3, the feedback closed-loop mechanism: the difficult example is input into the multi-task discrimination network, and the confidence output of the forged attack branch is obtained in real time. If the confidence of the difficult example is >0.9 for 3 consecutive iterations, it is judged as a high-confidence forgery and the loss decrease is <5%. The system automatically identifies the difficult example as a stubborn high-risk attack sample.

[0021] Step 4, Multimodal Hard Example Enhancement and Sample Sampling Probability Improvement: For the stubborn high-risk attack samples obtained in Step 3, multi-channel sample enhancement is performed, which includes randomly adjusting the illumination between 50 and 800 lux, simulating local occlusion with an occlusion area between 10% and 20%, channel fusion perturbation with complementary visible and infrared spectral bands, and image style transfer with parameter perturbation of ±20%, which automatically increases the sampling probability of the sample to 2 to 3 times the base probability of the same type.

[0022] Step 5, Continuous Looping and Distribution Adaptive Balancing: Continuously monitor the performance of difficult examples, dynamically update sampling weights and perturbation parameters, and achieve adaptive alignment between the training set distribution and the actual attack and defense threat distribution.

[0023] For steps 2, 3, and 4 above, a further refined technical solution is as follows: In the dynamic difficult example generation stage, firstly, for the difficult example samples selected in step 1 with the top 20% loss ranking and confidence level below 0.7, an adversarial attack generation method based on projective gradient descent (PGD) is adopted. The input samples are iteratively perturbed within the pixel space, with the perturbation amplitude ε limited to [0.01, 0.05] within the L∞ norm range. The single-step step size α is 0.01, and each iteration has a maximum of 10 steps. The direction of each perturbation is constrained by the joint gradient guidance of the multi-task network for mask region segmentation and forgery detection branches, ensuring that the perturbation preferentially focuses on the mask and forgery-sensitive areas. Simultaneously, the difficult example samples are processed by ResNet-50 or the backbone YOLOv5. The feature extraction module is fed into the PCA dimensionality reduction module, where the high-dimensional principal components are perturbed. Specifically, while retaining the first 80% of the cumulative variance contributing principal components, each principal component value is perturbed with Gaussian noise with a standard deviation of 0.05. The perturbed features are then inversely transformed back to the pixel domain, generating complex, blurred-edge transition, and abnormally enhanced local texture adversarial samples. Furthermore, the original category information and pixel masks of the mask-wearing area are injected into the generated samples to guide the perturbation to focus on the key areas with the highest risk of forgery. In the discrimination feedback closed-loop mechanism of step 3, the difficult sample and its dynamically generated forgery adversarial antibodies are jointly input into the multi-task discrimination network. The system synchronously collects the forgery attack detection branch's forgery of the difficult sample in each training round. The confidence output curve shows that if the confidence of the difficult example is greater than 0.9 for three consecutive epochs and the corresponding loss value decreases by less than 5%, a persistent high-risk attack label is automatically triggered. The relevant feature distribution, confidence change trend, and loss change parameters are stored in the background database for subsequent difficult example library management and risk modeling. In step 4, during multimodal difficult example enhancement and sample sampling probability improvement, the system automatically calls a multi-channel sample enhancement strategy. For difficult examples determined to be high-risk, firstly, during physical imaging, the LED supplementary lighting system is adjusted to achieve random illumination perturbation in the illuminance range of 50–800 lux. Then, in the image preprocessing stage, 10%–20% of the face area is randomly selected for local occlusion simulation (occlusion objects can be superimposed). The system uses random shapes and occlusion materials, including black and white blocks and colored bands. It further inputs the original visible light and infrared spectral images into a multi-channel fusion module, employing cross-domain channel exchange and feature perturbation algorithms to introduce sub-pixel-level displacement within ±10 pixels in the spatial dimension. During image style transfer, it sets the color saturation, brightness, and contrast parameters to a range of ±20% from their original values, enhancing the diversity of lighting, style, and imaging conditions of the samples. Finally, based on the hard example library statistics, the system automatically increases the sampling probability of the training samples of the aforementioned high-risk samples to 2 to 3 times the base probability of similar samples, ensuring that they frequently participate in the model weight optimization process in subsequent training epochs. This achieves a continuous improvement in the robustness of the model and its ability to defend against forgery attacks through hard examples.

[0024] The technical advantages of the facial recognition system based on a face mask detection model of the present invention are as follows:

[0025] This invention constructs a dedicated data acquisition and enhancement module for banking service windows, integrating multi-channel high-definition camera acquisition, automatic facial key point and pose selection, multi-dimensional sample enhancement, and dynamic generation of forged samples. Combined with an improved YOLOv5 backbone structure, multi-scale feature fusion, spatial and channel attention mechanisms, multi-task branch discrimination, and a feature extraction and discrimination network for forgery attack assistance, it achieves accurate detection of facial regions under both masked and unmasked states, mask type identification, forged region segmentation, and joint protection against deepfake and image restoration attacks. Furthermore, the system incorporates frequency domain analysis and Pat... The chGAN-type multi-scale adversarial network, automatic confidence screening, and event tracing log mechanism enable accurate identification and automatic risk blocking of complex forgery attacks and high-risk difficult cases. At the same time, it innovatively integrates multimodal high-precision acquisition, dynamic difficult case mining and closed-loop feedback enhancement, automatic format conversion and hardware adaptation, and online sample distribution adaptive sampling technology. This achieves collaborative optimization and continuous adaptation throughout the entire process of model training, inference, deployment, and business security protection, improving the robustness, generalization ability, and attack and defense security level of the face recognition system in real-world banking scenarios against mask occlusion, forgery attacks, image mutations, and complex environmental interference. Attached Figure Description

[0026] Figure 1 This is a flowchart of a facial recognition system based on a face mask detection model according to the present invention.

[0027] Figure 2 The data acquisition and enhancement module of this invention captures facial images in a banking service window scenario;

[0028] Figure 3 This invention relates to a set of frontal facial images of a user wearing a green mask, simultaneously acquired through an integrated CMOS image sensor and near-infrared illumination component in a banking service window environment.

[0029] Figure 4 This invention proposes a system model for data label distribution analysis of facial mask detection and recognition tasks;

[0030] Figure 5 This is a graph showing the changes in various loss functions and performance indicators during the training and verification process of the system proposed in this invention.

[0031] Figure 6 This is a visualization of the multi-category confusion matrix of the system of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] like Figure 1 As shown, the present invention provides a facial recognition system based on a face mask detection model, comprising:

[0034] The data acquisition and enhancement module is used to acquire facial mask images of people in a banking service window environment through multi-channel high-definition cameras. It performs multi-scale preprocessing, sharpness filtering, and automatic facial key point localization on the acquired images, and performs pixel-level segmentation and annotation of mask wearing status and mask type. Simultaneously, it uses a generative adversarial network to generate various types of forged attack images, and dynamically expands the training set using an automatic hard example sampling mechanism. The feature extraction and multi-task neural network discrimination module adopts an improved YOLOv5 network backbone structure and integrates multi-scale feature pyramids and attention mechanisms. It integrates multiple task branches for mask detection, face detection, mask type recognition, and forged region segmentation. A forged recognition auxiliary path is added to the detection head layer to reconstruct the mask area. Face images are restored using an image inpainting network, and the restored images are compared with a standard database of unmasked faces for feature consistency verification. A high-dimensional distribution alignment algorithm is used to identify potential forgery samples. The forgery attack detection and risk blocking module uses frequency domain analysis and a local PatchGAN sub-network to perform multi-scale forgery detection on the input images. For identified aggressive features, it automatically blocks recognition requests and records detailed event logs. The system deployment and model adaptive training module supports automatic export of models in ONNX, OpenVINO, and TensorRT formats and multi-platform inference deployment during the training process. Based on real-time data distribution monitoring and the OHEM weighting mechanism for difficult examples, the training strategy and sample distribution are adjusted to achieve continuous adaptive iterative optimization of the model.

[0035] It should be noted that the data acquisition and enhancement module specifically includes: for face images collected in banking business window scenarios, using facial key point detection and pose estimation algorithms to automatically select front-facing mask images that meet preset quality requirements, and using a forgery attack sample generation strategy based on generative adversarial networks to continuously enhance and expand the original training data in multiple dimensions such as mask wearing deformation, partial occlusion, image blurring, and image synthesis, while using an adaptive sampling mechanism to weight sample difficult-to-identify samples.

[0036] In the process of acquiring facial images for banking service window scenarios, the data acquisition and enhancement module first integrates multi-channel high-definition visible light and near-infrared cameras with a resolution of ≥1920×1080 pixels and a frame rate of ≥60 frames per second to simultaneously capture images under ambient illumination conditions of 10–1000 lux. The data is then processed in real-time via an embedded edge computing terminal, utilizing technologies based on HRNet and FAN (Face Alignment). The algorithm employs a facial landmark detection and 3D pose estimation algorithm (based on a network) to perform frame-by-frame facial feature localization and head pose angle (Pitch, Yaw, Roll) filtering within ±15°. It removes side-view, low-quality, and motion-blurred samples, retaining only high-quality, unobstructed, mask-wearing images as raw data. Then, based on a conditional generative adversarial network (such as cGAN or StyleGAN2), the algorithm automatically generates images with variations in mask type, irregular mask wearing (e.g., exposed nose, unilateral slippage), partial facial occlusion (occlusion area of ​​5%–20% of the total facial area), different lighting and skin tone conditions, image resolution downsampling (128×128 to 1024×1024), and Gaussian blur. The dataset includes multi-dimensional and multi-level forgery attack samples such as (standard deviation range of 2-5 pixels) and JPEG compression artifacts (compression rate of 20%-80%). All original and enhanced samples are automatically labeled according to mask wearing status, image quality level, and distribution of difficult examples. An adaptive sampling mechanism based on loss value and confidence is adopted to dynamically increase the sampling probability of difficult examples with an accuracy of less than 85% or a loss ranking in the top 20% to 2-4 times the base probability. This ensures that difficult examples can participate in the model optimization process at a high frequency in each training epoch, thereby forming a continuously dynamic, balanced, and realistic multimodal, multi-morphological high-quality face mask dataset that closely reflects the distribution of actual attack and defense threats. This provides rich and challenging multi-sample support for subsequent deep model training.

[0037] When acquiring facial images in banking service window scenarios, the data acquisition and enhancement module first integrates at least two CMOS cameras with a resolution of 1920×1080 pixels or higher, a frame rate of 60 frames per second or higher, and supporting dual-spectrum imaging of visible light and near-infrared wavelengths of 780 to 900 nanometers. These cameras acquire color and infrared facial images in parallel under ambient illumination of 10–1000 lux. The module then runs high-performance HRNet and FAN (Face Alignment) based on an embedded ARM architecture edge computing device. The Network algorithm extracts at least 68 facial key point coordinates in real time for each frame. It uses spatial pose regression to perform full detection and filtering of head pose angles (Pitch, Yaw, Roll). High-quality frontal masked face images are automatically selected if Pitch, Yaw, and Roll are all within ±15°, eye distance is at least 60 pixels, key facial areas are unobstructed, and image blur is below a specified threshold (e.g., BRISQUE score less than 35). Low-quality samples caused by motion blur, strong backlighting, side face occlusion, or camera offset are automatically removed. Then, the algorithm is used... Conditional Generative Adversarial Networks (StyleGAN2-cGAN structure) take "original image + attribute vector" as input. The attribute space incorporates mask type (e.g., medical, N95, cloth), wearing position (complete coverage, nose exposed, unilateral offset), local occlusion area (mouth, nose, chin, cheek, occlusion area 5%–20%), illumination color temperature (2700K, 4000K, 6500K), skin color type (Lab values ​​of 5 common races), resolution gradient (128×128, 256×256, 512×512, 1024×1024), and height. The system employs eight main axes, including sparse blur (standard deviation 2 / 3 / 4 / 5 pixels) and JPEG compression (20%–80%), each containing multiple sub-attributes. This enables batch amplification of single samples to generate fake mask and real face attack samples under arbitrary combinations of the aforementioned spatial conditions. Furthermore, a patch-level discriminator is introduced to perform authenticity detection on local fusion and mask edge and skin tone transition details in the generated results, further enhancing the difficulty of forgery detection and improving sample authenticity. After data augmentation, the system integrates image quality grading algorithms based on BRISQUE and NIQE to automatically grade all original and... Forged samples are scored and categorized in multiple dimensions. At the same time, an adaptive sampling scheduling mechanism for difficult examples is established in conjunction with the training process of the detection model. For difficult-to-identify samples with model accuracy below 85%, loss ranking in the top 20%, and confidence below 0.6, their sampling probability in training rounds is increased to 2 to 4 times the base probability in real time. Combined with the sample index table and difficult example category labels, the participation frequency of samples is dynamically adjusted to ensure that multiple types of difficult examples and high-complexity forged attack samples participate in high-frequency mixed participation in each round of training. This significantly improves the robustness, generalization and security protection capabilities of the model against real forgery attacks and defenses in bank window scenarios.

[0038] For example, such as Figure 2 As shown, in a banking service window scenario, an integrated high-definition camera synchronously captured a frontal image of a user wearing a blue medical mask with a resolution of 1920×1080 pixels and a frame rate of 60 frames per second. The ambient illumination was 300 lux. After processing with HRNet and FAN algorithms, the system automatically detected 68 facial key points in the image and determined that the head pose angles Pitch=8°, Yaw=4°, and Roll=2°, all within the ±15° limit. The eye distance was 88 pixels, and the blur score was 22 points (below the threshold of 35 points). There was no facial occlusion or abnormal lighting. After quality screening, it was selected as a high-quality frontal mask image. Subsequently, the image was input into the StyleGAN2-cGAN conditional generative adversarial network, with the attribute vector set to "mask type = medical, wearing position = nose exposed, partial occlusion area = mouth and nose, occlusion area = 12%, illumination color temperature = 4000K, skin tone type = Asian, resolution = 512×512, Gaussian blur standard deviation = 3 pixels, JPEG compression rate = 40%". The system automatically generated 10 enhanced and fake samples with different combinations of the above features, including 1 fake mask image with the nose not completely occluded, 2 samples with approximately 13% occlusion on the jaw and cheeks respectively, 3 low-resolution (128×128, 256×256, 512×512) and Gaussian blurred samples, and 4 images generated under different compression rates and color temperatures. The enhanced samples were then processed by a patch-level discriminator to determine the authenticity of edges and transitions before being entered into the dataset. During the training phase, the system monitors the model's detection and recognition accuracy in real time. If a user's forged sample in a given batch is misjudged by the model, with an accuracy rate below 85%, ranking in the top 20% of losses, and a model confidence level below 0.6, the sampling probability of that sample is automatically increased from the basic 1 / 100 to 1 / 30, ensuring its high-frequency participation in subsequent training. This cycle repeats, automatically optimizing the participation frequency of difficult examples and forged attack samples in each training round, achieving adaptive alignment between the training set distribution and the actual attack and defense threat distribution. Ultimately, when faced with various mask-wearing anomalies, low-quality forged, or composite attack samples appearing in real business scenarios, the model can still accurately detect, identify, and block anomalies with high confidence, achieving a significant improvement in security level and attack and defense robustness. This technical solution effectively improves the practicality, generalization, and security protection capabilities of facial recognition systems in banking scenarios with mask occlusion and complex attack environments.

[0039] It should be noted that the data acquisition and enhancement module includes: an integrated CMOS image sensor with a frame rate of ≥60 frames per second and a near-infrared illumination component with a wavelength range of 780 to 900 nanometers, used to simultaneously acquire face image data in the color visible light and infrared spectral bands within an ambient illumination range of ≤50 lux to ≥1000 lux, and ensuring real-time alignment of multi-source data acquisition based on a distributed timestamp calibration mechanism with an error of no more than 10 milliseconds; and cross-domain feature mapping with a spatial resolution of no less than 640×480 pixels and an error of less than 1 pixel. The consistency alignment algorithm performs pixel-level registration of visible light images and infrared data and then inputs them into the model backbone. At the same time, it adopts an enhancement strategy for detected fake attack samples with a confidence level greater than or equal to 0.85. It performs simulation operations on the samples with an interpolation mixing amplitude of 10% to 30%, a style transfer perturbation with a parameter perturbation range of ±20%, and a local occlusion area of ​​5% to 15% of the total image area. It dynamically adjusts the sample sampling weight to achieve a ratio of attack samples to real samples of 1:3 to 1:1, so as to achieve adaptive balanced sampling of the distribution of multi-source data, attack samples and real samples.

[0040] It should be specifically noted that the data acquisition module integrates a dynamic difficult case generation and discrimination feedback linkage mechanism, the process of which includes:

[0041] Step 1, Loss Distribution Tracking and Hard Example Selection: After each training iteration, the individual loss of the batch samples is statistically analyzed, and the samples with the top 20% loss values ​​and confidence scores below 0.7 are automatically marked as hard example candidates.

[0042] Step 2, Dynamic Difficult Example Generation: For the selected difficult examples, adversarial perturbation based on PGD is combined with the principal component perturbation method in the feature space to generate fake samples with complex structure, blurred edges and local texture mutations. At the same time, the perturbation is jointly guided according to the category features of the original sample, the mask wearing area and the fake area.

[0043] Step 3, the feedback closed-loop mechanism: the difficult example is input into the multi-task discrimination network, and the confidence output of the forged attack branch is obtained in real time. If the confidence of the difficult example is >0.9 for 3 consecutive iterations, it is judged as a high-confidence forgery and the loss decrease is <5%. The system automatically identifies the difficult example as a stubborn high-risk attack sample.

[0044] Step 4, Multimodal Hard Example Enhancement and Sample Sampling Probability Improvement: For the stubborn high-risk attack samples obtained in Step 3, multi-channel sample enhancement is performed, which includes randomly adjusting the illumination between 50 and 800 lux, simulating local occlusion with an occlusion area between 10% and 20%, channel fusion perturbation with complementary visible and infrared spectral bands, and image style transfer with parameter perturbation of ±20%, which automatically increases the sampling probability of the sample to 2 to 3 times the base probability of the same type.

[0045] Step 5, Continuous Looping and Distribution Adaptive Balancing: Continuously monitor the performance of difficult examples, dynamically update sampling weights and perturbation parameters, and achieve adaptive alignment between the training set distribution and the actual attack and defense threat distribution.

[0046] For steps 2, 3, and 4 above, a further refined technical solution is as follows: In the dynamic difficult example generation stage, firstly, for the difficult example samples selected in step 1 with the top 20% loss ranking and confidence level below 0.7, an adversarial attack generation method based on projective gradient descent (PGD) is adopted. The input samples are iteratively perturbed within the pixel space, with the perturbation amplitude ε limited to [0.01, 0.05] within the L∞ norm range. The single-step step size α is 0.01, and each iteration has a maximum of 10 steps. The direction of each perturbation is constrained by the joint gradient guidance of the multi-task network for mask region segmentation and forgery detection branches, ensuring that the perturbation preferentially focuses on the mask and forgery-sensitive areas. Simultaneously, the difficult example samples are processed by ResNet-50 or the backbone YOLOv5. The feature extraction module is fed into the PCA dimensionality reduction module, where the high-dimensional principal components are perturbed. Specifically, while retaining the first 80% of the cumulative variance contributing principal components, each principal component value is perturbed with Gaussian noise with a standard deviation of 0.05. The perturbed features are then inversely transformed back to the pixel domain, generating complex, blurred-edge transition, and abnormally enhanced local texture adversarial samples. Furthermore, the original category information and pixel masks of the mask-wearing area are injected into the generated samples to guide the perturbation to focus on the key areas with the highest risk of forgery. In the discrimination feedback closed-loop mechanism of step 3, the difficult sample and its dynamically generated forgery adversarial antibodies are jointly input into the multi-task discrimination network. The system synchronously collects the forgery attack detection branch's forgery of the difficult sample in each training round. The confidence output curve shows that if the confidence of the difficult example is greater than 0.9 for three consecutive epochs and the corresponding loss value decreases by less than 5%, a persistent high-risk attack label is automatically triggered. The relevant feature distribution, confidence change trend, and loss change parameters are stored in the background database for subsequent difficult example library management and risk modeling. In step 4, during multimodal difficult example enhancement and sample sampling probability improvement, the system automatically calls a multi-channel sample enhancement strategy. For difficult examples determined to be high-risk, firstly, during physical imaging, the LED supplementary lighting system is adjusted to achieve random illumination perturbation in the illuminance range of 50–800 lux. Then, in the image preprocessing stage, 10%–20% of the face area is randomly selected for local occlusion simulation (occlusion objects can be superimposed). The system uses random shapes and occlusion materials, including black and white blocks and colored bands. It further inputs the original visible light and infrared spectral images into a multi-channel fusion module, employing cross-domain channel exchange and feature perturbation algorithms to introduce sub-pixel-level displacement within ±10 pixels in the spatial dimension. During image style transfer, it sets the color saturation, brightness, and contrast parameters to a range of ±20% from their original values, enhancing the diversity of lighting, style, and imaging conditions of the samples. Finally, based on the hard example library statistics, the system automatically increases the sampling probability of the training samples of the aforementioned high-risk samples to 2 to 3 times the base probability of similar samples, ensuring that they frequently participate in the model weight optimization process in subsequent training epochs. This achieves a continuous improvement in the robustness of the model and its ability to defend against forgery attacks through hard examples.

[0047] like Figure 3 As shown, in a banking window environment, a set of frontal facial images of users wearing green masks were simultaneously acquired using an integrated CMOS image sensor and near-infrared illumination components. The resolution was 1920×1080 pixels, and the frame rate was 60fps. Visible light and infrared channel images were acquired simultaneously under ambient illumination of 120 lux and 700 lux, respectively, with an alignment error of less than 8 milliseconds after timestamp calibration. The system used HRNet and FAN algorithms to extract 68 facial key points, determining the head pitch to be 6°, yaw to be 3°, roll to be 1°, eye distance to be 86 pixels, and a BRISQUE score of 20. The images had low blur and no occlusion, and were selected for inclusion in a high-quality raw library. When training a multi-task model for facial recognition and forgery detection, a sample in a certain batch was misclassified by the model, resulting in a loss value of 0.38, placing it in the top 15% of the current batch's loss ranking, and the model confidence was only 0.59 (below 0.7). The system automatically marked it as a difficult example. Subsequently, the dynamic hard example generation stage begins. The PGD method is applied to this hard example, with an L∞ perturbation amplitude ε=0.03 and a step size α=0.01, executing 8 steps. Perturbations are applied to pixels concentrated in the nose and chin areas of the mask. The perturbation direction for each step is automatically determined based on the joint gradient of the mask region segmentation and forgery detection branches. Simultaneously, the sample is fed into a ResNet-50 feature extractor and undergoes PCA dimensionality reduction, retaining the first 80% of the cumulative variance principal components. Gaussian noise with a standard deviation of 0.05 is added to each principal component, and inverse transformation is used to restore it to a forgery enhancement sample. Finally, a set of high-risk adversarial samples with blurred edges and abrupt texture changes at the interface between the mask and the face is generated.

[0048] The original hard case and its forgery were jointly input into the discrimination network. During three consecutive epochs of training, the forgery attack detection branch consistently showed a confidence level greater than 0.92 for this hard case, with a loss reduction of less than 4%, classifying it as a persistent high-risk attack sample. The system automatically stored its feature distribution and discrimination trend. For this hard case, in subsequent sampling and enhancement stages, the system adjusted LED illumination to achieve random lighting variations (measured at 65–320 lux). A randomly shaped gray occlusion band with an occlusion area of ​​14% was superimposed on the mouth and nose region of the image. The visible light and infrared images of this sample were then fed into a multi-channel fusion module. After pixel-level registration, a spatial ±6 pixel local channel perturbation was added, along with ±18% random transformations of brightness, saturation, and contrast, creating diverse enhanced samples. The system increased the sampling probability of this type of hard case from 1 / 60 to 1 / 25, allowing it to frequently participate in model parameter optimization in subsequent training epochs.

[0049] Technical Results: Through the above process, the model achieves focused training, structural diversification, and continuous high-frequency optimization for high-risk face samples that are difficult to identify and vulnerable to forgery attacks. This not only improves the robustness of facial recognition under complex lighting conditions at bank windows and malicious attacks, but also significantly enhances its defense and adaptive capabilities against real business threats such as partial forgery, occlusion, and adversarial attacks. This enables the system to continuously, dynamically, and accurately adapt to security attack and defense challenges in actual deployment, ensuring the security of facial identity verification in financial transactions.

[0050] It should be noted that the feature extraction and multi-task neural network discrimination module includes: based on the YOLOv5 backbone structure, it integrates spatial attention module and channel attention module. The multi-scale features output by the backbone feature extraction layer are fused through FPN and PAN paths and then fed into the mask detection branch, face detection branch, mask type recognition branch, and auxiliary branch for fake region detection, respectively. Each branch is equipped with an independent loss function and weight normalization parameters. During the training phase, the information collaboration and generalization ability between multi-task branches are optimized through joint loss.

[0051] It should be specifically explained that the spatial attention module introduces channel recalibration and spatial adaptive weighting mechanisms into the feature maps of each layer output by the backbone network. It automatically generates spatial weight masks by utilizing the response statistics within the local receptive field, and combines global average pooling and max pooling features to redistribute spatial information through pointwise convolution.

[0052] By integrating spatial attention and channel attention mechanisms into the feature maps of each layer of the YOLOv5 backbone feature extraction network, the system can automatically focus on key information related to the discrimination task, such as face and mask regions, dynamically suppress background, occlusion, and irrelevant noise features, and generate adaptive spatial weight masks through the fusion of local receptive field statistics and global pooling. This enables feature representation to achieve information enhancement and redundancy removal at different scales and in different regions, significantly improving the robustness and discrimination accuracy of multi-task branches such as mask detection, face detection, mask type recognition, and forgery region detection in complex environments. It also strengthens the model's ability to perceive local anomalies, forgery details, and subtle feature changes, effectively improving the system's generalization performance and practical application security in multi-source, highly variable banking scenarios.

[0053] It should be noted that the forgery attack detection and risk blocking module specifically includes: for the input mask-removed image, firstly, multi-scale frequency domain analysis is performed through wavelet transform and discrete cosine transform to extract local texture features and edge noise features. Then, the analysis results are input into the PatchGAN type adversarial network for secondary forgery discrimination, and a forgery confidence threshold is used for filtering. When the detected forgery confidence is greater than the preset threshold, the recognition request is automatically blocked, and an event tracing log is automatically generated and stored in the background, supporting data tracing of each attack behavior at the time, image feature and model output level.

[0054] It should be specifically noted that in the forgery attack detection and risk blocking module, the discriminative structure of the PatchGAN-type adversarial sub-network consists of multiple cascaded convolutional layers. It uses a method of gradually expanding the receptive field to distinguish between real and fake probabilities for each small region of the input face image. In the discriminative output layer, the sigmoid activation function is used to normalize the local forgery score. During the training process, the model's ability to distinguish between deepfake, face-swapping, and image fusion attacks is enhanced by introducing alternating inputs of real samples and highly realistic forged samples. This ensures that the model outputs interpretable feature heatmaps to support business security auditing.

[0055] Multi-scale frequency domain analysis of unmasked face images using wavelet transform and discrete cosine transform can fully capture fine-grained local texture anomalies and edge forgery traces, providing rich anomaly feature representations for subsequent discrimination. Inputting the frequency domain analysis results into a PatchGAN-type adversarial discriminant subnetwork composed of multiple convolutional layers enables fine-grained judgment of the real and forged probabilities of various local regions of the image. Sigmoid normalization significantly improves the sensitivity and robustness of forgery detection. Alternating inputs of real and highly realistic forged samples during training give the model strong discriminative capabilities against deepfakes, face-swapping, and fusion attacks, greatly reducing the risk of being bypassed by highly complex forged images. Simultaneously, the PatchGAN discriminator outputs interpretable feature heatmaps, assisting security personnel in intuitively locating attack areas and analyzing the model's decision criteria. Combined with automatic event tracing logs, it supports tracing the source of each attack and security auditing, enabling refined control and traceable protection of face recognition security in high-risk scenarios such as banking service windows.

[0056] It should be noted that the system deployment and model adaptive training module specifically includes: during the model training and inference deployment phases, it integrates automated format conversion and hardware adaptation mechanisms, supporting the automatic export of PyTorch-trained models to ONNX, OpenVINO, TensorRT, TensorFlow Lite, and PaddlePaddle formats according to business needs, and automatically selecting the optimal inference backend based on the actual deployment environment, supporting dynamic model loading and updates; at the same time, it dynamically adjusts the sampling probability of training samples through adaptive weighting of sample difficulty and Online Hard Example Mining strategies, combined with real-time data collection distribution.

[0057] It should be specifically noted that in the system deployment and model adaptive training module, the automated format conversion and hardware adaptation mechanism includes: after the model training is completed, automatically analyzing the chip architecture, memory bandwidth and computing power of the target deployment platform, prioritizing the selection of inference engines that support FP16 or INT8 acceleration for model quantization and compression, and realizing model weight export, parameter reordering and inference interface redirection through a fully automated script. At the same time, the model metadata is automatically loaded and verified during the deployment phase.

[0058] By integrating automated format conversion and hardware adaptation mechanisms into the system deployment and model adaptive training modules, after model training, the system can automatically analyze the target platform's chip architecture, memory bandwidth, and computing power based on different business needs and actual deployment environments. It prioritizes inference engines supporting low-precision acceleration features such as FP16 or INT8, achieving efficient quantization compression and automatic export of models in multiple formats (such as ONNX, OpenVINO, TensorRT, TensorFlow Lite, and PaddlePaddle), effectively improving model inference speed and resource utilization. Through fully automated scripts, it achieves efficient export of model weights, parameter reordering, and inference interface redirection, significantly simplifying cross-platform deployment and reducing manual adaptation and maintenance costs. During deployment, it automatically loads and verifies model metadata, ensuring the correctness and consistency of the model across multiple platforms. Furthermore, combining real-time data distribution, it employs adaptive weighting based on sample difficulty and Online HardExample. The mining strategy dynamically adjusts the sampling probability of training samples, which can continuously optimize the model's adaptability to new scenarios, new threats, and high-risk difficult examples, enabling continuous iteration and online evolution of model performance. This significantly improves the overall system's deployment flexibility, inference efficiency, robustness, and practicality in security attacks and defenses under complex scenarios.

[0059] like Figure 4The diagram shows the data label distribution analysis of the system proposed in this invention for the face mask detection and recognition task. Its core function is to visualize and statistically analyze the label attributes in the training dataset, specifically illustrating the distribution of the number of instances of various targets (e.g., "mask" indicating wearing a mask, "face" indicating not wearing a mask), the spatial location (x, y coordinates) of the bounding boxes in the image, the size (width and height) distribution of the bounding boxes, and the clustering hotspots of all bounding boxes in the image. Such charts are commonly used for dataset quality checks and data distribution analysis before training object detection models (such as YOLOv5), serving as important data evidence for improving the model's efficient learning and generalization capabilities. The top-left bar chart shows the instance distribution of the "mask" and "face" labels, helping to analyze whether the categories are balanced. The top-right hotspot diagram, marked in red, shows the overlapping clustering areas of all bounding boxes in the image space, reflecting the location hotspots of the main targets in the image (commonly seen when faces are concentrated in the center of the image). The scatter plot in the lower left corner shows the distribution density of the center points (x, y coordinates) of all labeled bounding boxes in the standardized image space, verifying whether the spatial distribution of targets in the image is reasonable and uniform. The scatter plot in the lower right corner shows the width and height distribution of the target boxes, assessing the diversity of target sizes, which directly affects anchor generation and the model's ability to detect small and large targets. This plot is one of the fundamental bases for the design and optimization of the face mask detection system. Through these statistical plots, it is possible to discover whether there are problems such as class imbalance or abnormal target distribution in the dataset, thereby guiding subsequent technical decisions in core aspects such as data augmentation, sampling equalization, and model parameter adjustment. If this set of plots reveals that the mask / face categories are extremely imbalanced or that targets are overly concentrated in a certain area, targeted technical means such as adaptive sampling, fake sample generation, or spatial distribution augmentation can be adopted to significantly improve the model's generalization and robustness to real business scenarios.

[0060] like Figure 5The figure shows the curves of various loss functions and performance indicators during the training and validation process of the system proposed in this invention. By arranging them horizontally, it shows the changing trends of core indicators such as box_loss (boundary box regression loss), obj_loss (target confidence loss), cls_loss (class loss), precision, recall, mAP_0.5, and mAP_0.5:0.95 (mean precision) in the model training and validation stages over the number of epochs. It intuitively reflects the convergence process of model training, the dynamic trajectory of generalization ability, and the improvement of detection performance. It can comprehensively evaluate the effectiveness and stability of the innovative technologies proposed in this invention, such as the multi-task neural network structure, feature enhancement, and multimodal fusion, for practical face mask recognition and forgery detection tasks. It is an indispensable performance evaluation basis in the engineering application and continuous optimization of facial recognition systems, and provides strong quantitative support for the technical effect and practical value of the patent innovation points.

[0061] like Figure 6 The diagram shown is a visualization of the multi-class confusion matrix of the system of this invention. It is used to intuitively present the accuracy and confusion of the system's classification prediction for categories such as "mask (wearing a mask)," "face (not wearing a mask)," and background false positives (FP) in actual mask wearing detection and face recognition tasks. The horizontal axis represents the true label (True), and the vertical axis represents the model's prediction result (Predicted). The value in each cell represents the prediction proportion (or accuracy) of the model in that category, and the shade of blue reflects the magnitude of the value. For example, 0.77 in the upper left corner means that 77% of the samples correctly identified as "mask" are true "mask" samples; 0.97 in the upper right corner means that 97% of the samples that are true "background FP" are incorrectly predicted as "mask" samples; and 0.03 in the middle right corner means that 3% of the true "face" samples are incorrectly identified as "background FP" samples. In addition, the "background FN" row at the bottom shows the cases where the system failed to correctly identify (missed detection) the categories of "mask" or "face." This confusion matrix reveals in detail the model's classification performance, common confusion points, and error distribution in multi-class detection scenarios. It provides important guidance for the continuous optimization of key algorithms, precise adjustment of sampling weights, and enhancement of anti-counterfeiting robustness. It is also an essential technical performance support for the present invention to achieve highly reliable mask recognition and face verification in high-security scenarios such as bank windows. It provides clear, quantitative, and practically applicable technical evidence for the patent's innovation.

[0062] In summary, the system proposed in this invention achieves high-precision, multi-task, and robust end-to-end recognition of facial mask wearing status, type, and deep forgery behavior in high-security scenarios such as banks by integrating innovative mechanisms such as simultaneous acquisition of multimodal high-resolution visible light and near-infrared images, precise localization and pose screening of facial key points, generation of multi-dimensional masks and forgery samples, multi-task spatial-channel attention fusion deep network discrimination, dynamic hard example mining and enhancement against multi-source high-risk forgery attacks, frequency-spatial domain joint discrimination based on PatchGAN, and full-process adaptive sampling training. This greatly improves the model's discrimination ability and generalization adaptability in complex environments, various occlusions, and high-imitation attacks, effectively ensuring the security, practicality, and intelligence level of facial recognition services. It provides system-level technical support and continuous self-learning optimization capabilities for actual bank counter identity verification and anti-counterfeiting protection.

[0063] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0064] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A facial recognition system based on a face mask detection model, characterized in that, include: The data acquisition and enhancement module is used to acquire facial mask images in the banking service window environment through multi-channel high-definition cameras. It performs multi-scale preprocessing, sharpness filtering, and automatic facial key point localization on the acquired images, and performs pixel-level segmentation and annotation of mask wearing status and mask type. Simultaneously, it uses a generative adversarial network to generate various types of forgery attack images, and dynamically expands the training using an automatic hard example sampling mechanism. The feature extraction and multi-task neural network discrimination module adopts an improved YOLOv5 network backbone structure and integrates multi-scale feature pyramids and attention mechanisms. It integrates multiple task branches for mask detection, face detection, mask type recognition, and forgery region segmentation. A forgery recognition auxiliary path is added to the detection head layer to analyze the facial images in the mask region. The system employs various mechanisms, including: feature restoration via image inpainting networks, feature consistency verification between the restored images and a standard database of unmasked faces, and high-dimensional distribution alignment algorithms to identify potential forgeries; a forgery attack detection and risk blocking module using frequency domain analysis and local PatchGAN sub-networks for multi-scale forgery detection of input images, automatically blocking recognition requests and recording detailed event logs for identified malicious features; and a system deployment and adaptive model training module supporting automatic export of models in ONNX, OpenVINO, and TensorRT formats and multi-platform inference deployment during training. It adjusts training strategies and sample distribution based on real-time data distribution monitoring and the OHEM weighting mechanism for difficult examples, enabling continuous adaptive iterative optimization of the model.

2. The facial recognition system based on a face mask detection model according to claim 1, characterized in that, The feature extraction and multi-task neural network discrimination module includes: based on the YOLOv5 backbone structure, it integrates spatial attention module and channel attention module. The multi-scale features output by the backbone feature extraction layer are fused by FPN and PAN paths and then fed into the mask detection branch, face detection branch, mask type recognition branch and auxiliary branch for fake region detection. Each branch is equipped with an independent loss function and weight normalization parameters. During the training phase, the information collaboration and generalization ability between multi-task branches are optimized by joint loss.

3. A facial recognition system based on a face mask detection model according to claim 2, characterized in that, The spatial attention module introduces channel recalibration and spatial adaptive weighting mechanisms into the feature maps of each layer output by the backbone network. It automatically generates spatial weight masks by utilizing the response statistics within the local receptive field and combines global average pooling and max pooling features to redistribute spatial information through pointwise convolution.

4. The facial recognition system based on a face mask detection model according to claim 1, characterized in that, The forgery attack detection and risk blocking module specifically includes: for the input mask-removed image, firstly, multi-scale frequency domain analysis is performed through wavelet transform and discrete cosine transform to extract local texture features and edge noise features. Then, the analysis results are input into the PatchGAN type adversarial network for secondary forgery discrimination, and a forgery confidence threshold is used for filtering. When the detected forgery confidence is greater than the preset threshold, the recognition request is automatically blocked, and an event tracing log is automatically generated and stored in the background, supporting data tracing of each attack behavior at the time, image feature and model output levels.

5. A facial recognition system based on a face mask detection model according to claim 4, characterized in that, In the forgery attack detection and risk blocking module, the discriminative structure of the PatchGAN-type adversarial sub-network consists of multiple cascaded convolutional layers. It uses a progressively expanding receptive field to distinguish between real and fake probabilities for each small region of the input face image. In the discriminative output layer, the sigmoid activation function is used to normalize the local forgery score. During training, the model's ability to distinguish between deepfake, face-swapping, and image fusion attacks is enhanced by introducing alternating inputs of real samples and highly realistic forgery samples. This ensures that the model outputs interpretable feature heatmaps to support business security auditing.

6. A facial recognition system based on a face mask detection model according to claim 1, characterized in that, The system deployment and model adaptive training module specifically includes: during the model training and inference deployment phases, it integrates automated format conversion and hardware adaptation mechanisms, supporting the automatic export of PyTorch-trained models to ONNX, OpenVINO, TensorRT, TensorFlow Lite, and PaddlePaddle formats according to business needs, and automatically selecting the optimal inference backend based on the actual deployment environment, supporting dynamic model loading and updates; at the same time, it dynamically adjusts the sampling probability of training samples through adaptive weighting of sample difficulty and Online Hard Example Mining strategies, combined with real-time data collection distribution.

7. A facial recognition system based on a face mask detection model according to claim 6, characterized in that, In the system deployment and model adaptive training module, the automated format conversion and hardware adaptation mechanism includes: after the model training is completed, automatically analyzing the chip architecture, memory bandwidth and computing power of the target deployment platform, selecting an inference engine that supports FP16 or INT8 acceleration for model quantization and compression, and realizing model weight export, parameter reordering and inference interface redirection through a fully automated script. At the same time, the model metadata is automatically loaded and verified during the deployment phase.

8. A facial recognition system based on a face mask detection model according to claim 1, characterized in that, The data acquisition and enhancement module specifically includes: for face images collected in banking business window scenarios, using facial key point detection and pose estimation algorithms to automatically select front-facing mask images that meet preset quality requirements, and using a forgery attack sample generation strategy based on generative adversarial networks to continuously enhance and expand the original training data in multiple dimensions such as mask wearing deformation, partial occlusion, image blurring, and image synthesis. At the same time, it uses an adaptive sampling mechanism to weighted sample for difficult-to-identify samples.

9. A facial recognition system based on a face mask detection model according to claim 8, characterized in that, The data acquisition and enhancement module includes: an integrated CMOS image sensor with a frame rate of ≥60 frames per second and a near-infrared illumination component with a wavelength range of 780 to 900 nanometers, used to simultaneously acquire face image data in the color visible light and infrared spectral bands within an ambient illumination range of ≤50 lux to ≥1000 lux, and ensuring real-time alignment of multi-source data acquisition based on a distributed timestamp calibration mechanism with an error of no more than 10 milliseconds; and cross-domain feature mapping with a spatial resolution of no less than 640×480 pixels and consistency alignment with an error of less than 1 pixel. The algorithm performs pixel-level registration of visible light images and infrared data and inputs them into the model backbone. It also employs an enhancement strategy for detected forged attack samples with a confidence level greater than or equal to 0.

85. The algorithm performs simulation operations on the samples, including interpolation mixing with an amplitude of 10% to 30%, style transfer perturbation with parameter perturbation range of ±20%, and local occlusion area of ​​5% to 15% of the total image area. The algorithm dynamically adjusts the sample sampling weights to achieve a ratio of attack samples to real samples of 1:3 to 1:1, thereby achieving adaptive balanced sampling of the distribution of multi-source data, attack samples, and real samples.

10. A facial recognition system based on a face mask detection model according to claim 8, characterized in that, The data acquisition module integrates a dynamic difficult case generation and discrimination feedback linkage mechanism, the process of which includes: Step 1, Loss Distribution Tracking and Hard Example Selection: After each training iteration, the individual loss of the batch samples is statistically analyzed, and the samples with the top 20% loss values ​​and confidence scores below 0.7 are automatically marked as hard example candidates. Step 2, Dynamic Difficult Example Generation: For the selected difficult examples, adversarial perturbation based on PGD is combined with the principal component perturbation method in the feature space to generate fake samples with complex structure, blurred edges and local texture mutations. At the same time, the perturbation is jointly guided according to the category features of the original sample, the mask wearing area and the fake area. Step 3, the feedback closed-loop mechanism: the difficult example is input into the multi-task discrimination network, and the confidence output of the forged attack branch is obtained in real time. If the confidence of the difficult example is >0.9 for 3 consecutive iterations, it is judged as a high-confidence forgery and the loss decrease is <5%. The system automatically identifies the difficult example as a stubborn high-risk attack sample. Step 4, Multimodal Hard Example Enhancement and Sample Sampling Probability Improvement: For the stubborn high-risk attack samples obtained in Step 3, multi-channel sample enhancement is performed, which includes randomly adjusting the illumination between 50 and 800 lux, simulating local occlusion with an occlusion area between 10% and 20%, channel fusion perturbation with complementary visible and infrared spectral bands, and image style transfer with parameter perturbation of ±20%, which automatically increases the sampling probability of the sample to 2 to 3 times the base probability of the same type. Step 5, Continuous Looping and Distribution Adaptive Balancing: Continuously monitor the performance of difficult examples, dynamically update sampling weights and perturbation parameters, and achieve adaptive alignment between the training set distribution and the actual attack and defense threat distribution.

Images