Privacy protection face recognition method based on five-sense physical mask and dynamic confrontation

Abstract

The application discloses a privacy protection face recognition method based on five sense organs physical mask and dynamic confrontation and belongs to the field of computer vision and information security.The method comprises the following steps: five key points of a face are extracted to generate a five sense organs physical mask, residual information is obtained through reversible frequency domain transformation and a generator, and the protected residual error is combined with the mask to be sent into a recognizer to complete face verification; the five sense organs mask shields the non-core privacy area of the face, a pixel-level attacker is introduced to construct an active confrontation defense closed loop, and the technical defects that existing privacy face recognition technology is easy to leak face visual clues, passive protection is easy to be cracked by reconstruction attack, and recognition accuracy and training efficiency are difficult to be considered are solved, the face data anti-reconstruction attack capability is greatly improved under the premise of ensuring the face recognition accuracy, the model training algorithm cost is reduced, and the method can be widely applied to various face verification scenes such as access control, payment and public security.

Description

Technical Field

[0001] This invention relates to a privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques, belonging to the fields of computer vision and information security. Background Technology

[0002] With the widespread application of facial recognition technology in scenarios such as access control, payment, identity verification, public safety, and mobile terminals, significant privacy risks exist in the entire process of facial image acquisition, transmission, storage, and model training. If unauthorized attackers obtain the visual representation of a facial image or model, they can reconstruct a user's complete facial appearance, deduce the user's identity attributes, and even forge facial samples to impersonate the user. How to reduce the visualization and reconstructability of facial representations while ensuring the accuracy of facial recognition has become a core challenge that urgently needs to be addressed in the field of privacy-preserving facial recognition.

[0003] Existing privacy-preserving facial recognition technologies are mainly categorized into image encryption, feature encryption, pixel perturbation, partial facial occlusion, feature compression, and generative facial anonymization. While these methods can reduce the risk of direct leakage of original facial images to some extent, they generally suffer from several inherent flaws: First, the facial representation after encryption and perturbation processing cannot simultaneously achieve both facial recognition accuracy and model training efficiency; there is a clear trade-off between the two performance metrics. Second, most existing solutions only generally reduce the overall visual information of the face, while still fully preserving non-facial visual information such as background, skin regions, facial contours, and hair texture. This type of information can be extracted by attackers and used for facial reconstruction attacks. Third, existing protection methods are mostly passive privacy masking, failing to incorporate attack networks that simulate facial reconstruction capabilities into the model training loop. Consequently, the protection effect is significantly reduced when facing learning-based pixel reconstruction attacks such as U-Net.

[0004] MinusFace is a face recognition scheme that relies on generative models to achieve privacy protection. Its core idea is to process face images through reversible frequency domain transformation, use a generator to output a reconstructed face representation, and complete face recognition based on the residual between the original and reconstructed frequency domain representations. While this scheme achieves basic privacy protection by using residual representations that "have recognition value but lack intuitive visual information about the face," it still has two key drawbacks: First, this scheme uses complete face images and full-image residuals as the model's learning objects, consuming a large amount of computational power in non-identity core areas such as the background and skin. Furthermore, visual cues remaining in these non-core areas can still be exploited by reconstruction attackers. Second, the training process only co-optimizes the generator and recognizer networks, without introducing pixel-level attackers to build an adversarial training loop. It cannot actively simulate or suppress the attacker's face reconstruction capabilities during the training phase. When attackers use U-Net-like image reconstruction models to crack the residual representations, the reliability of the original scheme's privacy protection will significantly decrease. Summary of the Invention

[0005] To address the current limitations of facial recognition technology in simultaneously achieving high accuracy, efficient training, and robust privacy protection, this invention provides a privacy-preserving facial recognition method based on physical masking of facial features and dynamic adversarial techniques. The technical solution is as follows: Step 1: Obtain the face image to be identified, extract five facial key points from the face image, determine the core region of interest of the face based on the five facial key points, and generate the corresponding physical mask of the facial features; Step 2: Input the face core region of interest image into a pre-trained reversible transform encoder to obtain a high-dimensional frequency domain representation, and then input the high-dimensional frequency domain representation into a pre-trained generator to obtain a regenerated frequency domain representation; Step 3: Calculate the high-dimensional residual based on the high-dimensional frequency domain representation and the regenerated frequency domain representation. After randomly shuffling the channel dimensions of the high-dimensional residual, obtain the spatial domain pixel residual by mapping it through an invertible transform decoder. Step 4: Multiply the spatial domain pixel residual with the facial physical mask to generate the protected residual; Step 5: Input the protected residual into the pre-trained recognizer, and the recognizer outputs the facial identity features to complete the facial recognition.

[0006] This invention generates a physical mask of facial features based on five key facial points, retaining only the effective information of core identity feature areas such as eyes, nose, and mouth, while masking non-core areas such as facial background, skin, face shape, and hair. This significantly reduces visual clues that can be exploited by attackers, improves the privacy protection capabilities of facial data from the source, and effectively resists various facial reconstruction attacks.

[0007] Secondly, residual representations are constructed by combining frequency domain transformation, generator reconstruction, and random channel shuffling. Then, protected residuals are generated by combining masking. This not only weakens the visual information of the face image but also fully preserves the key features used for identity recognition. This strengthens privacy protection while ensuring that the recognition accuracy of face recognition is not affected.

[0008] Meanwhile, the model's computation focuses on the core areas of the facial features, reducing redundant calculations caused by ineffective areas, effectively lowering computational overhead and improving overall processing efficiency. The entire process has clear logic and strong interpretability, requiring no complex full-image encryption or global perturbation operations, making deployment and implementation easy. This invention balances recognition performance, privacy and security, and operational efficiency, and can be adapted to various facial recognition application scenarios such as access control, payment, public safety, and mobile terminals, significantly improving its practicality and versatility.

[0009] Optionally, the training process for the recognition network includes: The pixel-level attacker is updated individually every preset number of iterations. Gradient clipping is performed on the protected residual, the original image of the facial region of interest, and the physical mask of the facial features. The reconstruction loss of the pixel-level attacker is expressed as:

[0010] in, This indicates the protected residual. This represents a pixel-level attack network. This represents the original image of the region of interest for a face. Indicates the valid region mask; Subsequently, an adversarial update of the generator is performed, freezing the parameters of the pixel-level attacker and switching it to evaluation mode. Batch normalization statistics are locked, allowing gradients to be backpropagated through the attacker's computational graph to the protected residuals and the generator. A stable adversarial loss is constructed using the attacker's reconstruction loss, calculated as follows:

[0011] in, To counter the border, For temperature parameters, and These are the lower and upper bounds of the numerical value, respectively. The total loss for generator updates is expressed as:

[0012] in, , , These represent the generation constraints respectively. Identify constraints Combating losses The weighting coefficients.

[0013] This alternating adversarial training mechanism offers several technical advantages over existing MinusFace-like schemes that only train the generator and recognizer collaboratively. First, it introduces a pixel-level attacker to construct a complete adversarial training loop, updating the network alternately in A-step and G-step phases. In the A-step phase, gradient clipping is performed on the protected residual and the original ROI image, optimizing the attacker independently and continuously strengthening its face reconstruction capabilities. This actively simulates reconstruction attack behavior in a real-world environment during the training phase. In the G-step phase, the attacker parameters are frozen and batch normalization statistics are locked, allowing gradients to be backpropagated to the generator. This forces the generator to actively optimize the protected residual, suppressing the attacker's ability to reconstruct the original face image, achieving proactive privacy defense and overcoming the shortcomings of traditional methods that rely on passive protection and are easily compromised by learning-based reconstruction attacks.

[0014] Second, a segmented constraint loss function system is designed. The attacker reconstruction loss is only used for attacker iteration and adversarial term construction and does not directly participate in generator optimization. At the same time, a stable adversarial loss with truncation and smooth mapping is adopted. Extreme abnormal losses are filtered by the clamp function and gradients are smoothed by softplus to avoid numerical oscillations, gradient explosion or vanishing problems in adversarial training, thereby improving the stability and convergence speed of the training process.

[0015] Third, the system employs a clear division of labor among multiple loss types, with weighted and collaborative optimization. Generation constraints rely on facial feature masks to focus on optimizing facial feature regions, recognition constraints ensure face differentiation capabilities, and adversarial constraints continuously suppress privacy leakage risks. These three elements are weighted to balance recognition accuracy and privacy protection needs, overcoming the inherent contradiction between performance and privacy security in existing technologies. The entire alternating adversarial update logic is lightweight, highly portable, and adaptable to various distributed training environments, significantly improving the model's robustness against pixel-level reconstruction attacks such as U-Net.

[0016] Optionally, the training of the recognition network further includes: The pixel-level attacker is separated from the DDP backbone model and managed separately. After the attacker is updated, the gradients of each process are manually synchronized. The checkpoints independently save the attacker parameters and optimizer status. Two-stage training is adopted. The first stage is full-module collaborative training according to the GRA mechanism. The second stage freezes the generator, cuts off the attacker's computation link, and only fine-tunes the recognizer. When the loss weight is 0, the attacker's forward computation is automatically skipped. In the early stages of training, periodically detect key points, ROI bounding boxes, and mask area anomalies. If the percentage of anomalies exceeds the limit, an alarm is triggered and a visual sample is saved to investigate data and mask failure issues.

[0017] In this way, the pixel-level attacker is separated from the DDP backbone model and managed independently. Manual gradient synchronization avoids errors caused by unused parameters in the DDP model. Simultaneously, attacker weights and optimizer states are stored separately. This ensures stable operation of multi-GPU distributed training and facilitates training interruption recovery and independent evaluation of white-box attack effects. A two-stage training and adaptive computation skipping logic is implemented. The first stage fully conducts GRA collaborative adversarial training. The second stage freezes the generator, cuts off the attacker's forward link, and only fine-tunes the recognition branch. When the loss weight is reset to zero, the attacker's computation is skipped directly, significantly reducing iterative computation overhead and adapting to the needs of rapid fine-tuning and lightweight deployment. Finally, an early training consistency self-check mechanism is added, periodically verifying key points, ROIs, and mask anomalies. Exceeding limits triggers automatic alarms and retains visualized samples. This allows for rapid location of data problems such as key point registration errors, image enhancement synchronization deviations, and mask failures, promptly eliminating training interference and effectively improving the robustness and fault diagnosis efficiency of the entire model training process.

[0018] Optionally, the expression for generating the constraint is:

[0019] in, This represents a spatially reconstructed image.

[0020] Thus, the generation constraint loss only calculates the reconstruction error in the effective area of ​​the physical mask of the facial features, and blocks irrelevant areas such as the face background and skin. This guides the generator to optimize only the core identity area of ​​the facial features, reducing visual clues that can be exploited by attacks in non-critical areas. At the same time, pixel mean normalization and numerical protection terms are introduced to avoid training fluctuations caused by differences in face size and empty masks, thus balancing privacy constraint effect and training stability.

[0021] Optionally, the recognition constraint adopts a face recognition classification loss, expressed as:

[0022] in, This represents the classification loss calculation function. Represents the classification head network, Represents the recognizer network. This indicates the sample's true identity label.

[0023] Thus, by employing a face recognition-specific classification loss as a recognition constraint and relying on the protected residual features extracted by the recognizer to complete identity classification supervision, the model can be precisely constrained to retain the distinguishable features of facial features, thereby maintaining stable recognition accuracy while weakening the visual information of the face. Optionally, the weights in the total loss function updated by the generator are: , , .

[0024] This reasonable balance between the optimization efforts of generation constraints, recognition constraints, and adversarial constraints, appropriately strengthening facial feature reconstruction constraints to prioritize recognition accuracy, and reasonably setting adversarial loss weights to continuously suppress face reconstruction attacks, avoids performance imbalance caused by a single loss dominating training, and allows the model to possess both excellent recognition capabilities and privacy protection effects. Optionally, the generator adopts a U-Net structure.

[0025] Optionally, the identifier employs an IR_18 network.

[0026] In a second aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial methods as described in any of the preceding claims.

[0027] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial methods as described in any of the preceding claims. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the overall process of the privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques of the present invention.

[0030] Figure 2 This is a schematic diagram of the alignment ROI and facial physical mask generation based on five key points in this invention.

[0031] Figure 3 This is a flowchart of the generator-identifier-attacker adversarial training process of the present invention.

[0032] Figure 4 This is an example diagram of the main loss function and gradient flow direction in the GRAM-MF network of this invention.

[0033] Figure 5 This is an example diagram of distributed training and checkpoint management of the GRAM-MF network of this invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0035] Example 1: This embodiment provides a privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques, including the following steps: Step 1: Obtain the face image to be recognized, extract five facial key points from the face image, determine the core region of interest of the face based on the five facial key points, and generate the corresponding physical mask of the facial features; Step 2: Input the face core region of interest image into the pre-trained reversible transform encoder to obtain a high-dimensional frequency domain representation. Then input the high-dimensional frequency domain representation into the pre-trained generator to obtain the regenerated frequency domain representation. Step 3: Calculate the high-dimensional residual based on the high-dimensional frequency domain representation and the regenerated frequency domain representation. After randomly shuffling the channel dimensions of the high-dimensional residual, obtain the spatial domain pixel residual by mapping it through the reversible transform decoder. Step 4: Multiply the spatial domain pixel residuals with the facial feature physical mask to generate protected residuals; Step 5: Input the protected residual into the pre-trained recognizer, and the recognizer outputs the facial identity features to complete the facial recognition.

[0036] This embodiment generates a physical mask of facial features based on five facial key points, retaining only the effective information of core identity feature areas such as eyes, nose, and mouth, while masking non-core areas such as facial background, skin, face shape, and hair. This significantly reduces visual clues that can be exploited by attackers, improves the privacy protection capability of facial data from the source, and effectively resists various facial reconstruction attacks.

[0037] Secondly, residual representations are constructed by combining frequency domain transformation, generator reconstruction, and random channel shuffling. Then, protected residuals are generated by combining masking. This not only weakens the visual information of the face image but also fully preserves the key features used for identity recognition. This strengthens privacy protection while ensuring that the recognition accuracy of face recognition is not affected.

[0038] Meanwhile, the model's computation focuses on the core areas of the facial features, reducing redundant calculations in ineffective areas, effectively lowering computational overhead and improving overall processing efficiency. The entire process is logically clear and highly interpretable, requiring no complex full-image encryption or global perturbation operations, making deployment and implementation easy. This embodiment balances recognition performance, privacy and security, and operational efficiency, making it adaptable to various facial recognition application scenarios such as access control, payment, public safety, and mobile terminals, significantly improving its practicality and versatility.

[0039] Example 2: This embodiment provides a detailed training process for the network in a privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques, specifically including the following steps.

[0040] Step 1: Construct the GRAM-MF network framework.

[0041] The framework includes generators Recognizer Pixel-level attack tool Reversible Transform Encoder Reversible Transform Decoder and classification heads .

[0042] Among them, generator A U-Net structure can be used for image regeneration on a high-dimensional representation in the frequency domain; the recognizer IR_18 or other face recognition backbone networks can be used, and the input layer can be adjusted to receive the spatial domain residual after the physical mask is applied; attacker Using the U-Net image-to-image structure, the protected residual is input and the reconstruction result of the original ROI face image is output.

[0043] Step 2: Obtain training samples.

[0044] Each training sample includes face images. Identity tags and five facial landmarks The five key points include the left and right eyes, the tip of the nose, and the left and right corners of the mouth. If no key points are provided for the sample, the ArcFace standard five-point template scaled to the current input resolution is used as a fallback key point. When randomly flipping the training image horizontally, the key point coordinates are flipped simultaneously, and the semantic order of the left and right eyes and the left and right corners of the mouth are swapped to ensure consistency between image enhancement and key point positions.

[0045] Step 3: Generate aligned ROIs based on key points.

[0046] like Figure 2 As shown, for each sample, the bounding box of the face core region is calculated based on the minimum bounding range of the five key points, and then calculated according to a preset ratio. Expand outwards. In this embodiment... =0.15. The bounding box is aligned according to a preset grid size. In this embodiment, the grid size is 8 pixels, and the bounding box is further restricted to the image area. When key point anomalies cause the bounding box to degenerate, the entire input image is used as a fallback ROI.

[0047] Step 4: Construct a batch ROI container and a valid region mask.

[0048] For ROIs within the same batch, the maximum side length of the clipping region is calculated and rounded up to an integer multiple of the U-Net downsampling factor. In this embodiment, U-Net includes 4 downsampling operations, so the container side length is a multiple of 16. Each non-square ROI is pasted into the upper left corner of the square container, and other areas are filled with zeros; simultaneously, a valid region mask roi_valid_mask is generated to mark the true ROI region, preventing subsequent reconstruction loss from being diluted by the padding region.

[0049] Step 5: Generate physical masks for facial features.

[0050] Transform the five keypoints to the ROI coordinate system, generate a binary circular region centered on each keypoint, and take the union of the five circular regions as the physical mask. In this embodiment, the radii of the left and right eye regions are 0.12 and 0.12, respectively. The radius of the nasal region is 0.10. The radii of the left and right corners of the mouth are 0.15 respectively. ,in Width of the ROI. Mask. The core area of ​​the facial features is assigned a value of 1, while other areas are assigned a value of 0. This physical mask serves as hard attention, ensuring that the model retains effective residual information only in the core facial feature areas.

[0051] Step 6: Generate the frequency domain regenerated representation.

[0052] Input the ROI image into the reversible transform encoder This yields a high-dimensional frequency domain representation. For example... Figure 1 As shown, the reversible encoder in this embodiment For DCT transformation, This represents 192 channels; Input generator To obtain the generated frequency domain regenerated representation and potential characteristics ; via decoder Will Restored to spatial domain regenerated image And align the numerical range to the input image range.

[0053] Step 7: Calculate and randomly shuffle the residuals along the channel dimension.

[0054] Calculate high-dimensional residuals Subsequently The channel dimensions are randomly shuffled and then processed by the decoder. The scrambled high-dimensional residuals are mapped to spatial domain pixel residuals. The channel shuffling process can be performed on the GPU by obtaining the channel permutation index for each sample through random noise sorting, and then using a gather operation to reduce CPU loop and inter-device copy overhead.

[0055] Step 8: Generate and identify protected residuals.

[0056] Spatial domain pixel residuals Physical mask of facial features Multiply to obtain the protected residual. .like If the spatial dimensions differ from the recognition input dimensions, then it is interpolated to a preset size; in this embodiment, the recognition input dimensions are 112×112. Input Recognizer Obtain identity characteristics Then Input category header , and obtain the identity classification output.

[0057] Step 9: Calculate and generate constraints and identify constraints.

[0058] The generation constraint compares the ROI image and the regenerated image only within the valid region of the physical mask. This prevents regions other than facial features from dominating generator training. This constraint is used to avoid the model retaining identity cues through irrelevant regions and to ensure the generator focuses on processing the core facial feature regions. The expression for the generation constraint is:

[0059] The recognition constraint uses face recognition classification loss. In this embodiment, ArcFace or distributed cross-entropy loss is used, which can be expressed as follows:

[0060] Step 10: Construct the pixel-level attacker and its reconstruction loss.

[0061] attack weapon With protected residuals Input: Restored Image The attacker's goal is to reconstruct the original ROI image within the valid region of the real ROI, and its reconstruction loss is:

[0062] in This is the valid region mask (roi_valid_mask). This loss is calculated only within the real ROI region to avoid interference from padding regions during attacker training and evaluation.

[0063] Step 11: Execute the A-step attacker update.

[0064] During the training process, every The attacker is updated once per iteration step. In this embodiment... In A-step, only the attacker is updated. Protected residuals Original ROI image and effective region mask All gradients are detached to prevent the gradients lost by the attacker from flowing back to the generator and recognizer. When the attacker loss is lower than a preset early stopping threshold, the current attacker update is skipped to avoid adversarial oscillations caused by excessive attacker updates. In this embodiment, the attacker early stopping threshold is 0.2, and the attacker learning rate is 0.01.

[0065] Step 12: Execute the G-step generator to resist updates.

[0066] Freeze Attack Device in G-Step The parameters are set, and the attacker is put into evaluation mode to lock the batch normalized statistics; at the same time, gradients are allowed to pass through the attacker's computation graph and backpropagate to the protected residuals. and generator-related parameters. Using an attacker to... Reconstructing stable loss to mitigate reconstruction error:

[0067] in To counter the border, For temperature parameters, and These are the lower and upper bounds, respectively. In this embodiment, Take 0.45, Take 0.20, Take 1.0e-4, Take 2.0; For smooth activation function, This represents the numerical truncation function. Compared to directly minimizing the negative attacker loss, this stable adversarial loss avoids numerical instability and training oscillations.

[0068] The total loss is a weighted average of the generation constraints, identification constraints, and adversarial constraints:

[0069] In this embodiment, , , Damage to attack equipment Instead of being directly added to the total loss of the generator update, it is used to train the attacker in the A-step and to construct adversarial terms in the G-step. In this way, the attacker continuously enhances its recovery capabilities, while the generator actively suppresses information that the attacker can exploit under recognition constraints.

[0070] Step 13: Manage attackers during distributed training.

[0071] To prevent interference from DDP wrapping and unused parameter detection during independent A-step updates of the attacker, the attacker is physically separated from the backbone model and managed separately by the training task. After the attacker is updated, the attacker gradients in each distributed process are manually all-reduce synchronized to prevent attacker parameter drift across different devices. In addition to saving the generator, recognizer, and classification head parameters, training checkpoints also separately save the attacker parameters and attacker optimizer state, facilitating interruption recovery and white-box attack evaluation.

[0072] Step 14: Set up phased training and zero-overhead path.

[0073] In the first stage, the generator, recognizer, and attacker are trained collaboratively according to the GRA mechanism. In the second stage, the generator from the first stage can be loaded, and only the recognizer or classification head is trained. At this time, the attacker update and adversarial terms are physically cut off, and the attacker forward process is not executed to reduce computational overhead. If the attacker loss weight or adversarial loss weight is set to 0, the corresponding attacker forward computation is also skipped.

[0074] Step 15: Perform a training consistency self-check.

[0075] In the early stages of training, keypoints, ROI bounding boxes, and mask areas can be periodically checked for anomalies, including keypoints going out of bounds, ROI areas being too small or too large, and bounding boxes touching the edges. If the proportion of anomalies exceeds a threshold, an alarm is output and the visualization results are saved. This mechanism is used to detect data keypoint mismatches, synchronization errors, or mask failures, thereby improving the robustness of the training process.

[0076] This embodiment introduces a hard attention mechanism through physical masks of facial features, limiting the effective training area to core identity feature regions such as the eyes, nose, and mouth. This reduces interference from non-core areas such as background, skin, hair, and facial edges, allowing for more concentrated model computational resources and accelerating training and convergence. This mechanism does not rely solely on the model's implicit learning of attention; instead, it explicitly zeros out residual information outside the facial features in the spatial domain using binary physical masks, directly cutting off a large area of ​​non-core visual cues that attackers could exploit. Therefore, it offers clear, interpretable, and controllable privacy protection.

[0077] In the white-box recovery attack experiment, the original MinusFace and the method of this embodiment used the same attacker settings, both using U-Net as the pixel-level white-box attacker, and training the attacker to recover the original ROI image from the protected residual under the condition of freezing the attacked model. Based on the attack recovery results of 5 demonstration examples, the average PSNR of the recovered image after the attack of the original MinusFace was 21.62dB, and the average SSIM was 0.7570; the average PSNR of the recovered image after the attack of the method of this embodiment was reduced to 17.77dB, and the average SSIM was reduced to 0.6561. Compared with the original method, the average PSNR of the method of this embodiment was reduced by 3.85dB, a reduction of about 17.79%; the average SSIM was reduced by 0.1009, a reduction of about 13.32%. According to the relationship between PSNR and mean squared error, the above PSNR reduction corresponds to an increase of about 2.43 times in attack recovery error, indicating that it is more difficult for the attacker to recover a high-quality face image from the protected residual generated by the method of this embodiment, thereby effectively improving privacy protection capabilities.

[0078] This embodiment reuses U-Net as a pixel-level attacker and incorporates the attacker into the training loop. The A-step enables the attacker to continuously learn to recover the original ROI image from the protected residuals, while the G-step, by freezing the attacker, pushes the generator to generate protected residuals that are more difficult to recover. This generator-recognition-attacker collaborative adversarial training mechanism can proactively discover and eliminate information in the protected residuals that can be exploited by the attacker. Compared to methods that simply rely on random shuffling or non-visualized residuals, it has a stronger defense against learning-based attackers.

[0079] This embodiment employs mechanisms such as stable adversarial loss, attacker update frequency control, attacker early stopping threshold, G-step parameter freezing, batch normalization statistic locking, and effective ROI loss calculation to reduce the risk of oscillations during adversarial training. In particular, replacing the direct maximization of attacker error with a softplus-form boundary-based adversarial loss avoids gradient numerical anomalies and improves training stability.

[0080] This embodiment further improves training efficiency and engineering usability at the implementation level. Tests were conducted under identical hardware and training configurations: a vGPU-48GB×4 card environment, TFR-MS1M training data, an input size of 112×112, an IR_18 backbone network, 24 training epochs, and a batch size of 64. The original MinusFace had an average training time of 0.3623 sec / batch in stage 1, with a total training time of 146.345081 hours; the method in this embodiment reduces the average training time to 0.3101 sec / batch, and the total training time to 126.440896 hours. Therefore, the method in this embodiment reduces the training time per batch by 0.0522 seconds, a reduction of approximately 14.41%; and the total training time by 19.904185 hours, a reduction of approximately 13.60%, resulting in an overall training speed approximately 1.16 times faster than the original method.

[0081] The aforementioned improvement in training efficiency mainly comes from three aspects: First, this embodiment introduces a hard attention mechanism through physical masks of facial features, limiting the effective training area to core identity feature regions such as the eyes, nose, and mouth. This concentrates the model's computational resources on facial feature regions that contribute more to identity discrimination, reducing interference from non-core regions such as background, skin, hair, and facial edges, thereby reducing the overhead of learning invalid features and accelerating model convergence. Second, the recognizer can directly receive the protected residual in the three-channel spatial domain in the first stage, instead of the original 192-channel high-dimensional residual, thus reducing the input overhead of the recognition backbone. Third, the random shuffling of channels is completed in parallel on the GPU, reducing the overhead of generating per-sample per-period permutation indexes by the CPU. In addition, in the second stage or when the attacker weights are turned off, the attacker forward pass is completely skipped, ensuring zero additional overhead.

[0082] This embodiment also incorporates mechanisms such as keypoint synchronization enhancement, standard template fallback, ROI degradation fallback, effective region masking, training consistency self-checking, independent storage and restoration of attacker states, and attacker optimizer state storage and restoration. These mechanisms improve the robustness, reproducibility, and deployment convenience of training with large-scale face data. In summary, this embodiment addresses the shortcomings of existing methods, including insufficient privacy protection, low training efficiency, and limited defense against learning-based recovery attacks. It achieves reduced recovery attack quality, shorter training time, and improved engineering training stability, thus balancing privacy protection, training efficiency, and the usability of face recognition.

[0083] Some steps in the embodiments of the present invention can be implemented using software, and the corresponding software program can be stored in a readable storage medium, such as an optical disc or a hard disk.

[0084] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques, characterized in that, Includes the following steps: Step 1: Obtain the face image to be identified, extract five facial key points from the face image, determine the core region of interest of the face based on the five facial key points, and generate the corresponding physical mask of the facial features; Step 2: Input the face core region of interest image into a pre-trained reversible transform encoder to obtain a high-dimensional frequency domain representation, and then input the high-dimensional frequency domain representation into a pre-trained generator to obtain a regenerated frequency domain representation; Step 3: Calculate the high-dimensional residual based on the high-dimensional frequency domain representation and the regenerated frequency domain representation. After randomly shuffling the channel dimensions of the high-dimensional residual, obtain the spatial domain pixel residual by mapping it through an invertible transform decoder. Step 4: Multiply the spatial domain pixel residual with the facial physical mask to generate the protected residual; Step 5: Input the protected residual into the pre-trained recognizer, and the recognizer outputs the facial identity features to complete the facial recognition.

2. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 1, characterized in that, The training process of the recognition network includes: The pixel-level attacker is updated individually every preset number of iterations. Gradient clipping is performed on the protected residual, the original image of the facial region of interest, and the physical mask of the facial features. The reconstruction loss of the pixel-level attacker is expressed as: in, This indicates the protected residual. This represents a pixel-level attack network. This represents the original image of the region of interest for a face. Indicates the valid region mask; Subsequently, an adversarial update of the generator is performed, freezing the parameters of the pixel-level attacker and switching it to evaluation mode. Batch normalization statistics are locked, allowing gradients to be backpropagated through the attacker's computational graph to the protected residuals and the generator. A stable adversarial loss is constructed using the attacker's reconstruction loss, calculated as follows: in, To counter the border, For temperature parameters, and These are the lower and upper bounds of the numerical value, respectively. The total loss for generator updates is expressed as: in, , , These represent the generation constraints respectively. Identify constraints Combating losses The weighting coefficients.

3. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 2, characterized in that, The training of the recognition network also includes: The pixel-level attacker is separated from the DDP backbone model and managed separately. After the attacker is updated, the gradients of each process are manually synchronized. The checkpoints independently save the attacker parameters and optimizer status. Two-stage training is adopted. In the first stage, the entire module is trained collaboratively according to the GRA mechanism. In the second stage, the generator is frozen, the attacker's computing link is cut off, and only the recognizer is fine-tuned. When the loss weight is 0, the attacker's forward calculation is automatically skipped. In the early stages of training, key points, ROI bounding boxes, and mask areas are periodically detected to prevent anomalies. If the percentage of anomalies exceeds the limit, an alarm is triggered and a visual sample is saved to investigate data and mask failure issues.

4. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 2, characterized in that, The expression for the generated constraint is: in, This represents a spatially reconstructed image.

5. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 2, characterized in that, The recognition constraint adopts the face recognition classification loss, which is expressed as: in, This represents the classification loss calculation function. Represents the classification head network, Represents the recognizer network. This indicates the sample's true identity label.

6. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 2, characterized in that, The weights in the total loss function updated by the generator are: , , .

7. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 1, characterized in that, The generator adopts a U-Net structure.

8. The privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial techniques according to claim 1, characterized in that, The identifier uses an IR_18 network.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial methods as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the privacy-preserving face recognition method based on physical masking of facial features and dynamic adversarial methods as described in any one of claims 1 to 8.

Images