Method for realizing security evaluation of face recognition system by physical domain counter-sticker attack

Abstract

The present application belongs to the field of artificial intelligence information security technology, and discloses a method for realizing security evaluation of a face recognition system by physical domain countermeasure sticker attack, which comprises collecting user face images and target face images, and making countermeasure stickers according to the collected images, and modeling physical environment (illumination, contrast), face image self change (rotation, scaling, etc.) and sticker change, inputting the countermeasure face after modeling and the target face into a pre-trained face recognition model for countermeasure optimization to obtain a final countermeasure patch, and finally printing and pasting the patch at a specific position of the face to attack the face recognition system for security evaluation. The present application also relates to an evaluation system for verifying security of a face recognition system by countermeasure sticker attack. The method for realizing security evaluation of a face recognition system by physical domain countermeasure sticker attack greatly simplifies the operation process of attack and evaluation, and has positive guiding significance for the field of face recognition security.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence information security technology, and in particular relates to a method and system for security assessment of a face recognition system that uses physical domain countermeasures against sticker attacks. Background Technology

[0002] Currently, with the rapid development of the information age, facial recognition technology is widely used in various key areas, such as gate identity verification, attendance tracking, and video surveillance. However, while providing convenience, facial recognition also brings significant security risks. Increasing research indicates that facial recognition (FR) systems are susceptible to adversarial examples, meaning that adversarial examples can cause FR systems to make incorrect identifications with high confidence, posing a significant security threat. Researching adversarial attacks on facial recognition can better assess the security of facial recognition systems deployed in real-world environments. However, most current adversarial attacks are at the digital level, effectively attacking white-box facial recognition systems. In reality, most facial recognition systems are black-box, and digital attacks are largely impossible to implement. The few physical adversarial attacks have low robustness in complex environments and are overly reliant on equipment, making it difficult to assess the security of facial recognition systems in real-world environments.

[0003] In summary, the problems with existing technologies are:

[0004] (1) Most existing adversarial attacks on facial recognition are based on digital attacks and can only be white-box attacks.

[0005] (2) Physical domain face recognition adversarial attacks have low robustness in complex environments and are overly dependent on devices.

[0006] The difficulty and significance of solving the above technical problems:

[0007] The main challenges in solving the above problems lie firstly in how to effectively attack facial recognition systems in a black-box environment in a real-world setting, and secondly in how to overcome the attack robustness issues in complex real-world environments. The significance of this invention lies in its ability to effectively attack facial recognition systems in a black-box environment while maintaining an effective attack rate in various complex environments. This allows for better assessment of the security level of facial recognition systems deployed in real-world environments, faster detection of vulnerabilities, and enables facial recognition systems to implement corresponding defensive measures against these vulnerabilities. Summary of the Invention

[0008] To address the problems existing in the prior art, this invention provides a method and system for security assessment of a face recognition system that uses physical domain-based anti-sticker attacks.

[0009] This invention is implemented as follows: A method for security assessment of a face recognition system against sticker attacks using the physical domain includes:

[0010] S1, Collect the face x to be attacked and the target face x. t Injected into the face recognition system to be evaluated, ensuring that the face to be attacked can be recognized normally under normal circumstances, and that the target face cannot be attacked;

[0011] S2, select the form of adversarial patch δ, and use the masking algorithm to generate digital domain face adversarial examples x. adv1 ;

[0012] S3, x adv1 The second-stage adversarial face sample x is obtained after physical world changes such as lighting, scaling, and rotation. adv2 ;

[0013] S4 uses total variation loss and unprintable fractional loss to reduce the adversarial loss in the digital-to-physical domain conversion process; finally, iterative gradient optimization is performed using an improved Expectation Transformation (EOT) algorithm combined with cosine similarity loss to obtain the final adversarial face image x. adv And the anti-sticker δ;

[0014] S5 prints out and cuts out the optimized adversarial stickers, then pastes them onto the corresponding positions of the real face. The face is then used to attack the target facial recognition system, and the security level of the current facial recognition system is determined based on the success rate of the attack.

[0015] Furthermore, in steps S2 and S3, a masking algorithm and a corresponding physical transformation module are used to obtain the first and second stage adversarial face samples:

[0016] x adv1 = (1-M)οx+MοT A (δ+v),

[0017] x adv2 =T B (x adv1 ),

[0018] Where “ο” indicates the multiplication of corresponding matrix elements, T A The sticker transformation module includes: 1) Deformation of stickers when pasted on real-world surfaces, including out-of-surface bending and 3D rotation; parabolic transformation operator is used to simulate out-of-surface bending, and three-dimensional affine transformation is used to simulate 3D rotation; 2) Position perturbation, which makes it difficult to paste the stickers precisely at the designed position, such as random rotation and translation.

[0019] Furthermore, the T BFor the face transformation module: the attacker also experienced a series of physical world changes such as different affine transformations and lighting contrast; including randomly generating adversarial face images with rotation, scaling, translation, contrast and brightness.

[0020] Furthermore, M is a mask for resisting stickers, which adopts a 0-1 binary encoding form; v~N(μ,σ) is random Gaussian noise with a mean of μ and a standard deviation of σ.

[0021] Furthermore, in step S4, the total adversarial loss in the adversarial example optimization stage is: L adv =L cos +αL TV +βL NPS ;

[0022] Where L cos L TV L NPS These represent the cosine similarity loss, total variation loss, and unprintable fraction loss, respectively, with α and β being the loss weights.

[0023] Furthermore, in step S5, when the target face recognition system identifies the adversarial sample as someone else or makes a recognition error (cosine similarity is below the threshold), the attack is considered successful. The attack rate is calculated using the formula FAR = NA / N * 100%, where NA represents the number of successful attacks and N represents the total number of attacks.

[0024] Furthermore, attack success rates of FAR>90%, FAR>85%, FAR>80%, FAR>70%, FAR>60%, and FAR>50% indicate security levels of F, E, D, C, B, and A, respectively.

[0025] Another objective of this invention is to provide a system for security assessment of a face recognition system that counters sticker attacks in the physical domain, comprising:

[0026] Face capture module: used to capture the user's face images. The captured images will first undergo face detection to remove face images without faces or with poor quality. Then, a pre-trained feature extraction model will be used to extract the feature vector of each face and store it in the face database.

[0027] Adversarial Example Generation Module: Initializes adversarial stickers and corresponding masks, then randomly selects a face image from the face data as the attack face, selects the form of adversarial patch, generates adversarial face examples through mask algorithm and its corresponding physical transformation module, inputs them into a pre-trained face recognition model for adversarial optimization, and obtains the final adversarial face examples and adversarial stickers after optimization.

[0028] Attack Evaluation Module: Print and cut the adversarial stickers generated by the adversarial sample optimization module, then paste the stickers on specific parts of the face, and finally attack the target face recognition model or system. Calculate the attack success rate by comparing the cosine similarity between the attacking face and the target face in the face database, and output the security level of the corresponding target face recognition model or system.

[0029] The aforementioned anti-sticker attack verification facial recognition system security assessment system constructs robust adversarial facial samples based on facial image data, effectively maintains a good attack rate under various physical environments, and finally calculates the attack success rate to obtain the security assessment level of the facial recognition system under test.

[0030] Another object of the present invention is to provide a computer device including a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for implementing a security assessment of a face recognition system against sticker attacks in the physical domain.

[0031] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for implementing a security assessment of a face recognition system against sticker attacks in the physical domain.

[0032] Another objective of this invention is to provide an information data processing terminal, which is used to implement a system for security assessment of a face recognition system against sticker attacks in the physical domain.

[0033] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0034] First, if a facial recognition system makes a mistake or even misidentifies an individual, it can lead to incalculable economic losses. Therefore, researchers should focus on improving the accuracy of facial recognition models while also paying attention to the model's resistance to attacks. To better evaluate the security and resistance of facial recognition models, adversarial attacks against facial recognition have been extensively studied. Depending on the attack domain, adversarial attacks can be divided into digital domain attacks and physical domain attacks. Digital domain attacks require direct access to the digital input of the facial recognition system. However, facial recognition systems typically capture faces using devices such as cameras to form digital images, making it difficult for attackers to perturb these images. Compared to digital domain attacks, physical domain attacks can attack facial recognition systems in real-world scenarios using only physical adversarial patches or physical lighting, posing a greater threat. For example, when an attacker wears adversarial eyeglass frames or a mask, the facial recognition system may mistakenly identify them as someone else. Attacks based on physical lighting are less physically feasible due to their reliance on equipment and potential harm to the human body. The physical adversarial patching method involves printing and pasting adversarial perturbations generated in the digital domain onto specific locations on the face. The camera then captures the facial image and inputs it into the facial recognition model to launch the attack. Due to its simplicity, lack of reliance on external devices, and harmlessness to the human body, this method is the most feasible and effective, and is widely used in the security assessment of facial recognition systems.

[0035] However, applying physical adversarial patch-based attacks to real-world facial recognition systems presents two main challenges. First, there's the challenge of effectively converting adversarial perturbations from the digital domain to the physical domain without losing their offensive capabilities. In real-world scenarios, attacking facial recognition systems involves numerous uncontrollable factors, such as ambient light intensity, facial posture, distance, and viewpoint. These uncontrollable factors can reduce or even eliminate the effectiveness of adversarial perturbations in the digital domain during the conversion process. Second, there's the challenge of improving the transferability of adversarial examples in black-box scenarios. Existing physical domain attack methods primarily target white-box attacks, meaning they only show good attack effectiveness when the model structure and parameters of the target model are known. However, in real-world scenarios, facial recognition models are generally black-box, meaning their parameters and architecture are unknown. This hinders the feasibility of physical domain attacks. Therefore, improving the transferability of physical domain attacks in black-box scenarios is a key focus of this invention.

[0036] This invention enhances the fault tolerance of adversarial examples in complex environments and black-box scenarios by fully modeling different environments, locations, and printer colors, and combining this with a self-learning strategy. The created adversarial examples maintain a high attack rate in various complex and changing physical environments, and can also effectively attack in black-box environments. Furthermore, the attack only requires cutting out the adversarial sticker and pasting it onto a specific part of the face, without relying on specific physical equipment. This invention plays a positive role in evaluating the security levels of various facial recognition systems, enabling timely identification of security vulnerabilities in target facial recognition systems. It provides a more comprehensive security assessment of real-world facial recognition systems and offers research directions for facial recognition defense, driving the continuous development and advancement of facial recognition systems towards greater security and practicality.

[0037] Secondly, to make facial recognition technology more practical and secure, a systematic security assessment of its key applications is essential. This invention fully considers uncontrollable factors in real-world environments, such as location and environmental factors, to maximize the robustness of physical domain attacks in complex environments. Furthermore, real-world facial recognition systems are typically black-box systems, and white-box systems generally have low success rates for physical domain attacks and are difficult to transfer to other models. To address this, this invention significantly improves the transferability of physical domain attacks in black-box environments. These two improvements further enhance the practical value and feasibility of physical domain attacks on facial recognition. This invention demonstrates innovative effects in both robustness against physical domain attacks in complex environments and transferability of physical domain attacks in black-box settings, enabling better evaluation of the anti-attack robustness of facial recognition systems and providing guidance for facial recognition defense research.

[0038] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0039] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0040] This invention primarily targets two groups of people. First, researchers studying adversarial attacks on faces can use the system to generate digital domain adversarial patches for digital domain attacks or print them for physical domain attacks. This allows them to assess the robustness of a face recognition model or system against attacks in the digital or physical domains, helping to expose vulnerabilities and providing insights for future face recognition defense research. Second, it targets individuals with face privacy protection needs. They can use this invention to generate unique adversarial face samples for posting on social media platforms to meet work requirements. These samples appear to the naked eye as merely minor patches added to certain areas, effectively protecting the face from misuse by malicious actors that threaten property security without affecting recognition.

[0041] Fourth, the method for assessing the security of a face recognition system using physical domain adversarial sticker attacks provided by this invention addresses the shortcomings of existing face recognition system security assessment methods and offers a new and more challenging approach to evaluating the security of face recognition systems. This method generates adversarial stickers to simulate real-world physical attacks, thereby comprehensively assessing the robustness and security of the face recognition system.

[0042] First, this invention ensures that the face to be attacked can be correctly identified under normal circumstances, while the target face remains unaffected by the attack, by collecting data on the face to be attacked and the face to be targeted. Next, it generates adversarial face examples in both the digital and physical domains using a masking algorithm and a physical transformation module, fully considering various changes that occur in the real physical world, such as lighting, scaling, and rotation.

[0043] In the adversarial example optimization stage, this invention employs total variation loss and unprintable fractional loss to reduce the adversarial loss during the digital-to-physical domain conversion process. Simultaneously, it combines cosine similarity loss for iterative gradient optimization, thereby obtaining the final adversarial sticker. This optimization method not only improves the generation efficiency of adversarial stickers but also enhances their effectiveness in practical applications.

[0044] Finally, this invention involves printing and cutting optimized adversarial stickers, which are then affixed to corresponding positions on a real human face to attack the target facial recognition system. The attack success rate is used to determine the security level of the current facial recognition system, providing a strong basis for system design and improvement.

[0045] Compared with the prior art, the present invention has the following significant technical advancements:

[0046] 1. Realism: This invention simulates physical attacks in the real world and fully considers various changes that occur in the actual environment, making the evaluation results more realistic and reliable.

[0047] 2. Comprehensiveness: This invention comprehensively evaluates the robustness and security of the face recognition system by generating adversarial stickers, taking into account not only the recognition performance under normal conditions, but also the performance under attack.

[0048] 3. High efficiency: The optimized algorithm used in this invention can quickly generate effective adversarial stickers, improving the efficiency of evaluation.

[0049] 4. Practicality: The evaluation method provided by this invention can provide strong support for the design and improvement of face recognition systems, and help improve the security and robustness of the system.

[0050] This invention provides a method for security assessment of face recognition systems by using physical domain countermeasures against sticker attacks, which solves the problems in the prior art and provides new ideas and methods for improving the security of face recognition systems. Attached Figure Description

[0051] Figure 1 This is a flowchart of a method for security assessment of a face recognition system based on physical domain countermeasures against sticker attacks, provided in an embodiment of the present invention.

[0052] Figure 2 This is a diagram showing the results of an escape attack in a complex environment, as provided in an embodiment of the present invention.

[0053] Figure 3 This is a graph showing the evaluation results of escape attacks in a black-box environment provided in an embodiment of the present invention.

[0054] Figure 4 This is a diagram showing the evaluation results of impersonation attacks in a black-box environment provided by an embodiment of the present invention.

[0055] Figure 5 This is a visualization of the physical countermeasure patch provided in this embodiment of the invention.

[0056] Figure 6 This is a visualization of the escape attack results under normal conditions compared with existing methods, provided by an embodiment of the present invention.

[0057] Figure 7 This is a visualization of the impersonation attack results under normal conditions compared with existing methods, provided by an embodiment of the present invention.

[0058] Figure 8 This embodiment of the invention provides a system structure block diagram for security assessment of a face recognition system that utilizes physical domain countermeasures against sticker attacks. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0060] like Figure 1 As shown in the figure, the method for security assessment of a face recognition system against sticker attacks using physical domains provided by an embodiment of the present invention includes the following steps:

[0061] S1, Collect the face x to be attacked and the target face x. t Injected into the face recognition system to be evaluated, ensuring that the face to be attacked can be recognized normally under normal circumstances, and that the target face cannot be attacked;

[0062] S2, select the form of adversarial patch δ, and use the masking algorithm to generate digital domain face adversarial examples x. adv1 ;

[0063] S3, x adv1 The second-stage adversarial face sample x is obtained after physical world changes such as lighting, scaling, and rotation. adv2 ;

[0064] S4 uses total variation loss and unprintable fractional loss to reduce the adversarial loss in the digital-to-physical domain conversion process; finally, iterative gradient optimization is performed using an improved Expectation Transformation (EOT) algorithm combined with cosine similarity loss to obtain the final adversarial face image x. adv And the anti-sticker δ;

[0065] S5 prints out and cuts out the optimized adversarial stickers, then pastes them onto the corresponding positions of the real face. The face is then used to attack the target facial recognition system, and the security level of the current facial recognition system is determined based on the success rate of the attack.

[0066] In steps S2 and S3 provided in this embodiment of the invention, a masking algorithm and a corresponding physical transformation module are used to obtain the first and second stage adversarial face samples:

[0067] x adv1 = (1-M)οx+MοT A (δ+v),

[0068] x adv2 =T B (x adv1 ).

[0069] The T provided in the embodiments of the present invention AThe sticker transformation module includes: 1) Deformation of stickers when pasted on real-world surfaces, including out-of-surface bending and 3D rotation; parabolic transformation operator is used to simulate out-of-surface bending, and three-dimensional affine transformation is used to simulate 3D rotation; 2) Position perturbation, which makes it difficult to paste the stickers precisely at the designed position, such as random rotation and translation.

[0070] The T provided in the embodiments of the present invention B For the face transformation module: the attacker also experienced a series of physical world changes such as different affine transformations and lighting contrast; including randomly generating adversarial face images with rotation, scaling, translation, contrast and brightness.

[0071] In this embodiment of the invention, M is a mask for resisting stickers, which adopts the form of 0-1 binary encoding; v~N(μ,σ) is random Gaussian noise with a mean of μ and a standard deviation of σ.

[0072] In step S4 provided in this embodiment of the invention, the total adversarial loss in the adversarial example optimization stage is: L adv =L cos +αL TV +βL NPS ;

[0073] Where L cos L TV L NPS These represent the cosine similarity loss, total variation loss, and unprintable fraction loss, respectively, with α and β being the loss weights.

[0074] In step S5 provided in this embodiment of the invention, when the target face recognition system identifies the adversarial sample as someone else or makes a recognition error (cosine similarity is below the threshold), the attack is determined to be successful. The attack rate is calculated using the formula FAR = NA / N*100%, where NA represents the number of successful attacks and N represents the total number of attacks.

[0075] Furthermore, attack success rates of FAR>90%, FAR>85%, FAR>80%, FAR>70%, FAR>60%, and FAR>50% indicate security levels of F, E, D, C, B, and A, respectively.

[0076] like Figure 8 As shown in the figure, the system for physical domain countermeasures against sticker attacks to achieve security assessment of a face recognition system provided by an embodiment of the present invention includes:

[0077] Face capture module: used to capture the user's face images. The captured images will first undergo face detection to remove face images without faces or with poor quality. Then, a pre-trained feature extraction model will be used to extract the feature vector of each face and store it in the face database.

[0078] Adversarial Example Generation Module: Initializes adversarial stickers and corresponding masks, then randomly selects a face image from the face data as the attack face, selects the form of adversarial patch, generates adversarial face examples through mask algorithm and its corresponding physical transformation module, inputs them into a pre-trained face recognition model for adversarial optimization, and obtains the final adversarial face examples and adversarial stickers after optimization.

[0079] Attack Evaluation Module: Print and cut the adversarial stickers generated by the adversarial sample optimization module, then paste the stickers on specific parts of the face, and finally attack the target face recognition model or system. Calculate the attack success rate by comparing the cosine similarity between the attacking face and the target face in the face database, and output the security level of the corresponding target face recognition model or system.

[0080] The aforementioned anti-sticker attack verification facial recognition system security assessment system constructs robust adversarial facial samples based on facial image data, effectively maintains a good attack rate under various physical environments, and finally calculates the attack success rate to obtain the security assessment level of the facial recognition system under test.

[0081] The method for security assessment of face recognition systems using physical domain-based anti-sticker attacks provided by this invention is a technical solution for systematically assessing potential weaknesses in face recognition systems. Its detailed working principle is as follows:

[0082] 1. Face data acquisition and recognition verification (S1): Collect the target face (x) and the person to be attacked (x) for the attack. t This step ensures that the facial recognition system can correctly identify and distinguish the target face under normal conditions where it has not been attacked. This step verifies the system's identification capability under normal conditions.

[0083] 2. Adversarial Example Generation (S2-S3): First-stage digital domain adversarial examples (x) are generated using a masking algorithm. adv1 Then, by simulating real-world physical changes (such as lighting, scaling, rotation, etc.), a second-stage adversarial face sample (x) is generated. adv2 These changes ensure that adversarial examples can adapt to different physical environments.

[0084] 3. Adversarial Example Optimization (S4): To reduce the adversarial loss during the digital-to-physical domain conversion process, this method introduces total variation loss and unprintable fraction loss, and performs iterative gradient optimization by combining the Expectation Transformation (EOT) algorithm with cosine similarity loss to obtain high-quality adversarial face examples.

[0085] 4. Adversarial Sticker Creation and Practical Application (S5): Print out the optimized adversarial stickers and affix them to the corresponding positions on actual human faces. Then, launch an attack on the facial recognition system. The success of the attack is determined by whether the system misidentifies the face with the adversarial sticker as the target face, and the security level of the system is assessed accordingly.

[0086] 5. Security Level Classification: The system's security level is assessed by calculating the Success Rate of Attack (FAR) under different conditions. Specifically, different FAR values ​​correspond to different security levels, thus providing system administrators with targeted security hardening recommendations.

[0087] This invention provides a comprehensive assessment of the vulnerability of facial recognition systems to physical adversarial attacks, offering a scientific basis for improving the resilience and reliability of facial recognition technology. This method not only helps understand the strategies attackers employ and the potential weaknesses of systems, but also guides the development of more secure facial recognition applications.

[0088] An embodiment of the present invention provides a computer device, the computer device including a memory and a processor, the memory storing a computer program, the computer program being executed by the processor causing the processor to perform the steps of the method for implementing a security assessment of a face recognition system against physical domain sticker attacks.

[0089] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor performs the steps of a method for assessing the security of a face recognition system against sticker attacks in the physical domain.

[0090] This invention provides an information data processing terminal, which is used to implement a system for security assessment of a face recognition system by resisting sticker attacks in the physical domain.

[0091] The method for security assessment of a face recognition system based on physical domain-based adversarial sticker attacks provided by this invention is characterized by the following steps:

[0092] Attack preparation phase:

[0093] S1, Collect the face x to be attacked and the target face x. t Injected into the face recognition system to be evaluated, ensuring that the face to be attacked can be recognized normally under normal circumstances, and that the target face cannot be attacked.

[0094] S2, select the form of adversarial patch δ, and use the masking algorithm to generate digital domain face adversarial examples x. adv1 .

[0095] S3, x adv1The second-stage adversarial face sample x is obtained after physical world changes such as lighting, scaling, and rotation. adv2 .

[0096] S4 uses total variation loss and unprintable fractional loss to reduce the adversarial loss in the digital-to-physical domain conversion process. Finally, an improved Expectation Transformation (EOT) algorithm combined with cosine similarity loss is used for iterative gradient optimization to obtain the final adversarial face example image x. adv And the anti-sticker δ.

[0097] Attack assessment phase:

[0098] S5 prints out and cuts out the optimized adversarial stickers, then pastes them onto the corresponding positions of the real face. The face is then used to attack the target facial recognition system, and the security level of the current facial recognition system is determined based on the success rate of the attack.

[0099] Preferably, for steps S2 and S3, two forms of countermeasure patches are provided: rectangular stickers and countermeasure eyeglass frames. The rectangular stickers are 900×400 pixels in the digital domain and 13cm×5.8cm in the physical domain. Since the countermeasure eyeglass frame is a planar image (digital domain), in a real-world environment, only a lensless eyeglass frame (16.5×6.4cm) is needed to wear the printed countermeasure eyeglass frame accessory.

[0100] Finally, the second-stage adversarial face samples are obtained using the following formula:

[0101] x adv1 = (1-M)οx+MοT A (δ+v),

[0102] x adv2 =T B (x adv1 ),

[0103] Where “ο” indicates the multiplication of corresponding matrix elements, T A For sticker transformation module, T B Let M be the face transformation module, and M be the mask for adversarial stickers, which is encoded in 0-1 binary form. v~N(μ,σ) is random Gaussian noise with mean μ and standard deviation σ.

[0104] For T A This includes: 1) Deformation of stickers when pasted on real-world surfaces, including out-of-plane bending and 3D rotation. Parabolic transformation operators are used to simulate out-of-plane bending, and 3D transformations are used to simulate rotation; 2) Positional perturbations, making it difficult to paste the stickers precisely at the designed position, such as random rotation and translation.

[0105] Specifically, a parabolic transformation operator is used to simulate out-of-plane bending, and a three-dimensional transformation is used to simulate rotation. The specific coordinate transformation formulas are as follows:

[0106]

[0107] z'=ax' 2

[0108] The parameter 'a' controls the parabolic velocity. This formula maps each coordinate point (x, y, z) of the sticker to (x', y, z'), simulating out-of-plane bending and 3D transformation. Furthermore, the sticker's position is perturbed, with random rotations and translations, because it's difficult to precisely place the anti-parallel sticker in the designed location.

[0109] For T B The attackers also subjected the characters to a series of physical world variations, including different affine transformations and changes in lighting contrast. This included randomly generating adversarial face images with rotation, scaling, translation, contrast, and brightness adjustments.

[0110] For the affine transformation module, an affine transformation with four parameter factors is adopted: a rotation factor, a scaling factor, and two translation factors. To make the position transformation more uniform, a random transformation strategy is used. Specifically, let U(a,b) be a uniformly distributed random sampling function from a to b, and η be a hyperparameter. Then the four parameter factors can be expressed as follows:

[0111] t x =U(-ηW,-ηW)

[0112] t y =U(-ηH,-ηH)

[0113] θ = U(-ηπ / 2, -ηπ / 2)

[0114] s=U(1-η,1+η)

[0115] Where t x t y θ and s are the incremental translation factor, incremental rotation factor, and incremental scaling factor, respectively. W and H are the width and height of the input face image.

[0116] After obtaining the above four factors, the transformation matrix T1 can be expressed as follows:

[0117]

[0118] For the illumination contrast module, brightness and contrast can be adjusted by designing a mapping curve. The mapping curve can be expressed as T² = k × p + b, where p is the pixel value, the slope k and deviation b control the image contrast, and b alone controls the image brightness. Finally, T² can be obtained. B =T2(T1(x)).

[0119] Preferably, for step S4, the following formula is used for adversarial optimization:

[0120] L adv =L cos +αL TV +βL NPS

[0121] Where α and β are the loss weights.

[0122] To apply the improved EOT algorithm, a self-stepping learning strategy is introduced. cos It can be represented as:

[0123]

[0124] Where N is the number of transformations, w i It is the transformation weight, R(w) i ) = w i / 2-w i It is a self-stepping regularizer, where λ>0 is the age parameter and it increments with the number of iterations. It is the transformation function for random sampling, cos(f(t) i (x)),f(x t The cosine similarity is calculated between two feature vectors.

[0125] To reduce the differences in pixel values ​​around the sticker (δ) and enable a smooth transition of the sticker in the physical environment, a total variation loss L is introduced. TV :

[0126]

[0127] To reduce printing errors, color values ​​are first uniformly sampled across the three RGB channels in the digital domain to obtain a digital domain color palette C. Next, C is printed to obtain the printed color palette C'. Finally, C' is photographed using a camera or similar device to re-obtain the digital domain RGB color palette C', which is the printable color palette. Additionally, the color mapping relationship from the initial digital domain to the second-transformed digital domain RGB color palette, C → C', is also obtained. The unprintable fractional loss L is given below. NPS Definition:

[0128]

[0129] To efficiently optimize adversarial face examples, an alternating search strategy is adopted:

[0130] First, with the adversarial sticker δ fixed, the optimization of the weight w can be achieved with the following objectives:

[0131]

[0132] Because R(w) i Since w is a convex function, the optimal solution for w can be directly obtained as w. * =1-L cos / λ.

[0133] Secondly, with a fixed transformation weight w, the objective against the sticker δ can be optimized as follows:

[0134]

[0135] Because of t i Since (x) is differentiable, the above equation can be solved using the stochastic gradient descent algorithm.

[0136] Preferably, for step S5, when the face recognition system under test identifies the adversarial sample as someone else or makes a recognition error (cosine similarity is below the threshold), the attack is considered successful. The attack rate is calculated using the formula FAR = NA / N*100%, where NA represents the number of successful attacks and N represents the total number of attacks.

[0137] Furthermore, attack success rates of FAR>90%, FAR>85%, FAR>80%, FAR>70%, FAR>60%, and FAR>50% indicate security levels of F, E, D, C, B, and A, respectively.

[0138] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0139] Example 1

[0140] like Figure 1 The diagram shows a flowchart of the security assessment method for a face recognition system based on physical adversarial stickers according to the present invention. To verify the effectiveness of the method, the present invention designed escape attack and impersonation attack experiments in both normal and complex environments. An escape attack reduces the recognition accuracy of the same person, while an impersonation attack identifies the attacker's face as a face specified by the attacker.

[0141] Based on the needs of relevant experiments, this paper introduces cosine similarity as an attack evaluation index.

[0142] Cosine similarity: This accurately measures the Euclidean distance between the feature vectors of two faces in cosine space. Currently, most face recognition feature learning is based on angle learning; therefore, using cosine similarity can accurately determine whether two faces are the same face or the degree of similarity between them. Cosine similarity can be expressed as:

[0143]

[0144] Attack evaluation experiments in normal and complex environments

[0145] This experiment collected 10 faces from each of the four different identities and used ResNet101+Arcface as the target face recognition model for actual evaluation. It achieved a recognition accuracy of 99.77% on the LFW test set. Tables 1 and 2 present the average results of 10 escape and impersonation attacks under normal conditions (normal lighting, no head pose changes, and a viewing distance of 50cm). To evaluate complex environments, 12 additional photos were taken for the three different identities under different environments: normal, head facing left, head facing right, head facing lower left, head facing upper right, head facing upper left, head facing lower right, head facing up, head facing down, dark, relatively dark, and bright, and were numbered Environment1 to 12 respectively. Figure 2 The average results of 10 escape attacks for the three identities (01, 02, 03) after wearing rectangular stickers in a complex environment are given. "×" represents the baseline cosine similarity, "o" represents the adversarial cosine similarity, and different colors represent different identities.

[0146] As shown in Table 1, under normal conditions, wearing rectangular stickers or adversarial glasses frames can significantly reduce the cosine similarity of the same identity. For example, in Table 1, ID=01, wearing a rectangular sticker reduced the similarity by 0.55, and wearing adversarial glasses frames reduced it by 0.49.

[0147] As shown in Table 2, under normal circumstances, wearing rectangular stickers or adversarial glasses frames can significantly improve the cosine similarity between two different identities. For example, in Table 2, for the attack pair (01→03, Fig. 3-12), the cosine similarity increases from 0.12 to 0.42 and 0.35.

[0148] Table 1. Escape Attack Assessment under Normal Environments (Cosine Similarity)

[0149]

[0150] Table 2. Impersonation attack assessment under normal conditions (cosine similarity)

[0151]

[0152] from Figure 2 The results show that the attack is most effective in Environment 1. While the adversarial cosine similarity decreases to varying degrees in other environments, the cosine similarity remains significantly lower than the baseline even after applying the physics adversarial patch, indicating that the escape attack is still effective. Therefore, considering scaling, translation, rotation, lighting, and contrast variations during the attack process, combined with robust learning strategies, can greatly improve the robustness of physics domain attacks in complex and variable environments.

[0153] Attack assessment experiment in a black-box environment

[0154] This experiment will use multiple face recognition models trained with different backbone networks. Specifically, ResNet50+Arcface, ResNet34+Arcface, and MobileFaceNet (MobileNet+Arcface) will be used as the target face recognition models for actual evaluation, while ResNet101+Arcface will be used as the baseline model. Other settings are the same as in the previous experiment.

[0155] Figure 3 and Figure 4 The transferability evaluation under different black-box face recognition models is presented. Figure 3 Regarding escape attacks, Figure 4 Targeting impersonation attacks.

[0156] Experimental results show that this method improves the transferability of adversarial examples to a certain extent and can still effectively carry out attacks in a black-box environment.

[0157] Figure 3 Evaluation results of evasion attacks in a black-box environment.

[0158] Figure 4 Evaluation results of impersonation attacks in a black-box environment.

[0159] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0160] To verify the practical effect of this invention, three experimenters evaluated the performance of the proposed Physical Domain Attack Algorithm (SCLA) using the tenth order mean cosine similarity of escape attack and impersonation attack under normal conditions (sufficient lighting, viewing distance of 50cm, and no head rotation). The traditional Adv-Hat (rectangular sticker) and Adv-Glasses (adversarial glasses frame) physical domain attack algorithms were used as the baseline for the comparative experiment (the optimization algorithm adopted EOT, and the experimental settings were based on their papers). The proposed SCLA-Hat and SCLA-Glasses were compared with Adv-Hat and Adv-Glasses, respectively, and the actual evaluation results were visualized.

[0161] The physical domain adversarial patch generated by this invention takes the form of, for example... Figure 5 As shown.

[0162] The visualization effect of escape attack is as follows Figure 6 .

[0163] Visual representation of impersonation attacks, such as Figure 7 .

[0164] like Figure 6 In the evaluation of escape attack results, compared with the traditional Adv-Hat and Adv-Glasses, the SCLA-Hat and SCLA-Glasses methods proposed in this invention achieve lower mean cosine similarity for the same identity. For example, for the target face 01, SCLA-Hat reduces the cosine similarity by 0.55 compared to the baseline, while Adv-Hat only reduces it by 0.45; SCLA-Glasses reduces it by 0.49, while Adv-Glasses only reduces it by 0.40.

[0165] like Figure 7 In the evaluation of impersonation attack results, compared with the traditional Adv-Hat and Adv-Glasses, the SCLA-Hat and SCLA-Glasses methods proposed in this invention achieve higher mean cosine similarity for different identities. For example, for the target face 03, SCLA-Hat improves the cosine similarity by 0.30 compared to the baseline, while Adv-Hat only improves by 0.22; SCLA-Glasses improves by 0.23, while Adv-Glasses only improves by 0.18.

[0166] In summary, the Physical Domain Attack Algorithm (SCLA) provided by this invention has superior escape and impersonation attack performance compared to traditional Adv-Hat and Adv-Glasses physical domain attack algorithms, and is more conducive to evaluating the security status of face recognition systems in real-world environments.

[0167] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for security assessment of a face recognition system against sticker attacks using physical domain, characterized in that, Includes the following steps: S1, Collect the face x to be attacked and the target face x. t Injected into the face recognition system to be evaluated, ensuring that the face to be attacked can be recognized normally under normal circumstances, and that the target face cannot be attacked; S2, select the form of adversarial patch δ, and use the masking algorithm to generate digital domain face adversarial examples x. adv1 ; S3, x adv1 The second-stage adversarial face sample x is obtained after physical world changes including lighting, scaling, and rotation. adv2 ; S4 uses total variation loss and unprintable fractional loss to reduce the adversarial loss in the digital-to-physical domain conversion process; finally, iterative gradient optimization is performed using an improved Expectation-Over-The-Object (EOT) algorithm combined with cosine similarity loss to obtain the final adversarial face image x. adv And adversarial sticker delta; in order to apply the improved EOT algorithm, a self-paced learning strategy is introduced. It can be represented as: ； Where N is the number of transformations. It is a weight transformation. It is a self-synchronizing regularizer, where λ>0 is the age parameter and increases with the number of iterations. It is a transformation function for random sampling. It calculates the cosine similarity between two feature vectors; S5 prints out and cuts out the optimized adversarial stickers, then pastes them onto the corresponding positions of the real face. The face is then used to attack the target facial recognition system, and the security level of the current facial recognition system is determined based on the success rate of the attack.

2. The method for security assessment of a face recognition system based on a physical domain-based approach to resist sticker attacks, as described in claim 1, is characterized in that... In steps S2 and S3, a masking algorithm and a corresponding physical transformation module are used to obtain the first and second stage adversarial face samples: ， x adv2 =T B (x adv1 ) , Where "ο" indicates the element-wise multiplication of the corresponding matrix, T A The sticker transformation module includes: 1) Deformation of stickers when pasted on real-world surfaces, including out-of-surface bending and 3D rotation; parabolic transformation operator is used to simulate out-of-surface bending, and three-dimensional affine transformation is used to simulate 3D rotation; 2) Position perturbation, which makes it difficult to paste the stickers precisely at the designed position, such as random rotation and translation. The T B For the face transformation module: the attacker also experienced a series of different affine transformations and changes in the physical world such as lighting contrast; This includes randomly generating adversarial face images with rotation, scaling, translation, contrast, and brightness adjustments; M is a mask for resisting stickers, which adopts the form of 0-1 binary encoding; v~N (μ,σ) is random Gaussian noise with mean μ and standard deviation σ.

3. The method for security assessment of a face recognition system based on a physical domain-based approach to resist sticker attacks, as described in claim 1, is characterized in that... In step S4, the total adversarial loss during the adversarial example optimization phase is: ; in These are cosine similarity loss, total variation loss, and unprintable fraction loss, respectively. and This is the loss weight.

4. The method for security assessment of a face recognition system based on a physical domain-based approach to resist sticker attacks, as described in claim 1, is characterized in that... In step S5, when the target face recognition system identifies the adversarial sample as someone else or makes a mistake due to the cosine similarity being lower than the threshold, the attack is considered successful. The attack rate is calculated using the formula FAR=NA / N*100%, where NA represents the number of successful attacks and N represents the total number of attacks. Furthermore, attack success rates of FAR>90%, FAR>85%, FAR>80%, FAR>70%, FAR>60%, and FAR>50% indicate security levels of F, E, D, C, B, and A, respectively.

5. A system for implementing a method for security assessment of a face recognition system against physical domain sticker attacks as described in any one of claims 1-4, characterized in that, The system for assessing the security of a face recognition system by countering sticker attacks in the physical domain includes: Face capture module: used to capture the user's face images. The captured images will first undergo face detection to remove face images without faces or with poor quality. Then, a pre-trained feature extraction model will be used to extract the feature vector of each face and store it in the face database. Adversarial Example Generation Module: Initializes adversarial stickers and corresponding masks, then randomly selects a face image from the face data as the attack face, selects the form of adversarial patch, generates adversarial face examples through mask algorithm and its corresponding physical transformation module, inputs them into a pre-trained face recognition model for adversarial optimization, and obtains the final adversarial face examples and adversarial stickers after optimization. Attack Evaluation Module: Print and cut the adversarial stickers generated by the adversarial sample optimization module, then paste the stickers on specific parts of the face, and finally attack the target face recognition model or system. Calculate the attack success rate by comparing the cosine similarity between the attacking face and the target face in the face database, and output the security level of the corresponding target face recognition model or system. The aforementioned anti-sticker attack verification facial recognition system security assessment system constructs robust adversarial facial samples based on facial image data, effectively maintains a good attack rate under various physical environments, and finally calculates the attack success rate to obtain the security assessment level of the facial recognition system under test.

6. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for security assessment of a face recognition system against physical domain sticker attacks as described in any one of claims 1-4.

7. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method for security assessment of a face recognition system against physical domain sticker attacks as described in any one of claims 1-4.

8. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the system described in claim 5 for security assessment of a face recognition system against sticker attacks in the physical domain.

Images