Facial Recognition Security Testing via Adversarial Light Projections
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Facial recognition systems are vulnerable to physical adversarial attacks, such as presentation attacks using printed photos or 3D masks, and physical adversarial attacks using adversarial machine learning, which can deceive the system into misclassifying images, necessitating a robust evaluation method to assess their security.
Innovation Solution
A method and system that generate and project light patterns onto a tester to simulate adversarial conditions, using transformed images of targets and testers, processed through neural networks to evaluate the robustness of facial recognition systems, enabling systematic evaluation of their resistance to such attacks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If facial recognition systems use standard image capture and comparison methods, then the system is easy to operate and implement, but the system becomes vulnerable to presentation attacks and physical adversarial attacks
Solution Approach 1:
The patent introduces an intermediary evaluation system that acts as a mediator between the facial recognition system and adversarial attacks. This evaluation system includes a projector that displays adversarial patterns and a camera that captures the combined effect, allowing security testing without modifying the core facial recognition system. The intermediary setup enables comprehensive security evaluation while keeping the original system intact and relatively simple.
2Measurement precision
If the system projects light patterns onto physical subjects to simulate adversarial conditions, then the evaluation becomes more realistic and accurate, but the device complexity and setup requirements increase
Solution Approach 1:
The patent uses digital copying and simulation approaches where adversarial patterns are generated computationally and then projected onto physical subjects. Instead of creating complex physical adversarial objects, the system creates digital representations (adversarial images) that are then optically projected. This copying approach maintains evaluation accuracy while reducing the complexity of physical setup compared to creating actual physical adversarial devices.
3Reliability
If multiple transformed images are generated and processed through neural networks to create light patterns, then the robustness evaluation becomes more comprehensive, but the computational time and processing requirements increase
Solution Approach 1:
The patent performs preliminary processing of adversarial patterns by generating multiple transformed images and processing them through neural networks before the actual projection and evaluation phase. This preliminary action includes creating the adversarial images, applying transformations, and generating the light patterns in advance. By doing this preparation work beforehand, the system can conduct comprehensive robustness evaluations without excessive time delays during the actual testing phase.
Data Source
AI summary
Embodiments of the invention are directed to systems, methods, and devices for testing the security of a facial recognition system (FRS). A target image depicting a target person enrolled in the FRS and a tester image depicting a tester may be obtained. A plurality of transformed images may be generated from an image of the target person or the tester image. A processed tester image (e.g., one that is likely to cause the FRS to misclassify) may be identified using the plurality of transformed images, the tester image, and the target image. Data representing a light pattern can be generated using the processed tester image and the light pattern can be projected onto a second person. Another image may be captured of the second person with the light pattern as projected. This image may be provided to the FRS and a remedial action may be performed based on the corresponding output.