Face-Swap Recognition via Head Area Occlusion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing face-swap recognition solutions lack generalization in identifying synthetic faces generated by various automatic face-swap algorithms, requiring specific models for each algorithm, which limits their effectiveness in recognizing improperly used public figures in videos.

Innovation Solution

A method involving face detection, extension of the face area to a head area, occlusion of parts of the face, and use of a pre-trained face-swap recognition model to determine if a face in an image is swapped with a preset face, improving recognition by focusing on periphery features without needing specific models for each algorithm.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If specific face-swap recognition models are developed for each automatic face-swap algorithm, then recognition accuracy for that specific algorithm is improved, but device complexity and model maintenance burden increase

Engineering Contradiction:
Improveface-swap recognition accuracyVSAvoidmodel complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies universality by designing a single face-swap recognition model that can identify synthetic faces generated by multiple different automatic face-swap algorithms simultaneously. Instead of creating separate specialized models for each algorithm, the universal model processes images from various face-swap sources using a unified architecture, thereby reducing device complexity while maintaining broad recognition capability across different algorithm types

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent segments the face area into multiple regions (e.g., left eye region, right eye region, nose region, mouth region) and applies different occlusion strategies to each segment. This segmentation allows the model to focus on specific facial features that are commonly manipulated by face-swap algorithms, improving recognition accuracy without requiring complex overall model structures

Inventive Principle:
Principle #1Segmentation

2Measurement precision

If the entire face area is analyzed for face-swap detection, then recognition coverage is improved, but recognition precision decreases due to irrelevant information

Engineering Contradiction:
Improveface-swap detection precisionVSAvoidfacial feature information
Core Design Contradiction:
Measurement precisionVSLoss of information

Solution Approach 1:

The patent extracts and isolates specific facial feature regions (eye regions, nose, mouth) from the complete face image by applying occlusion masks to other areas. This extraction process removes irrelevant background and peripheral information that could interfere with detection, allowing the model to focus computational resources on the most discriminative facial features where face-swap manipulations typically occur

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent applies local quality by treating different facial regions with different levels of analysis and occlusion. Critical regions such as the eye areas and mouth receive focused attention through selective occlusion of surrounding areas, while less critical regions are occluded to reduce noise. This localized approach enhances detection precision by optimizing the analysis depth for each facial region based on its relevance to face-swap detection

Inventive Principle:
Principle #3Local quality

3Device complexity

If a universal face-swap recognition model is used for all algorithms, then device complexity is reduced, but recognition accuracy for specific algorithms decreases

Engineering Contradiction:
Improvemodel structure complexityVSAvoidalgorithm-specific recognition accuracy
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent employs parameter changes by adjusting occlusion ratios and occlusion patterns based on the specific face-swap algorithm being detected. The universal model incorporates adaptive parameters that modify the degree and pattern of facial region occlusion according to the characteristics of different algorithms, allowing it to maintain high recognition accuracy across various algorithms while preserving a single unified model structure

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS11403799B2Method and apparatus for recognizing face-swap, device and computer readable storage medium
Publication Date: 2022.08.02 BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD
  • US11403799B2 patent drawing
  • US11403799B2 patent drawing
  • US11403799B2 patent drawing

AI summary

A method and apparatus for recognizing face-swap, a device and a computer readable storage medium are provided, and relates to the field of computer technology. An implementation of the method includes: performing face detection on a to-be-recognized image to determine a face area in the to-be-recognized image; extending the face area to a head area in the to-be-recognized image, to generate an image of the head area; occluding at least one part of the face area in the image of the head area, to obtain an occlusion image; and determining, based on the occlusion image and a pre-trained face-swap recognition model, a face-swap recognition result of the to-be-recognized image, the face-swap recognition model being used to determine whether a face in the to-be-recognized image is swapped with a preset face.