Multi-Person Face Replacement in Messaging Videos
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current messaging applications lack the capability to perform sophisticated video editing, such as replacing faces with those of users, requiring third-party software and limiting user interaction.
Innovation Solution
A method and system for generating personalized videos on computing devices, allowing users to replace faces in pre-generated videos with their own faces, using facial expression analysis and synthesis to create personalized videos in real-time, which can be sent via communication chats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If third-party video editing software is used to replace faces in videos, then sophisticated editing capability is achieved, but device complexity and ease of operation deteriorate
Solution Approach 1:
The patent merges face replacement functionality directly into the messaging application, combining video editing capabilities with communication features. This integration eliminates the need for separate third-party editing software while providing sophisticated face replacement functionality within the messaging ecosystem.
Solution Approach 2:
The system automatically detects faces in videos and enables users to replace them with their own faces through simple user interface interactions. The automated face detection and synthesis processes reduce the operational burden on users, making sophisticated editing accessible without requiring manual frame-by-frame processing.
2Adaptability or versatility
If third-party video editing software is used to replace faces in videos, then sophisticated editing capability is achieved, but ease of operation deteriorates
Solution Approach 1:
The system automatically detects faces in videos and enables users to replace them with their own faces through simple user interface interactions. The automated face detection and synthesis processes reduce the operational burden on users, making sophisticated editing accessible without requiring manual frame-by-frame processing.
Solution Approach 2:
The messaging application is enhanced to perform multiple functions including video transmission, face detection, face replacement synthesis, and communication. This multi-functionality consolidates what would traditionally require separate applications into a single unified platform, improving ease of operation.
3Adaptability or versatility
If real-time face replacement synthesis is implemented, then user interaction and creativity are enhanced, but processing time and computational resources increase
Solution Approach 1:
The system pre-processes and stores facial feature data, expressions, and synthesis models before the actual video generation is needed. By preparing facial templates and expression mappings in advance, the real-time face replacement process is accelerated, reducing the computational burden during actual video generation.
Solution Approach 2:
The system uses parameter-based face synthesis where facial expressions and features are represented as adjustable parameters rather than processing entire video frames. By manipulating facial parameter vectors and expression weights, the system achieves real-time synthesis with reduced computational complexity compared to pixel-level processing.
Data Source
AI summary
Provided are systems and methods for providing personalized videos featuring multiple persons. An example method includes providing an option enabling a user to select a video that includes at least one frame having a target face and a further target face, receiving an image of a source face associated with the user and a further image of a further source face, modifying the image of the source face to generate a first image of a modified source face that adopts a facial expression of the target face, modifying the further image of the further source face to generate a second image of a modified further source face that adopts a further facial expression of the further target face, and replacing, in the at least one frame, the target face with the first image and the further target face with the second image to generate a modified personalized video.