Selfie Video Personalization with Preprocessed Actor Face Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing messaging applications lack the ability to create personalized videos that integrate a user's self-image seamlessly with pre-recorded stock videos, limiting the creativity and engagement in video sharing.
Innovation Solution
A system and method for generating personalized videos by capturing a user's face image, analyzing facial parameters, and replacing the face of actors in pre-recorded videos with the user's face, using a parametric face model and deep neural networks to ensure photorealistic integration, while allowing for additional modifications like adding accessories or changing skin tone.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a user's face image is captured and integrated with stock videos using parametric face models and deep neural networks, then the personalization and photorealistic quality of videos is improved, but the device complexity and processing requirements increase
Solution Approach 1:
The system performs preliminary actions by pre-processing stock videos to identify and segment actor faces, creating masks and bounding boxes in advance. This preparation work is done before the user actually creates their personalized video, so when the user wants to create a video, the system already has pre-processed templates ready to quickly integrate the user's face image without requiring complex real-time processing
Solution Approach 2:
The system uses copying by creating and storing parametric face models from stock video actors. These models include extracted facial features, expressions, and characteristics that can be copied and applied to the user's face image. The deep neural networks create reusable templates that can be rapidly instantiated multiple times without re-processing the original stock videos each time
2Loss of time
If facial parameters are analyzed and face replacement is performed in real-time, then the user interaction speed is improved, but the measurement precision and processing accuracy may be compromised
Solution Approach 1:
The system performs preliminary face detection, landmark identification, and parameter extraction on stock video actors before user interaction. This pre-processing creates ready-to-use facial models and masks that can be quickly matched and applied during user video creation, avoiding the need for time-consuming real-time analysis while maintaining high precision through pre-computed accurate measurements
Solution Approach 2:
The system replaces complex real-time mechanical processing with pre-computed data structures. Instead of performing heavy facial analysis computations during user interaction, the system substitutes these with pre-generated facial models, masks, and parameter sets that were computed in advance using deep neural networks, achieving both speed and accuracy
Data Source
AI summary
Provided are systems and methods for forming personalized videos including a self-image of a user. An example method includes displaying a plurality of videos to a user, where at least one video of the plurality of videos is generated based on a source video featuring a subject facing a recording device, where the subject is associated with a marker indicating a location for insertion of a user-specific content, providing a user interface configured to enable selection of a video from the plurality of videos, and upon determining that the user has selected the video, generating a personalized video by combining the selected video with the user-specific content.