Real-Time Facial Mask Overlay via Remote Image Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current facial detection technologies lack the ability to dynamically apply customizable media effects, such as facial masks, in real-time video applications, especially when pre-generated effects are not available, and do not adequately address data privacy concerns.
Innovation Solution
A system that allows users to generate and apply media effects, like facial masks, by searching and filtering publicly available images, using a client-server architecture, where image searching, filtering, and selection are performed remotely to conserve local resources, and ensures data privacy through user consent and secure storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If image searching and filtering are performed locally on the client device, then media effects can be applied in real-time, but local computational resources and processing power are consumed
Solution Approach 1:
The patent introduces a server as an intermediary between the client device and the image database. The server performs image searching, filtering, and selection operations remotely, returning only the necessary target images to the client. This mediator approach allows real-time media effect generation without consuming significant local computational resources, as the heavy processing occurs on the server side.
2Adaptability or versatility
If publicly available images are searched and used for media effects, then customizable facial masks can be generated, but data privacy concerns arise
Solution Approach 1:
The patent implements a user consent mechanism where the system requests and receives user permission before collecting or processing personal data. The server obtains explicit authorization from users whose images may be used for media effects, and provides feedback about data usage purposes. This feedback loop ensures data privacy is maintained while still enabling customizable media effects through publicly available images.
3Manufacturing precision
If multiple images are processed and filtered to find suitable target images, then higher quality media effects can be achieved, but processing time increases
Solution Approach 1:
The patent applies preliminary filtering criteria to quickly eliminate unsuitable images before detailed processing. The server performs initial filters on image characteristics (such as face detection, image quality metrics) to identify promising candidates, then applies more sophisticated filtering only to these pre-selected images. This preliminary action reduces the overall processing time while maintaining high media effect quality by focusing computational effort on the most suitable images.
Data Source
AI summary
Exemplary embodiments relate to the application of media effects such as facial mask overlays, to visual data (such as a video or photo). Publicly-available images may be found and mapped to a mask. In the mapping process, a user may type in the name of a celebrity or public figure, and a system may perform a public image search. In some embodiments, candidate images may be filtered in order to remove images unsuitable for use in masks. Typically, only a single forward-facing image is required for mapping. However, multiple images may be used to provide different angles and allow the user to turn their head while the mask is applied. Mask generation may involve: extracting facial features from the image; mapping the facial features to the user's video; blending/recoloring of either or both of the image or the person's face; and applying the mask in real-time/on the fly.