Adaptive Model Mask for Real-Time Foreground Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image processing methods struggle to extract the foreground human object from selfie images in real-time due to the computational time required by traditional identification algorithms, leading to delays in applying digital visual effects.
Innovation Solution
An adaptive model-based human segment feature is developed, utilizing a motion sensor and camera pose data to adjust pre-defined masks into adaptive masks, enabling efficient real-time extraction of the foreground object during image capture, previewing, and editing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional identification algorithms are used to extract the foreground human object, then the extraction accuracy is improved, but the computation time increases causing real-time processing to fail
Solution Approach 1:
The patent segments the image processing task into two parts: (1) using a pre-defined model mask for rapid foreground extraction, and (2) using traditional identification algorithms only for refinement. This segmentation allows the system to achieve both real-time processing speed and acceptable extraction accuracy by applying complex algorithms only where necessary rather than to the entire image.
Solution Approach 2:
The patent applies preliminary action by using a pre-defined model mask before applying traditional identification algorithms. The model mask pre-identifies the general location and shape of the foreground object, which then guides the subsequent refinement process. This preliminary segmentation reduces the computational burden of the detailed analysis that follows.
2Reliability
If traditional identification algorithms are used to extract the foreground human object, then the extraction completeness is improved, but the processing speed decreases causing delays in visual effect application
Solution Approach 1:
The processing pipeline is segmented into a fast initial extraction phase using the model mask and a slower refinement phase using traditional algorithms. This ensures that most of the extraction is completed rapidly while maintaining completeness through the subsequent refinement step that fills in any missed details.
Solution Approach 2:
The patent implements a dynamic processing approach where the system adapts between two processing modes: a fast mode using the model mask for real-time preview and visual effects, and a more thorough mode using traditional algorithms when complete extraction is critical. This dynamic switching allows the system to maintain productivity while ensuring reliability when needed.
3Ease of operation
If real-time extraction of foreground object is implemented, then the user interface responsiveness is improved, but the computational complexity increases
Solution Approach 1:
The patent introduces a model mask as an intermediary between the raw image input and the final extracted foreground object. This intermediary structure simplifies the computational task by providing a pre-organized framework that guides the extraction process, reducing the overall computational complexity while enabling real-time processing and maintaining UI responsiveness.
Data Source
AI summary
An image segment method is provided in this disclosure. The method is suitable for an electronic apparatus including a first camera and a motion sensor. The method includes steps of: providing at least one pre-defined model mask; fetching pose data from the motion sensor, the pose data being related to an orientation or a position of the first camera; adjusting one of the at least one pre-defined model mask into an adaptive model mask according to the pose data; and, extracting an object from an image captured by the first camera according to the adaptive model mask.