Shared-Encoder Portrait Effects for Fast Mobile Switching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for generating portrait effects on non-professional electronic devices, such as smartphones and tablets, are computationally complex, time-consuming, and result in poor-quality effects, with significant delays in switching between effects like Bokeh, High Key, and Low Key.
Innovation Solution
A method and system using an encoder and multiple decoders within a single DNN model to extract and generate portrait effects in a single inference, leveraging low and high-level features to process images efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple separate neural networks are used for different portrait effects (Bokeh, High Key, Low Key), then each effect can be processed with dedicated complexity, but the overall processing time increases and switching between effects becomes delayed
Solution Approach 1:
The patent merges multiple separate neural networks into a single unified neural network that can generate multiple portrait effects (Bokeh, High Key, Low Key) simultaneously. The encoder extracts features from the input image once, and multiple decoders process these shared features to produce different effects in parallel, eliminating the need for sequential processing of separate networks.
Solution Approach 2:
The unified neural network is designed with multi-functionality to handle multiple portrait effects through a single model. The encoder component serves all effect types universally by extracting general image features, while effect-specific decoders apply specialized transformations. This universal architecture allows instantaneous switching between effects without reprocessing.
2Measurement precision
If multiple separate neural networks are used for different portrait effects, then each effect can be optimized independently, but memory utilization increases due to redundant feature extraction
Solution Approach 1:
The patent combines multiple neural networks into a shared architecture where the encoder is common to all effect types. This merging eliminates redundant feature extraction operations, as the same encoder processes the input image once and shares its output with multiple decoders, significantly reducing memory utilization compared to independent networks.
Solution Approach 2:
The unified network is segmented into an encoder portion and multiple decoder portions. The encoder handles universal feature extraction, while decoders handle effect-specific processing. This segmentation allows efficient resource allocation where memory-intensive feature extraction is performed once, and lighter effect-specific transformations are applied separately.
3Manufacturing precision
If computational methods with multiple modules are used for portrait effects, then processing can be detailed and precise, but the computational complexity increases significantly
Solution Approach 1:
The patent merges multiple computational modules into a unified neural network architecture. Instead of separate modules for depth estimation, segmentation, matting, and blending that each require independent processing, the unified network integrates these functions into a single computational flow with shared features, reducing overall computational complexity.
Solution Approach 2:
The unified neural network enables continuous processing where feature extraction and effect generation occur in a single uninterrupted inference. The encoder continuously provides features to multiple decoders simultaneously, maintaining continuous useful action without the interruptions and repeated processing that occur in modular sequential approaches.
Data Source
AI summary
A method and a system for generation of a plurality of portrait effects in an electronic device are provided. The method includes feeding an image captured from the electronic device into an encoder pre-learned using a plurality of features corresponding to the plurality of portrait effects and extracting, using the encoder, at least one of one or more low level features and one or more high level features from the image. The method includes generating, for the image, one or more first portrait effects of the plurality of portrait effects by passing the image through one or more first decoders. The method includes generating, for the image, one or more second portrait effects of the plurality of portrait effects by passing the image through one or more second decoders, wherein each of the one or more first portrait effect, and the one or more second portrait effects is generated in a single inference.