Portrait Effect Generation with Shared Encoder and Multiple Decoders
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computational methods for generating portrait effects on non-professional electronic devices, such as smartphones, are time-consuming and result in poor-quality images due to high computational complexity, with separate neural networks for each effect leading to delays in switching between effects like Bokeh, High Key, and Low Key.
Innovation Solution
A method and system using an encoder and multiple decoders within a single deep neural network model to extract both low-level and high-level features from images, enabling the generation of multiple portrait effects in a single inference, including Bokeh and studio lighting effects, by pre-learning features corresponding to these effects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If separate neural networks are used for each portrait effect, then each effect can be processed independently, but the computational complexity increases and processing time is extended
Solution Approach 1:
The patent applies a single universal neural network model that can generate multiple different portrait effects (Bokeh, High Key, Low Key, etc.) by processing the same input image through different effect-specific decoders. This multi-functional approach eliminates the need for separate neural networks for each effect, thereby reducing computational complexity while maintaining the ability to generate diverse portrait effects.
2Reliability
If separate neural networks are used for each portrait effect, then each effect can be optimized independently, but the processing time increases and switching between effects becomes delayed
Solution Approach 1:
The patent merges multiple separate neural networks into a single unified model that processes all portrait effects simultaneously. The encoder processes the input image once and extracts features that are then distributed to multiple decoders, each responsible for a specific effect type. This combining approach enables parallel processing of different effects, significantly reducing processing time and enabling seamless switching between effects while maintaining high generation quality.
3Ease of manufacture
If computational methods are used to achieve portrait effects on non-professional devices, then the effects can be generated without professional hardware, but the image quality becomes poor compared to professional cameras
Solution Approach 1:
The patent replaces the mechanical/optical portrait effect generation capability of professional DSLR cameras with a computational neural network system. Instead of relying on physical aperture control and optical lenses to create Bokeh and studio lighting effects, the system uses deep learning models to synthesize these effects computationally, achieving professional-quality results on standard mobile devices with regular cameras.
4Adaptability or versatility
If multiple separate neural networks are used for different portrait effects, then each network can be specialized, but the overall system complexity and resource consumption increase
Solution Approach 1:
The patent implements a single universal neural network that handles multiple portrait effects through shared encoder-decoder architecture. The encoder extracts common features from the input image that are reused across different effect types, and the decoders apply effect-specific transformations. This universal design reduces computational resource consumption compared to running multiple separate specialized networks, while still maintaining the capability to generate diverse portrait effects.
Data Source
AI summary
A method and a system for generation of a plurality of portrait effects in an electronic device are provided. The method includes feeding an image captured from the electronic device into an encoder pre-learned using a plurality of features corresponding to the plurality of portrait effects and extracting, using the encoder, at least one of one or more low level features and one or more high level features from the image. The method includes generating, for the image, one or more first portrait effects of the plurality of portrait effects by passing the image through one or more first decoders. The method includes generating, for the image, one or more second portrait effects of the plurality of portrait effects by passing the image through one or more second decoders, wherein each of the one or more first portrait effect, and the one or more second portrait effects is generated in a single inference.