Mobile Stylized Image Generation With Distilled Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face inefficiencies in generating high-resolution stylized images on mobile devices due to processing resource limitations, often requiring centralized servers and being limited to low-resolution outputs on handheld devices.
Innovation Solution
A lightweight neural network, generated through model distillation of a generative adversarial network, is deployed on mobile devices to create high-resolution artistic images in real time, utilizing constraints like content, style, and blur to enhance performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional neural networks are deployed on centralized servers to generate stylized images, then image processing accuracy is improved, but device complexity and resource requirements increase
Solution Approach 1:
The patent extracts the essential stylization functionality from complex conventional neural networks and implements it through a simplified color space conversion process. The system separates the stylization task into independent color channel operations in the LAB color space, eliminating the need for large-scale neural network deployments on centralized servers while maintaining processing accuracy.
Solution Approach 2:
The patent creates a simplified computational model that copies only the necessary stylistic transformation logic from complex neural networks. By representing style transfers as color space conversions rather than full neural network inferences, the system replicates essential functionality with dramatically reduced computational requirements and device complexity.
2Manufacturing precision
If time- and computing-resource-intensive methods are used to create stylized images, then manufacturing precision is improved, but productivity deteriorates
Solution Approach 1:
The patent replaces the mechanical computational process of conventional neural network inference with an optimized color space transformation approach. By working in the LAB color space and applying targeted color channel adjustments, the system achieves high-quality stylization with significantly reduced computational time and resource consumption, thereby improving processing speed without sacrificing stylization quality.
Solution Approach 2:
The patent changes the parameter representation from raw RGB values to LAB color space parameters, enabling more efficient computation. This parameter transformation allows the system to achieve precise color-based stylization through simpler mathematical operations rather than intensive neural network computations, resolving the contradiction between processing speed and stylization quality.
3Ease of operation
If conventional systems are implemented on handheld devices, then ease of operation is improved, but manufacturing precision deteriorates due to resource constraints
Solution Approach 1:
The patent employs a lightweight computational approach that uses minimal processing resources, analogous to using simple, inexpensive tools rather than complex equipment. The color space conversion method requires only basic arithmetic operations on pixel values, making it suitable for execution on resource-constrained handheld devices while maintaining acceptable stylization quality, thus enabling portability without severe precision loss.
4Manufacturing precision
If high-resolution stylized images are generated on mobile devices, then manufacturing precision is improved, but use of energy increases
Solution Approach 1:
The patent applies partial action by focusing computational effort only on the color channels that contribute most to stylistic transformation in the LAB color space. Rather than processing all pixel data through intensive algorithms, the system selectively modifies specific color channels (particularly the a and b channels) to achieve stylization, thereby reducing overall energy consumption while maintaining high-resolution output quality.
Data Source
AI summary
Methods, systems, and non-transitory computer readable media are disclosed for generating artistic images by applying an artistic-effect to one or more frames of a video stream or digital images. In one or more embodiments, the disclosed system captures a video stream utilizing a camera of a computing device. The disclosed system deploys a distilled artistic-effect neural network on the computing device to generate an artistic version of the captured video stream at a first resolution in real time. The disclosed system can provide the artistic video stream for display via the computing device. Based on an indication of a capture event, the disclosed system utilizes the distilled artistic-effect neural network to generate an artistic image at a higher resolution than the artistic video stream. Furthermore, the disclosed system tunes and utilizes an artistic-effect patch generative adversarial neural network to modify parameters for the distilled artistic-effect neural network.