Guided Neural Style Transfer Using Multi-Map Image Guidance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current graphics processing methods struggle with high performance image processing tasks, particularly in non-photorealistic or photorealistic rendering, due to a lack of effective guidance in Deep Neural Network (DNN) based approaches, making them difficult for end users to utilize effectively.
Innovation Solution
A guided neural network model is employed for image processing, utilizing neural guidance maps, semantic guidance maps, and position guidance maps to enhance the style transfer task, with compact convolution operations to improve efficiency and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If DNN based methods are used for image processing, then ease of operation is improved, but guidance quality deteriorates
Solution Approach 1:
The patent introduces guidance maps as intermediary structures that bridge the gap between simple DNN operations and high-quality guidance. The guidance map generator creates intermediate representations (neural guidance maps, semantic guidance maps, position guidance maps) that provide structured guidance information to the style transfer network, enabling both ease of use and high guidance quality simultaneously
2Manufacturing precision
If traditional mathematical models are used for style transfer, then manufacturing precision is improved, but ease of operation deteriorates
Solution Approach 1:
The patent replaces complex mathematical modeling mechanisms with neural network-based approaches. Instead of requiring users to manually construct mathematical models for style and lighting, the system uses DNNs to automatically learn and capture these relationships from data, substituting the mechanical/mathematical modeling process with a more user-friendly neural computation approach
3Manufacturing precision
If guided neural network model is implemented, then guidance quality is improved, but device complexity increases
Solution Approach 1:
The patent segments the complex image processing task into distinct functional modules: a guidance map generator that creates different types of guidance maps (neural, semantic, position), and a style transfer network that consumes these guidance maps. This segmentation allows each module to be optimized independently and simplifies the overall system architecture despite the increased functionality
Data Source
AI summary
The present disclosure provides an apparatus and method of guided neural network model for image processing. An apparatus may comprise a guidance map generator, a synthesis network and an accelerator. The guidance map generator may receive a first image as a content image and a second image as a style image, and generate a first plurality of guidance maps and a second plurality of guidance maps, respectively from the first image and the second image. The synthesis network may synthesize the first plurality of guidance maps and the second plurality of guidance maps to determine guidance information. The accelerator may generate an output image by applying the style of the second image to the first image based on the guidance information.