Style-Transfer Neural Network with NPR Generator for Stroke Imitation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image-style-conversion systems struggle to accurately and consistently imitate artistic strokes, particularly in rendering subtle strokes and varying artistic styles, and often require labor-intensive paired-ground-truth drawings for training neural networks.
Innovation Solution
Integration of a non-photorealistic rendering (NPR) generator with a style-transfer neural network to generate stylized images that resemble specific stroke styles without the need for paired-ground-truth images, allowing for the selection of multiple styles and flexible input types, including natural photographs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional image-style-conversion systems use paired-ground-truth drawings for training neural networks, then the accuracy of stroke imitation improves, but the time and labor required for data preparation increases significantly
Solution Approach 1:
The system uses style transfer to copy the artistic style characteristics from reference images to generate stylized output images, eliminating the need for manual paired-ground-truth drawings. The neural network learns style representations and applies them to convert input images into target artistic styles automatically.
Solution Approach 2:
The system enables self-service by allowing the neural network to automatically learn and imitate artistic styles without requiring manual annotation or pairing of ground-truth drawings. The style transfer process is autonomous, taking only input images and style references to generate stylized outputs.
2Illumination intensity
If conventional systems use rendering techniques with gradients to make strokes more visible, then stroke visibility improves, but the realism of stroke depiction deteriorates
Solution Approach 1:
The system applies local quality by preserving the natural variation in stroke characteristics across different regions of the image. Instead of uniformly enhancing all strokes with gradients, the neural network selectively renders strokes with appropriate visibility and realism based on local artistic style characteristics and image content.
Solution Approach 2:
The system uses parameter changes by dynamically adjusting stroke rendering parameters through neural network learning. The network learns optimal stroke depiction parameters from style references and applies them to generate realistic yet visible strokes, avoiding the need for fixed gradient-based enhancement that compromises realism.
3Measurement precision
If conventional image-style-conversion systems are designed to imitate a specific artistic style, then the accuracy of style imitation improves, but the flexibility to handle different styles and input types decreases
Solution Approach 1:
The system implements universality by designing a neural network architecture that can handle multiple artistic styles and input image types through a unified style transfer framework. The network learns style representations that can be applied across different styles (pencil sketches, paintings, drawings) and input types (photographs, illustrations, sketches) without requiring style-specific models.
Solution Approach 2:
The system uses dynamics by making the style transfer process adaptive and flexible. The neural network dynamically adjusts style application based on the input image characteristics and selected target style, allowing real-time switching between different artistic styles and input types while maintaining accurate style imitation for each combination.
4Manufacturing precision
If conventional systems use vector lines from input images to resemble pencil strokes, then the structural accuracy improves, but the ability to convert natural images without distinct lines deteriorates
Solution Approach 1:
The system replaces the mechanical approach of extracting vector lines with a neural network-based approach. Instead of requiring distinct lines to be converted to vectors, the neural network directly learns to generate pencil stroke representations from pixel-based input images, preserving structural accuracy while accepting any image type including natural photographs without distinct lines.
Data Source
AI summary
This disclosure relates to methods, non-transitory computer readable media, and systems that integrate (or embed) a non-photorealistic rendering (“NPR”) generator with a style-transfer-neural network to generate stylized images that both correspond to a source image and resemble a stroke style. By integrating an NPR generator with a style-transfer-neural network, the disclosed methods, non-transitory computer readable media, and systems can accurately capture a stroke style resembling one or both of stylized edges or stylized shadings. When training such a style-transfer-neural network, the integrated NPR generator can enable the disclosed methods, non-transitory computer readable media, and systems to use real-stroke drawings (instead of conventional paired-ground-truth drawings) for training the network to accurately portray a stroke style. In some implementations, the disclosed methods, non-transitory computer readable media, and systems can either train or apply a style-transfer-neural network that captures a variety of stroke styles, such as different edge-stroke styles or shading-stroke styles.