Stroke Sequence Planning for Complex Sketch and Paint Recreation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle with accurately recreating intricate details of complex artworks due to challenges in representing and sequencing strokes, especially in complex sketches involving shading and textures, and face high computational demands.
Innovation Solution
A method and system that converts an input image to a sketch, constructs stroke sequences based on vector curves, organizes strokes into clusters, determines a shortest path through these clusters, and generates a paint sequence to recreate the image, using techniques like TSP and RGB clustering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing methods are used to extract and sequence strokes from complex artworks, then stroke-level information can be obtained, but the accuracy deteriorates over complex sketches involving shading and textures
Solution Approach 1:
The patent segments the image processing into multiple independent streams: one stream processes the sketch image to extract strokes, while another stream processes the original image to obtain corresponding stroke information. This segmentation allows each stream to specialize in specific tasks, improving overall accuracy for complex sketches with shading and textures by handling different aspects of stroke extraction separately rather than attempting to process everything in a single unified manner.
2Reliability
If neural painting techniques employing reinforcement learning are used to generate stroke sequences, then non-photo-realistic image recreation can be achieved, but computational demands increase significantly
Solution Approach 1:
The patent replaces the computationally intensive reinforcement learning mechanism with a more efficient stream-based processing approach. Instead of using neural networks with reinforcement learning to generate stroke sequences, the system uses parallel processing streams that independently analyze sketch and original images, then combine results through a simplified sequencing mechanism. This substitution dramatically reduces computational demands while maintaining image recreation quality.
3Extent of automation
If deep reinforcement learning is used for neural painting, then stroke sequences can be generated, but the process becomes computationally demanding and lacks inherent sequence order
Solution Approach 1:
The patent performs preliminary analysis in separate processing streams before final stroke sequence generation. Each stream independently processes specific aspects of the image data (sketch features, original image features, color information) and prepares intermediate representations that facilitate straightforward sequence assembly. This preliminary action breaks down the complex task into manageable steps, eliminating the need for computationally intensive reinforcement learning while ensuring proper sequence order through the structured multi-stream architecture.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Existing sketch generation techniques have disadvantages such as less accuracy over complex sketches, incapable of scaling for complex sketches involving shading and textures, and high computational demands of deep reinforcement learning and lack an inherent sequence order while generating strokes. Embodiments disclosed herein provide a method and system which converts an input image to a sketch, and further obtains an associated sequence of strokes. Further, a sketch sequencing to orchestrate the sequence of strokes is performed, during which a stroke sequence is generated for a constructed sketch of strokes. Based on the generated stroke sequence, the input image is recreated. Further, a paint sequence for the recreated image is generated, wherein by executing the paint sequence, the recreated image is painted to match color of the input image.