Sketch-and-Text Image Generation with Boundary-Edge Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in generating detailed and creative images using text-to-image models, particularly for complex products like vehicles, leading to inefficient design cycles and manual alterations.
Innovation Solution
A drawing system that iteratively generates images and sketches using models guided by sketches and text, incorporating a feedback loop to enhance design creativity by computing intersections between boundary and edge information, segmenting images, and modifying sketches based on text inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a text-to-image model is used to generate design images, then design creativity is enhanced, but the generated images lack detailed features and require manual alterations
Solution Approach 1:
The system implements a feedback mechanism where the generated image is processed through segmentation and edge detection models, and the extracted boundary and edge information is fed back to guide subsequent image generation iterations, enabling continuous refinement of detail accuracy while maintaining creativity
Solution Approach 2:
The patent introduces intermediary models (segmentation model and edge detection model) that act as mediators between the text-to-image model and the final output, extracting structural information to guide the generation process and improve detail accuracy without directly altering the creative generation capability
2Manufacturing precision
If manual alterations are made to generated designs to meet design parameters, then detail accuracy is improved, but design iterations and time increase
Solution Approach 1:
The system performs preliminary actions by automatically executing segmentation and edge detection on generated images, and using the extracted information to guide subsequent generation iterations, thereby preemptively addressing detail accuracy requirements without requiring manual intervention in each iteration cycle
3Productivity
If a text-to-image model generates images for complex products, then design efficiency is improved, but the model cannot track specific themes and produces haphazard renders
Solution Approach 1:
The system uses feedback from segmentation and edge detection models to continuously monitor and refine theme consistency throughout the generation process, ensuring that specific design themes are tracked accurately while maintaining high design efficiency through automated iterative refinement
Solution Approach 2:
The patent applies segmentation to break down complex product designs into distinct boundary and edge components, allowing the model to track and maintain specific themes for each segment while generating the overall design, thereby improving both theme consistency and handling capability for complex products
Data Source
AI summary
Systems, methods, and other embodiments described herein relate to generating images and sketches iteratively using models guided by sketches and text for a design. In one embodiment, a method includes generating an image from a sketched stroke and text inputted to a learning model. The method also includes segmenting the image to identify boundary information with a segmentation model and extracting edge information from the image using an edge model. The method also includes rendering an estimated sketch of the image by computing an intersection between the boundary information and the edge information.