Multi-Modal Design System for Sketch Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing design systems using learning models for generating product designs face challenges in efficiently producing robust and viable designs due to difficulties in extracting and understanding salient features from design sketches, and relying on parsable inputs that may lead to erroneous outputs and increased development cycles.
Innovation Solution
A design system that utilizes a transformer model and a learning model to generate designs by processing text and sketch-based inputs, which includes estimating analogical suggestions, generating images, and manipulating modified sketches to refine design parameters, thereby reducing development cycles and improving design accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If learning models rely on parsable inputs to augment design sketches, then design goals can be conformed, but difficulties remain in extracting and understanding salient features from design sketches
Solution Approach 1:
The patent combines multiple input modalities (text prompts, sketch-based inputs, and parsed design parameters) into a unified multi-modal learning framework. This integration allows the system to leverage the strengths of each input type while compensating for their individual weaknesses, thereby improving both feature extraction accuracy and design understanding reliability simultaneously
2Adaptability or versatility
If learning models generate visual prototypes from text prompts, then design exploration is enhanced, but erroneous outputs occur with subjective design parameters
Solution Approach 1:
The patent introduces an intermediary processing layer that translates subjective design parameters (e.g., 'relaxing') into objective, quantifiable features through multi-modal learning. This intermediary mechanism bridges the gap between subjective human perception and objective machine processing, enabling accurate generation of designs with subjective requirements
3Manufacturing precision
If learning models iterate to enhance generated images, then design accuracy improves, but development cycles are extended
Solution Approach 1:
The patent performs preliminary processing of design parameters and sketch features before the main image generation process. By pre-extracting and organizing salient features from sketches and pre-processing text prompts, the system reduces the computational burden during iterative refinement, thereby maintaining high design accuracy while reducing overall development cycle time
Data Source
AI summary
Systems, methods, and other embodiments described herein relate to generating designs using learning models for analogics that process text and sketch-based inputs. In one embodiment, a method includes estimating analogical suggestions using a transformer model for a text prompt having design parameters. The method also includes generating an image using a learning model for an expression selected from the analogical suggestions and a sketched stroke inputted. The method also includes manipulating a modified sketch by the learning model and the modified sketch is derived from a sketched conversion of the image by an edge model.