LLM Editable Design Generation with Retrieval-Based Asset Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital design applications struggle to automatically create or edit designs effectively, often requiring users to manually select templates and perform numerous edits, leading to substandard results and increased cognitive burden.
Innovation Solution
A retrieval-based design generation system using a pre-trained large language model (LLM) that processes input prompts to generate editable designs by selecting design assets from a database, incorporating multimodal data understanding and generating model representation data for rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users manually select templates and perform numerous edits in existing digital design applications, then they can create custom designs, but the process requires increased user effort and cognitive burden leading to substandard results
Solution Approach 1:
The system enables self-service by allowing the LLM to automatically generate designs based on text prompts without requiring manual template selection or editing. The model autonomously selects design elements, arranges them, and creates complete designs, freeing users from tedious manual operations while maintaining creative control through prompt engineering.
Solution Approach 2:
The patent replaces the mechanical manual editing process with an automated LLM-based system. Instead of users mechanically selecting and editing templates, the system uses natural language processing and generative AI to automatically create designs, substituting human manual operations with intelligent automation that understands design principles and asset relationships.
2Adaptability or versatility
If existing applications provide multiple design templates and categories, then users have more design options, but the complexity of navigating and selecting appropriate templates increases cognitive burden
Solution Approach 1:
The system extracts the complexity of template navigation and selection from the user interface. Instead of presenting users with multiple templates and categories to manually browse, the LLM internally processes the design requirements and automatically selects appropriate design elements, extracting the complex decision-making process from the user-facing interface and handling it autonomously.
Solution Approach 2:
The LLM serves as an intermediary between the user's simple text prompt and the complex design generation process. It translates high-level design intentions into specific design element selections and arrangements, mediating between user simplicity and system complexity without requiring users to directly interact with the complex template library.
3Ease of operation
If the system automatically generates designs using LLM, then user effort is reduced, but the ability to edit individual design elements may be compromised
Solution Approach 1:
The system segments the design into discrete editable elements including text, images, videos, and other media components. Each element is independently generated and can be individually selected and modified by users after generation, allowing fine-grained editing while maintaining the benefits of automated creation. The segmentation enables both automation at the macro level and precision editing at the micro level.
Data Source
AI summary
Systems and methods for generating an editable design using an auto-regressive pre-trained large language model (LLM) are disclosed. The method includes: receiving a prompt to generate the editable design; sequentially generating a set of tokens of model representation data for the editable design, each token in the set of tokens defining an attribute of the editable design; for each token in the set of tokens, determining whether the token is a predicted special token associated with a design asset or a non-special token; upon determining that the token is a non-special token, providing the non-special token as an input to the LLM to generate a next token in the set of tokens; upon determining that the token is a predicted special token: replacing the predicted special token with a replacement special token associated with a design asset stored in a design asset library; and providing the replacement special token as the input to the LLM to generate the next token in the set of tokens.