Visual Prompting for Automatic Design Deliverable Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods using generative AI for design deliverables often result in outputs that differ from the user's intent, requiring repetitive prompt modifications and image format limitations, which are difficult for non-experts to navigate.
Innovation Solution
A visual interaction method between users and generative AI, allowing input of prompts in image format, automatic review of deliverables, and modification of design elements through a code editing interface, with semantic analysis for prompt specificity and compliance with design and security guidelines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If users input detailed text prompts to specify design deliverables, then the precision of design intent is improved, but the ease of operation deteriorates for non-expert users
Solution Approach 1:
The patent introduces an image upload interface as an intermediary between the user and the generative AI system. Instead of requiring users to craft detailed text prompts, they can directly upload reference images that convey their design intent. The system then processes these images to generate appropriate design deliverables, eliminating the need for users to understand complex prompt engineering while maintaining high precision in capturing their intent.
Solution Approach 2:
The patent replaces the text-based prompt input mechanism with an image-based input mechanism. By substituting the mechanical process of text composition and prompt engineering with a visual image upload process, the system makes the interface more intuitive and accessible to non-expert users while preserving or enhancing the precision of design intent communication.
2Manufacturing precision
If users repeatedly modify prompts to achieve desired deliverables, then the manufacturing precision of design output is improved, but the loss of time increases significantly
Solution Approach 1:
The patent enables users to perform preliminary action by uploading reference images that pre-encode their design intent. Instead of iteratively modifying text prompts to gradually approach the desired outcome, the user's initial image upload already contains the complete design specification, allowing the generative AI to produce accurate results in a single pass or with minimal iterations.
Solution Approach 2:
The patent utilizes image copying as a direct representation of design intent. By allowing users to upload existing images as prompts, the system captures the exact visual characteristics and design elements the user desires, eliminating the need for repeated textual descriptions and modifications to achieve the desired output precision.
3Productivity
If generative AI provides completed image format deliverables, then the productivity of design output is improved, but the ease of modification deteriorates
Solution Approach 1:
The patent introduces dynamic modification capabilities that allow users to interact with and adjust generated images after initial creation. The system enables post-generation editing where users can modify specific elements, parameters, or characteristics of the delivered images, transforming the static delivery process into a dynamic, iterative refinement process that maintains both high productivity and ease of modification.
Solution Approach 2:
The patent applies segmentation by allowing users to modify specific portions or elements of the generated images rather than requiring complete regeneration. This enables targeted adjustments to particular design elements while preserving the rest of the image, making the modification process more efficient and easier to control while maintaining the overall productivity benefits of AI-generated content.
4Reliability
If the system provides comprehensive review and modification interfaces, then the reliability of design deliverables is improved, but the device complexity increases
Solution Approach 1:
The patent implements automated review mechanisms that provide feedback on generated deliverables to ensure they meet design guidelines and requirements. The system automatically evaluates the output images against predefined criteria and provides guidance for improvements, enhancing reliability without requiring complex manual review interfaces, as the feedback loop is integrated into the generation process itself.
Data Source
AI summary
A method and system for automatically generating design deliverables are provided. According to the aforementioned and other embodiments of the present disclosure, a method for automatically generating design deliverables performed by a computing system is provided. The method may comprise receiving a first prompt from a user terminal and transmitting the first prompt to a generative artificial intelligence (AI) system, transmitting at least some of a plurality of images associated with the first prompt to the user terminal, receiving a user's selection input for at least one of the plurality of images associated with the first prompt from the user terminal and transmitting a first image corresponding to the user's selection input to the generative AI system and receiving a second image generated by the generative AI system based on the first image and the first prompt.