Fragment-Based Design Search Using Embeddings for Faster Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional design tools lack the ability to efficiently search for designs that are similar to a screenshot, sketch, or other representations, requiring designers to manually browse through thumbnails and individual designs, which is time-consuming and resource-intensive.
Innovation Solution
An interactive graphic design system (IGDS) that utilizes fragment-based design search, where design interfaces are represented as interconnected nodes in a graph structure, allowing users to search for visually or functionally similar design fragments through a searchable repository using embeddings generated by machine learning models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If designers manually browse through thumbnails and individual designs to find similar designs, then they can search for design similarities, but the process becomes time-consuming and resource-intensive
Solution Approach 1:
The patent replaces the manual mechanical browsing process with an automated image recognition system using convolutional neural networks. The system automatically extracts visual features from design screenshots and performs similarity comparisons, substituting human manual inspection with automated computational analysis, thereby eliminating time loss while maintaining search capability
Solution Approach 2:
The patent introduces an intermediary image recognition system that acts as a mediator between the design database and the user search query. This intermediary automatically processes design screenshots, extracts visual features, and retrieves similar designs without requiring manual user interaction, thus resolving the contradiction between search precision and time efficiency
2Measurement precision
If designers manually browse through individual designs to find similar designs, then they can evaluate design similarities, but the process becomes resource-intensive
Solution Approach 1:
The patent extracts only the essential visual features from design screenshots using pre-trained convolutional neural networks. Instead of processing entire high-resolution images, the system extracts key visual characteristics (colors, shapes, layouts) and uses only these extracted features for similarity comparison, thereby reducing computational resource consumption while maintaining evaluation accuracy
Solution Approach 2:
The patent performs preliminary feature extraction and indexing of design screenshots in advance using pre-trained models. By pre-processing and storing extracted visual features in an indexed format, the system avoids repeated heavy computational processing during actual search operations, thus reducing real-time resource consumption while preserving evaluation precision
Data Source
AI summary
A network computer system provides interactive graphic design system instructions for performing fragment-based design search. The network computer system receives a search query specifying a representation of one or more design elements. The network computer system matches one or more embeddings associated with the representation to a set of embeddings for a set of design fragments, wherein each of the one or more design fragments corresponds to a subset of one or more hierarchical structures representing one or more design interfaces. The network computer system generates a set of search results using the set of embeddings and outputs the set of search results in a response to the search query.