Graph Convolutional Network for UX Layout Structural Similarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current UX design processes face challenges in effectively searching for similar user interface layouts, as traditional methods rely on visual comparisons that fail to capture structural details, leading to inaccurate matching and inefficient reuse of successful design elements.
Innovation Solution
The use of machine learning techniques to encode user experience layouts into search embeddings, allowing for the identification of structurally similar layouts by generating a graph representation of UX layouts and processing them with a graph convolutional network (GCN) to produce search embeddings that can be compared for similarity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If visual comparison methods are used to search for similar user interfaces, then the search process is simple to implement, but the matching accuracy is low because visual similarity does not capture structural details
Solution Approach 1:
The patent replaces traditional visual comparison methods with machine learning-based structural analysis. Instead of comparing pixel-level visual appearances, the system uses graph convolutional networks to analyze the structural relationships between UI components, thereby improving matching accuracy while accepting increased system complexity.
Solution Approach 2:
The patent introduces graph representations as an intermediary between UI layouts and similarity comparison. By converting UI layouts into graph structures that capture component relationships, the system enables more accurate structural analysis while maintaining a clear processing pipeline through the graph convolutional network.
2Reliability
If traditional visual search methods are used, then the implementation is straightforward, but the ability to identify structurally similar layouts is poor
Solution Approach 1:
The patent performs preliminary structural analysis by converting UI layouts into graph representations before comparison. This preprocessing step captures the essential structural relationships early, enabling more reliable similarity identification and reducing the need for repeated design iterations.
Solution Approach 2:
The system replaces manual or simple visual inspection methods with automated graph-based structural analysis using machine learning. This substitution significantly improves the reliability of structural similarity identification, allowing designers to confidently reuse proven layouts across different projects.
3Loss of information
If image-based comparison is used, then the search process is fast, but the information captured about user interface functionality is limited
Solution Approach 1:
The patent segments UI layouts into individual components and their relationships, representing them as graph nodes and edges. This segmentation captures detailed structural information about component arrangements and interactions, going beyond what image-based methods can provide while organizing information in a manageable graph structure.
Solution Approach 2:
The patent transitions from two-dimensional image representations to graph-based structural representations, adding a new dimension of analysis. By representing UI layouts as graphs with nodes and edges, the system captures functional relationships and structural patterns that are invisible in traditional image formats.
Data Source
AI summary
Embodiments are disclosed for learning structural similarity of user experience (UX) designs using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise generating a representation of a layout of a graphical user interface (GUI), the layout including a plurality of control components, each control component including a control type, geometric features, and relationship features to at least one other control component, generating a search embedding for the representation of the layout using a neural network, and querying a repository of layouts in embedding space using the search embedding to obtain a plurality of layouts based on similarity to the layout of the GUI in the embedding space.