Automated UX Design Evaluation Using Deep Learning

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional methods for evaluating user experience (UX) design are inefficient, costly, biased, and difficult to quantify, making it challenging to determine the effectiveness of UX designs across different platforms.

Innovation Solution

An automated evaluation system using graph theory and sequential deep learning modeling that filters out background information from user interface screens, creates weighted flow graphs, and trains models with historical data to predict the success of UX designs based on confidence scores, evaluating UX designs at three levels: UI object, transition logic, and sequential flow.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional methods are used to evaluate UX design, then evaluation can be performed, but the process is inefficient and costly

Engineering Contradiction:
Improveevaluation efficiencyVSAvoidevaluation time
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent replaces traditional manual evaluation methods with an automated deep learning-based system. The system uses convolutional neural networks to automatically analyze UI screenshots, extract features, and evaluate UX design quality, eliminating the need for manual human evaluation and significantly improving efficiency and reducing time loss.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The evaluation system is self-service capable, automatically performing the entire evaluation process without human intervention. The system autonomously processes UI screenshots, extracts meaningful features, compares them against learned patterns from successful designs, and provides evaluations, allowing the system to serve itself rather than requiring manual operations.

Inventive Principle:
Principle #25Self-service

2Measurement precision

If traditional evaluation methods are used, then UX design can be assessed, but the results are biased and difficult to quantify

Engineering Contradiction:
Improveevaluation quantifiabilityVSAvoidevaluation bias
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent replaces subjective human judgment with an objective automated system based on deep learning. The convolutional neural network processes UI screenshots through multiple convolutional layers and fully connected layers to produce quantifiable evaluation scores, eliminating human bias and providing reliable, repeatable measurements that are difficult to manipulate or interpret subjectively.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Quantity of substance

If background information is included in UI screen analysis, then complete information is preserved, but computational resources are consumed unnecessarily

Engineering Contradiction:
Improveinformation completenessVSAvoidcomputational resource consumption
Core Design Contradiction:
Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

The patent extracts and removes background information from UI screenshots before analysis. The system identifies and eliminates redundant background elements, retaining only the essential foreground objects and interface elements that are relevant to UX evaluation. This extraction process reduces the amount of data that needs to be processed by the deep learning model, significantly decreasing computational resource consumption while preserving the completeness of meaningful information.

Inventive Principle:
Principle #2Taking out (Extraction)

Data Source

PatentUS11249736B2AI-assisted UX design evaluation
Publication Date: 2022.02.15 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US11249736B2 patent drawing
  • US11249736B2 patent drawing
  • US11249736B2 patent drawing

AI summary

A method and system of evaluating a user experience (UX) design are provided. A UX design is received. All objects that are identified to be part of a background of the input UI screen are removed to create a filtered input UI screen. The input UI screen is assigned to a cluster. A target UI screen of the input screen is determined and its background removed, to create a filtered target UI cluster. The target UI screen is assigned to a cluster. The filtered input UI screen is used as an input to a deep learning model to predict a target UI cluster. The predicted target UI cluster is compared to the filtered target UI cluster based on the clustering. Upon determining that the filtered target UI cluster is similar to the target UI screen, the UX design is classified as being successful.