Machine Learning Model for Automated Visual UI Validation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current web design testing across multiple platforms is manual, tedious, and prone to errors due to the human eye's inability to consistently identify visual defects, making it challenging to ensure high-quality user experiences.

Innovation Solution

A validation platform utilizing a machine learning model to perform a pixel-by-pixel comparison of user interface information and design information, generating defect reports and correction recommendations, and automatically generating code to address identified issues.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual visual testing is used to validate user interfaces across multiple platforms, then human judgment and flexibility are applied, but the process is tedious, error-prone, and time-consuming

Engineering Contradiction:
Improvevalidation accuracyVSAvoidtesting time
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent replaces manual visual inspection with an automated machine learning-based validation system. The machine learning model automatically compares user interface renders across multiple platforms against design specifications, eliminating the need for human testers to manually review each platform. This substitution of mechanical manual inspection with automated intelligent processing directly addresses the contradiction by maintaining high validation accuracy while dramatically reducing testing time.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The validation system performs self-service by automatically generating validation reports, identifying defects, and providing correction recommendations without requiring human intervention. The machine learning model autonomously compares user interface information with design information, generates defect reports, and even suggests code corrections, enabling the system to validate itself across multiple platforms efficiently.

Inventive Principle:
Principle #25Self-service

2Productivity

If automated validation systems are implemented, then speed and consistency improve, but complexity of the system increases

Engineering Contradiction:
Improvevalidation speedVSAvoidsystem complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces a machine learning model as an intermediary component that bridges the gap between raw user interface information and validation results. This intermediary model processes visual data, compares it with design specifications, and generates meaningful defect reports. By using the machine learning model as a mediator, the system achieves automated high-speed validation without requiring direct complex interactions between all system components, thus managing complexity while improving productivity.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates digital copies of user interface renders across multiple platforms and compares these copies against stored design information. Instead of physically testing each platform, the system creates and compares visual representations (copies) of the user interfaces. This copying approach enables rapid automated validation while keeping the physical system structure relatively simple, as it works with data representations rather than requiring complex hardware configurations.

Inventive Principle:
Principle #26Copying

3Measurement precision

If human eyes are used to identify visual defects, then contextual understanding is applied, but consistency and precision are insufficient

Engineering Contradiction:
Improvedefect detection precisionVSAvoiddetection consistency
Core Design Contradiction:
Measurement precisionVSReliability

Solution Approach 1:

The patent changes the detection parameter from human visual perception to machine learning-based pixel-level analysis. The machine learning model processes user interface information at a granular level, comparing individual pixels and visual elements against design specifications. This parameter change from macro human perception to micro computational analysis enables precise defect detection while ensuring consistent results across multiple platforms, as the machine learning model applies uniform criteria without human variability.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS10984517B2Utilizing a machine learning model to automatically visually validate a user interface for multiple platforms
Publication Date: 2021.04.20 CAPITAL ONE SERVICES LLC
  • US10984517B2 patent drawing
  • US10984517B2 patent drawing
  • US10984517B2 patent drawing

AI summary

A device receives user interface information associated with a user interface to be provided for a particular platform, and receives design information for a design of the user interface to be provided for the particular platform. The device receives a request to visually compare the user interface information and the design information, and utilizes, based on the request, a trained machine learning model to visually compare the user interface information and the design information. The device generates information, indicating defects in the user interface information, based on utilizing the trained machine learning model to visually compare the user interface information and the design information, where the defects include user interface information that does not visually match the design information. The device provides the information indicating the defects in the user interface information.