Automated Visual UI Change Detection via Differential Image Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automation testing tools are inadequate for efficiently detecting visual changes in user interfaces, leading to over-testing and a lack of automated processes to quickly identify discrepancies in software application UI changes.
Innovation Solution
A computer system and method that compares baseline and recent UI screenshots by matching test steps, determining differences, and generating a differential image to highlight changes, thereby streamlining the analysis of UI changes and errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review of UI changes is performed, then visual differences can be detected, but the analysis efficiency is low and time-consuming
Solution Approach 1:
The patent replaces manual visual inspection (mechanical human review process) with an automated image comparison system that uses computational algorithms to detect UI changes. The system automatically captures screenshots, compares them against baseline images, and generates differential reports, eliminating the need for manual review while maintaining detection accuracy.
Solution Approach 2:
The patent introduces an intermediary automated comparison system between the UI changes and the reviewer. This system processes screenshots and generates differential images that highlight changes, serving as a mediator that prepares information for efficient human review rather than requiring direct manual inspection of all UI elements.
2Reliability
If exhaustive testing is performed to detect all UI changes, then comprehensive coverage is achieved, but over-testing occurs and resources are wasted
Solution Approach 1:
The patent extracts and compares only the relevant UI elements that have actually changed between test runs. By capturing screenshots and using image comparison algorithms to identify differential regions, the system extracts only the necessary information about UI changes rather than performing exhaustive analysis of entire interfaces, thus avoiding over-testing.
Solution Approach 2:
The patent changes the testing approach from parameter-based DOM element verification to image-based visual comparison. This parameter change enables the system to detect actual visual differences in UI rendering without being constrained by DOM structure, allowing comprehensive coverage of visual changes while reducing unnecessary testing of unchanged elements.
3Extent of automation
If existing automation testing tools are used, then test execution is automated, but visual changes are not effectively detected due to DOM-based locating
Solution Approach 1:
The patent replaces DOM-based element location and verification with image-based visual comparison. By capturing screenshots and comparing pixel-level visual representations of UI elements, the system achieves both automation and precise detection of visual changes that DOM-based tools miss when elements are repositioned or visually altered.
Solution Approach 2:
The patent creates visual copies (screenshots) of the UI interface and compares these copies against baseline images. This copying approach allows automated detection of visual changes without requiring direct interaction with or knowledge of the underlying DOM structure, enabling precise visual change detection while maintaining full automation.
Data Source
AI summary
Methods and apparatuses for automated verification of visual user interface (UI) changes include a computing device that receives a baseline test report comprising a plurality of test steps associated with execution of a test against a software application. The computing device receives a recent test report comprising a plurality of test steps associated with execution of a test against the software application. The computing device matches each test step of the baseline report with a corresponding test step of the recent report. For each pair of matched steps, the computing device compares a first UI screenshot captured during execution the baseline report to a second UI screenshot captured during execution of the recent report. The computing device determines differences between the first UI screenshot and the second UI screenshot, generates a differential image based upon the differences, and stores the differential image in a data store.