Automated UI Testing via Dynamic Region Image Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual user interface testing for detecting errors in application interfaces is expensive and prone to human error, often missing existing errors in the application.
Innovation Solution
Automated user interface testing using screenshots captured during automated tests to identify dynamic and static regions of the interface, comparing these to detect changes and errors, and generating alerts or corrective actions for errors found.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual user interface testing is used to detect errors in application interfaces, then human reviewers can identify interface issues, but the process is expensive and prone to human error
Solution Approach 1:
The patent creates visual copies (images) of the user interface at different states and compares these copies to detect errors. Instead of manual visual inspection, the system automatically generates and compares interface images, eliminating human error while maintaining detection accuracy.
Solution Approach 2:
The patent replaces the mechanical human review process with an automated image comparison system. The computer automatically captures interface images, processes them, and compares them to identify discrepancies, substituting manual labor with automated computational processes.
2Reliability
If manual testing is used to review user interfaces, then interface errors can be detected, but the process is time-consuming and expensive
Solution Approach 1:
The system performs self-testing by automatically capturing its own interface images and comparing them without requiring external human reviewers. The automated system serves itself, continuously monitoring and detecting interface errors independently, which improves both reliability and productivity.
Solution Approach 2:
The patent implements continuous automated testing that can run repeatedly and continuously, comparing interface images across multiple states and time points. This continuous action ensures reliable error detection while significantly improving productivity compared to intermittent manual reviews.
3Productivity
If automated image comparison is used to test user interfaces, then testing efficiency improves, but the system must handle complex image processing and region identification
Solution Approach 1:
The patent segments the user interface into distinct regions (static regions and dynamic regions) and compares these segmented areas separately. This segmentation simplifies the overall image processing complexity by breaking down the interface into manageable, comparable units while maintaining high testing speed.
Solution Approach 2:
The system applies different comparison strategies to different regions of the interface based on their local characteristics. Static regions are compared for exact matches while dynamic regions allow for expected changes, optimizing both processing speed and accuracy without requiring uniform complex processing across the entire interface.
Data Source
AI summary
In an example, a plurality of images of an interface of an application may be identified. An image of the plurality of images may be compared with a second image of the plurality of images to determine a first set of regions, of the interface, in which the second image is different than the first image. Based upon the first set of regions, a set of dynamic regions of the interface may be determined. A fourth image of the interface may be identified. Based upon the set of dynamic regions, a first portion of the fourth image may be compared with a first portion of the first image to determine user interface testing information associated with the interface of the application.