Automated UI Regression Testing via ML Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current regression testing methods, especially for customer-facing software applications, are often manual and costly, failing to efficiently detect and reconcile unintended changes to the user interface (UI) across various software application builds.
Innovation Solution
An automated system that generates state instance maps for both current and previous software application versions, uses image classification techniques to identify differential features, and employs a machine learning model to classify these features as intentional or unintentional, thereby reducing the need for manual verification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual regression testing is used, then testing accuracy can be maintained, but testing time and cost increase significantly
Solution Approach 1:
The patent creates a virtual copy of the UI automation framework that captures visual states of application interfaces. Instead of manual testing, the system generates screenshots and state instance maps that replicate UI conditions, enabling automated comparison between versions without human intervention in each testing step.
Solution Approach 2:
The patent replaces manual mechanical testing operations with an automated image processing system. The differential detection engine uses image processing algorithms to automatically compare screenshots from different application versions, substituting human visual inspection and manual comparison with computational methods.
2Productivity
If automated image comparison is used, then testing speed increases, but false detection of intentional changes occurs
Solution Approach 1:
The patent applies local quality analysis by examining specific regions of interest within UI screenshots. The system identifies and focuses on particular UI elements that have changed, rather than treating the entire interface uniformly. This allows the system to distinguish between significant changes in critical areas and minor variations in less important regions.
Solution Approach 2:
The patent implements feedback mechanisms where the system provides classified information about detected changes to users. The machine learning model analyzes differential features and provides feedback classifications (intentional vs. unintentional changes), allowing the testing process to adapt and focus on truly problematic areas while filtering out false positives.
3Reliability
If all detected differential features are reviewed manually, then detection completeness is maintained, but labor cost increases
Solution Approach 1:
The patent enables the system to self-service by automatically classifying and prioritizing detected differential features using machine learning models. The system performs self-evaluation of changes, automatically determining which features require human review and which can be dismissed, thereby reducing the quantity of features that need manual labor while maintaining detection reliability.
Solution Approach 2:
The patent applies partial action by having the automated system perform the bulk of the analysis work - generating state instance maps, detecting differential features, and classifying them through machine learning. Only a subset of potentially problematic features requires human review, rather than requiring manual examination of all detected changes, thus reducing labor requirements while maintaining adequate detection completeness.
Data Source
AI summary
Systems, computer program products, and methods are described herein for automated regression testing. The present invention is configured to generate, using a regression testing engine, a second state instance map for a second version of an application; generate, using the regression testing engine, a first state instance map for a first version of the application; initiate a differential detection engine on the first state instance map and the second state instance map; determine, using the differential detection engine, one or more differential features in the second version of the application; initiate a machine learning model on the one or more differential features in the second version of the application; and classify, using the machine learning model, the one or more differential features into one or more classes.