Automated Regression Identification via Data Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Manual identification of regressions in application performance after code changes is inefficient for quality assurance engineers, as it involves checking multiple factors to determine if a defect is a regression or a new defect.
Innovation Solution
An automated system that captures and correlates data points during test executions, using a directed acyclic graph (DAG) to identify regressions by comparing test steps and application behaviors between different versions of an application under test (AUT).
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual identification of regressions is performed by quality assurance engineers, then accuracy in determining whether a defect is a regression can be maintained through careful checking of multiple factors, but productivity decreases due to the time-consuming nature of manual analysis
Solution Approach 1:
The patent replaces the manual mechanical process of quality assurance engineers analyzing test results with an automated computer-implemented system. The system automatically identifies regressions by comparing test executions across different application versions, eliminating the need for manual checking while maintaining identification accuracy through systematic data correlation and version comparison algorithms.
Solution Approach 2:
The system enables self-service automation where the regression identification process performs itself without human intervention. The computer-implemented system automatically captures test data, correlates data points across versions, and identifies regressions autonomously, freeing quality assurance engineers from repetitive manual analysis tasks while preserving the thoroughness of multi-factor verification.
2Measurement precision
If comprehensive data collection and analysis is performed to accurately identify regressions, then measurement precision improves, but device complexity increases due to the need for sophisticated data correlation systems
Solution Approach 1:
The patent segments the complex regression identification process into distinct functional modules: a data capture component that collects test execution data, a data correlation component that links data points across versions, and a regression identification component that analyzes correlations to detect regressions. This modular segmentation manages system complexity by organizing comprehensive data analysis into manageable, independent functional units that can be developed and maintained separately.
Solution Approach 2:
The patent introduces a directed acyclic graph (DAG) as an intermediary data structure to manage the complexity of correlating test data across multiple application versions. The DAG serves as a mediator that systematically organizes and links data points from different versions, enabling accurate regression identification without requiring direct complex comparisons between all possible version pairs, thus reducing overall system complexity.
Data Source
AI summary
Example implementations relate to automatically identifying regressions. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also include a regression identification engine to automatically identify, based on the correlated data points, a regression between a first version of the application under test and a second version of the application under test.