Regression Test Control Automation for Software Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current regression testing methods require manual effort to identify and ignore insignificant differences in software outputs, such as version numbers and timestamps, which can lead to inefficiencies and increased workload, especially when dealing with dynamic data.
Innovation Solution
A system and method that automatically configure regression controls to ignore allowable differences by comparing outputs from multiple executions of software, identifying and storing dynamic changes as insignificant, and using these settings to filter out expected variations during subsequent tests.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual processes are used to identify and ignore insignificant differences in regression testing, then testing accuracy can be maintained, but operator workload and time consumption increase significantly
Solution Approach 1:
The system performs self-service by automatically comparing output data sets, identifying differences, and configuring regression controls without requiring manual operator intervention. The automated system serves itself to determine which differences are insignificant and should be ignored, eliminating the need for operators to manually review and mark each difference.
Solution Approach 2:
The system performs preliminary action by executing the software multiple times before actual regression testing to collect baseline output data and pre-identify insignificant differences. This preliminary data collection and analysis enables the system to configure regression controls in advance, so that when actual testing occurs, the system can automatically ignore known insignificant differences without requiring manual setup.
2Reliability
If regression controls are updated manually to accommodate dynamic changes, then testing reliability is maintained, but the complexity and time required for maintenance increase
Solution Approach 1:
The system uses feedback by automatically comparing current output against stored baseline data, identifying differences, and using this information to configure regression controls. This feedback loop enables the system to adapt to dynamic changes automatically, maintaining testing reliability while reducing manual maintenance complexity through automated control configuration.
Solution Approach 2:
The system performs preliminary action by collecting baseline output data and pre-identifying insignificant differences before actual regression testing. This preliminary configuration of regression controls with embedded ignore rules eliminates the need for manual updates during maintenance, reducing complexity while maintaining reliability.
3Difficulty of detecting and measuring
If all differences in output are treated as potential errors, then detection sensitivity is high, but false positive rates increase due to dynamic data changes
Solution Approach 1:
The system applies local quality by treating different parts of the output data differently. Instead of uniformly treating all differences as potential errors, the system identifies specific local differences (such as timestamps, version numbers, and other dynamic elements) that are known to be insignificant. These localized differences are excluded from error detection, while other differences continue to be treated as potential errors, maintaining high detection sensitivity for actual bugs.
Solution Approach 2:
The system extracts and separates insignificant differences from the set of all detected differences. By identifying and extracting dynamic elements like timestamps and version information, the system removes them from the list of potential errors. This extraction process reduces false positives while maintaining the ability to detect actual errors in the output.
Data Source
AI summary
A method and system for regression testing of a computer software. The method and system execute the software for a first time to create a first data set; execute the software for a second time after the first time to create a second data set; compare the two data sets to determine differences between the two data sets; and store the differences as allowable differences to be ignored in a subsequent regression testing of the software.