Automated Test Case Definition Generation from Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional test case management systems require manual updates and are prone to errors, making it time-consuming to maintain up-to-date test cases and definitions as changes occur in software development, especially when multiple team members modify test case code and source code.
Innovation Solution
An automated system that generates test case definitions from test results, storing associations between test case definitions and results in a database, allowing for dynamic management and reducing the need for manual updates, while enabling hierarchical groupings and drill-down capabilities for test results analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual updates are used to maintain test case definitions, then flexibility and control are improved, but time consumption and error rate increase
Solution Approach 1:
The system automatically generates test case definitions by extracting information from version control system commits and test execution results. The automated test case management system serves itself by monitoring code changes, identifying affected test cases, and updating definitions without human intervention, thereby eliminating manual errors and freeing up time while maintaining high accuracy through systematic automated tracking.
2Productivity
If automated generation of test case definitions is implemented, then time efficiency and consistency are improved, but system complexity increases
Solution Approach 1:
The system introduces an intermediary layer between version control systems and test case management. This intermediary automatically monitors commits, analyzes changed files, determines affected test cases based on file paths and patterns, and updates test case definitions accordingly. This mediator component handles the complexity of integrating multiple systems while presenting a simplified interface for test case maintenance, thereby improving productivity without requiring end users to manage the underlying complexity.
3Loss of information
If test case definitions are automatically updated from code changes, then up-to-date accuracy is improved, but risk of incorrect updates increases
Solution Approach 1:
The system implements feedback mechanisms where test execution results are continuously monitored and fed back into the test case definition management. When automated updates are made based on code changes, the system executes tests to verify correctness and detects any regressions. This feedback loop allows the system to identify incorrect automated updates and trigger alerts for manual review, thereby maintaining up-to-date accuracy while mitigating the risk of incorrect updates through continuous validation.
Data Source
AI summary
Computing systems, database systems, and related methods are provided for managing test cases. One method involves a server receiving a first test result message, automatically generating a first test case definition based on a subset of the fields of metadata associated with the first test result message, and creating an entry in the database that maintains an association between the first test case definition and the first test results. The method continues by receiving a second test result message, automatically generating a second test case definition based on the subset of the fields of metadata associated with the second test result message, and updating a database to maintain an association between the first test case definition and the second test results when the second text case definition matches the first test case definition. Relationships between test results that share a common definition are thereby maintained in an automated manner.