Automated Test Failure Analysis Using Change-Based Cause Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies are unable to specify the cause of test failures in software testing, which can be attributed to the test target software, test data, test script, or test environment.
Innovation Solution
An analysis device and method that calculates failure cause scores for test target software, test data, test script, and test environment based on the presence or absence of changes, and estimates the cause of test failures using a score calculation unit and estimation unit.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated test execution is implemented without human intervention, then test efficiency is improved, but the ability to identify test failure causes deteriorates
Solution Approach 1:
The invention implements feedback mechanisms by automatically collecting test execution results, change information, and error logs, then feeding this data back through score calculation units that analyze correlations between changes and test failures. The estimation unit generates cause identification results that feedback to the testing process, enabling automated failure cause specification without human intervention while maintaining diagnostic capability.
Solution Approach 2:
The invention introduces intermediary components including the score calculation unit and estimation unit that act as mediators between automated test execution and failure cause identification. These intermediaries automatically process test results, calculate scores based on change information, and estimate failure causes, bridging the gap between automated execution and diagnostic capability without requiring human experts.
2Measurement precision
If manual analysis of test failures is performed by experts, then accurate cause identification is achieved, but time consumption and cost increase
Solution Approach 1:
The invention enables self-service by implementing automated score calculation and estimation units that independently analyze test failures without requiring expert intervention. The system automatically collects necessary data, calculates failure cause scores based on change information, and generates cause identification results, allowing the testing system to diagnose its own failures efficiently and accurately.
Solution Approach 2:
The invention replaces the mechanical process of manual expert analysis with an automated computational system. The score calculation unit and estimation unit use algorithmic processing to analyze test failures, substituting human expert mechanics with automated information processing that achieves comparable accuracy while significantly reducing time consumption.
3Measurement precision
If comprehensive change information is collected for all test elements, then failure cause analysis accuracy is improved, but system complexity increases
Solution Approach 1:
The invention applies segmentation by dividing the complex information collection system into distinct modules: a score calculation unit that processes different types of change information separately, and an estimation unit that synthesizes these scores. Change information is segmented into categories (test target software changes, test data changes, test script changes, test environment changes), allowing systematic processing of comprehensive data without overwhelming system complexity.
Data Source
AI summary
An analysis device specifies a cause when a test of test target software fails in a test environment that is an environment for executing a test. In the test, execution of test target software and confirmation of an execution result are executed by a test script without human intervention. Test data is data read into the test target software and the test script when the test is executed. The analysis device includes: a score calculation unit that calculates a failure cause score for each of test target software, test data, a test script, and a test environment based on the presence or absence of a change; and an estimation unit that estimates, based on a failure cause score, whether the test has failed due to test target software, test data, a test script, or a test environment.