Software Testing Stability Index Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in optimizing software testing, particularly for CAD software, as they struggle to reduce unnecessary tests when software versions are updated, due to varying user data and usage standards, leading to increased man-hours and inefficiencies.
Innovation Solution
A system that groups test data based on similar test results and stability indices, selecting only necessary data for testing new software versions, using a processor to calculate test result stability and module stability indices, and storing these indices for efficient test case selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the developer conducts comprehensive tests covering all user usages each time a new release is made, then the quality of the CAD software can be guaranteed, but the testing time and man-hours increase significantly
Solution Approach 1:
The system performs preliminary actions by automatically executing tests using multiple versions of test data before the developer conducts manual testing. Test result information is acquired and analyzed in advance, allowing the system to predict which test data may cause failures and prioritize those for manual testing, thereby reducing the overall testing time while maintaining software quality.
Solution Approach 2:
The system introduces an intermediary automated testing mechanism that acts as a mediator between the software update and manual quality verification. By acquiring test result information from automated tests and analyzing it to determine priority orders, the system bridges the gap between rapid software releases and thorough quality assurance, reducing the time burden on developers.
2Reliability
If the user conducts tests using user data each time a new release is made, then the usage-specific quality can be verified, but the testing workload and man-hours increase
Solution Approach 1:
The system implements feedback by acquiring test result information from automated tests and using it to determine the priority order of test data. This feedback mechanism allows the system to learn from previous test outcomes and automatically prioritize test cases that are more likely to reveal usage-specific issues, thereby reducing the user's testing workload while maintaining usage-specific quality verification.
Solution Approach 2:
The system enables self-service by automatically analyzing test result information and determining priority orders without requiring manual intervention from the user. The system serves itself by autonomously identifying which test data should be prioritized for manual testing, thereby reducing the user's testing workload while ensuring usage-specific quality is verified.
3Reliability
If all test data is tested each time a new release is made, then comprehensive coverage is achieved, but the testing process becomes inefficient and time-consuming
Solution Approach 1:
The system applies segmentation by dividing test data into priority levels based on analyzed test result information. Instead of treating all test data equally, the system segments test cases into high-priority and low-priority groups, allowing the testing process to focus on critical areas first. This segmentation maintains comprehensive coverage while improving testing efficiency by reducing the time spent on lower-priority test cases.
Solution Approach 2:
The system implements partial action by selectively testing only the most critical test data first, based on the determined priority order. Rather than executing all tests uniformly, the system performs partial testing on high-priority cases that are most likely to reveal issues, thereby achieving effective test coverage with reduced time and resource expenditure.
Data Source
AI summary
A method and system for processing test results from testing operation of software. A test result of pass, fail status, or unperformed is received for each test case of a test performed for each release of the software. A group to which each test belongs is ascertained which determines a group identifier of the group to which each test belongs. A test result stability index is calculated for each test case as being proportional to a total number of consecutive releases that include and are prior to the last release of the software such that the test result for each of the consecutive releases denotes a pass. The group identifier and the test result stability index are stored in a hardware storage unit.