Oracle-Free Software Testing via Two-Layer Covering Arrays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing computer software lack efficiency in detecting errors, especially in complex conditions, and often require a test oracle, which can be costly and time-consuming.
Innovation Solution
A process involving the creation of equivalence classes, primary and secondary covering arrays, and result vectors to test software without a test oracle, using a two-layer covering array approach to detect errors by analyzing self-consistency of result entries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional software testing methods are used, then test coverage can be achieved, but test efficiency is low and error detection capability is insufficient
Solution Approach 1:
The patent segments the testing process into two distinct layers: a primary covering array that provides broad test coverage, and a secondary covering array that focuses on equivalence class validation. This segmentation allows each layer to specialize in different aspects of testing, improving both error detection capability and test efficiency by avoiding redundant testing efforts.
Solution Approach 2:
The patent introduces a two-dimensional testing structure by creating a secondary covering array that operates in addition to the primary covering array. This dimensional expansion allows the system to test not only individual test cases but also the consistency across equivalence classes, thereby enhancing error detection without proportionally increasing test complexity.
2Measurement precision
If a test oracle is used to verify test results, then accuracy of error detection improves, but time consumption and cost increase
Solution Approach 1:
The patent implements self-service testing by enabling the test system to verify its own results through the secondary covering array. The secondary array tests whether equivalent inputs produce equivalent outputs, allowing the system to self-validate test consistency without requiring an external test oracle. This reduces time consumption while maintaining error detection accuracy.
Solution Approach 2:
The secondary covering array acts as an intermediary mechanism that bridges the gap between primary test execution and final verification. Instead of directly using a time-consuming test oracle, the system introduces this intermediate layer that provides efficient equivalence class validation, reducing the time burden while preserving accuracy.
3Reliability
If comprehensive test coverage is achieved through traditional methods, then reliability improves, but device complexity and human insight requirements increase
Solution Approach 1:
The patent creates a secondary covering array that is structurally similar to but distinct from the primary covering array. This copying approach allows the system to reuse the framework and structure of the primary array while adding equivalence class validation capabilities, thereby improving verification reliability without proportionally increasing overall system complexity.
Solution Approach 2:
The two-layer covering array structure serves multiple functions simultaneously: the primary array provides broad test coverage while the secondary array validates equivalence classes. This multi-functionality allows a single testing framework to achieve comprehensive verification without requiring separate specialized processes, reducing overall complexity while maintaining high reliability.
Data Source
AI summary
A process for testing a program includes: receiving a variable comprising a plurality of input values; producing a plurality of equivalence classes for the input values; producing a representative value per equivalence class; producing, by a processor, a primary covering array comprising a plurality of primary vectors; producing a secondary covering array comprising a plurality of secondary vectors; providing the secondary vectors to the program; and producing a result vector comprising a plurality of result entries to test the program. A computer system for testing the program includes: a memory; and a processor, in communication with the memory, wherein the computer system is configured to perform the process for testing the program. A computer program product for testing the program includes: a non-transitory computer readable storage medium readable by a processor and storing program code for execution by the processor to perform the process.