Atomic Scenario Testing with Binary Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated testing of large, complex software applications spanning multiple computing hosts and involving multiple code sections and user interface documents is inefficient due to time-consuming sequential processing of scenarios, especially when identifying faults in a large number of scenarios.
Innovation Solution
A method and system for atomic testing that generates a listing of interacting scenarios likely to cause failures, using a binary search strategy to identify the smallest subset of scenarios causing failures, and updates this listing dynamically during testing, optimizing the search for failures by pre-specifying suspected interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If sequential processing of scenarios is used to ensure thorough testing, then testing completeness is improved, but testing time increases substantially
Solution Approach 1:
The patent segments the scenario testing process into three distinct phases: composition phase (creating test buckets), execution phase (running scenarios), and problem determination phase (analyzing failures). This segmentation allows each phase to be optimized independently, reducing overall testing time while maintaining completeness.
Solution Approach 2:
The patent performs preliminary actions by pre-composing test buckets and pre-identifying probable areas of conflict before execution. Scenarios are organized into test buckets in advance, and conflict patterns are pre-analyzed, eliminating the need for exhaustive sequential testing during the actual test run.
2Measurement precision
If all scenario combinations are tested to identify interacting scenarios causing failure, then failure detection accuracy is improved, but the number of tests required becomes unmanageable
Solution Approach 1:
The patent applies partial action by focusing testing efforts on the most probable areas of conflict rather than testing all possible scenario combinations. By identifying and prioritizing scenarios with higher conflict probability, the system achieves effective failure detection with a manageable subset of tests.
Solution Approach 2:
The system performs self-service by automatically analyzing scenario interactions and identifying probable areas of conflict without requiring exhaustive manual testing. The automated conflict analysis engine examines scenario relationships and prioritizes testing based on predicted conflict likelihood, reducing the burden of comprehensive testing.
3Measurement precision
If comprehensive scenario testing is performed to identify root cause, then problem determination accuracy is improved, but the time required for problem determination becomes substantial
Solution Approach 1:
The patent implements feedback mechanisms where test results are continuously analyzed to refine the identification of probable areas of conflict. Failed scenarios provide feedback that updates the conflict probability models, allowing the system to rapidly narrow down root causes in subsequent testing iterations without exhaustive analysis.
Solution Approach 2:
The system dynamically adjusts the testing strategy based on observed failures and conflict patterns. As testing progresses, the probable areas of conflict are refined and updated, allowing the problem determination process to adapt and focus on the most likely root causes, significantly reducing analysis time while maintaining accuracy.
Data Source
AI summary
Atomic testing of a multiplicity of scenarios includes generating a listing of interacting scenarios which are likely to cause a failure, and testing ones of the scenarios not included in the listing according to a binary search strategy to identify a subset of the scenarios as a source of failure among the scenarios. Additionally, the listing can be updated with newly identified interacting scenarios which are likely to cause a failure.