Navigation Map-Guided Testing for Software Change Resilience
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated testing methods, such as scripted and random exploration testing, are costly to author and maintain, and cannot guarantee that important scenarios are exercised during testing, with traditional scripted tests failing due to software changes and random exploration tests being non-deterministic.
Innovation Solution
Navigation augmented testing uses a navigation map built from historical data to guide testing agents, allowing for lightweight scripted tests that adapt to software changes by determining routes to target states and actions, reducing the need for explicit step authoring.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional scripted testing is used, then testing coverage and reliability are improved, but authoring and maintenance costs increase significantly
Solution Approach 1:
The patent segments the test script into two distinct parts: navigation instructions (high-level intent) and action instructions (specific validations). The navigation map handles the complex path-finding separately from the test logic, allowing each component to be developed and maintained independently. This segmentation reduces the complexity of authoring and maintaining complete scripted tests.
Solution Approach 2:
The navigation map serves as an intermediary between the test author's high-level navigation instructions and the testing agent's low-level execution steps. Instead of directly specifying every interaction step, the test author provides navigation goals, and the navigation map intermediates by computing the actual path through the UI, thereby reducing authoring complexity while maintaining reliability.
2Ease of manufacture
If random exploration testing is used, then authoring and maintenance costs are reduced, but testing determinism and scenario coverage deteriorate
Solution Approach 1:
The system dynamically adapts between two extremes: when navigation instructions are provided, the testing becomes deterministic and targeted; when navigation maps are used alone, the testing remains flexible and adaptive. This dynamic approach allows the system to maintain ease of authoring while ensuring scenario coverage reliability through the combination of high-level navigation goals with the underlying navigation map guidance.
3Manufacturing precision
If complete step-by-step scripting is required, then testing precision is improved, but adaptability to software changes deteriorates
Solution Approach 1:
The navigation map performs preliminary action by pre-computing navigation paths through the UI based on historical data and current state. When software changes occur, the navigation map can be updated with new historical data, and it automatically recalculates paths without requiring changes to the test scripts themselves. This preliminary computation separates the adaptive navigation logic from the precise test validation logic.
4Loss of time
If navigation augmented testing is implemented, then test authoring time is reduced, but system complexity increases
Solution Approach 1:
The navigation map operates autonomously by automatically computing navigation paths based on historical testing data and current UI state without requiring manual intervention. The system self-updates by learning from historical data, eliminating the need for manual maintenance of navigation logic. This self-service capability reduces authoring time while the modular architecture keeps system complexity manageable.
Data Source
AI summary
Navigation augmented testing (NAuT) of a software product under test. In examples, NAuT uses a testing agent to execute steps of a navigation augmented test. The navigation augmented test is a lightly scripted test where one or more steps may be condensed into a navigation instruction to navigate to a specified target element (e.g., a state where the target element is available and/or otherwise accessible for an interaction). According to examples, a navigation map is built from data collected in a previous random exploration testing run. The navigation map includes information about relationships between elements and/or actions performed on the elements and different pre- and post-action states. The testing agent uses the navigation map to determine a route including one or more steps for the testing agent to execute to produce a state where the specified target element is available and/or otherwise accessible for performing a subsequent action.