GUI Walkthrough Testing With Automated Trace Validation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in automatically testing whether walkthroughs and augmented functionalities in applications match the specific Graphical User Interface (GUI) layout, particularly when there are changes in GUI versions, platforms, or user roles, leading to potential failures and the need to determine the cause of failed tests.

Innovation Solution

A software test layer is executed over a software augmentation layer to test the walkthrough or augmented functionality, using a sequence of automated user actions to mimic user activity, monitor GUI interactions, and generate an interleaved trace display of events to ensure alignment with the GUI layout, without requiring access to the application's internal data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual intervention is used to acquire GUI elements for testing walkthroughs, then the testing process can be performed, but the process requires significant manual effort and time

Engineering Contradiction:

Improvetesting reliabilityVSAvoidmanual intervention time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs self-testing by automatically executing walkthroughs and generating trace displays without requiring manual intervention. The automated testing framework independently acquires GUI elements, executes test cases, and generates test results, enabling the system to test itself autonomously

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

Manual mechanical operations (clicking, navigating, observing) are replaced with automated computational processes. The testing framework uses programmatic control to simulate user interactions and automatically capture GUI state changes, substituting human operators with software agents

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If walkthroughs are tested manually to ensure accuracy with GUI layouts, then testing can be performed, but productivity is reduced due to manual effort

Engineering Contradiction:

ImproveGUI layout matching accuracyVSAvoidtesting productivity

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

Manual visual inspection and comparison operations are replaced with automated image processing and DOM comparison algorithms. The system programmatically captures screenshots, compares them against expected layouts, and validates GUI element positions and properties using computational methods

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The automated testing framework enables continuous testing of walkthroughs against GUI layouts without interruption. Multiple test cases can execute sequentially or in parallel, and the system can re-run tests automatically when GUI changes are detected, maintaining continuous validation

Inventive Principle:

Principle #20Continuity of useful action

3Adaptability or versatility

If automated testing is implemented without access to application internal data or API, then independence is maintained, but testing capability is limited

Engineering Contradiction:

Improvetesting independenceVSAvoidtesting capability

Core Design Contradiction:

Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The system uses the walkthrough itself as an intermediary to access application functionality. By executing the walkthrough's prescribed actions and observing the resulting GUI state changes, the testing framework indirectly queries application behavior without requiring direct API access or internal data

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system creates visual copies (screenshots) and structural copies (DOM snapshots) of the GUI state to analyze and compare. These copies serve as testable representations of the application state, enabling verification without direct access to internal application data

Inventive Principle:

Principle #26Copying

4Measurement precision

If GUI element acquisition is performed manually, then element identification can be achieved, but the process is time-consuming and error-prone

Engineering Contradiction:

ImproveGUI element identification accuracyVSAvoidelement acquisition time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

Manual visual identification and selection of GUI elements is replaced with automated DOM parsing and element location algorithms. The system programmatically traverses the DOM tree, identifies target elements based on their structural properties and attributes, and automatically extracts element information without human intervention

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS12619521B2Automated testing of walkthroughs

Publication Date: 2026.05.05 WALKME

AI summary

A product, system and method comprising: obtaining a test of a walkthrough of an application, wherein the application comprises a Graphical User Interface (GUI), wherein the walkthrough is defined by a sequence of steps and associated advancement conditions, wherein the test comprises a sequence of automated user actions to be provided to the GUI that is configured to invoke the sequence of steps of the walkthrough; executing, at least partially, the test on the walkthrough, wherein said executing comprises providing the sequence of automated user actions in a manner that is configured to cause the sequence of steps to be invoked; generating a trace display of the test execution, wherein the trace display comprises a visual display of events including both steps of the walkthrough and automated user actions of the test; and outputting the trace display.