UI Element Identification via URI Fragment Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing test automation tools struggle to accurately record and replay user interactions with user interface (UI) elements due to poor semantic understanding and flexibility issues with HTML content generated by UI frameworks, making it difficult to distinguish between different UI elements and play back recorded actions effectively.
Innovation Solution
A method and system that utilize a uniform resource identifier (URI) with fragments to identify and resolve UI elements within a UI hierarchy, using a URI parser and resolver to validate syntax, determine resolution contexts, and delegate resolution to specific UI technology resolvers, enabling precise identification and highlighting of targeted UI elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If basic recorder is used to record low-level UI events, then recording capability is provided, but semantic understanding is poor and elements cannot be distinguished
Solution Approach 1:
The URI is segmented into multiple fragments, where each fragment corresponds to a specific UI element or attribute. This segmentation allows the system to precisely identify and distinguish between different UI elements by breaking down the identification process into manageable parts, thereby improving measurement precision without losing semantic information
Solution Approach 2:
The URI acts as an intermediary that bridges the gap between low-level UI events and semantic understanding. By encoding UI element information in a structured URI format with meaningful fragments, the system preserves semantic information while enabling precise element identification and differentiation
2Adaptability or versatility
If HTML content is generated by UI framework, then flexibility is provided, but difficulty in collecting information for playback arises
Solution Approach 1:
The URI scheme provides a universal mechanism that works across different UI frameworks and technologies. By using a standardized URI format with fragments that can represent various UI attributes, the system maintains flexibility for different frameworks while simplifying information collection through a consistent interface
Solution Approach 2:
The URI parameters and fragments are designed to capture essential UI element information in a structured format. By changing the parameter structure to include meaningful fragments, the system makes information collection easier while preserving the flexibility needed for diverse UI frameworks
3Measurement precision
If URI parsing and resolution is implemented, then precise UI element identification is achieved, but system complexity increases
Solution Approach 1:
The URI parsing process is segmented into distinct steps: validation, fragment extraction, and resolution. Each step handles a specific aspect of the identification process, making the overall complex task more manageable while maintaining high precision in UI element identification
Solution Approach 2:
The resolution context acts as an intermediary layer that manages the complexity of resolving URI fragments to actual UI elements. By introducing this intermediate structure, the system can handle complex identification logic in an organized way without exposing the full complexity to the user or test scripts
Data Source
AI summary
A method includes executing instructions recorded on a non-transitory computer-readable storage media using at least one processor. The method includes identifying a targeted user interface (UI) element using a uniform resource identifier (URI), where the URI includes one or more URI fragments with each of the URI fragments being associated with a specific UI element, parsing the URI identifying a UI element to validate a format syntax of the URI, determining from a current session an initial resolution context, resolving each of the URI fragments within a UI hierarchy in a current resolution context, where the current resolution context is the initial resolution context for a first URI fragment and after resolving a last of the URI fragments, returning a reference to the targeted UI element.