Browser Extension Interface Navigation Recording
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for recording and replicating user interface navigation processes are cumbersome and prone to user error, often requiring significant user involvement and relying on network traffic that may not effectively capture interactions with specific UI states.
Innovation Solution
A computer system uses a browser extension to observe and record user interactions, organizing them into an interface navigation recording that can recreate the process, including filtering and optimizing interactions to generate steps for replication, and verifying successful recreation through UI elements and network interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If explicit user designation of actions is used to collect information for replicating interface navigation, then the computer system can replicate the user's interface navigation process, but the process becomes burdensome for the user and has a high risk of user error
Solution Approach 1:
The system automatically records interface navigation processes by itself without requiring user designation of actions. The browser extension captures user interactions with UI elements automatically, converting them into reproducible steps without user intervention, thus eliminating user burden while maintaining accuracy
Solution Approach 2:
The patent replaces the manual mechanical process of explicit user designation with an automated observation system. The browser extension uses programmatic methods to monitor and capture user interactions, substituting human effort with automated software that records navigation processes accurately
2Loss of information
If network traffic is used to identify interactions between device and host system, then the computer system can capture interactions, but it does not effectively capture user's interactions with specific UI state
Solution Approach 1:
The patent segments the monitoring approach into two parts: network traffic monitoring and UI element interaction monitoring. The browser extension specifically captures interactions with UI elements (buttons, links, forms) by observing DOM manipulations and user events, separating this from general network traffic to ensure precise UI state capture while maintaining complete interaction data
Solution Approach 2:
The browser extension acts as an intermediary between the user's browser and the recording system. It injects into the browser process to directly observe UI element interactions and network traffic, mediating between these two data sources to capture both complete interaction data and precise UI state information that neither source alone could provide
3Ease of operation
If conventional recording methods are used, then the process can be captured, but the method is intrusive and less robust in mimicking user behavior
Solution Approach 1:
The patent replaces intrusive manual recording methods with automated browser extension-based observation. The extension runs within the browser process, automatically capturing user interactions with UI elements through programmatic monitoring of DOM events and user actions, reducing intrusiveness while improving robustness
Solution Approach 2:
The system creates an accurate copy of user behavior by observing and recording actual user interactions with UI elements. Instead of requiring users to manually designate actions or using indirect network traffic analysis, the system copies the actual user navigation path by monitoring real user events, producing a more reliable and less intrusive recording
Data Source
Figure 1
Figure 2~3B
Figure 4A~4B
AI summary
A computer system records and recreates an interface navigation process performed by a user with a host system. The computer system observes the user's interactions with the various UI elements during an interface navigation process by using a browser extension for the browser application in which the user is performing the interface navigation process. The browser extension then stores information about the interactions the user is performing and the UI elements that they are being performed on. The browser extension sends the stored interactions to the computer system, which processes the interactions to generate steps of the interface navigation recording. In some embodiments, the computer system also identifies one or more UI elements present in the final UI state of the interface navigation process as "verifiers" that can be used to determine whether the recreation of the interface navigation process is successful.