Screen Gesture Capture and Replay for Touch Interface Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current monitoring systems fail to adequately capture and accurately recreate user experiences on touch screen devices, particularly on smaller devices, due to their inability to effectively capture and replay screen gestures such as zoom, scroll, and orientation changes.
Innovation Solution
A capture system that captures client events including screen gestures like scroll, touch, and pinch operations, and a replay system that uses this data to recreate the original application session, simulating the physical display environment and user interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current monitoring systems only capture HTML for webpages, then the system complexity remains low, but the measurement precision of user interactions deteriorates
Solution Approach 1:
The patent segments the capture process into multiple components: HTML capture, gesture capture, and display state capture. Each component handles specific aspects of user interaction, allowing the system to achieve comprehensive measurement precision while managing complexity through modular architecture. The segmentation enables independent optimization of each capture module.
Solution Approach 2:
The patent adds new dimensions to the capture system by incorporating gesture data and display state information alongside traditional HTML content. This multi-dimensional approach captures user interactions from multiple angles (visual content, touch gestures, display transformations), significantly improving measurement precision without proportionally increasing system complexity.
2Measurement precision
If the system captures comprehensive client events including gestures, then the measurement precision improves, but the loss of information increases due to data volume
Solution Approach 1:
The patent extracts only the essential elements needed for accurate recreation: HTML content, gesture events, and display states. By taking out only these critical components rather than capturing all possible data, the system achieves high measurement precision while minimizing information loss and storage requirements. Irrelevant data is excluded from the capture process.
Solution Approach 2:
The system discards redundant or unnecessary data during capture while recovering and preserving only the essential information needed for accurate recreation. This selective approach manages data volume by discarding superfluous information while recovering and maintaining the critical elements that enable faithful user experience reconstruction.
3Reliability
If the replay system simulates physical display environment, then the reliability of user experience recreation improves, but the device complexity increases
Solution Approach 1:
The patent creates simplified copies of the physical display environment by capturing display states (zoom levels, scroll positions, orientations) and reproducing them in the replay system. Rather than simulating the entire physical environment, the system copies the essential visual and interaction states, achieving high reliability while managing complexity through targeted replication of critical display characteristics.
Data Source
AI summary
A capture system may capture client events for an application session. Some client events may contain display information associated with screen gestures. The screen gestures may be associated with any user input that changes how images are displayed during the application session. For example, the screen gestures may comprise one or more of a scroll gesture, a touch start gesture, a touch move gesture, a touch end gesture, and/or a pinch gesture. In another example the screen gesture may comprise a reorientation of a device operating in the application session. A replay system may replay the application session based on the captured client events to recreate images displayed during the application session in response to the screen gestures.