Touch Gesture Replay for Non-Responsive Interaction Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current solutions for developing touch-enabled applications fail to provide insight into user interactions, particularly when users encounter issues due to unresponsive elements on the screen, leading to frustration and confusion, as they cannot identify non-anticipated user actions or gestures.
Innovation Solution
A method for capturing and interpreting touch events on a touchscreen, comparing them against expected application responses, and generating reports and replays to understand user interactions, including non-responsive gestures, by sending gesture information, application events, and screen layouts to a service for analysis and visualization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If current event capture solutions are used to monitor user actions, then previously coded user actions can be captured, but insight into unanticipated user intentions and non-responsive gestures cannot be obtained
Solution Approach 1:
The system segments user interactions into distinct gesture types (taps, swipes, pinsches, rotations) and separately tracks application responses. This segmentation allows comprehensive capture of both anticipated and unanticipated user actions, enabling identification of non-responsive gestures while maintaining manageable system complexity through structured data organization.
Solution Approach 2:
The system introduces an intermediary analysis layer that compares captured gestures against expected application responses. This intermediary comparison mechanism identifies mismatches between user intentions and application behavior, providing insight into non-responsive gestures without requiring direct complex analysis of every possible user action scenario.
2Ease of operation
If button sizes are changed to accommodate easy navigation, then navigation ease is improved, but identification of user struggle becomes difficult without proper tools
Solution Approach 1:
The system provides feedback to developers by capturing and analyzing user gestures alongside application responses. When users interact with elements of various sizes, the system records the gesture characteristics and application behavior, enabling detection of whether users are struggling with element sizes or layout configurations, thus providing measurable feedback on ease of operation.
3Adaptability or versatility
If designers add non-button elements like logos to the screen layout, then design flexibility is improved, but detection of user tapping on these elements becomes impossible
Solution Approach 1:
The system implements universal gesture capture that applies to all screen elements regardless of their functional designation. Whether an element is coded as a button, logo, or other non-interactive element, the system captures the same gesture data, enabling unified analysis of user interactions across all visual elements and preserving design flexibility while maintaining comprehensive action detection.
4Loss of information
If comprehensive gesture capture is implemented to analyze all user interactions, then insight into user intentions is improved, but system complexity and data processing requirements increase
Solution Approach 1:
The system captures comprehensive gesture data (excessive action) including all touch events, gesture types, and application responses, then processes only the relevant portions to identify non-responsive gestures. This approach ensures no user interaction insight is lost while managing complexity by focusing analysis on the comparison between captured gestures and expected responses rather than processing every possible interaction scenario.
Data Source
AI summary
In gesture reporting and replay, touch events are captured from an operating system of a client device for touch actions on a touchscreen. The touch events are interpreted as gestures. Any application events are captured from an application in response to the gestures. A given gesture and any given application event in response to the given gesture are compared against an expected application event. When there is a match, the given gesture is identified as a responsive gesture. When there is no match, the given gesture is identified as a non-responsive gesture. The gesture information for the given gesture, any application event in response to the given gesture, and a screen layout displayed on the touchscreen at the time of the given gesture are sent to a service. For the information received, the service replays the given gesture and generates a report.