Mobile App Session Replay via UI State Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in recording user activity on mobile applications in a performant and scalable manner, making it difficult to generate insightful session replays that can improve debugging and user experience analysis.
Innovation Solution
The implementation of an experience analytics system that includes a client-side and server-side architecture, utilizing an experience analytics client and server, which records user interactions through scripts and APIs, and processes data to generate session replays, providing insights into user behavior and application performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If comprehensive user activity data is recorded on mobile applications, then session replay quality and debugging insights are improved, but data usage and network bandwidth consumption increase
Solution Approach 1:
The patent extracts and records only the essential state changes of UI elements rather than complete screen captures. By identifying and recording only the minimal necessary data (element identifiers, state parameters, timestamps) needed to reproduce user sessions, the system achieves high-fidelity session replays while dramatically reducing data transmission requirements.
Solution Approach 2:
Instead of recording complete screen images and processing them to extract changes, the patent inverts the approach by directly monitoring and recording only the state changes of UI elements through event listeners and state management integration. This captures the essence of user interactions with minimal data overhead.
2Reliability
If detailed user interaction data is captured for session replays, then debugging capability is improved, but processing complexity and computational resources increase
Solution Approach 1:
The patent segments the user interface into discrete, identifiable elements and tracks state changes individually for each element. By dividing the complex UI into manageable components with unique identifiers, the system can efficiently monitor and record only the specific elements that change during user interactions, reducing overall processing complexity while maintaining comprehensive debugging capability.
3Measurement precision
If state change data is transmitted frequently for real-time session replay, then session replay accuracy is improved, but network bandwidth consumption increases
Solution Approach 1:
The patent implements periodic batching of state change data transmissions. Instead of transmitting every state change immediately, the system accumulates state changes in batches and transmits them at intervals or when a threshold is reached. This maintains session replay accuracy by capturing all state changes while significantly reducing network bandwidth consumption through consolidated transmissions.
Data Source
AI summary
The subject technology receives, from a first client device, a set of requests for recording user activity detected on a mobile application executing on the first client device. The subject technology determines that a first request from the set of requests includes a change in position for an element of a screen of the mobile application. The subject technology determines a previous timestamp of a set of previous properties including a previous change in position for the element of the screen. The subject technology determines a transition time between a current timestamp and the previous timestamp. The subject technology generates, using a style sheet language, a linear transition based at least in part on the transition time. The subject technology adds the linear transition to a style of a markup language element corresponding to the element.