Screen Event Capture via Structured Data Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for assessing customer service representative performance through screen recording face issues such as network congestion, excessive resource consumption, and inefficiency due to the need for manual analysis of bitmap screenshots, which are not susceptible to automatic analysis.
Innovation Solution
A system and method for capturing screen events in a data format that allows for filtering and analysis, decomposing events into components, and storing content-related data, enabling real-time or offline analysis with low bandwidth and resource consumption, and automatic data processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If screen recording is implemented using bitmap screenshots, then performance assessment data is captured, but network bandwidth is excessively consumed causing network congestion
Solution Approach 1:
The screen recording is segmented into discrete event components (mouse clicks, keyboard strokes, window changes, application events) rather than continuous bitmap capture. Each event is captured as structured data with specific attributes, allowing selective transmission and analysis while dramatically reducing network bandwidth consumption.
Solution Approach 2:
The recording format is changed from bitmap images to structured event data with parameters (event type, coordinates, timestamp, application context). This parameter-based representation maintains analytical value while reducing data size and network transmission requirements.
2Loss of information
If heavy software implementation is used for screen recording, then comprehensive performance data is captured, but workstation resources are excessively consumed slowing down representative performance
Solution Approach 1:
The essential performance assessment information is extracted from the screen in the form of discrete events (mouse movements, clicks, keyboard input, window focus changes, application state changes). Only these critical event parameters are captured and transmitted, leaving the heavy bitmap rendering and processing requirements at the client workstation.
Solution Approach 2:
The mechanical process of continuous bitmap screen capture and transmission is replaced with an event-driven data collection mechanism that generates structured records of user interactions. This substitution eliminates the computational overhead of continuous screen rendering while preserving all necessary performance metrics.
3Loss of information
If bitmap screenshots are used for performance assessment, then visual record of screen activity is obtained, but automatic analysis is not possible requiring manual supervisor review
Solution Approach 1:
The screen activity is transformed from visual bitmap format to structured parameter-based event data (event type, coordinates, timestamp, application context, control element information). This structured format enables automated parsing, filtering, and analysis of performance metrics without requiring manual supervisor review.
Solution Approach 2:
An event data structure serves as an intermediary between the screen capture mechanism and the analysis system. This structured format acts as a universal interface that enables automated processing while preserving all necessary information for performance assessment.
4Loss of information
If screen recording is implemented on low bandwidth connections, then performance data can be captured, but network performance deteriorates and recording becomes impractical
Solution Approach 1:
The recording data is segmented into discrete, compact event records that can be transmitted efficiently over low bandwidth connections. Each event is represented by minimal structured data rather than large bitmap images, making the recording practical even on constrained network connections.
Data Source
AI summary
A system and method for capturing screen events, and processing these events to form data in a data format which is susceptible to analysis. Preferably, the method also comprises analyzing the data in order to extract useful information about the user of the screen and computational device which generated these screen events. More preferably, the method of the present invention is able to assess the quality of the performance of the user whose interaction with the computational device caused the screen events to be generated.