Screen Recording Usability Evaluation via Visual Feature Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for evaluating software usability through screen recordings are inefficient and error-prone, particularly when dealing with dynamically changing user interfaces, as they require manual segmentation and high computational power for frame-by-frame analysis.
Innovation Solution
A method that detects visual features in screen recordings, creates metadata, and clusters similar screenshots using a threshold similarity value, reducing computational power and memory usage by comparing metadata instead of entire screenshots, allowing for real-time preparation and efficient analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If frame-by-frame analysis of entire screen recordings is performed, then evaluation accuracy is improved, but time consumption and computational resources increase significantly
Solution Approach 1:
The patent segments the screen recording into meaningful sections based on visual feature changes rather than analyzing every frame. By detecting when visual features change significantly, the system divides the continuous recording into discrete segments that can be evaluated independently, reducing the total number of frames that need detailed analysis while preserving evaluation accuracy.
Solution Approach 2:
The patent extracts only the essential visual features from each frame rather than processing the entire image data. By identifying and extracting key visual elements that indicate user interaction or interface changes, the system reduces the data volume requiring analysis while maintaining the ability to accurately evaluate usability issues.
2Reliability
If manual segmentation of screen recordings is performed, then evaluation reliability is improved, but productivity decreases
Solution Approach 1:
The system performs automatic segmentation based on detected visual feature changes, making the segmentation process self-service rather than requiring manual intervention. The algorithm autonomously identifies meaningful sections by monitoring for changes in visual features, eliminating the need for manual review while maintaining reliable evaluation through objective, consistent segmentation criteria.
3Measurement precision
If entire screenshots are compared for similarity, then clustering accuracy is improved, but computational power and memory usage increase
Solution Approach 1:
The patent extracts and compares only the detected visual features from each screenshot rather than comparing the entire image data. By focusing comparison efforts on specific feature elements that are most relevant for identifying similar interface states, the system achieves accurate clustering while significantly reducing computational requirements and memory usage.
4Speed
If monitoring pixel change rate is used for segmentation, then segmentation speed is improved, but computing power requirements increase and reliability decreases
Solution Approach 1:
The patent applies different analysis criteria to different parts of the screen recording based on local visual features. Rather than using a uniform pixel change rate threshold across the entire image, the system identifies specific regions with meaningful visual changes and focuses segmentation decisions on those local areas, improving both reliability and efficiency by avoiding unnecessary analysis of unchanged regions.
Data Source
AI summary
Provided is a screen recording preparation method for evaluating software usability using additional data of a user session carried out by a computing system includes at least the following steps in an initial phase: loading a screen recording including a number of N time-ordered screenshots of the user session from a storage medium and detecting visual features within each of the N screenshots, wherein each of the features is attributed a position within the respective screenshot and a score identifying the feature, and creating a set of metadata containing an identifier of each of the N screenshots and the features detected within each of the N screenshots and setting a threshold similarity value. The method also comprises a cycling phase for treating the metadata. Also, a computing system, a computer program and a computer-readable storage medium perform a method according to an embodiment of the present invention.