Automated Video Game Testing via KPI Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Testing large video games is complex and time-consuming due to their massive size and the challenge of comparing performance across multiple sessions, as the data generated during each session can exceed several gigabytes, making it impractical to store and compare.
Innovation Solution
A method and system that processes test data by extracting key performance indicators (KPIs) from a byte stream, reducing the data volume significantly, allowing for aggregation and comparison across sessions, and using machine learning algorithms to select and perform automated test procedures based on performance trends.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If the entire byte stream from video game test sessions is stored and analyzed, then complete performance data is available, but the data volume becomes unmanageably large (exceeding several gigabytes per session)
Solution Approach 1:
The patent extracts only the essential KPI data points from the complete byte stream of test session data. By identifying and isolating specific performance indicators (frame rate, CPU usage, memory consumption, etc.), the system retains critical performance information while discarding unnecessary contextual data, thereby reducing data volume from gigabytes to manageable sizes.
Solution Approach 2:
The patent segments the continuous byte stream into discrete KPI data objects. This segmentation process divides the monolithic data stream into separate, identifiable performance metric units that can be individually processed, stored, and compared across sessions, making the data more manageable while preserving complete performance information.
2Measurement precision
If all test session data is stored for comparison, then comprehensive performance analysis is possible, but storage requirements become impractical
Solution Approach 1:
The system extracts only the KPI-related data objects from the complete test session byte streams. By selectively extracting only the performance indicators needed for comparison (frame rate, processing time, resource consumption), the patent maintains measurement precision for performance analysis while dramatically reducing storage requirements compared to storing entire sessions.
Solution Approach 2:
The patent changes the parameter of data representation from raw byte stream format to structured KPI data object format. This parameter transformation consolidates multiple data points into compact, standardized representations that preserve comparison accuracy while reducing the effective storage footprint through efficient data encoding and aggregation.
3Measurement precision
If manual analysis of test sessions is performed, then detailed performance review is possible, but the process becomes time-consuming
Solution Approach 1:
The patent implements automated feedback mechanisms that continuously monitor KPI data objects during test sessions and immediately compare them against predefined thresholds and historical trends. This automated feedback loop provides detailed performance analysis without requiring manual intervention, as the system automatically identifies anomalies, generates reports, and suggests test procedures based on the KPI data.
Solution Approach 2:
The testing system performs self-service analysis by automatically processing KPI data objects, comparing them across sessions, and generating performance reports without human intervention. The system uses machine learning algorithms to autonomously determine test procedures and optimize testing based on the analyzed KPI trends, significantly reducing manual testing time while maintaining analysis depth.
4Reliability
If the game engine is extensively documented and reviewed, then proper usage can be ensured, but the complexity of testing increases
Solution Approach 1:
The patent introduces KPI data objects as intermediary structures that bridge the game engine's internal operations and the testing framework. These standardized KPI objects serve as mediators that automatically capture performance information from the game engine without requiring direct complex interactions with the engine's internal APIs, thereby ensuring correct usage while simplifying the testing complexity.
Solution Approach 2:
The patent replaces manual review and analysis mechanisms with automated machine learning-based systems. Instead of requiring developers to manually review game engine usage patterns and documentation, the system uses AI algorithms to automatically analyze KPI data, identify optimization opportunities, and verify correct engine usage, thereby reducing testing complexity while maintaining reliability.
Data Source
AI summary
Embodiments disclosed herein include a system that is capable of processing test data across multiple sessions of a video game. In some cases, the tests are performed over multiple video games that share a game engine. The generated test data may be analyzed as the test is being performed and key performance indicators may be extracted from the test data reducing the test data by at least an order of magnitude. Further, the extracted key performance indicators are used to automatically conduct further testing, such as regression testing, based on an analysis of the key performance indicators with respect to trend data generated from prior tests of the video game or of a shared game engine used by multiple video games.