Automated End-to-End Testing for User Experience Measurement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional non-functional testing of computer systems focuses on system performance without considering the user experience, which limits the ability to optimize the system for improved user interaction.
Innovation Solution
A method that involves executing performance test cases by a testing tool, applying a load to the computer system, monitoring user experience parameters, aggregating the results, and comparing expected versus actual results to evaluate the impact of the load on user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional non-functional testing focuses on system performance metrics, then system performance can be evaluated, but user experience cannot be properly assessed
Solution Approach 1:
The patent combines multiple monitoring tools and testing tools into an integrated testing system that simultaneously monitors system performance metrics and user experience parameters. This merging allows comprehensive evaluation of both system performance and user experience through unified test cases that capture both dimensions, resolving the contradiction between measurement precision and system complexity.
Solution Approach 2:
The testing system is designed with multi-functional capabilities where test cases can evaluate both system performance and user experience through a single execution framework. The system universally handles multiple types of measurements (performance metrics and experience parameters) using standardized test case structures, enabling comprehensive evaluation without proportionally increasing complexity.
2Reliability
If load testing is applied to evaluate system performance, then performance data can be collected, but the impact on individual user experience remains unmeasured
Solution Approach 1:
The patent segments user experience measurement into individual user-specific parameters that can be monitored independently during load testing. Each user's experience metrics are captured and evaluated separately, allowing reliable assessment of how load affects individual users while maintaining efficient automated testing through structured segmentation of monitoring data.
Solution Approach 2:
The testing system implements feedback mechanisms where monitored user experience parameters are compared against expected results, and this feedback is used to evaluate the reliability of user experience under load. The automated comparison and evaluation processes maintain testing efficiency while enabling comprehensive reliability assessment through structured feedback loops.
3Measurement precision
If comprehensive user experience parameters are monitored for each user, then user-specific experience can be evaluated, but data aggregation and processing complexity increases
Solution Approach 1:
The patent uses virtual users that replicate real user behavior patterns to generate and monitor experience parameters. These virtual copies allow comprehensive collection of user-specific experience data without requiring actual user participation, reducing data processing complexity while maintaining measurement precision through standardized virtual user models that capture essential experience metrics.
Data Source
AI summary
A method for testing a computer system. The method includes executing by a testing tool a performance test case associated with a service of the computer system, and applying by the testing tool a load to the computer system in response to executing a sequence of actions of the performance test case. The method additionally includes monitoring by a monitoring tool experience parameters of the computer system that define a user experience of the service as the performance test case is executed, and aggregating by the monitoring tool the experience parameters. The method further includes communicating by the monitoring tool and actual result to the testing tool, and comparing by the testing tool the expected result of the performance test case with the actual result of the performance test case to evaluate the impact of the applied load to the experience of the service.