End-user Device Testing via Local Script Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional user interface testing methods fail to account for real-world variables such as deployment on new devices, inexperienced users, and changing network conditions, leading to subjective and cumbersome problem identification and resolution processes.
Innovation Solution
An error reporting system that generates reports by monitoring user input and executing test scripts locally on a computing device, capturing screen recordings, and obtaining user feedback to objectively assess user interface functionality under current conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional controlled environment testing is used, then testing can be conducted in a stable and manageable setting, but the testing fails to account for real-world variables such as new computing devices, inexperienced users, operating system updates, and current network conditions
Solution Approach 1:
The system enables end-users to automatically perform testing functions on their own devices without requiring developer intervention. Users download test scripts, execute them locally on their computing devices, and automatically report results back to the service provider, allowing the system to self-test across diverse real-world environments.
Solution Approach 2:
Test scripts are prepared and configured in advance by the service provider, containing pre-defined test cases and evaluation criteria. These scripts are distributed to end-users before actual testing occurs, enabling systematic testing across multiple devices and conditions without requiring real-time analysis or interpretation.
2Adaptability or versatility
If end-users are relied upon for problem reporting, then real-world user perspectives can be captured, but the reporting is limited to subjective perceptions and is prone to human error
Solution Approach 1:
The system implements automated feedback loops where test scripts objectively evaluate user interface elements and automatically generate reports based on measured performance metrics. This feedback mechanism replaces subjective user perceptions with quantifiable, objective data about actual interface behavior and functionality.
Solution Approach 2:
The system replaces the manual, subjective human reporting process with an automated computational system. Test scripts mechanically execute predefined test cases, automatically detect issues, and generate standardized reports, eliminating human error and subjectivity from the problem identification process.
3Measurement precision
If developers manually investigate and diagnose problems reported by end-users, then detailed analysis can be performed, but the process is tedious and cumbersome
Solution Approach 1:
The system enables automated self-diagnosis where test scripts execute comprehensive evaluations and automatically generate detailed problem reports with specific issue locations and nature. This eliminates the need for developers to manually investigate each reported problem, as the system performs the diagnostic work autonomously.
Solution Approach 2:
Test scripts are pre-configured with comprehensive test cases that cover potential issues before they occur. When problems arise, these pre-prepared scripts automatically execute relevant tests and immediately generate diagnostic reports, eliminating the time-consuming manual investigation process.
4Reliability
If comprehensive testing across multiple devices and conditions is performed, then real-world performance can be accurately assessed, but the complexity and resources required for testing increase significantly
Solution Approach 1:
The testing system is divided into independent, modular test scripts that can be executed separately on individual devices. Each script handles a specific aspect of testing, allowing the overall testing process to be distributed across multiple devices without requiring complex centralized coordination or management.
Solution Approach 2:
The system uses standardized test scripts that can be copied and distributed to numerous end-user devices. These identical script copies execute the same comprehensive test suite locally on each device, enabling scalable testing across diverse environments without requiring the service provider to manage or configure testing on each individual device.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An error reporting system is described that generates, for a user interface being output by a computing device, a report using one or more test scripts for the user interface locally at the computing device. In response to detecting input at an error reporting control included as part of the user interface, the error reporting system obtains test scripts designated for testing the user interface and generates test results by executing the test scripts at the computing device outputting the user interface. Screen displays output by the computing device are recorded during execution of the test scripts to capture a visual appearance of the user interface and are aggregated together with the test results and feedback from a user of the computing device to generate the report. The report is then transmitted to a service provider associated with the user interface.