Crowd-Sourced Application Testing via Execution Script Replay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional software application testing methods are labor-intensive, resource-consuming, and may not exhaustively verify all execution paths or input combinations, leading to potential undetected errors, especially with the increased pace of software updates in mobile computing.
Innovation Solution
The method involves instrumenting software applications on end-user devices to collect real-life execution data, which is used to generate and execute automated testing scripts that replicate actual user interactions, allowing for dynamic and comprehensive testing of updated application versions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional manual testing methods are used, then testing can be performed with simple tools, but testing is labor-intensive and cannot exhaustively verify all execution paths
Solution Approach 1:
The system instruments the application itself to automatically generate execution data and testing scripts. The application under test performs self-diagnosis and self-documented behavior capture, eliminating the need for external manual testing effort while achieving exhaustive path verification through automated replay of actual user interactions.
Solution Approach 2:
The system creates copies of actual user execution paths by instrumenting the application to capture real interaction sequences. These copied execution paths are then replayed automatically as test scripts, allowing exhaustive verification of all possible user interaction combinations without requiring manual recreation of each test scenario.
2Productivity
If the pace of software updates is increased, then productivity improves, but testing cannot keep up with the update frequency
Solution Approach 1:
The system performs preliminary instrumentation of the application during development, capturing execution data as it naturally occurs during real usage. This preliminary data collection enables rapid generation of replay scripts that can be executed immediately against updated versions, allowing testing to keep pace with frequent updates without time loss.
Solution Approach 2:
The system enables continuous testing by automatically generating and executing replay scripts against each update iteration. The instrumentation remains active throughout the update cycle, continuously capturing new execution paths and immediately converting them into test cases, ensuring no testing time is lost between updates.
3Reliability
If comprehensive testing of all execution paths is performed, then reliability improves, but resource consumption increases
Solution Approach 1:
The application under test instruments itself to automatically generate execution data and testing scripts, eliminating the need for external testing infrastructure and manual effort. This self-service approach achieves comprehensive path verification while minimizing external resource consumption, as the application's own execution data becomes the test input.
Solution Approach 2:
The system captures actual user execution paths that naturally occur during real usage, focusing testing resources on the specific paths that matter most. By instrumenting to capture real-world execution rather than attempting to predefine all possible paths, the system achieves effective comprehensive testing with reduced resource overhead.
Data Source
AI summary
A method is disclosed including instrumenting a first version of an application on a plurality of end user devices and receiving execution data for the first version of the application from at least some of the plurality of end user devices. The execution data may be generated by the instrumentation in response to an execution of the first version of the application by the at least some of the end user devices. The method further includes automatically generating execution scripts based on the received execution data. The execution scripts may be configured to reproduce the execution of the first version of the application by the at least some of the end users devices. The method further includes automatically executing at least one of the execution scripts on an updated version of the application.