Cloud-Based Mobile App Testing via Gesture Playback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The proliferation of mobile applications across various devices and platforms has made functional testing challenging due to the complexity of touch and gesture-based user interfaces, leading to time-consuming and error-prone manual testing processes, with existing automation methods relying on fragile optical recognition techniques and requiring jailbreaking or tethering.
Innovation Solution
A cloud-based testing platform that captures and plays back gesture-based user inputs on mobile devices without tethering or jailbreaking, using a TouchTest Driver component library to record and validate touch gestures within the mobile app, enabling precise testing of continuous touch gestures and internal state changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual testing is used to test mobile applications across different devices and platforms, then testing coverage can be comprehensive, but the testing process becomes extremely time-consuming and labor-intensive
Solution Approach 1:
The patent uses optical recognition techniques to capture images of the mobile device screen and processes these images to identify UI elements and their positions. This allows the testing system to automatically detect and interact with UI elements without requiring physical presence or manual operation, significantly reducing testing time while maintaining comprehensive coverage across multiple devices and platforms
Solution Approach 2:
The patent introduces an intermediary testing system that acts as a bridge between the test controller and the mobile application under test. This intermediary system includes components for capturing screen images, processing them to identify UI elements, and automatically performing test actions, thereby eliminating the need for manual testing operations while ensuring thorough testing coverage
2Extent of automation
If optical recognition techniques are used for automated testing, then testing automation is achieved, but the technique becomes fragile and brittle to software revisions
Solution Approach 1:
The patent implements a feedback mechanism where the testing system continuously captures screen images during test execution, processes these images to identify UI elements, and uses this information to dynamically adjust test actions. This feedback loop allows the system to adapt to software revisions by detecting changes in UI element positions or characteristics and automatically adjusting the testing approach, thereby maintaining testing stability despite automation
Solution Approach 2:
The patent employs dynamic image processing techniques that can adapt to changes in the mobile application interface. The system processes captured images in real-time to identify UI elements regardless of their exact positions, and can adjust test actions based on the detected element locations. This dynamic approach makes the automated testing system more resilient to software revisions while maintaining high automation levels
3Extent of automation
If jailbreaking is performed to enable functional test automation, then access to internal application state is obtained, but the device security and integrity are compromised
Solution Approach 1:
The patent introduces an intermediary layer that interfaces with the mobile application through authorized channels such as public APIs or documented interfaces. This intermediary testing system can observe and interact with the application without requiring jailbreaking, thereby maintaining device security and integrity while still enabling automated testing functionality through legitimate access methods
4Extent of automation
If tethering is used to connect mobile device to computer for testing, then optical recognition can be performed, but the testing setup becomes complex and less flexible
Solution Approach 1:
The patent captures images of the mobile device screen and processes these images to perform optical recognition and UI element identification. By using image copying and processing techniques, the system can achieve automated testing capabilities without requiring physical tethering connections, thereby simplifying the testing setup while maintaining the ability to perform optical recognition and interaction
Data Source
AI summary
A system for worldwide cloud-based testing of mobile applications includes a plurality of mobile computing devices geographically distributed across a plurality of different countries. One or more servers are connected with the mobile computing devices over one or more networks. The one or more servers run a program for playback of a test composition which includes a timed sequence of clips on the mobile computing devices. The timed sequence of clips include one or more messages each of which instructs a precise gesture-based action applied to at least one object displayed on the touch-sensitive display screen. The precise gesture-based action corresponds to a functional aspect of the mobile application and consists of a plurality of data elements previously captured from inside the mobile application while the mobile application was subject to the gesture-based user input entered via the touch-sensitive display screen.