Touchscreen Test Platform for Dynamic UI Target Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional testing platforms struggle to effectively test touch screen devices due to the dynamic nature of input controls, which can change positions on the screen, making it difficult to determine and engage these controls consistently.
Innovation Solution
A robotic testing platform that utilizes a camera to dynamically recognize the location of target features on a touch screen, calibrates a coordinate system, and engages the screen using a robot arm equipped with a moveable tip to simulate user interactions, including the use of pattern recognition and optical character recognition to locate and interact with dynamic input controls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional testing platforms with fixed-location inputs are used, then the device structure is simple and easy to test, but the testing platform cannot adapt to touch screen devices where input controls change position dynamically
Solution Approach 1:
The patent replaces traditional mechanical position-based input methods with an optical recognition system. A camera captures images of the touch screen display, and image processing algorithms automatically identify the coordinates of input controls regardless of their position. This substitution allows the testing platform to adapt to dynamically changing control locations without requiring complex mechanical repositioning mechanisms.
Solution Approach 2:
The patent introduces an image processing system as an intermediary between the testing platform and the touch screen device. This intermediary layer captures visual information from the screen, processes it to locate input controls, and translates it into actionable coordinates for the testing robot. This mediator enables the system to handle dynamic control positions while maintaining a relatively simple overall structure.
2Productivity
If manual methods are used to determine input control locations, then the testing can be flexible, but the testing efficiency and repeatability are reduced
Solution Approach 1:
The testing system performs self-positioning and self-calibration through automated image recognition. The robot automatically captures images of the touch screen, processes these images to determine the locations of input controls, and adjusts its own movement and engagement actions accordingly. This self-service capability eliminates the need for manual intervention in locating controls, significantly improving testing efficiency and repeatability while keeping the system relatively simple.
Solution Approach 2:
The system implements a feedback loop where the camera continuously monitors the touch screen display, the image processing system analyzes the current state to locate input controls, and this information feeds back to control the robot's actions. This closed-loop feedback mechanism enables automatic adaptation to changing control positions, improving both efficiency and repeatability without requiring complex pre-programming or manual setup.
3Measurement precision
If the testing system uses fixed coordinate systems, then the system is simple to implement, but it cannot accurately engage dynamically positioned target features
Solution Approach 1:
The patent transitions from a static coordinate system to a dynamic one that adapts to the actual display content. The system captures real-time images of the touch screen, processes these images to identify the current positions of input controls, and dynamically adjusts the coordinate references accordingly. This dynamic approach ensures accurate engagement of target features regardless of their changing positions while maintaining implementation simplicity through automated image processing.
Solution Approach 2:
The system changes the coordinate parameters based on the actual display state rather than using fixed predetermined coordinates. By capturing images and processing them to extract the actual positions of input controls, the system dynamically updates its coordinate parameters to match the current display configuration. This parameter change strategy enables accurate target engagement while keeping the overall system relatively simple through automated recognition.
Data Source
AI summary
A touch screen testing platform may be used to engage a dynamically positioned target feature being displayed on a touch screen enabled device during a testing protocol. The platform may record imagery displayed by the touch screen device and then analyze the imagery to locate the target feature within a reference coordinate system. The platform may recognize that the target feature is missing from the imagery and respond by causing the touch screen device to scroll through a command menu and/or toggle through virtual screens. Once located, the platform may instruct a robotic device tester to select the target feature by contacting the touch screen at the identified location using a conductive tip designed to simulate a user's fingertip, Prior to running a test, the camera may be focused to a point that is offset from the display screen of the touch screen device.