Robotic Touch Screen Tester With Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional testing platforms are inadequate for testing touch screen enabled devices, as they cannot accurately simulate human interaction with input controls that can be placed anywhere on the screen, leading to inefficiencies in testing durability, battery performance, and screen sensitivity.
Innovation Solution
A robotic device tester with a controller and various tips that can calibrate to a touch screen's coordinate system, use object recognition, and perform multi-touch operations to simulate human interaction, allowing for repeatable and cost-effective testing of touch screen enabled devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional testing platforms are used to test touch screen devices, then the testing process is simplified, but the testing accuracy and ability to simulate human interaction deteriorates
Solution Approach 1:
The patent replaces traditional mechanical testing platforms with a robotic system that uses computer vision and automated image processing to detect and interact with touch screen elements. The robotic device captures images of the touch screen, processes them to locate input controls, and automatically touches the screen at the correct positions, thereby maintaining testing simplicity while significantly improving testing accuracy and simulation of human interaction.
2Measurement precision
If manual testing of touch screen devices is performed to locate input controls, then testing accuracy improves, but productivity and cycle time deteriorate
Solution Approach 1:
The system employs automated image processing and object recognition algorithms that enable the robotic device to independently locate input controls on the touch screen without human intervention. The robotic device captures the screen display, processes the image to identify clickable elements, and automatically determines their positions, thereby maintaining high accuracy while dramatically improving productivity and reducing testing cycle time.
3Productivity
If automated testing is implemented without object recognition to locate input controls, then productivity improves, but measurement precision and reliability deteriorate
Solution Approach 1:
The patent introduces an image processing and object recognition system as an intermediary between the robotic device and the touch screen. This intermediary captures the screen display, identifies input controls through pattern recognition, and provides precise location information to the robotic device. This enables the system to maintain high productivity through automation while ensuring accurate detection of input controls that may be dynamically positioned on the screen.
Data Source
AI summary
A touch screen testing platform may be used to perform repeatable testing of a touch screen enabled device using a robotic device tester and a controller. Prior to running a test, the controller and/or robot may be calibrated to determine a planar surface of the touch screen and to establish a relative coordinate system across the touch screen. The controller may then be programmed to allow a robot to engage the touch screen using known input zones designated using the coordinate system. The platform may employ object recognition to determine and interact with content rendered by the device. The platform may use various types of tips that engage the touch screen, thereby simulating human behavior. The platform may perform multi-touch operations by employing multiple tips that can engage the touch screen simultaneously.