Embedded UI State Testing Using Visual Descriptors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for testing embedded systems face challenges in accurately processing images captured by robots due to varying conditions such as lighting, image sharpness, and display settings, leading to longer processing times and potential test failures.
Innovation Solution
A method using numerical descriptors to represent images of the user interface, allowing for quick and accurate identification of the device's state by comparing current descriptors with pre-stored images, and interacting with identified elements to measure response time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional image processing methods are used to capture and analyze UI images, then the system can identify device states, but the processing time increases and test reliability decreases due to varying lighting, sharpness, and display conditions
Solution Approach 1:
The patent extracts only the essential features from UI images by converting them to numerical descriptors, rather than processing the entire image data. This extraction approach removes irrelevant visual variations (lighting, sharpness) while retaining the core state identification information, thereby reducing processing time without sacrificing accuracy
Solution Approach 2:
The patent transforms image data from pixel-space to descriptor-space by changing the representation parameters. Instead of comparing raw pixels which are sensitive to environmental conditions, the system uses numerical descriptors that are invariant to lighting and display variations, improving both speed and reliability of state identification
2Reliability
If traditional image processing methods are used to capture and analyze UI images, then the system can identify device states, but test reliability decreases due to varying lighting, image sharpness, and display settings
Solution Approach 1:
The patent changes the representation parameters from raw pixel values to numerical descriptors that are inherently more robust to environmental variations. This parameter transformation makes the state identification process reliable across different lighting conditions, display settings, and image sharpness levels
Solution Approach 2:
The patent creates a simplified numerical copy of the visual state rather than working with the original complex image data. These numerical descriptors serve as reliable representations that capture the essential state information while being immune to harmful visual variations in the original images
3Reliability
If manual testing is performed on dozens of devices to ensure thorough testing, then test coverage improves, but the time and resources required increase significantly
Solution Approach 1:
The patent enables the testing system to automatically perform comprehensive tests across multiple devices without requiring manual intervention for each device. The automated robot with state identification capabilities can independently navigate, interact with UI elements, and verify device states, achieving both thoroughness and efficiency
Solution Approach 2:
The patent creates a universal testing system that can adapt to test multiple different device types and models using the same core technology. The numerical descriptor approach and automated robot can handle various UI layouts and device configurations, making the testing process both thorough and scalable across product lines
Data Source
AI summary
A method of testing an embedded system of the device using a testing robot, a central control unit and a device under test. The device under test may be in different states, wherein the states are determined using the testing robot with a visual sensor. After the state of the device is determined, the testing robot interacts with the device under test and changes its state to a new state.