Dynamic HMI Test Script Generation via Image Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional HMI testing methods are unreliable and time-consuming, as they rely on manual screenshot capturing and verification, leading to undetected design errors and increased time-to-market due to frequent requirement changes and high effort in writing and rewriting test-scripts for numerous screen transitions.
Innovation Solution
An auto script generation system that dynamically generates automation test scripts for HMI testing by utilizing a Transition Logic Database and View Database to identify HMI components and determine reference images, reducing manual intervention and error probability through automated validation of screen transitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual screenshot capturing and verification is used for HMI testing, then the testing process can be performed with simple tools, but the reliability of testing is poor and design errors are not properly detected
Solution Approach 1:
The patent introduces an automated testing system that acts as an intermediary between the HMI product and manual testing processes. This system includes components for automatic screenshot capturing, image processing modules for comparing screenshots against design requirements, and automated test script generation. The intermediary system bridges the gap between simple manual testing and complex automated validation, enabling reliable detection of design errors such as width, height, and font-size mismatches without requiring extensive manual intervention.
2Reliability
If separate test-scripts are written for each HMI screen transition, then comprehensive testing coverage is achieved, but the time and effort required for script creation becomes excessive
Solution Approach 1:
The patent implements self-service automation where the testing system automatically generates test scripts by capturing screenshots during HMI screen transitions and processing them through image recognition algorithms. The system autonomously creates comprehensive test coverage for all screen transitions without requiring manual script writing for each transition. The automated script generation includes extracting HMI component information, generating validation rules, and producing executable test scripts, thereby eliminating the excessive time and effort associated with manual script creation while maintaining comprehensive testing coverage.
3Adaptability or versatility
If manual test-script rewriting is performed for modified HMI requirements, then the testing adapts to changes, but significant delay is introduced in time to market
Solution Approach 1:
The patent implements a dynamic testing system that automatically adapts to HMI requirement changes through continuous screenshot capturing and image processing. When HMI requirements are modified, the system dynamically updates test scripts by comparing new screenshots against updated design specifications using image recognition and difference analysis algorithms. This dynamic adaptation mechanism eliminates the need for manual test-script rewriting, allowing the testing system to automatically adjust to changes in HMI screens, components, and transitions, thereby maintaining testing adaptability while significantly reducing the time delay associated with manual updates.
4Ease of manufacture
If conventional HMI testing methods are used, then the testing process is simple to implement, but manual errors are introduced and quality is compromised
Solution Approach 1:
The patent replaces manual mechanical testing operations with automated computational systems. The system uses automated screenshot capturing tools to record HMI screens, image processing algorithms to analyze and compare screenshots against design requirements, and automated validation logic to detect errors in width, height, font-size, and other HMI parameters. This substitution of manual mechanical processes with automated computational mechanisms maintains ease of implementation through centralized system control while dramatically improving testing quality by eliminating manual errors and providing consistent, objective validation of HMI design compliance.
Data Source
AI summary
Disclosed herein is a system and a method for generating automation test scripts dynamically for Human Machine Interface (HMI) testing. Test configurations are generated for a Device Under Test (DUT) (115) based on a screen transition dataset upon receiving a user request (105). Based on test configurations and the screen transition dataset, HMI components located in a test path are identified for transitions from source HMI screens to destination HMI screens. Based on the HMI components, reference images are determined for the destination HMI screens from a screen design dataset. The test configurations are executed on the DUT (115) via an external simulating device (113) and transition associated with the test path is validated based on matching the real time destination HMI screen image with the reference image. Based on a validation result, automation test report (117) is generated for the DUT (115).