GUI Test Reference Image Masking for Scale Invariance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Differences in screen resolutions and formats between various System Under Test (SUT) configurations lead to erroneous test failures due to non-scale-invariant reference images, causing image analysis to fail in identifying expected image objects.
Innovation Solution
A method that involves receiving a reference image, identifying text characters within its boundary, defining a mask region, applying transformations such as scaling, rotation, or skewing to the unmasked portion of the reference image, and matching the transformed image with the SUT's GUI to ensure accurate image recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If reference images are used for automated test descriptions, then image analysis can identify expected image objects in GUI, but differences in screen resolutions and formats between various SUT configurations cause erroneous test failures
Solution Approach 1:
The patent segments the reference image into multiple scaled versions (e.g., 100%, 50%, 25% sizes) and stores them in a cascade structure. During testing, the system selects and compares appropriate segmented versions based on the actual GUI image size, enabling accurate matching across different screen resolutions and device configurations.
Solution Approach 2:
The patent changes the size parameter of reference images by creating multiple scaled versions. This parameter transformation allows the same reference image to adapt to different display resolutions and screen formats, resolving the contradiction between maintaining test accuracy and achieving cross-device compatibility.
2Productivity
If the same test description is executed on multiple devices with different screen resolutions, then testing coverage increases, but image matching fails due to non-scale-invariant reference images
Solution Approach 1:
The patent performs preliminary action by pre-generating multiple scaled versions of reference images before actual testing begins. This advance preparation ensures that when tests are executed on devices with different screen resolutions, the appropriate scaled version is already available for immediate comparison, maintaining both testing efficiency and image matching accuracy.
Solution Approach 2:
The patent introduces dynamics by making the reference image selection adaptive rather than static. The system dynamically selects the appropriate scaled version of the reference image based on the actual size of the GUI image being tested, enabling the same test description to work accurately across devices with varying screen resolutions.
Data Source
AI summary
A method for performing an automated computer test that causes a test computer system to control a graphical user interface (GUI) of a system under test (SUT). The method includes: a) receiving a reference image representative of an expected image on the GUI of the SUT, the reference image having an image boundary; b) identifying whether one or more text characters are present within the image boundary; c) responsive to any identified text characters in step b), defining a mask region within the reference image boundary based on the identified text characters; d) identifying a transformed expected image on the GUI of the SUT using one or more transforms of the reference image, the identifying comprises matching an unmasked portion of a transformed reference image to a portion of the GUI; e) responsive to the identification of a transformed expected image, performing subsequent operations and/or verification steps on the SUT.