Computer Vision Template Matching Across UI Scales and Themes

Solution Overview

Problem

Existing digital guidance systems face challenges in providing optimized template matching for user interfaces across different scales and resolutions, leading to increased time and resource consumption, especially when adapting to varying software application versions and themes.

Innovation Solution

A method involving grayscale conversion, background type detection and inversion, Gaussian filtering, scale adjustment, and reduced resolution to optimize template matching, utilizing system metadata for efficient scale estimation and theme agnostic detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple templates are created at different scales to achieve scale-invariant template matching, then template matching accuracy across different resolutions is improved, but computational time and system resource consumption increase

Engineering Contradiction:

Improvetemplate matching accuracyVSAvoidcomputational time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent applies Gaussian filtering and noise reduction to the template image before performing template matching. This preliminary processing prepares the template in advance to be more robust against scale variations and noise, reducing the need for extensive multi-scale template creation and subsequent computational iterations, thereby decreasing computational time while maintaining matching accuracy

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent modifies template matching parameters including applying different scaling factors, adjusting noise thresholds, and varying match evaluation criteria. By dynamically changing these parameters based on the specific matching context, the system achieves accurate template detection across different scales without requiring exhaustive multi-scale template generation, thus reducing computational resource consumption

Inventive Principle:

Principle #35Parameter changes

2Reliability

If multiple templates are created at different scales to achieve scale-invariant template matching, then template matching accuracy across different resolutions is improved, but system resource utilization increases

Engineering Contradiction:

Improvetemplate matching accuracyVSAvoidsystem resource utilization

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent applies Gaussian filtering and noise reduction to the template image before performing template matching. This preliminary processing prepares the template in advance to be more robust against scale variations and noise, reducing the need for extensive multi-scale template creation and subsequent computational iterations, thereby decreasing computational time while maintaining matching accuracy

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent modifies template matching parameters including applying different scaling factors, adjusting noise thresholds, and varying match evaluation criteria. By dynamically changing these parameters based on the specific matching context, the system achieves accurate template detection across different scales without requiring exhaustive multi-scale template generation, thus reducing computational resource consumption

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If template matching is performed on high-resolution images to maintain detection precision, then template location accuracy is improved, but computational complexity increases

Engineering Contradiction:

Improvetemplate location accuracyVSAvoidcomputational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies Gaussian filtering and noise reduction to the template image before performing template matching. This preliminary processing prepares the template in advance to be more robust against scale variations and noise, reducing the need for extensive multi-scale template creation and subsequent computational iterations, thereby decreasing computational time while maintaining matching accuracy

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent modifies template matching parameters including applying different scaling factors, adjusting noise thresholds, and varying match evaluation criteria. By dynamically changing these parameters based on the specific matching context, the system achieves accurate template detection across different scales without requiring exhaustive multi-scale template generation, thus reducing computational resource consumption

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12405812B2Computer vision template matching system and method

Publication Date: 2025.09.02 WHATFIX PTE LTD

AI summary

Provided herein are systems and methods for performing optimized template matching within a digital adoption platform used in conjunction with an underlying computer application. In some embodiments, a method includes capturing an image and template from a user interface of the underlying application, converting the image and template to grayscale, detecting a background type of the image and template and inverting one of the background types if there is a mismatch, applying a Gaussian filter to the image and template, dividing the scale of the image by the scale of the template to determine a starting scale, reducing a resolution of at least the image, applying a template matching algorithm to the image and template using the starting scale to locate the template in the image, and remapping the coordinates of the template to reflect its original location on the image.