Visual Testing Server Automating Set-Top Box UI Verification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current visual testing of content presentation in software applications, such as set-top boxes, is inefficient and labor-intensive due to reliance on manual tagging and the need for lengthy verification processes, which delays testing until code changes are approved.

Innovation Solution

A visual testing server utilizing a trained machine learning model to detect and verify screen elements automatically, allowing for the creation and execution of test scenarios independently of the underlying code, through REST APIs and image analysis, enabling efficient and consistent testing across different platforms.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual tagging of displayed features is used for visual testing, then testing can be performed, but the process becomes lengthy and repetitive

Engineering Contradiction:

Improvetesting accuracyVSAvoidtesting time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical tagging operations with an automated machine learning-based image recognition system. The ML model automatically detects and tags UI elements in screenshots, eliminating the need for manual human tagging while maintaining or improving detection accuracy through consistent algorithmic application.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The visual testing system performs self-service by automatically executing test scenarios, capturing screenshots, detecting UI elements, and comparing actual versus expected states without requiring continuous manual intervention. The system autonomously iterates through test cases and generates test results.

Inventive Principle:

Principle #25Self-service

2Reliability

If comprehensive black box testing is performed manually, then thorough verification is achieved, but developers must wait for code approval before testing can begin

Engineering Contradiction:

Improveverification thoroughnessVSAvoidtesting efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system enables preliminary testing actions by allowing test scenarios to be defined and prepared in advance based on design specifications, without waiting for code implementation or approval. When code is ready, the pre-prepared test scenarios can be executed immediately, reducing waiting time while maintaining comprehensive verification coverage.

Inventive Principle:

Principle #10Preliminary action

3Reliability

If manual verification of code changes is performed, then changes can be validated, but developers and testers must spend significant time on repetitive verification

Engineering Contradiction:

Improvechange validationVSAvoidoperational simplicity

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent replaces manual verification operations with automated machine learning-based detection and comparison systems. The ML model automatically validates UI element properties against expected values, eliminating repetitive manual verification while maintaining thorough validation through systematic algorithmic checking of multiple test scenarios.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Reliability

If traditional visual testing workflows are used, then testing can be performed, but the process requires lengthy back-and-forth between developers and testers

Engineering Contradiction:

Improvetesting completenessVSAvoiditeration time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The automated visual testing system enables continuous execution of test scenarios without interruption or manual handoff between developers and testers. The system continuously captures screenshots, detects UI elements, compares results against expectations, and generates test reports in an unbroken workflow, eliminating idle time and communication delays.

Inventive Principle:

Principle #20Continuity of useful action

Solution Approach 2:

The system implements automated feedback loops where test results are immediately generated and can trigger automatic notifications or issue tracking. This continuous feedback mechanism eliminates the need for manual back-and-forth communication, as the system autonomously reports test outcomes and can alert relevant stakeholders immediately upon detecting failures.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS12035004B2Visual testing based on machine learning and automated workflow

Publication Date: 2024.07.09 DISH NETWORK LLC

AI summary

An example method for visual testing of programmed display of content includes obtaining a workflow of test scenarios for visual testing of a display controlled by a set-top box (STB) device. The method also includes obtaining images that capture content displayed on the display, feeding the images to a trained machine learning model to detect display elements, and performing visual testing based on the detected display elements in accordance with visual expectations specified by the test scenarios.