UI Screenshot Validation to Block Fake Action Prompts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Malicious software can trick users into triggering unintended actions by presenting fake user interface prompts on compromised computing devices, which are difficult to detect and prevent using existing hardware-protected prompts due to limited rendering capabilities and manufacturer-specific secure-world environments.
Innovation Solution
A computing device captures a screenshot of the user interface and sends it to a trusted server for text-based validation, using character recognition to determine if the text matches expected prompts, controlling device actions based on the validation response.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a TEE or secure-world environment is used to present UI prompts, then security against malicious software is improved, but device complexity and manufacturing requirements worsen
Solution Approach 1:
The patent introduces a server as an intermediary that performs text validation of UI prompts. Instead of requiring each device to have complex TEE infrastructure, the device captures a screenshot of the UI prompt and sends it to the server, which validates the text content and returns a verification result. This mediator approach maintains security while reducing device complexity.
Solution Approach 2:
The patent replaces the mechanical/hardware-based TEE system with a software-based solution using screenshot capture and server-side text validation. Instead of relying on hardware isolation and secure-world environments, the system uses image processing and character recognition to verify UI prompt authenticity, substituting physical security mechanisms with computational ones.
2Reliability
If TEE is used to render UI prompts, then security is improved, but ease of manufacture worsens due to manufacturer-specific configurations
Solution Approach 1:
The patent creates a universal validation system that works across different device manufacturers and models. The server-based text validation approach is manufacturer-agnostic and does not require device-specific TEE configurations. Any device capable of capturing screenshots and communicating with the server can benefit from this security mechanism, making the solution universally applicable rather than manufacturer-specific.
3Reliability
If TEE with limited font-rendering capabilities is used, then security is improved, but ease of operation worsens
Solution Approach 1:
The patent uses screenshot capture to create a visual copy of the UI prompt for validation purposes. Instead of relying on the TEE's limited font-rendering capabilities to display complex layouts, the system captures the actual rendered output and sends it to the server for text extraction and validation. This copying approach bypasses the rendering limitations while maintaining security verification.
Data Source
AI summary
A computing device engages in text-based validation of a user interface (UI) presented on a display of the computing device, including (i) capturing a screenshot of the display when the UI is presented on the display, (ii) transmitting to a server a validation request providing the captured screenshot, and (Hi) receiving from the server, in response to the validation request, a validation response 2024/076457 based at least on (a) character recognition of text depicted by the screenshot and (b) a determination of whether the character-recognized text corresponds with an associated action. Further, the computing device uses the received validation response as a basis to control whether to allow the computing device to take the associated action in response to user input into the computing device when the UI is presented on the display.WO