UI Threat Detection Using Triggered Screenshot Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing devices with modest resources face challenges in detecting computing threats efficiently due to the need for large and complex models, which consume significant computing resources and bandwidth, and there is a need to conserve resources, especially in battery-powered devices.
Innovation Solution
Implementing a rules-based approach to generate screenshots of potentially critical display content, encoding the content, and transmitting or analyzing it remotely to minimize resource usage, using application-specific rules to reduce the number of screenshots generated, transmitted, and analyzed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If large and complex models are used for threat detection, then detection accuracy is improved, but processor loading and energy consumption increase
Solution Approach 1:
The system divides the threat detection task into two segments: a lightweight on-device model that performs initial screening of screenshots, and a larger remote model that provides comprehensive analysis for flagged cases. This segmentation allows the device to maintain low energy consumption while still achieving high detection accuracy through the remote model's capabilities.
Solution Approach 2:
The system performs preliminary threat assessment using a lightweight local model before engaging the more resource-intensive remote analysis. This preliminary action filters out benign cases, ensuring that the larger remote model only processes potentially threatening screenshots, thereby reducing overall energy consumption while maintaining detection accuracy.
2Reliability
If continuous monitoring and screenshot capture are performed, then threat detection capability is improved, but bandwidth consumption and data storage increase
Solution Approach 1:
Instead of analyzing every screenshot captured, the system applies partial action by using the lightweight local model to selectively identify only those screenshots that warrant further analysis. This approach maintains strong threat detection capability while significantly reducing bandwidth consumption and data storage requirements by transmitting only a subset of screenshots to the remote system.
Solution Approach 2:
The lightweight local model acts as an intermediary that filters screenshots before they are transmitted remotely. This intermediary layer prevents unnecessary data transmission and storage by identifying and blocking benign screenshots, thereby reducing bandwidth consumption and data storage requirements while maintaining effective threat detection through the remote model.
3Reliability
If comprehensive user interface monitoring is implemented, then security coverage is improved, but device complexity increases
Solution Approach 1:
The monitoring system is segmented into multiple layers: a comprehensive but lightweight local monitoring component that captures all UI events, and a more specialized remote analysis system that handles deep inspection. This segmentation allows the device to maintain comprehensive security coverage without overburdening the local system with complex analysis algorithms.
Solution Approach 2:
The local lightweight model serves as an intermediary between the comprehensive UI monitoring system and the remote analysis system. It processes the monitored data locally with minimal complexity, filtering and preparing only necessary information for remote transmission, thereby reducing the overall system complexity while maintaining comprehensive security coverage.
Data Source
AI summary
A system enables a method for detecting and mitigating a computing threat. The method includes monitoring, by a computing device, a user interface of the computing device. The computing device encodes an output of the user interface to generate a text encoding. The computing device analyzes the text encoding to detect one or more triggers and performs a screen capture of the user interface to generate a screenshot responsive to the detecting the one or more triggers. The computing device transmits the screenshot via a network. The computing device receives via the network an indication based on the screenshot and controls a function of the computing device based on the indication based on the screenshot.