GPU-Driven Screen Update Sensing for Low-Power Context Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Capturing and processing screen frames at high frequencies for user context analysis significantly impacts power consumption and battery life in computing devices, despite the use of specialized AI engines, due to excessive compute and bandwidth requirements.
Innovation Solution
Implementing an API-based mechanism in the GPU driver to asynchronously notify AI engines of screen updates, using HID events and graphical rendering information to determine active screens and changed content, thereby reducing unnecessary screen capture and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If screen frames are captured and processed frequently for user context analysis, then the accuracy and responsiveness of AI-based context processing is improved, but power consumption and battery life are significantly degraded
Solution Approach 1:
The system uses GPU-driven screen update events to self-regulate when screen capture is necessary. The GPU driver automatically generates events when screen content changes, eliminating the need for continuous polling or fixed-rate capturing. This self-service mechanism ensures captures occur only when meaningful updates happen, reducing power consumption while maintaining analysis accuracy.
Solution Approach 2:
The patent changes the capture frequency parameter dynamically based on screen update events rather than using a fixed high frequency. By adjusting the capture rate from continuous to event-driven, the system maintains responsiveness when needed while dramatically reducing power consumption during static periods.
2Loss of information
If screen capture frequency is increased to capture all user context changes, then completeness of context data is improved, but compute resources and memory requirements are excessively increased
Solution Approach 1:
The patent extracts only the essential information needed for context analysis by using GPU-driven events to identify and capture only meaningful screen changes. Instead of capturing entire frames continuously, the system extracts and processes only the portions of screen content that have actually changed, reducing memory requirements while maintaining context completeness.
3Speed
If continuous screen capture is performed at high frame rates, then responsiveness to user actions is improved, but bandwidth requirements and processing overhead are significantly increased
Solution Approach 1:
The system implements periodic screen capture triggered by GPU-driven update events rather than continuous capturing. This event-driven periodic action maintains responsiveness by immediately capturing when screen changes occur, while avoiding the processing overhead of continuous high-frame-rate capturing during static periods.
Data Source
AI summary
Various systems and methods for contextual capture and processing of screen capture data are disclosed. An example method for screen capture data processing in a computing device may include: determining an active screen of the computing device based on a user interaction event; identifying a graphics rendering event associated with a software application presented in the active screen; identifying screen capture data in a buffer (e.g., of graphics processing circuitry such as a GPU) that corresponds to the graphics rendering event; and communicating a contextual screen update event via an application programming interface (e.g., a an API operated by a GPU driver). The receipt of this contextual screen update event can be used by an AI engine to control whether to perform contextual processing on particular frames of the screen capture data.