On-Device Screen Activity Summarization for Privacy-Aware Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently processing screen activity data on computation devices while balancing resource usage, privacy, and ethical considerations, leading to high computational costs and potential privacy breaches.
Innovation Solution
A dynamic on-device screen activity data processing system that captures, summarizes, and transmits data using an intelligent agent equipped with AI models, optimizing resource usage and privacy by locally processing data before transmission to a remote server based on configurable policies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If screen activity data is captured and processed in detail, then monitoring accuracy and task optimization are improved, but computational resource usage and energy consumption increase
Solution Approach 1:
The patent segments the data processing workload between on-device AI models and remote servers. The on-device model performs initial processing and filtering of screen activity data, while more intensive analysis is performed remotely. This segmentation allows detailed monitoring where needed while conserving local computational resources.
Solution Approach 2:
The system applies partial processing action by selectively analyzing only certain portions of screen activity data based on configured policies and AI-determined priorities. Not all screen data is processed with the same level of detail,而是 only the most relevant portions are subjected to intensive analysis.
2Productivity
If more screen activity data is transmitted to remote servers, then task performance optimization is improved, but data transmission overhead and network usage increase
Solution Approach 1:
The patent extracts and transmits only the most relevant and processed screen activity data to remote servers, rather than transmitting all raw data. The on-device AI model filters and prioritizes data before transmission, reducing network overhead while maintaining the quality of data needed for task optimization.
Solution Approach 2:
Data processing and filtering actions are performed preliminarily on the device before transmission to the server. This preliminary processing reduces the volume and improves the quality of data that needs to be transmitted, thereby reducing network overhead while maintaining productivity benefits.
3Productivity
If comprehensive screen monitoring is implemented, then productivity monitoring and security are improved, but user privacy is compromised
Solution Approach 1:
The patent applies local quality by processing and analyzing data locally on the user's device using on-device AI models. This allows comprehensive monitoring capabilities to be maintained while keeping sensitive data processing local, thereby reducing privacy risks associated with transmitting all raw data to external servers.
Solution Approach 2:
The system provides self-service monitoring capabilities through on-device AI processing, enabling the device to autonomously analyze and prioritize its own screen activity data. This reduces the need for external server processing of sensitive data, thereby maintaining privacy while achieving monitoring objectives.
4Power
If on-device processing resources are increased, then data processing capability is improved, but device cost and complexity increase
Solution Approach 1:
The patent implements dynamic resource allocation where the on-device AI processing capability is adjusted based on current device conditions, policy configurations, and data priority assessments. This dynamic approach allows the system to maximize processing capability when needed while minimizing resource usage and apparent complexity during normal operation.
Solution Approach 2:
The on-device AI model serves multiple functions: it processes screen activity data, filters information, prioritizes content for transmission, and makes real-time decisions about data handling. This multi-functionality reduces the need for separate specialized components, thereby reducing overall device complexity while maintaining strong processing capability.
Data Source
AI summary
Method, system, and computer program product for dynamic on-device screen activity data processing. Data of screen activity on a display screen of a computation device is captured. Textual information of the screen activity is extracted from the data captured and stored locally in a database. Summarized data of the screen activity is generated using the textual information stored in the database and transmitted to a remote server configured to perform a task using summarized data received. Usage demand of on-device resources of the computation device for performing screen activity data processing operations is dynamically determined according to a policy defining at least one rule associated with at least one configurable parameter and representing a tradeoff between conserving usage of the on-device resources and optimization of the summarized data received at the remote server for the task.