Secure-Domain Perception Detection for Private User Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing user devices face challenges in securely processing user interactions while protecting privacy, as continuous data capture by microphones and cameras raises concerns about data accessibility and storage outside secure memory areas.
Innovation Solution
Implementing a secure system process that analyzes input data from cameras and microphones within a secured memory area, ensuring that captured images and audio are never allowed outside this area, and using face and audio embeddings to determine user interactions without exposing raw data to application processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If continuous data capture by microphones and cameras is implemented for user interaction detection, then user interaction detection capability is improved, but data security and privacy protection deteriorate
Solution Approach 1:
The system segments data processing into two distinct parts: a secure domain that captures and processes raw sensor data (camera and microphone), and an application domain that receives only processed results. This segmentation ensures that raw captured data remains isolated in the secure domain while interaction detection results are safely transmitted to applications, resolving the contradiction between continuous monitoring capability and data security.
2Adaptability or versatility
If raw captured data is made accessible to application processes for analysis, then data processing flexibility is improved, but privacy protection deteriorates
Solution Approach 1:
The system extracts only the essential interaction detection results from the raw captured data and transmits them from the secure domain to application processes. By taking out only the necessary information (interaction status, not raw data), the system maintains application processing flexibility while preventing privacy exposure, as applications receive processed results without access to sensitive raw captured data.
3Ease of operation
If captured data is stored outside secure memory areas for application access, then data accessibility is improved, but security protection deteriorates
Solution Approach 1:
The system introduces an intermediary mechanism (the secure domain's data transmission interface) that enables controlled data exchange between the secure memory area and application processes. This intermediary allows applications to access interaction detection results without direct access to raw captured data stored in secure memory, thereby improving data accessibility for legitimate purposes while maintaining security protection through controlled access channels.
Data Source
AI summary
Aspects of the subject technology provide for, processing by a secure system process at a device, captured video data and captured audio data, and determining whether a user is attempting to interact with the device. The data related to the captured video and audio data can be provided to an application process that interacts with the secure system process to determine identity information for the video data and audio data, and frame position data for the people detected in the video data. The system can utilize the video data and the audio data together to determine who in the frame was speaking and if they were directing their speech toward the device.