Out-of-process hit-testing for XR privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human-computer interface technologies fail to effectively balance user privacy and efficiency, particularly in extended reality systems that track user inputs like hand gestures and eye gaze, as they often require sharing sensitive interaction data with applications.
Innovation Solution
Implementing an out-of-process hit-testing system that filters and renders user input data within a trusted operating system process, providing privacy by restricting application access to preliminary interactions while allowing rendered feedback to the user, and redirecting confirmed user inputs to applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If user input data is shared with applications for interaction handling, then interaction efficiency is improved, but user privacy deteriorates due to exposure of sensitive interaction data
Solution Approach 1:
The system segments the hit-testing process into two distinct components: a trusted hit-testing service that performs privacy-preserving hit testing, and application processes that receive only the results. This segmentation allows the system to maintain privacy protection while enabling efficient interaction handling, as the trusted service filters and processes raw input data before sharing results with applications.
Solution Approach 2:
The trusted hit-testing service acts as an intermediary between the user input source and the applications. It receives raw user input, performs hit testing against UI elements, and provides results to applications without exposing the actual input data. This intermediary role enables applications to function efficiently while user privacy remains protected by the trusted mediator.
2Measurement precision
If all user inputs are processed by applications, then interaction accuracy is improved, but system resource consumption worsens due to redundant processing
Solution Approach 1:
The trusted hit-testing service performs preliminary hit testing and filtering of user inputs before they reach applications. By pre-processing the input data and identifying relevant UI elements, the system reduces the burden on applications to perform redundant processing, thereby conserving system resources while maintaining interaction accuracy through the trusted service's preliminary analysis.
3Object-affected harmful factors
If preliminary interactions are filtered from applications, then user privacy is improved, but application functionality deteriorates due to lack of interaction data
Solution Approach 1:
The trusted hit-testing service provides feedback to applications in the form of hit-test results that indicate which UI elements were targeted, without revealing the actual user input data. This feedback mechanism enables applications to respond appropriately to user interactions while maintaining privacy protection, as applications receive sufficient information to function correctly without accessing sensitive raw input data.
Data Source
Figure 1~2
Figure 3A~3B
Figure 4~5
AI summary
Out-of-process hit-testing can provide increased privacy and efficiency in computer user input systems. In an aspect, an application specifies a control style for a UI window to be managed separately from the application, such as by a system process running outside of the application process. When a user input is received at a location corresponding to a part of the application UI that is separate from the UI window for which the control style has been specified, the user input may be redirected to the UI window for which the control style has been specified.