XR Privacy via Keyed Feature Transforms
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Extended reality (XR) technologies face challenges in preserving user privacy by inadvertently collecting sensitive data, such as images of real-world objects, which can be accessed by servers and degrade the XR experience when attempts are made to obscure these data, limiting user interaction with virtual representations.
Innovation Solution
XR devices transform sensitive object features using a feature transform key, allowing servers to process 'stand-in' virtual objects without accessing original sensitive data, enabling secure data transmission and maintaining interactive capabilities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Object-affected harmful factors
If sensitive objects are identified and redacted at the server, then other clients cannot see sensitive objects, but the XR server still has access to sensitive information
Solution Approach 1:
The patent extracts only the necessary geometric features (dimensions, shape, position) from sensitive objects while leaving out all identifying visual information. This extracted feature set is transmitted to the server, allowing the server to process the object for multi-user interaction without having access to the actual sensitive visual data.
Solution Approach 2:
The patent creates a simplified copy or representation of the sensitive object that contains only the geometric features needed for interaction. This copy is transmitted to the server instead of the original sensitive data, allowing the server to work with a non-sensitive representation that preserves functionality while eliminating privacy risks.
2Reliability
If an XR client blurs sensitive objects, then the server cannot receive sensitive object information, but the server cannot process the sensitive object degrading the XR experience
Solution Approach 1:
The patent segments the object data into two distinct parts: visual information (kept private and blurred) and geometric features (transmitted to server for processing). This segmentation allows the server to receive and process the geometric data for enabling multi-user interaction while the visual data remains protected and blurred on the client side.
Solution Approach 2:
The patent introduces an intermediary geometric feature representation that acts as a mediator between the sensitive visual data and the server processing requirements. This intermediary contains only the necessary geometric information (dimensions, shape, position) without the sensitive visual content, allowing the server to process the object for interaction while privacy is maintained.
3Object-affected harmful factors
If blurred virtual objects are transmitted to other users, then privacy is protected, but the objects lack defined dimensions preventing collision detection and interaction
Solution Approach 1:
The patent applies different quality levels to different aspects of the object data. The visual information is heavily obfuscated (blurred) to protect privacy, while the geometric features (dimensions, shape, position) are transmitted with high precision to enable accurate collision detection and physical interactions. Each aspect receives the appropriate level of detail for its intended use.
Solution Approach 2:
The patent creates a specialized copy of the object data that contains only the geometric features needed for physics calculations and interactions. This copy includes precise dimensional information, shape data, and position coordinates, while the visual representation remains blurred. The server receives this geometric copy for processing without accessing the sensitive visual content.
Data Source
AI summary
Systems and methods are provided herein for providing privacy while allowing interactions with virtual representations of sensitive objects. This may be accomplished by an extended reality (XR) system receiving information associated with a real-world environment of a user. The XR system may transmit the received information to one or more devices (e.g., server, XR device, etc.) for processing. Before transmitting the received information, the XR system can use the received information to identify and categorize objects in the real-world environment to identify sensitive objects. To protect the privacy of the sensitive objects, the XR system transforms features of the sensitive objects using one or more transform keys prior to transmitting the information to the one or more devices. The XR system may also update the transform keys after a first time period to further protect the privacy of the sensitive objects.