Touchless Body UI for Augmented Reality Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing messaging applications lack the ability to provide user interface elements that can be activated without physical contact with the device, limiting user interaction, especially in scenarios where users want to engage with augmented reality features without holding the device, such as taking full-body photos or trying on virtual wearable items.
Innovation Solution
The implementation of a body UI AR component that uses a digital image sensor to detect collisions with user-selectable elements, allowing touchless operation by tracking anchor points on a human body image and triggering actions like recording or image capture when a hand object collides with these elements, enabling users to interact with messaging clients without physical contact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional touch-based UI elements are used, then the device can be operated with simple hardware, but users cannot interact with the device without physical contact, limiting AR feature accessibility
Solution Approach 1:
The patent introduces an intermediary system consisting of a camera/digital image sensor and image processing algorithms that mediate between the user's hand and the UI elements. The camera captures images, the system detects hand objects and their movements, and translates these into touch interactions with UI elements, enabling touchless operation without requiring complex hardware modifications to the device itself
Solution Approach 2:
The patent replaces the mechanical touch-based interaction system with an optical-based system. Instead of requiring physical contact between the user's finger and the screen, the system uses a camera to capture optical information about hand movements and translates these into digital signals that trigger UI actions, substituting mechanical interaction with optical detection and digital processing
2Adaptability or versatility
If the device is held in hand for operation, then traditional UI interaction is possible, but users cannot take full-body photos or try on virtual wearable items
Solution Approach 1:
The patent adds a spatial dimension to the interaction model by enabling the device to be positioned at a distance from the user rather than requiring constant hand-held contact. The camera captures the user's full body or upper body in the frame, and hand gestures in this extended spatial arrangement trigger UI actions, allowing users to take full-body photos and try on virtual items without holding the device
Solution Approach 2:
The system enables self-service operation where the user's own body (specifically hand gestures visible in the camera frame) serves as the control mechanism. The user can interact with the device by making gestures with their hand that is visible in the captured image, allowing the system to self-interpret and respond to user intentions without requiring physical contact or complex external controls
3Reliability
If touch-based UI elements are used, then the interface can be simple and direct, but users must physically touch the device which interrupts the AR experience
Solution Approach 1:
The patent implements a dynamic interaction system where the method of interaction changes based on the context. The system can operate in different modes: traditional touch-based interaction when the device is held close, and gesture-based interaction when the device is positioned at a distance for AR features. The interaction mode dynamically adapts to the user's physical arrangement with the device, maintaining reliability across different usage scenarios
Data Source
AI summary
The technical problem of providing user interface elements that can be activated without physical contact with any specific device is addressed by configuring an augmented reality component to display a touchless user selectable element that can be activated in response to detecting collision of an object from the output of a digital image sensor of a camera with a user selectable element. The augmented reality component detects collision of the touchless user selectable element with an object from the person image, such as a hand object, and, in response, triggers a predetermined action, such as, for example, starting or stopping recording of the output of a digital image sensor or capturing a still image of the output of a digital image sensor.