HMD Camera Rendering Virtual Body Parts for AR Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing electronic devices, such as head-mounted display (HMD) devices, struggle to effectively synthesize and display augmented reality (AR) images that include external body parts covered by other HMD devices, leading to blocked nonverbal expressions and impaired user interaction.
Innovation Solution
A head-mounted display (HMD) device equipped with a camera, communication circuit, display, memory, and processors, which controls the camera to obtain images, detects external HMD devices, receives information about covered body parts, performs rendering to create virtual images, and displays composite images that include both real and virtual elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If HMD devices are worn by multiple users in close proximity, then user interaction and communication are enhanced, but body parts become blocked from view and nonverbal expressions are lost
Solution Approach 1:
The patent creates virtual copies of body parts (eyes, mouth, etc.) that are blocked from direct view. The HMD device captures images of the user's body parts using onboard cameras, processes these images to generate virtual representations, and displays them to other users. This copying mechanism preserves nonverbal expression information even when the actual body parts are physically blocked by other HMD devices.
Solution Approach 2:
The patent introduces an intermediary system consisting of image capture devices, processing circuits, and display mechanisms. This intermediary captures the blocked body parts, transforms them into virtual image data, and transmits this data to other users' devices. The intermediary effectively bridges the gap created by physical blocking, allowing nonverbal expressions to be communicated without direct line-of-sight.
2Adaptability or versatility
If HMD devices capture and process images of external environment, then augmented reality experience is enhanced, but device complexity and processing requirements increase
Solution Approach 1:
The patent makes the HMD device's camera system and image processing capabilities serve multiple functions. The same cameras and processing circuits used for general augmented reality environment capture are also used to capture and process images of the user's own body parts. This multi-functionality reduces the need for separate dedicated hardware for expression capture, thereby managing device complexity while maintaining enhanced AR functionality.
Data Source
AI summary
A processor of a head-mounted display (HMD) device according to an embodiment controls a camera to obtain a first image of the outside of the HMD device. The processor detects a portion of the first image corresponding to an external HMD device. The processor receives information with respect to a body part covered by the external HMD device from the external HMD device through a communication circuit. The processor obtains a second image representing the body part by performing rendering on the body part based on the information. The processor displays the second image positioned in the portion of the first image and a composite image including the first image through at least one display.