AR/MR Communication Device Adapting Avatars to Local Environments
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current AR/MR communication technologies limit natural interaction by requiring users to focus on virtual content, restrict movement within a single room, and fail to accurately share real environment elements or facial expressions beyond hand gestures, making them less usable for non-conference room settings.
Innovation Solution
An AR/MR system that adapts virtual representations of users and objects to different environments by using in-facing and out-facing cameras to capture gestures and environmental data, allowing for immersive, bi-directional communication without the need for external sensors, and enabling avatars to interact naturally with their surroundings.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AR/MR communication uses virtual representations and avatars to enable remote interaction, then communication capability is improved, but natural interaction and immersion are reduced because users cannot accurately share real environment elements or facial expressions
Solution Approach 1:
The patent uses cameras to capture real-world visual information and creates virtual representations (avatars) that copy and transmit facial expressions, gestures, and environmental context. Multiple cameras capture different views including front-facing cameras for facial expressions and out-facing cameras for environmental context, which are then rendered as virtual representations in the remote user's field of view.
Solution Approach 2:
The system dynamically adjusts parameters of the virtual representation including position, orientation, and visual characteristics based on the user's head movements and environmental context. The virtual avatar's parameters are continuously updated to match the user's real-world actions and maintain natural interaction.
2Ease of operation
If AR/MR communication requires users to focus on virtual content, then virtual interaction is enabled, but user mobility is restricted within a single room
Solution Approach 1:
The system combines multiple functions into a single AR/MR device including cameras for capturing environmental data, sensors for tracking user movement, and display for showing virtual content. This multi-functional device enables both virtual interaction and environmental awareness simultaneously, allowing users to move freely while maintaining communication capability.
Solution Approach 2:
The virtual representation acts as an intermediary between the user and the remote environment. Instead of requiring users to directly interact with virtual content, the system mediates by projecting virtual avatars and environmental elements into the user's real-world view, allowing natural movement while maintaining interaction.
3Measurement precision
If AR/MR communication uses external sensors to capture environmental data, then environmental awareness is improved, but device complexity increases
Solution Approach 1:
The AR/MR device integrates multiple sensing functions into unified components. The cameras serve dual purposes: capturing images for virtual representation and sensing environmental geometry for spatial mapping. Inertial sensors track both user movement for avatar animation and environmental features for localization, reducing the need for separate specialized sensors.
Data Source
AI summary
Methods and apparatus are provided for adaptation between different environments in Augmented Reality or Mixed Reality (AR/MR) communication in an AR/MR system. The AR/MR system includes at least two communication devices such as a first and a second communication device. The first and second communication devices are in their respective physical environments and are associated with their respective users. The spatial and semantic characteristics of both the user and the physical environments are captured in order to adapt to the virtue representations of the users to each other's context, while still keep the focus of the conversation towards users' interaction. The embodiments herein will give a natural, immersive and bi-directional unintrusive communication, as if the other person would be present next to you, without disrupting either person's activity.