Video Synthesis Device for AR Avatar Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video distribution technologies require users to capture landscape images behind them using a selfie stick to synthesize computer graphics characters with their facial expressions, making it cumbersome to generate augmented reality video images with expressive avatars as backgrounds.
Innovation Solution
A video synthesis device and method that captures live-action and operator images, detects position and orientation, and synthesizes avatars in real space coordinates, allowing for the generation of augmented reality video images with expressive avatars integrated into live-action video frames, enabling users to capture landscapes in front of them without additional equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a selfie stick is used to capture landscape images behind the user, then the user can synthesize CG characters with the intended landscape as background, but the operation becomes cumbersome and requires additional equipment
Solution Approach 1:
The patent divides the imaging function into two separate imaging units: a first imaging unit for capturing the user's face and a second imaging unit for capturing the landscape. This segmentation allows each unit to specialize in its function, eliminating the need for a selfie stick while achieving the desired composition.
Solution Approach 2:
The video synthesis device integrates multiple functions into a single system: face tracking, landscape capture, coordinate system detection, and avatar synthesis. This multi-functional device replaces the need for separate equipment like selfie sticks, making the operation simpler while maintaining versatility.
2Productivity
If face-tracking technology is used to synthesize CG characters with user's facial expression, then the synthesis can be done in real-time with live-action video, but the background is limited to only what is behind the user
Solution Approach 1:
The patent introduces a coordinate system that maps real space to the video frame, allowing the avatar to be positioned in three-dimensional space rather than being constrained to a two-dimensional plane behind the user. This enables the avatar to appear in front of the user with the landscape as background.
Solution Approach 2:
The patent uses a coordinate system detector and position detector as intermediaries to bridge the gap between the user's face tracking and the landscape background. These detectors enable the system to understand spatial relationships and correctly position the avatar in the synthesized video.
3Adaptability or versatility
If the user captures images with a selfie stick to include both user and landscape, then the intended landscape can be captured as background, but the device complexity and setup requirements increase
Solution Approach 1:
The patent merges the face capture function and landscape capture function into a single video synthesis device with two imaging units. This combination eliminates the need for separate equipment like selfie sticks, reducing device complexity while maintaining the capability to capture both user and intended landscape.
Data Source
AI summary
A rear-facing camera captures a live-action video image while a front-facing camera captures an image of a distributor. An avatar controller controls an avatar based on the image of the distributor captured by the front-facing camera. A synthesizer arranges the avatar in a predetermined position of a real space coordinate system and synthesizes the avatar with the live-action video image. The face of the distributor captured by the front-facing camera is tracked and reflected on the avatar.