Augmented Reality Video Generation With Motion-Matched Virtual Cameras
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video creation tools fail to account for camera motion, limiting the quality and realism of background visual effects and integration of virtual foreground objects, as they do not leverage the sophisticated motion tracking capabilities of modern mobile devices.
Innovation Solution
An augmented reality application that tracks the physical camera's movement and orientation to dynamically update the virtual camera's position and orientation within a 3D model, allowing for realistic background and foreground elements that respond to camera motion, using chroma key technology and on-device rendering.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If simple one-to-one pixel replacement is used for chroma key background replacement, then the implementation is simple and fast, but the visual quality and realism deteriorate because camera motion is not accounted for
Solution Approach 1:
The patent applies dynamics by making the virtual background responsive to camera motion. Instead of using a static one-to-one pixel replacement, the system dynamically updates the virtual background position and orientation based on detected camera movement, creating a realistic effect that adapts to changing camera states while maintaining implementation feasibility through motion tracking integration
Solution Approach 2:
The system implements feedback by detecting camera motion and using this information to adjust the virtual background in real-time. The camera motion detection provides feedback about physical movement, which is then processed to update the virtual environment, creating a closed-loop system that maintains visual consistency between the physical camera movement and the rendered background
2Device complexity
If the computer-generated background stays fixed within the camera's frame of reference, then the implementation is simpler, but the realism deteriorates because it does not update as the physical camera moves through physical space
Solution Approach 1:
The patent transforms the static background into a dynamic one by continuously updating the virtual background position and orientation based on camera motion detection. This dynamic approach maintains realism while managing complexity through efficient motion tracking and rendering updates
Solution Approach 2:
The system creates a virtual copy of the camera's perspective in the rendered background. By replicating the camera's motion in the virtual environment, the system maintains the illusion of a physical set without requiring actual physical movement of background elements, achieving realism through virtual replication
3Ease of manufacture
If AR effects are limited to foreground objects only, then the implementation is simpler, but the versatility deteriorates because background detection and replacement remains difficult
Solution Approach 1:
The patent extracts the background replacement problem from the difficult general case and solves it specifically for chroma key scenarios. By focusing on chroma key backgrounds rather than attempting general background segmentation, the system achieves versatile background effects while maintaining implementation simplicity through the use of color-based keying
Solution Approach 2:
The system achieves multi-functionality by enabling both foreground AR objects and background replacement within a unified framework. The virtual camera approach allows the same rendering pipeline to handle both foreground virtual objects and background environment replacement, increasing versatility without proportionally increasing complexity
Data Source
AI summary
Systems and methods are described for generating an AR image are described herein. A physical camera is used to capture a video of a physical object in front of a physical background. The system then accesses data defining a virtual environment and selects a first position of a virtual camera in the virtual environment. While capturing the video, the system displays captured video of the physical object, such that the physical background is replaced with a view of the virtual environment from the first position of the virtual camera. In response to detecting a movement of the physical camera, the system selects a second position of the virtual camera in the virtual environment based on the detected movement. The system then displays the captured video of the physical object, wherein the view of the physical background is replaced with a view of the virtual environment from the second position of the virtual camera.