Room-Scale Virtual Sets With Real-Time View-Dependent Backgrounds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion capture technologies using LED walls for video game design face challenges in capturing multiple perspectives concurrently and integrating dynamic background changes relative to actor movements, limiting set and costume design, and requiring tools for immersive game level design and virtual landscape interaction.
Innovation Solution
A system comprising motion capture tracking sensors, a display wall projection system, and computing engines that modify virtual backgrounds in real-time based on actor perspectives, allowing for dynamic field of view adjustments and immersive game content generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional green screen chroma keying is used, then background can be easily replaced, but light spill onto subject and limited color choices for set/costume design occur
Solution Approach 1:
The system dynamically changes the background parameters (visual content, lighting conditions) displayed on LED walls in real-time based on camera position and actor location, replacing static green screens with adaptive virtual environments that provide ambient lighting without chroma keying limitations
Solution Approach 2:
LED walls serve as an intermediary between the physical mocap set and virtual backgrounds, providing real-time visual feedback and ambient lighting that eliminates the need for chroma keying while enabling unlimited background and color design options
2Adaptability or versatility
If LED walls are used for motion capture, then dynamic backgrounds and ambient lighting are provided, but capturing multiple perspectives concurrently and integrating dynamic background changes relative to actor movements is limited
Solution Approach 1:
The system divides the LED wall into multiple independently controllable segments or zones, each capable of displaying different visual content and lighting conditions, enabling simultaneous capture of multiple perspectives with customized backgrounds for each camera angle
Solution Approach 2:
The system implements real-time dynamic adjustment of background content and lighting on LED walls based on captured actor positions and camera movements, allowing backgrounds to change relative to actor movements and supporting multiple concurrent perspectives with adaptive visual environments
3Device complexity
If camera movements are restricted to dolly track or preselected shots, then rendering computer graphic content is simplified, but filming flexibility and immersive design are reduced
Solution Approach 1:
The system dynamically adjusts virtual background content, lighting conditions, and camera parameters in real-time based on actual camera position and orientation, eliminating the need for preselected shots and dolly track restrictions while maintaining rendering efficiency through adaptive rather than static pre-rendering
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables concurrent capture of multiple perspectives, enhances set and costume design freedom, and provides tools for immersive game level design and virtual landscape interaction, improving the realism and flexibility of video game content creation.
Implementation Method 1
A typical motion capture system uses one or more digital cameras to record the movement of the actor while illuminating a number of markers attached at a number of locations on a suit that the actor wears
Implementation Method 2
LED walls which are typically large screens made up of light emitting diodes to display video and any other visual content
Data Source
AI summary
A system adapted to generate video game content by real-time modifying a virtual background displayed on a display wall from a perspective of a physical actor is described. A first set of motion capture tracking sensors, coupled to the physical actor, is used to generate a first set of motion capture data. A first computing engine receives the first set of motion capture data and determines a first field of view of the physical actor based at least in part on the first set of motion capture data. A display wall projection system, that is at least 60 feet wide and 18 feet tall, defines a second field of view that is larger than the first field of view. A second computing engine configured generates video game content for display and transmits the video game content to the display wall projection system. The first computing engine transmits the first field of view to the second computing engine which then generates a first set of changes to the video game content transmitted to the display wall projection system based on the first field of view.