3D Projection Masking for Moving Performers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional projection systems fail to effectively project content onto moving objects, such as actors, that appears to move with and change based on the object's orientation and location, often resulting in unrealistic effects like 'blow-by' where content is projected onto the background instead of the object.
Innovation Solution
A system combining 3D tracking with 2D information to generate a mask of the object, allowing the content to be modified and projected only within the object's silhouette, thereby hiding latency and tracking errors, and ensuring the content adheres to the object's movement and orientation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional two-dimensional tracking is used to project content onto moving objects, then the system is simple to operate, but the content does not appear to stick to or move with the object, producing unrealistic effects
Solution Approach 1:
The patent transitions from two-dimensional tracking to three-dimensional tracking of the performer. The system captures depth information and uses it to calculate the performer's position and orientation in 3D space, allowing content to be projected with accurate spatial correspondence. This dimensional enhancement enables the content to appear to stick to and move with the performer realistically.
Solution Approach 2:
The patent introduces an intermediary processing system that receives both 2D camera data and 3D depth sensor data, combines them to generate a mask of the performer, and uses this mask to modulate the projected content. This intermediary processing layer reconciles the simplicity of 2D operation with the realism of 3D projection by acting as a mediator between the two approaches.
2Reliability
If three-dimensional tracking is used to track performer movement and modify content in real-time, then the content appears to move with and change based on performer orientation, but latency and computational errors increase
Solution Approach 1:
The system performs preliminary actions by pre-calculating the performer's mask from depth data and pre-positioning the projected content according to predicted performer location. The content is prepared in advance based on where the performer is expected to be, reducing the computational burden during real-time projection and minimizing latency.
Solution Approach 2:
The patent rushes through the computational process by using optimized algorithms that quickly process depth data to generate masks and transform content coordinates. The system skips unnecessary computational steps and uses efficient matrix operations to rapidly calculate the projected content position, delivering real-time performance despite the complexity of 3D tracking.
3Manufacturing precision
If content is projected without masking based on performer position, then the projection area covers the full scene, but content is projected onto the background instead of the performer, creating blow-by effects
Solution Approach 1:
The patent applies local quality by generating a mask that identifies only the specific regions corresponding to the performer's body. This mask has different properties (transparent vs. opaque) for different spatial locations, allowing the projection system to selectively apply content only where the performer is present. This localized approach ensures precise content placement on the performer while avoiding projection onto the background.
Data Source
AI summary
The present application relates generally to a projection system and method for projecting content onto a moving performer. In one embodiment, the system includes a projector that projects light onto the performer, a position sensor, such as a camera and/or depth sensor that captures position information about the performer, and a computer that modifies original content based on the position information of the performer. The computer uses the position information to generate a mask, such as a silhouette, of the performer, and then modifies the original content based on the mask of the performer. The modified content is then projected by the projector onto the performer.