Marker-Based Facial Capture System for Unencumbered Performance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Facial motion capture systems face challenges in capturing high-resolution facial performances while allowing actors to move freely, as traditional methods either restrict movement due to camera placement or cause discomfort with physical attachments, and markerless systems may still suffer from reduced range of motion and comfort issues.
Innovation Solution
A facial image capture system that uses a marker-based location detection system to automatically adjust the direction, zoom, and focus of video cameras to maintain a consistent view of the face, allowing for high-resolution image capture without physical camera attachments, and can switch between multiple actors and perspectives while predicting occlusions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If cameras are mounted on a head-mounted device directly in front of the performer's face, then the resolution of facial performance capture is improved, but the physical presence of cameras introduces discomfort and may physically interfere with actions
Solution Approach 1:
The patent uses markers placed on the performer's face as copies or proxies for the actual facial features being captured. These markers serve as surrogate objects that reflect light back to cameras, allowing high-resolution capture without requiring physical cameras to be mounted on the performer's head. The markers capture the essential information needed for facial performance reconstruction while eliminating the harmful physical presence of heavy camera equipment.
Solution Approach 2:
The patent introduces markers as intermediary objects between the performer's face and the cameras. These markers act as mediators that carry the facial performance information to the cameras without requiring direct physical contact or mounting on the performer. The markers reflect light from the cameras back to the sensors, enabling indirect but high-fidelity capture of facial movements while keeping the performer comfortable and unrestricted.
2Adaptability or versatility
If the performance volume is large with cameras fixed on a gantry, then the range of motion is improved, but the spatial resolution of facial performance capture deteriorates
Solution Approach 1:
The patent replaces the mechanical gantry-based camera positioning system with a marker-based optical tracking system. Instead of physically moving cameras on a mechanical gantry to maintain resolution, the system uses markers attached to the performer's face that can be detected by fixed cameras from multiple angles. This substitution eliminates the need for complex mechanical positioning while maintaining high spatial resolution through the marker's reflective properties and multi-camera geometry.
Solution Approach 2:
The patent transitions from a two-dimensional planar camera arrangement on a gantry to a three-dimensional spatial distribution of markers on the performer's face. By placing markers across the surface of the face in three-dimensional space, the system enables resolution maintenance from multiple camera angles simultaneously, allowing the performer to move freely within a large volume while preserving capture quality through the added spatial dimension of marker distribution.
3Loss of information
If markerless motion capture systems are used, then the density of recorded information is improved, but the range of motion and comfort issues persist
Solution Approach 1:
The patent applies local quality by placing markers only in specific locations on the face where they are most needed for capture, rather than covering the entire face or body. This selective placement provides sufficient information density for facial performance capture while minimizing the visual impact and physical constraints on the performer. The markers are strategically positioned to capture key facial movements without requiring full-face coverage or restrictive equipment.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables high-resolution, comfortable, and unencumbered capture of facial performances, allowing actors to move freely while generating accurate 3D models and animated performances, even in the presence of occlusions, by continuously adjusting camera positions and perspectives based on marker data.
Implementation Method 1
Marker-based motion capture systems record the trajectories of a set of dots or markers on the performer's face and body, typically using a set of cameras fixed to a gantry around the perimeter of a performance volume.
Implementation Method 2
A video camera may capture sequential images of a scene to which the video camera is directed.
Implementation Method 3
The camera control system may include a pan mirror in front of the lens that controls the horizontal direction to which the video camera is directed and/or a tilt mirror in front of the lens that controllably the vertical direction to which the video camera is directed.
Data Source
AI summary
A facial image capture system may capture images of a face of a person while the person is moving. A video camera may capture sequential images of a scene to which the video camera is directed. A marker-based location detection system may determine and generate information about the location of a marker worn on or close to the face of the person. A camera control system may automatically adjusts both the horizontal and vertical direction to which the video camera is directed so as to cause the sequential images of the camera to each be of the face of the person while the person is moving, based on the information about the location of the marker from the marker-based location detection system.