Synchronized Multi-Spectral Cameras for Markerless Facial Motion Capture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing motion capture systems interfere with an actor's performance by using head-mounted cameras and visible markers, and there is a need for improved methods that can accurately capture facial expressions without these obstructions.
Innovation Solution
A motion capture system using synchronized and aligned cameras, including a taking camera in the visible light spectrum and witness cameras in the infrared light spectrum, captures facial expressions without head-mounted cameras by employing invisible markers that are visible only to the witness cameras, allowing for precise alignment and synchronization with microsecond precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If head-mounted cameras and visible markers are used to capture facial expressions, then motion capture accuracy is improved, but actor performance and comfort deteriorate due to interference and obstruction
Solution Approach 1:
The patent extracts the camera from the actor's head and places it on the set, eliminating the interference of head-mounted equipment. The visible markers are replaced with infrared markers that are invisible to the actor and audience, achieving accurate capture without performance interference
Solution Approach 2:
The patent introduces infrared markers as an intermediary that bridges the gap between visible and invisible detection. These markers are invisible to human eyes but clearly detectable by infrared cameras, enabling accurate facial expression capture without visible obstructions
2Adaptability or versatility
If multiple camera systems capturing different light spectra are used, then content generation flexibility is improved, but system complexity and synchronization difficulty increase
Solution Approach 1:
The patent segments the camera system into distinct functional components: visible light cameras for capturing the actor's performance and infrared cameras for tracking invisible markers. Each camera type is optimized for its specific function, simplifying the overall system architecture while maintaining versatility
Solution Approach 2:
The patent implements synchronization mechanisms where camera triggers are coordinated through a central control system that receives feedback from each camera's frame rate and timing characteristics, ensuring precise alignment of captured images from different spectral ranges
3Measurement precision
If visible markers are used on the actor's face, then motion capture precision is improved, but image quality for content production deteriorates due to marker visibility
Solution Approach 1:
The patent applies different quality characteristics to different parts of the imaging system: infrared markers provide high-precision local tracking information on the actor's face, while the visible light capture maintains natural image quality without marker interference. Each system optimizes for its specific purpose
Data Source
AI summary
A motion capture system comprising: a master clock configured to repeatedly generate and output, at a frame rate, a primary clock signal that conveys when a video frame starts; a first camera configured to capture light within a first set of wavelengths and operably coupled to receive the master clock signal and initiate an image capture sequence on a frame-by-frame basis in fixed phase relationship with the primary clock signal to generate a first set of images at the frame rate from light captured within the first set of wavelengths; a synchronization module operably coupled to receive the master clock signal from the master clock and configured to generate a synchronization signal offset in time from and in a fixed relationship with the primary clock signal; and a second camera configured to capture light within a second set of wavelengths, different than the first set of wavelengths, and operably coupled to receive the synchronization signal and initiate an image capture sequence on the frame-by-frame basis in a fixed phase relationship with the synchronization signal to generate a second set of images at the frame rate from light captured within the second set of wavelengths; wherein the amount of time the synchronization signal is offset from the master clock signal aligns, within each frame, the image capture sequence of the second camera with the image capture sequence of the first camera.