Image Retargeting via Blendshape Regularization and Error Function Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for creating realistic 3D animations of human faces are limited in accurately modeling complex geometries and subtle expressions, leading to unrealistic animations due to sensitivity in recognizing emotional shifts and noise in marker tracking data.
Innovation Solution
A computer-implemented method for image retargeting that acquires motion capture data from markers on a subject, calculates blendshapes based on marker configurations, and evaluates an error function to determine suitability for retargeting, using a weighted combination of shape primitives and physiological characteristics to optimize and regularize the animation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large number of markers are used to improve performance capture accuracy, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The patent replaces the mechanical marker-based tracking system with a direct optical/image-based tracking system. Instead of using physical markers that need to be tracked, the system uses image processing and physiological models to directly determine facial feature positions and movements from video frames, eliminating the need for markers while maintaining or improving measurement precision.
Solution Approach 2:
The patent extracts and removes the markers from the system entirely. By using direct image analysis of facial features (eyes, eyebrows, mouth, nose) and applying physiological constraints, the system achieves accurate tracking without any physical markers, thereby reducing device complexity while maintaining measurement precision.
2Manufacturing precision
If smoothing filters and low-pass filters are applied to remove noise in animation curves, then manufacturing precision is improved, but loss of information increases
Solution Approach 1:
The patent applies physiological constraints and regularization during the motion capture and tracking process itself, before animation curve generation. By incorporating knowledge of how human facial muscles and features actually move into the tracking algorithm, the system produces smooth, realistic animation data from the start, eliminating the need for post-processing smoothing that would remove subtle expression information.
Solution Approach 2:
The patent changes the approach from filtering animation curves after generation to using constrained optimization during data acquisition. By formulating the tracking as an optimization problem with physiological constraints, the system directly generates clean, accurate motion data that inherently has the desired smoothness without requiring aggressive filtering that would lose subtle expression details.
3Reliability
If performance capturing with multiple cameras is used to create realistic facial animations, then realism is improved, but device complexity increases
Solution Approach 1:
The patent extracts and removes the complex multi-camera hardware system, replacing it with a single camera or even standard video input. The system achieves realistic facial animation by using advanced image processing, physiological models, and optimization algorithms to extract accurate motion data from simpler imaging sources, eliminating the need for complex multi-camera setups.
Solution Approach 2:
The patent replaces the mechanical multi-camera capture system with a computational approach using single-camera video input. By substituting physical capture complexity with computational complexity (image processing and optimization algorithms), the system achieves the same or better realism while dramatically reducing device complexity.
Data Source
AI summary
Systems and methods for image retargeting are provided. Image data may be acquired that includes motion capture data indicative of motion of a plurality of markers disposed on a surface of a first subject. Each of the markers may be associated with a respective location on the first subject. A plurality of blendshapes may be calculated for the motion capture data based on a configuration of the markers. An error function may be identified for the plurality of blendshapes, and it may be determined that the plurality of blendshapes can be used to retarget a second subject based on the error function. The plurality of blendshapes may then be applied to a second subject to generate a new animation.