Phase-Based Eulerian Motion Modulation via Complex Steerable Pyramids
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Eulerian video magnification methods support relatively small magnification factors and tend to significantly amplify noise when increasing the magnification factor, limiting their effectiveness in revealing small movements and introducing artifacts.
Innovation Solution
A phase-based approach using complex-valued steerable pyramids that directly manipulate phase variations in videos, allowing for larger magnification factors with reduced noise and fewer artifacts by modifying phase rather than amplitude, and employing temporal bandpassing to isolate specific frequency bands.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing Eulerian video magnification methods are used to increase magnification factor, then motion magnification capability is improved, but noise amplification and artifacts significantly increase
Solution Approach 1:
The patent changes the fundamental parameter being manipulated from amplitude to phase. By representing video frames using complex steerable pyramids and operating in the phase domain rather than amplitude domain, the method achieves motion magnification while avoiding the noise amplification that plagues amplitude-based approaches. The phase representation allows for larger magnification factors without the harmful side effects.
Solution Approach 2:
The patent introduces complex steerable pyramids as an intermediary representation between the original video and the magnified output. This complex-valued transform domain serves as a mediator that separates motion information (phase) from intensity information (amplitude), allowing selective manipulation of phase to achieve magnification while filtering out noise through temporal bandpassing.
2Measurement precision
If existing Eulerian methods increase magnification factor to reveal smaller movements, then detection sensitivity is improved, but processing stability deteriorates due to noise amplification
Solution Approach 1:
By transitioning from amplitude-based to phase-based representation, the patent fundamentally changes the parameter domain. This phase domain operation allows for enhanced detection sensitivity through larger magnification factors while maintaining processing stability because phase manipulation does not amplify noise in the same way amplitude manipulation does.
Solution Approach 2:
The patent applies temporal bandpassing to the phase component before magnification to pre-filter out noise and unstable frequency components. This preliminary filtering action ensures that subsequent magnification operations operate on cleaned, stable phase information, thereby maintaining processing stability while achieving high detection sensitivity.
3Measurement precision
If phase-based manipulation with complex steerable pyramids is used, then magnification factor capability is improved, but computational complexity increases
Solution Approach 1:
The patent segments the video processing task into distinct frequency bands using steerable pyramids with different scales and orientations. By decomposing the video into multiple sub-bands and processing each separately, the method achieves high magnification capability while managing computational complexity through localized, efficient filtering operations rather than global complex computations.
Solution Approach 2:
The patent applies temporal bandpassing to selectively filter only the frequency bands of interest rather than processing the entire frequency spectrum. This partial action approach focuses computational resources on the relevant temporal frequencies, achieving high magnification capability for specific motions while reducing overall computational complexity by ignoring irrelevant frequency components.
Data Source
AI summary
In one embodiment, a method of amplifying temporal variation in at least two images includes converting two or more images to a transform representation. The method further includes, for each spatial position within the two or more images, examining a plurality of coefficient values. The method additionally includes calculating a first vector based on the plurality of coefficient values. The first vector can represent change from a first image to a second image of the at least two images describing deformation. The method also includes modifying the first vector to create a second vector. The method further includes calculating a second plurality of coefficients based on the second vector.