Garment Animation Attention Map Correction for Temporal Coherence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems generate inaccurate and visually distracting animations of garments, often smoothing high-frequency patterns and causing unnatural motion, background motion, and facial expressions, leading to inefficient use of computing resources.
Innovation Solution
A garment animation system that modifies self-attention maps of a UNet based diffusion model using optical flow and cross-frame attention to suppress spurious motion, generating temporally coherent animations without additional training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional systems generate garment animations using standard diffusion models, then the animation generation process is simple, but the visual quality deteriorates with smoothed patterns and unnatural motion
Solution Approach 1:
The patent introduces an optical flow estimation module as an intermediary component that computes motion fields between consecutive frames. This mediator provides accurate motion information to guide the diffusion model, preventing spurious motion artifacts while maintaining pattern fidelity. The optical flow acts as a bridge between the input image sequence and the generated animation, enabling precise control over motion dynamics without requiring complex retraining of the diffusion model.
Solution Approach 2:
The patent segments the animation generation process into distinct functional modules: optical flow estimation, attention map modification, and frame generation. By dividing the complex task into separable components, each module can be optimized independently. The attention map modification step specifically targets regions with spurious motion, applying corrections only where needed rather than processing the entire image uniformly, thus improving visual quality efficiently.
2Loss of energy
If conventional systems use standard diffusion models for garment animation, then computing resource usage is high due to re-processing, but the animation quality is poor
Solution Approach 1:
The patent performs preliminary optical flow estimation and attention map modification before the diffusion model generates the animation frames. By pre-computing the motion fields and preparing corrected attention maps, the system avoids the need for multiple re-processing iterations to correct spurious motion. This preliminary preparation reduces redundant computing operations and energy consumption while ensuring high animation quality from the first generation pass.
Solution Approach 2:
The patent implements a feedback mechanism where the optical flow information from consecutive frames is used to correct the attention maps, which then guide the diffusion model generation. This closed-loop approach ensures that motion artifacts are suppressed and pattern fidelity is maintained without requiring extensive trial-and-error re-processing, thereby reducing computing resource consumption while preserving animation quality.
3Productivity
If the diffusion model generates animations independently without attention map correction, then the generation process is fast, but spurious motion and unnatural artifacts occur
Solution Approach 1:
The patent introduces attention map modification as an intermediary processing step that sits between the optical flow estimation and the diffusion model generation. This intermediary component adjusts the attention maps to suppress spurious motion in specific regions while preserving genuine garment motion. The modification is computationally efficient and does not significantly slow down the generation process, yet it dramatically improves temporal coherence and reduces unnatural artifacts in the final animation.
Data Source
AI summary
Embodiments are disclosed for generating an animated garment video. The method may include receiving a text prompt describing a garment by a diffusion model. The diffusion model generates an animation corresponding to the text prompt. The animation includes a sequence of frames generated by the diffusion model depicting the garment in motion. A frame of the sequence of frames is generated using a flow map of the frame, an attention map of a previous frame, and an attention map of the frame.