Video Retargeting via Motion Saliency Crop Paths
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video editing technologies face challenges in effectively retargeting multiple synchronized videos to preserve salient features and maintain a smooth viewing experience, particularly in sports events where camera views need to be seamlessly transitioned to avoid jerky changes.
Innovation Solution
The system employs a motion saliency engine to determine salient features in videos, a video retargeting engine to generate modified videos with a camera crop path that zooms in on these features, and a view selection engine to optimize the selection of views based on action scores, ensuring smooth transitions and adherence to cinematographic principles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multiple synchronized videos are retargeted to preserve salient features, then the viewing experience is enhanced by maintaining focus on action, but abrupt changes between camera views occur causing jerky transitions
Solution Approach 1:
The system performs preliminary actions by pre-calculating motion saliency scores and action scores for all videos before final view selection. The motion saliency engine analyzes motion content in advance to identify salient features, and the view selection engine pre-computes action scores based on these features, enabling smooth transitions without abrupt changes while preserving salient content.
Solution Approach 2:
The system implements feedback mechanisms where the motion saliency engine continuously monitors motion content and adjusts salient feature identification accordingly. The view selection engine uses action scores as feedback to guide camera view transitions, ensuring that transitions are smooth and maintain focus on salient features while avoiding jerky changes between different camera perspectives.
2Reliability
If camera crop paths are applied to zoom in on salient features, then focus on action is maintained, but sudden zooms cause jerky changes in viewing experience
Solution Approach 1:
The system applies dynamics by making camera crop paths adaptive and flexible rather than fixed. The view selection engine dynamically adjusts crop windows based on real-time motion saliency scores and action scores, allowing smooth zooming into salient features while avoiding sudden, jerky camera movements. The crop paths are continuously optimized based on the analyzed motion content.
Solution Approach 2:
The system changes parameters such as crop window size, position, and motion saliency thresholds dynamically during video processing. By adjusting these parameters based on the analyzed motion content and action scores, the system achieves smooth camera movements that focus on salient features without causing abrupt visual changes or jerky transitions.
3Manufacturing precision
If manual definition of protected areas is required, then precise control over pixel modification is achieved, but editing complexity and time consumption increase
Solution Approach 1:
The system performs self-service by automatically identifying salient features and determining optimal camera crop paths without requiring manual user input. The motion saliency engine autonomously analyzes motion content to find salient features, and the view selection engine automatically selects camera views based on action scores, eliminating the need for manual definition of protected areas while maintaining precise control over pixel modification through automated algorithms.
Data Source
AI summary
Methods and systems for video retargeting and view selection using motion saliency are described. Salient features in multiple videos may be extracted. Each video may be retargeted by modifying the video to preserve the salient features. A crop path may be estimated and applied to a video to retarget each video and generate a modified video preserving the salient features. An action score may be assigned to portions or frames of each modified video to represent motion content in the modified video. Selecting a view from one of the given modified videos may be formulated as an optimization subject to constraints. An objective function for the optimization may include maximizing the action score. This optimization may also be subject to constraints to take into consideration optimal transitioning from a view from a given video to another view from another given video, for example.