Dynamic Seam Adjustment for Video Stitching Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
When stitching multiple images captured by overlapping cameras, the movement of objects can result in unnatural movement artifacts due to shifting seams, particularly when the seam passes through prohibited regions set for moving objects, causing parallax issues.
Innovation Solution
An image processing device and method that dynamically sets prohibited regions based on the position and movement direction of objects within the overlap area, ensuring the seam does not pass through these regions, thereby stabilizing the object's appearance across frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If the seam is dynamically adjusted to avoid prohibited regions set for moving objects, then the object appearance is improved, but the seam position changes greatly causing unnatural movement artifacts
Solution Approach 1:
The patent applies preliminary action by detecting objects and setting prohibited regions in advance before seam calculation. The prohibited-region setting unit identifies moving objects and establishes prohibited regions based on their positions and movement directions, ensuring the seam calculation unit avoids these regions. This prevents the seam from passing through objects, maintaining stable object appearance while allowing dynamic seam adjustment to avoid prohibited regions.
2Productivity
If the seam passes through the overlap region to connect adjacent images, then the stitching is completed, but the moving object appears to move unnaturally due to parallax
Solution Approach 1:
The patent applies the taking out principle by extracting and removing the harmful effect of parallax-induced unnatural movement. The prohibited-region setting unit identifies regions where moving objects are detected and sets prohibited regions that exclude these object areas from seam passage. The seam calculation unit then calculates seams that avoid these prohibited regions, effectively removing the harmful parallax effect while maintaining stitching completion through alternative seam paths.
Solution Approach 2:
The patent uses an intermediary approach by introducing prohibited regions as a mediator between the seam calculation and object detection processes. The prohibited-region setting unit acts as an intermediary that receives object detection results and translates them into constrained regions that guide the seam calculation. This intermediary mechanism allows the seam to be calculated while respecting object boundaries, preventing unnatural movement artifacts without compromising stitching completeness.
3Reliability
If a fixed prohibited region is set for moving objects, then the seam avoids the object, but the seam cannot adapt to object movement causing unnatural appearance changes
Solution Approach 1:
The patent applies dynamics by making the prohibited region dynamically adjustable rather than fixed. The prohibited-region setting unit continuously monitors object positions and movement directions, updating the prohibited regions in real-time. This dynamic adjustment allows the seam calculation unit to adapt seam paths to moving objects, maintaining both reliable seam avoidance and adaptability to object movement, thereby preventing unnatural appearance changes.
Data Source
AI summary
To suppress an unnatural movement of a moving object at a connection point when a plurality of pieces of video data are connected to each other. A frame acquisition unit 22 acquires pieces of frame data captured at the same time, from the pieces of video data input from cameras C1 and C2. A prohibited-region setting unit 24 sets a prohibited region that is not set as a calculation target when a seam is calculated, based on a position of an object detected in an overlap region between adjacent pieces of frame data and a movement direction of the object. A seam calculation unit 25 calculates a seam between pieces of frame data adjacent to each other, without setting a pixel included in the prohibited region as the calculation target of seam calculation. A connection-frame output unit 26 connects pieces of frame data in accordance with the seam and outputs connection frame data 15.