Video Frame Object Segmentation for Motion Artefact Compensation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video stabilization methods fail to effectively compensate for camera motion and rolling shutter artefacts in video frames, leading to degraded video quality, as they either oversimplify motion compensation or do not account for the unique movements of multiple objects in a scene.
Innovation Solution
The method involves segmenting objects in video frames using three-dimensional pose estimation and depth information, modeling camera movement, and compensating motion artefacts individually for each segmented object based on its relative movement, allowing for selective stabilization of camera and object motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current video stabilization methods are used, then processing speed is maintained, but video quality is degraded due to ineffective motion compensation
Solution Approach 1:
The video frame is segmented into multiple regions of interest (ROIs) based on detected objects. Each ROI is processed independently with object-specific motion compensation parameters, allowing differentiated stabilization for different objects while maintaining overall video quality without excessive complexity
Solution Approach 2:
Different motion compensation parameters are applied to different regions of the video frame based on the detected objects and their relative movements. This allows each object to be stabilized according to its specific motion characteristics rather than applying a uniform stabilization approach to the entire frame
2Reliability
If simple motion compensation is applied, then processing complexity is reduced, but camera shake and rolling shutter artefacts are not effectively compensated
Solution Approach 1:
Objects are detected and their motion characteristics are analyzed in advance before applying motion compensation. This preliminary analysis allows the system to prepare object-specific compensation parameters, improving the accuracy of motion compensation for camera shake and rolling shutter artifacts
Solution Approach 2:
Different motion compensation parameters are derived and applied for each detected object based on its specific motion characteristics. This includes object-specific translation, rotation, and scaling parameters that adapt to the relative movement between objects and the camera
Data Source
Figure 1
Figure 2A
Figure 2B~3C
AI summary
An apparatus for or processing video frames is presented. The apparatus comprises at least one processing unit and at least one memory. The at least one memory stores program instructions that, when executed by the at least one processing unit, cause the apparatus to segment objects appearing in video frames, model movement of a camera recording the video frames, and individually compensate motion artefacts in the video frames for at least one segmented object based on relative movement of the camera and at least one segmented object.