Parallel Tracking and Clustering for Video Object Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital image and video editing on personal electronic devices is resource-intensive, making it challenging for users to apply desirable effects to captured images and videos, especially when dealing with dynamic salient objects.
Innovation Solution
A computer-implemented method and system that performs tracking, clustering, and segmentation operations in parallel to extract dynamic salient objects from digital images, allowing for their re-composition onto different images or videos, utilizing point tracking algorithms, motion clustering, and graph-based segmentation techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If digital image and video editing is performed on personal electronic devices, then users can apply effects to captured images and videos, but processing resources are excessively consumed
Solution Approach 1:
The patent segments the video processing task into two distinct components: (1) video object segmentation that extracts salient objects from video frames using computer vision algorithms, and (2) image composition that re-composes these extracted objects onto target images. This segmentation allows the computationally intensive segmentation operation to be performed once on video input, and the results to be reused across multiple frames, significantly reducing overall processing resource consumption while maintaining the ability to apply various effects.
2Adaptability or versatility
If video editing operations are performed on personal electronic devices, then users can enhance shareability of videos, but processing time increases
Solution Approach 1:
The patent performs preliminary video object segmentation on the video input to pre-identify and extract salient objects before the actual video editing or effect application. This preliminary action creates reusable object masks and segmentations that can be quickly applied during subsequent editing operations, eliminating the need to re-process the entire video frames for each editing operation and thereby significantly reducing processing time while maintaining versatility in effect application.
3Reliability
If traditional video editing methods are used, then complete video processing is achieved, but resource consumption and processing time are excessive
Solution Approach 1:
The patent extracts only the essential and computationally expensive video object segmentation operation from the complete video editing pipeline. By separating the segmentation task from subsequent editing operations, the system achieves reliable and complete video processing (maintaining all necessary editing capabilities) while dramatically improving productivity. The extracted segmentation results are then reused across multiple editing operations, avoiding redundant processing and reducing overall resource consumption and processing time.
Data Source
AI summary
Embodiments of the present disclosure include a computer-implemented method that receives a digital image input, the digital image input containing one or more dynamic salient objects arranged over a background. The method also includes performing a tracking operation, the tracking operation identifying the dynamic salient object over one or more frames of the digital image input as the dynamic salient object moves over the background. The method further includes performing a clustering operation, in parallel with the tracking operation, on the digital image input, the clustering operation identifying boundary conditions of the dynamic salient object. Additionally, the method includes combining a first output from the tracking operation and a second output from the clustering operation to generate a third output. The method further includes performing a segmentation operation on the third output, the segmentation operation extracting the dynamic salient object from the digital image input.