Video Foreground Edge Refinement for Segmentation Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing digital video segmentation methods produce visible artifacts at the contours of foreground subjects due to a rough approximation of foreground masks, causing blending of foreground and background, which affects the quality of the segmented video sequence.
Innovation Solution
A method that involves determining the edges of foreground subjects in video frames and modifying the pixel mask based on these edges to improve the accuracy of foreground segmentation, using techniques like edge detection and morphological operations to refine the segmentation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If simple pixel comparison is used for segmentation, then processing speed is improved, but segmentation precision deteriorates due to visible artifacts at contours
Solution Approach 1:
The patent divides the segmentation process into two distinct stages: first generating a rough foreground mask through simple pixel comparison for speed, then refining contour regions through edge detection and morphological operations for precision. This multi-stage segmentation approach resolves the contradiction by applying different processing strategies to different parts of the image.
Solution Approach 2:
The patent applies different processing quality to different regions: simple pixel comparison is sufficient for background regions, while edge detection and morphological refinement are applied specifically to contour regions where precision is critical. This local differentiation maintains overall processing speed while improving segmentation precision at critical boundaries.
2Manufacturing precision
If edge detection and morphological operations are added to refine segmentation, then segmentation precision is improved, but device complexity increases
Solution Approach 1:
The patent segments the processing pipeline into distinct modules: initial mask generation, edge detection, morphological operations, and final mask refinement. Each module performs a specific function, making the overall complex process more manageable and implementable through structured modular design.
Solution Approach 2:
The patent performs preliminary edge detection and morphological operations on the initial foreground mask before final segmentation. By preparing refined contour information in advance, the system reduces the complexity of subsequent processing steps and improves overall segmentation precision efficiently.
3Loss of time
If rough approximation of foreground mask is used, then processing time is reduced, but artifact generation increases at contours
Solution Approach 1:
The patent extracts and isolates contour regions from the overall image through edge detection, then applies specialized morphological refinement operations specifically to these extracted contour areas. This targeted approach eliminates artifacts at contours without requiring full-image processing, thus maintaining reduced processing time.
Solution Approach 2:
The patent introduces an intermediate foreground mask as a mediator between the initial rough segmentation and the final refined segmentation. This intermediate mask captures contour information through edge detection and morphological operations, serving as a bridge that reduces artifacts without requiring complete reprocessing of the entire image.
Data Source
AI summary
A method of image processing, includes: receiving at least one video frame of a video sequence, the at least one video frame including at least one foreground subject and a background; and processing the at least one video frame so as to separate the at least one foreground subject from the background. The processing includes: obtaining a reference image including the background; comparing the at least one video frame to the reference image; and generating a pixel mask as a result of the comparison, the pixel mask indicating whether a pixel of the at least one video frame belongs to the foreground subject or to the background. The method further comprises at least partially determining edges of the at least one foreground subject in the at least one video frame, and modifying the pixel mask based on the determined foreground subject edges.