Video Object Mask Generation via Gradient Infilling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video processing methods face challenges in generating fine object boundary masks for moving objects from image sequences, making it difficult to accurately extract moving objects from video content.
Innovation Solution
A method and system that aligns consecutive video frames, calculates absolute difference images, detects object contours, and uses gradient information to dynamically generate object masks by infilling object contours within bounding boxes, allowing for the extraction of moving objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional video processing methods are used to extract moving objects, then the process is simple, but the boundary extraction precision is poor
Solution Approach 1:
The patent segments the video processing into multiple distinct stages: frame differencing to identify moving regions, contour detection to define object boundaries, gradient calculation to enhance edge information, and mask generation to create precise object masks. This segmentation allows each stage to be optimized independently, achieving high boundary precision through the gradient-based refinement process while keeping the overall system manageable through modular design.
Solution Approach 2:
The patent performs preliminary actions by first calculating absolute difference images between consecutive frames to identify potential moving object regions before detailed contour analysis. This preliminary segmentation of the image into static and dynamic regions reduces the computational burden on subsequent processing stages and ensures that gradient calculations are focused only on relevant areas, improving both precision and efficiency.
2Measurement precision
If detailed contour detection and gradient analysis are performed, then the object extraction accuracy is improved, but the processing time increases
Solution Approach 1:
The patent applies local quality by performing gradient calculations and detailed contour analysis only within the bounding boxes of detected moving objects, rather than processing the entire image. This localized approach maintains high detection accuracy for moving objects while significantly reducing the computational time required, as static background regions are processed more simply through frame differencing.
Solution Approach 2:
The patent implements partial action by selectively applying computationally intensive gradient-based contour refinement only to regions identified as containing moving objects. The processing intensity is adjusted based on the presence and characteristics of moving objects: full gradient analysis for objects requiring high precision, and simplified differencing for other regions. This partial application of intensive processing maintains accuracy where needed while minimizing overall processing time.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
Various aspects of a method and system to process video content for extraction of moving objects from image sequences of the video content are disclosed herein. In an embodiment, the method includes determination of one or more object contours of one or more moving objects in the video content. A first object bounding box (OBB) that encompasses a first object contour of a first moving object is created based on the determined one or more object contours. A first object mask for the first moving object is generated in a first destination image frame, based on infilling of the first object contour in the created first OBB.