Video Segmentation Model Adjustment for Occlusion Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video image segmentation methods, such as one-shot video object segmentation (OSVOS) and on-line adaptive video object segmentation (OnAVOS), face challenges in predicting large changes in frames, leading to splitting of segmentation and poor performance under conditions like object occlusion and rotation, where only the initial and previous frames are used for fine-tuning the model.
Innovation Solution
A video image segmentation method that adjusts the image segmentation model based on an initial frame, a previous frame, and a mask image, determining whether the number of frames to be segmented exceeds a preset threshold, and if so, using a preset frame for further adjustment, thereby improving segmentation accuracy and handling occlusions by maintaining object features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If only the initial and previous frames are used for fine-tuning the model, then the model complexity is reduced, but the segmentation accuracy deteriorates under large changes like object occlusion and rotation
Solution Approach 1:
The patent segments the video frames into different types (initial frame, previous frame, and preset frames at specific time points). By dividing the frame selection into distinct categories and using different frames for different purposes, the system can maintain lower model complexity while improving segmentation accuracy through strategic frame selection.
Solution Approach 2:
The patent performs preliminary action by pre-selecting preset frames at specific time points (e.g., 60%-90% of total frames) before the actual segmentation process. These pre-selected frames are prepared in advance and stored, so when large changes occur, the model can immediately use these pre-prepared reference frames without requiring complex real-time adjustments.
2Measurement precision
If multiple frames are used for model adjustment, then the segmentation accuracy is improved, but the processing time and computational resources increase
Solution Approach 1:
The patent applies local quality by using different numbers of frames for different video segments. For videos with small changes between frames, only the initial and previous frames are used (2 frames). For videos with large changes, preset frames are additionally incorporated. This localized adaptation optimizes processing time while maintaining segmentation accuracy where needed.
Solution Approach 2:
The patent changes the parameter of frame count dynamically based on video characteristics. By adjusting the number of frames used for model fine-tuning according to the actual video content and change magnitude, the system achieves segmentation accuracy improvement without consistently incurring high processing costs for all videos.
3Reliability
If preset frames are extracted and used for model adjustment, then the handling of occlusion and rotation is improved, but the device complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-calculating and storing the frame numbers for preset frames (e.g., at 60%-90% of total frames) before processing. This pre-computation of frame indices simplifies the actual segmentation process, as the system only needs to retrieve pre-determined frame numbers rather than performing complex frame selection logic during real-time segmentation.
Solution Approach 2:
The patent introduces preset frames as intermediary reference points between the initial frame and subsequent frames. These intermediary frames serve as mediators that help the model understand large changes and occlusions without requiring complex direct comparisons between distant frames, thus improving reliability while managing complexity through a structured intermediate layer.
Data Source
AI summary
A video image segmentation method includes: obtaining an image segmentation model by performing machine learning based on a historical video image and a mask image of the historical video image; adjusting the image segmentation model based on an initial frame of an image to be segmented, a previous frame of a frame to be segmented and a mask image of the previous image; and performing segmentation on the image to be segmented using the image segmentation model after the adjusting.