Video Image Segmentation Neural Network Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image segmentation networks in computer vision only process a single image, leading to inconsistent predictions if accuracy is not perfect, especially in sequential video frames.
Innovation Solution
An image segmentation method that receives a first and a second image from a video stream, performs segmentation on the second image, determines similarity between the two images using a similarity calculation neural network, concatenates the results, and further infers to generate a label mask using a strategic neural network, incorporating previous frame information for improved consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional segmentation networks process only one image at a time, then the processing speed is fast, but the prediction consistency between frames deteriorates
Solution Approach 1:
The network is divided into three distinct modules: a segmentation neural network for generating label probability sets, a similarity calculation neural network for computing frame similarities, and a strategic neural network for final inference. This modular segmentation allows each component to specialize in a specific task, improving overall prediction consistency while maintaining manageable complexity through functional decomposition.
Solution Approach 2:
The similarity between the current frame and previous frame is calculated in advance before final segmentation inference. This preliminary similarity assessment allows the strategic neural network to incorporate temporal context proactively, ensuring consistent predictions across frames by preparing temporal relationship data before the final decision-making process.
2Measurement precision
If multiple images are processed to improve consistency, then the prediction accuracy improves, but the processing time increases
Solution Approach 1:
The patent extracts only the essential temporal information (similarity probability set between frames) rather than processing complete previous frame data. By extracting and utilizing only the relevant similarity metrics, the system achieves improved segmentation accuracy through temporal context while minimizing the additional processing time that would result from analyzing full frame sequences.
Solution Approach 2:
The similarity probability set acts as an intermediary that bridges current and previous frame information. Instead of directly processing multiple complete images, the system uses this intermediate representation to convey essential temporal relationships, enabling accurate multi-frame processing with reduced computational overhead and processing time.
3Stability of the object's composition
If only current frame is segmented, then the processing is simple, but the segmentation coherence across frames deteriorates
Solution Approach 1:
The system implements feedback by incorporating the similarity probability set (derived from previous frame information) into the current frame's segmentation process. The strategic neural network receives both the label probability set from current frame segmentation and the similarity probability set from temporal comparison, using this feedback loop to maintain segmentation coherence across frames while managing processing complexity through structured information flow.
Solution Approach 2:
The patent adds a temporal dimension to the segmentation process by integrating information from multiple time points. By concatenating feature maps from the segmentation network with similarity probability sets from temporal comparison, the system transitions from single-frame to multi-frame processing, achieving segmentation coherence across the temporal dimension while managing complexity through structured feature integration.
Data Source
AI summary
An image segmentation method is providing that includes the steps outlined below. A first image corresponding to a first time spot and a second image corresponding to a second time spot are received from a video stream, wherein the second time spot is behind the first time spot. Segmentation is performed on the second image by a segmentation neural network to generate a label probability set. Similarity determination is performed on the first image and the second image by a similarity calculation neural network to generate a similarity probability set. The label probability set and the similarity probability set are concatenated by a concatenating unit to generate a concatenated result. Further inference is performed on the concatenated result by a strategic neural network to generate a label mask.