Video Object Segmentation Using Iterative Tracking and Region Proposals

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current methods for analyzing media content, particularly for object detection and segmentation in videos, face limitations in semantic understanding and high computational complexity, despite advancements in neural networks.

Innovation Solution

A method that extracts region proposals, uses iterative tracking to determine object-specific representations, and performs object segmentation by combining static and motion information, with a confidence map to enhance classification confidence and reduce computational overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If neural networks are used for object detection and recognition, then recognition accuracy is improved, but computational complexity increases

Engineering Contradiction:
Improverecognition accuracyVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The video content is divided into multiple frames, and object detection is performed on individual frames rather than the entire video sequence. Region proposals are extracted from each frame independently, then tracked across frames to form coherent object representations. This segmentation approach reduces computational complexity by breaking down the complex task of video-wide object analysis into simpler frame-by-frame processing while maintaining recognition accuracy through temporal tracking.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Region proposals are pre-computed for each frame before performing detailed object classification. The system generates candidate regions in advance using efficient proposal generation algorithms, then applies neural network classification only to these pre-selected regions rather than all pixels. This preliminary action of generating region proposals reduces the computational burden of the neural network while preserving detection accuracy.

Inventive Principle:
Principle #10Preliminary action

2Manufacturing precision

If extensive pixel-level annotation is performed, then segmentation precision is improved, but time consumption increases

Engineering Contradiction:
Improvesegmentation precisionVSAvoidtime consumption
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The system extracts only the essential features needed for segmentation by focusing computational resources on region proposals rather than processing all pixels uniformly. By extracting region proposals that are likely to contain objects and applying neural network classification only to these regions, the system achieves high segmentation precision while avoiding the time-consuming process of annotating or processing every pixel in the video frames.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The system applies different processing quality to different regions of the video frames. High-quality neural network classification is applied only to region proposals that are likely to contain objects of interest, while other regions receive minimal or no processing. This local quality approach maintains segmentation precision for important regions while reducing overall time consumption by avoiding uniform high-quality processing across the entire frame.

Inventive Principle:
Principle #3Local quality

Data Source

PatentUS10319095B2Method, an apparatus and a computer program product for video object segmentation
Publication Date: 2019.06.11 NOKIA TECHNOLOGIES OY
  • US10319095B2 patent drawing
  • US10319095B2 patent drawing
  • US10319095B2 patent drawing

AI summary

The invention relates to a method and an apparatus implementing the method. The method comprises extracting region proposals from a media content; selecting a set of region proposals corresponding to an object in the media content; identifying objects of interest; determining an object-specific representation by an iterative tracking method; sampling positive examples from the set of tracked region proposal groups obtained from the iterative tracking method; and performing object segmentation.