Video Segmentation Using Time-Dependent Foreground Background Classification

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing adaptive scene modeling techniques face challenges in handling background merging and revealed background situations, leading to missed detections and spurious detections due to their reliance on per-frame learning rates and color gradients, which are not robust, especially in scenarios with multiple background modes.

Innovation Solution

The Time Dependent Foreground/Background (TDFB) method classifies visual elements as foreground or background based on motion and continuity strength values accumulated over time, using blob boundary characteristics and matched mode characteristics to update the scene model.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If per-frame learning rates and color gradients are used for scene modeling, then the system can adapt to background changes, but it produces missed detections and spurious detections in background merging and revealed background situations

Engineering Contradiction:

Improveadaptability to background changesVSAvoiddetection accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements dynamic scene modeling by maintaining multiple background modes for each visual element location, allowing the system to adapt to different background states (e.g., light on/off, different times of day). The scene model dynamically switches between modes based on the current frame characteristics, enabling reliable detection even when background changes occur. This resolves the contradiction by providing both adaptability through mode switching and reliability through maintaining multiple representative modes.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter representation from single-frame learning rates to multi-mode appearance models. Each visual element is represented by multiple modes with different appearance parameters (color, texture, shape), and the system selects the appropriate mode based on current conditions. This parameter change enables the system to handle background merging and revealed background situations reliably while maintaining adaptability to various background states.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If adaptive scene modeling updates the background model continuously, then it can learn background changes, but it may incorrectly classify foreground objects as background in background merging situations

Engineering Contradiction:

Improvebackground learning efficiencyVSAvoidforeground detection accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent performs preliminary classification by comparing the current frame against multiple pre-established background modes before updating the scene model. By having multiple modes ready in advance (representing different background states), the system can correctly identify foreground objects even when the background appears to have merged with the object. This preliminary action prevents incorrect classification while maintaining efficient background learning through selective mode updates.

Inventive Principle:

Principle #10Preliminary action

3Device complexity

If a single background mode is maintained for each visual element, then the model is computationally simple, but it cannot handle multiple background states (e.g., light on/off, different times)

Engineering Contradiction:

Improvemodel complexityVSAvoidhandling of multiple background modes

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent segments the background representation into multiple distinct modes for each visual element location. Instead of using a single background model, the system creates multiple mode models (e.g., one for light on, one for light off, one for daytime, one for nighttime). This segmentation allows the system to handle multiple background states accurately while keeping each individual mode computationally simple, thus resolving the contradiction between model complexity and adaptability.

Inventive Principle:

Principle #1Segmentation

4Ease of operation

If the scene model uses age-based classification with a fixed threshold, then it can distinguish foreground from background, but it produces spurious detections in revealed background situations

Engineering Contradiction:

Improveclassification simplicityVSAvoiddetection accuracy

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent replaces the fixed age-based classification with a dynamic mode-matching approach. Instead of using a fixed threshold, the system dynamically selects the appropriate background mode based on the current frame characteristics and compares the visual element against the selected mode. This dynamic approach maintains ease of operation through automated mode selection while significantly improving reliability by adapting the classification criterion to the specific background state, thereby eliminating spurious detections in revealed background situations.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10096117B2Video segmentation method

Publication Date: 2018.10.09 CANON KK

AI summary

A method for classifying a visual element in a video frame as either foreground or background, the video frame being characterized using a scene model comprising a plurality of modes, the method comprising the steps of: determining a blob boundary characteristic for a blob comprising at least the visual element; identifying a mode matched to the visual element; classifying the visual element and the matched mode as foreground dependent upon a match mode boundary characteristic of the matched mode; and updating the scene model dependent upon the blob boundary characteristic and the matched mode boundary characteristic.