Background Image Estimation Using Minimum Cost Labeling

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current background estimation techniques face challenges such as requiring the background to be visible for at least half the time, reliance on depth information that is not always available, and circular dependencies with optical flow-based methods, which complicates the removal of moving objects and camera motion in image sequences.

Innovation Solution

A method that formulates the background estimation as a minimum cost labeling problem, using pixel attributes like color and intensity, with cost functions that measure variance and motion boundary consistency to determine the background image, allowing for seamless copying from stationary areas and avoiding ambiguities without needing optical flow or depth computations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of manufacture

If median based approach is used to estimate background, then the method is simple to implement, but it requires the background to be visible for at least half the time which limits its generality

Engineering Contradiction:
ImproveEase of implementationVSAvoidGenerality of background estimation
Core Design Contradiction:
Ease of manufactureVSAdaptability or versatility

Solution Approach 1:

The patent changes the parameter threshold from 50% (median requirement) to a configurable minimum visibility threshold, allowing the background to be estimated even when visible for less than half the time. This parameter change enables the system to adapt to scenarios where background visibility is limited, thereby improving versatility while maintaining implementation simplicity.

Inventive Principle:
Principle #35Parameter changes

2Measurement precision

If depth based techniques are used for background estimation, then depth information can help separate foreground from background, but two input sequences are required which are not always available

Engineering Contradiction:
ImproveBackground separation accuracyVSAvoidNumber of input sequences required
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts and removes the dependency on depth information and multiple input sequences from the background estimation process. By formulating the problem as a labeling task that works with a single image sequence and pixel intensity values, it eliminates the need for complex depth computation and multiple sequence inputs, thereby reducing device complexity while maintaining adequate background separation accuracy.

Inventive Principle:
Principle #2Taking out (Extraction)

3Measurement precision

If optical flow based technique is used for background estimation, then accuracy of optical flow computations can be improved, but it creates a circular dependency that complicates the process

Engineering Contradiction:
ImproveOptical flow computation accuracyVSAvoidProcess complexity due to circular dependency
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

Instead of using optical flow to estimate background (which creates circular dependency), the patent inverts the approach by using a labeling-based method that directly estimates background without requiring optical flow computations. This inversion eliminates the circular dependency and process complexity while still achieving accurate background estimation that can subsequently improve optical flow computations.

Inventive Principle:
Principle #13The other way round (Inversion)

4Reliability

If background estimation is performed to remove moving objects, then the background can be reconstructed, but camera motion and scene brightness changes complicate the estimation process

Engineering Contradiction:
ImproveBackground reconstruction accuracyVSAvoidComplexity of handling camera motion and brightness changes
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the background estimation problem into independent pixel-level labeling tasks, where each pixel is labeled with the frame number corresponding to its background appearance. This segmentation allows the system to handle camera motion and brightness changes independently at each pixel location, reducing overall process complexity while maintaining reliable background reconstruction accuracy.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS7574038B1Method and apparatus for determining the background of an image sequence
Publication Date: 2009.08.11 ADOBE INC
  • US7574038B1 patent drawing
  • US7574038B1 patent drawing
  • US7574038B1 patent drawing

AI summary

One embodiment of the present invention provides a system that determines a background image for a sequence of image frames. During operation, the system receives a sequence of input image-frames, wherein an input image-frame associates pixels with pixel-attributes. The system then computes a labeling, wherein the labeling associates pixels in the output background image with input image-frames in the sequence of input image-frames. Next, the system determines the output background image using the sequence of input image-frames and the labeling.