Foreground Background Detection Using Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing background subtraction algorithms in computer vision face challenges such as illumination changes, camouflage, camera jitter, dynamic backgrounds, shadows, and noise, leading to incorrect assignment of pixels to foreground or background, resulting in false positives and negatives.
Innovation Solution
A method that uses semantic segmentation algorithms to calculate the probability of a pixel belonging to a foreground-relevant object, assigning it to the background set if the probability does not exceed a first threshold and to the foreground set if it exceeds both the threshold and a baseline probability, while updating the baseline probability and utilizing a background subtraction algorithm for pixels that fail to meet these criteria.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional background subtraction algorithms compare pixel values with a background model, then the detection process is simple and fast, but the accuracy deteriorates under illumination changes, camouflage, shadows, and dynamic backgrounds
Solution Approach 1:
The patent divides the detection process into multiple stages: first using a background subtraction algorithm for initial classification, then applying a semantic segmentation algorithm to pixels that fail the initial classification. This multi-stage segmentation approach improves accuracy by handling different pixel types with appropriate methods, resolving the contradiction between simple processing and high accuracy under challenging conditions.
Solution Approach 2:
The patent introduces an intermediary classification mechanism using semantic segmentation algorithms as a mediator between the simple background subtraction and the final pixel classification. This intermediary layer processes pixels that are ambiguous under illumination changes, shadows, or camouflage, improving overall accuracy without requiring the entire system to be complex.
2Reliability
If semantic segmentation algorithms are used to calculate pixel probabilities, then false positives and negatives are reduced, but the computational load increases
Solution Approach 1:
The patent applies semantic segmentation algorithms only partially - specifically to pixels that fail the initial background subtraction classification, rather than to all pixels in the image. This selective application maintains reliability for ambiguous pixels while reducing overall computational energy consumption compared to applying semantic segmentation to the entire image.
Solution Approach 2:
The patent implements local quality by applying different processing strategies to different pixels: simple background subtraction for clear pixels and semantic segmentation for ambiguous pixels. This localized approach ensures high reliability where needed while minimizing computational energy consumption in regions where simple methods suffice.
3Measurement precision
If multiple algorithms are combined for pixel classification, then detection accuracy improves, but the processing time increases
Solution Approach 1:
The patent segments the processing pipeline into two fast paths: one for pixels that clearly belong to background (using only background subtraction) and another for ambiguous pixels (using both background subtraction and semantic segmentation). This segmentation reduces average processing time while maintaining high precision for difficult cases.
Solution Approach 2:
The patent performs preliminary classification using the fast background subtraction algorithm before applying the slower semantic segmentation algorithm. This preliminary action filters out easily classifiable pixels, so that the computationally intensive semantic segmentation is only applied to pixels that truly need it, reducing overall processing time while maintaining high precision.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
The present invention concerns a method for assigning a pixel to one of a foreground pixel set and a background pixel set. In this method, if a first condition is met the pixel is assigned to the background pixel set, and if the first condition is not met and a second condition is met, the pixel is assigned to the foreground pixel set. The method comprises a step (S100) of calculating a probability that the pixel belongs to a foreground-relevant object according to a semantic segmentation algorithm, the first condition is that this probability that the pixel belongs to a foreground-relevant object does not exceed a first predetermined threshold, and the second condition is that a difference between this probability that the pixel belongs to a foreground-relevant object and a baseline probability for the pixel equals or exceeds a second predetermined threshold.