Video Processing Device Using LUT-Based Foreground-Background Separation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In image processing, the background difference method struggles to accurately separate foreground from background when feature vectors of input and background pixels are close, especially in complex imaging situations, and requires real-time processing for high-resolution, high-frame-rate images.
Innovation Solution
An image processing device employing a feature extractor, quantization unit, learning unit, LUT generator, and boundary corrector that uses machine learning for nonlinear identification, reduces feature gradations, and performs boundary correction to classify pixels accurately and rapidly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a simple threshold value is used in the background difference method, then processing speed is improved, but identification accuracy deteriorates when feature vectors of input pixels and background pixels are close
Solution Approach 1:
The patent pre-calculates and stores optimal threshold values in a lookup table (LUT) during an offline learning phase. This preliminary action allows the system to quickly retrieve pre-determined thresholds during real-time processing, achieving both high speed and high accuracy without performing complex calculations during image processing.
Solution Approach 2:
The patent dynamically adapts threshold values based on the specific feature vector combinations encountered. Instead of using a fixed uniform threshold, the system selects from multiple pre-calculated thresholds corresponding to different feature vector patterns, allowing the threshold to dynamically adjust to match the actual image content and achieve optimal separation accuracy.
2Device complexity
If a uniform threshold value is set for the entire image, then device complexity is reduced, but identification accuracy deteriorates in regions with small feature vector differences
Solution Approach 1:
The patent applies different threshold values to different regions of the image based on their specific feature vector characteristics. By dividing the feature space into multiple regions and assigning appropriate thresholds to each, the system achieves locally optimized separation accuracy without requiring complex real-time calculations for each pixel.
Solution Approach 2:
The system pre-determines and stores multiple threshold values corresponding to different feature vector combinations in a lookup table. This preliminary calculation phase allows the simple uniform threshold approach to be replaced with a more sophisticated multi-threshold system that maintains simplicity during actual processing while achieving high local accuracy.
3Measurement precision
If the number of threshold value combinations is increased to optimize separation, then identification accuracy is improved, but processing time increases
Solution Approach 1:
The patent performs the computationally intensive task of calculating and optimizing multiple threshold value combinations during an offline learning phase. The results are stored in a lookup table, transforming the complex optimization problem into a simple table lookup operation during real-time processing, thereby eliminating the time penalty while maintaining high accuracy.
Solution Approach 2:
Instead of performing complex threshold optimization calculations during actual image processing, the system creates a copy of the optimized threshold relationships in the form of a lookup table. This copying approach allows the system to reuse pre-computed optimal thresholds without repeating the expensive calculation process, achieving both accuracy and speed.
4Device complexity
If linear identification is used in the nearest neighbor classifier, then device complexity is reduced, but identification accuracy deteriorates when vector distribution in feature space is complicated
Solution Approach 1:
The patent transforms the classification problem from a linear identification task to a threshold-based classification task in a pre-processed feature space. By changing the approach from direct linear classification to threshold comparison of pre-calculated feature differences, the system achieves better accuracy for complex vector distributions while maintaining algorithmic simplicity.
Data Source
AI summary
Accurate and rapid identification can be performed even when feature vectors of input pixels and background pixels are close. An image processing device includes: a feature extractor which extracts features of pixels from an input image and an input image of a different time; a quantization unit which reduces the number of gradations of features extracted by the feature extractor; a learning unit 107 which learns combinations of the features of pixels through a machine learning algorithm that performs nonlinear identification and constructs a network according to machine learning; an LUT generator 109 which generates an LUT that replaces arithmetic operations of the network constructed by the learning unit 107; a foreground region estimator 103 which rapidly estimates similarity of each pixel of the input image to a foreground by referring to the LUT generated by the LUT generator 109; and a boundary corrector 121 which classifies regions similar to the foreground estimated by the foreground region estimator 103 into the foreground, a background, and an unclassified region including boundary pixels of the foreground and the background and performs boundary correction only on the unclassified region.