Anomalous Pixel Detection via Spatio-Temporal Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting dead or defective pixels in video analysis are inefficient due to high computational requirements and the challenge of identifying pixels that momentarily appear or disappear during transcoding or up-conversion, leading to residual point noise (RPN) that can cause blurring and be difficult to notice, especially in bright backgrounds.
Innovation Solution
A spatio-temporal approach using spatial and temporal filters to identify outlier pixels by computing pixel differences within frames and across successive frames, dynamically adding pixels to a candidate list based on divergence, and removing them if values remain invariant, thereby detecting RPN and defective pixels effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If computational resources are increased to detect defective pixels at every location throughout every frame, then detection accuracy is improved, but computational cost becomes unreasonable
Solution Approach 1:
The patent divides the video processing task into two distinct stages: a fast first pass that processes only key frames to identify candidate defective pixels, and a slower second pass that verifies these candidates. This segmentation allows the system to achieve high detection accuracy while minimizing overall computational resource usage by avoiding full-frame analysis of every video frame.
Solution Approach 2:
The patent applies partial action by performing comprehensive defective pixel detection only on key frames rather than every frame, and performing verification only on candidate pixels identified in the first pass. This selective approach achieves sufficient detection accuracy without the excessive computational cost of analyzing every pixel in every frame.
2Measurement precision
If spatial filtering is applied to identify outlier pixels, then defective pixel detection is improved, but false positives from normal scene variations increase
Solution Approach 1:
The patent merges two detection approaches: spatial filtering (comparing each pixel to its neighbors) and temporal filtering (tracking pixel consistency across multiple frames). By combining these methods, the system achieves more reliable defective pixel detection, as true defective pixels will exhibit both spatial outlier characteristics and temporal invariance, while normal scene variations will not consistently appear as outliers across multiple frames.
Solution Approach 2:
The patent implements feedback through a two-pass verification process where candidates identified in the first pass are re-examined in a second pass. This feedback mechanism allows the system to confirm or reject candidate pixels based on additional evidence, significantly reducing false positives while maintaining high detection accuracy.
3Reliability
If temporal filtering is used to remove pixels that change between frames, then false positives are reduced, but defective pixels that momentarily disappear are missed
Solution Approach 1:
The patent performs preliminary spatial outlier detection on key frames before applying temporal filtering. This preliminary action identifies candidate defective pixels based on their spatial characteristics, creating a candidate list that can then be verified through temporal analysis. This approach ensures that even if a defective pixel momentarily disappears or becomes less obvious in subsequent frames, it was already flagged as a candidate based on its initial spatial outlier status.
4Manufacturing precision
If interpolation processing is applied to up-convert video, then video quality is improved, but defective pixels become blurred and harder to detect
Solution Approach 1:
The patent performs defective pixel detection on the original, un-upconverted video before any interpolation processing is applied. By conducting the detection preliminarily on the source material where defective pixels appear as sharp, high-contrast outliers, the system can identify defective pixels with high accuracy. The detection results can then be applied to the upconverted video, avoiding the problem of blurred defective pixels that would result from detecting after interpolation.
Data Source
Figure 1
Figure A1A~A1B
Figure 2
AI summary
A video noise analyzer for detecting residual point noise in a video generates a list of candidate defective pixels by joining results of a temporal invariance detector and a spatial outlier detector. A data store is structured to contain data describing the candidate defective pixels and/or the blocks from which the pixels were selected. In other embodiments, a video noise analyzer includes a first salient analyzer structured to compare pixels within a block to a mean value of other pixels within the same block, a first threshold processor to generate a first pixel candidate list including pixels that differ significantly from their neighboring pixels, a second salient analyzer structured to generate a difference value between a subject pixel, its neighboring pixels, and neighbors of the neighboring pixels, and an identifier to label as a candidate dead pixel those pixels having a frequency occurrence of visual saliency greater than a predefined frequency threshold.