ISP Flicker Mitigation via Pixel Data Correction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video streams captured by cameras often exhibit noticeable flicker artifacts due to unsynchronized frame capture rates with light source frequencies, particularly from LEDs and fluorescents, which can lead to imperfect responses in systems relying on video information, such as Advanced Driver Assistance Systems (ADAS).

Innovation Solution

A post-processing solution using an image signal processor (ISP) to selectively correct pixel data representing flickering light sources by identifying and tracking light sources across frames, modifying image data to mitigate flicker effects, particularly in regions where luminance and color characteristics vary significantly.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If frame capture rate is not synchronized with light source flicker frequency, then video capture can be performed at standard rates, but noticeable flicker artifacts appear in the video

Engineering Contradiction:

Improvevideo capture rateVSAvoidflicker artifacts

Core Design Contradiction:

ProductivityVSObject-affected harmful factors

Solution Approach 1:

The patent detects the harmful flicker artifacts in captured video frames and uses image processing techniques to identify and correct the flickering regions. By analyzing color characteristics and luminance variations across frames, the system converts the harmful flicker effect into detectable patterns that can be corrected through selective pixel value modification, thereby eliminating the artifact while maintaining standard capture rates

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

Solution Approach 2:

The patent modifies pixel parameters (color values, luminance) in identified flickering regions to correct the flicker artifacts. By changing the parameter representation of affected pixels based on neighboring frame data and color space transformations, the system eliminates visible flicker without requiring synchronization of capture rate with light source frequency

Inventive Principle:

Principle #35Parameter changes

2Speed

If exposure windows are shortened to increase frame rate, then video capture speed increases, but frame to frame luminance variation due to flicker increases

Engineering Contradiction:

Improveframe capture speedVSAvoidluminance consistency

Core Design Contradiction:

SpeedVSStability of the object's composition

Solution Approach 1:

The patent implements a feedback mechanism by analyzing luminance and color characteristic variations across consecutive frames to detect flicker patterns. The system uses this feedback information to identify affected regions and applies corrective processing to stabilize luminance consistency, allowing high frame rates to be maintained without sacrificing luminance stability

Inventive Principle:

Principle #23Feedback

3Object-affected harmful factors

If post-processing correction is applied to remove flicker, then flicker artifacts are eliminated, but processing complexity and computational load increase

Engineering Contradiction:

Improveflicker artifact removalVSAvoidimage processing complexity

Core Design Contradiction:

Object-affected harmful factorsVSDevice complexity

Solution Approach 1:

The patent applies local quality processing by identifying specific flickering regions in the video frame and applying correction only to those areas rather than processing the entire image. By segmenting the frame and targeting only affected pixels and regions for correction, the system reduces overall processing complexity while effectively eliminating flicker artifacts from light sources

Inventive Principle:

Principle #3Local quality

4Object-affected harmful factors

If selective modification of pixel data is performed, then flicker in specific regions is corrected, but risk of affecting other image areas increases

Engineering Contradiction:

Improveflicker correction accuracyVSAvoidimage data integrity

Core Design Contradiction:

Object-affected harmful factorsVSReliability

Solution Approach 1:

The patent uses segmentation to divide the video frame into distinct regions, identifying which areas contain flickering light sources versus normal scene content. By segmenting the image and applying correction only to identified flicker regions based on color and luminance analysis, the system ensures accurate flicker correction while preserving the integrity and natural appearance of other image areas

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11700457B2Flicker mitigation via image signal processing

Publication Date: 2023.07.11 SAMSUNG ELECTRONICS CO LTD

AI summary

Disclosed is an image processing method employing a post processing technique that may eliminate flicker artifacts from captured video in real time. An example method involves receiving from an image sensor a sequence of input frames representing a scene and identifying and tracking a light source represented within the input frames. Flickering of the light source may be identified by detecting frame to frame color characteristic variation of a region representing the light source. A sequence of output frames may be generated with the flickering removed by selectively modifying image data of pixels within the region.