Temporal Filtering Independent Color Channels Image Signal Processor

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing technologies face challenges in efficiently handling image data, particularly in real-time corrections and enhancements, especially in devices with constrained resources, where noise filtering and resource management are critical.

Innovation Solution

An image signal processor is configured to perform temporal filtering on independent color channels of image data, determining specific filter weights for blending pixel values between current and reference image frames, and optionally applying spatial filtering before storing the filtered pixels as a new reference frame.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If temporal filtering is applied to all color channels, then noise reduction is improved, but processing complexity and resource consumption increase

Engineering Contradiction:
Improvenoise reductionVSAvoidprocessing complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the color processing into independent channels (e.g., luminance and chrominance channels in YCbCr color space). Temporal filtering is selectively applied to specific channels based on their noise characteristics and importance, rather than uniformly to all channels. This segmentation allows the system to reduce noise in critical channels while avoiding unnecessary processing in less sensitive channels, thereby reducing overall processing complexity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent applies different filtering strategies to different color channels based on their local quality requirements. The luminance channel, which carries brightness information critical for perceived image quality, receives stronger temporal filtering. In contrast, chrominance channels carrying color information receive reduced or no filtering, as excessive filtering can cause color artifacts and the human eye is less sensitive to color noise. This localized quality-based approach optimizes the balance between noise reduction and processing efficiency.

Inventive Principle:
Principle #3Local quality

2Measurement precision

If temporal filtering with motion compensation is used, then filtering accuracy is improved, but computational load increases

Engineering Contradiction:
Improvefiltering accuracyVSAvoidcomputational load
Core Design Contradiction:
Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The patent implements partial motion compensation by applying it only to specific color channels or specific regions of the image where motion is detected, rather than uniformly to the entire image and all channels. This partial action approach maintains filtering accuracy in areas where motion occurs while reducing computational load in static areas, achieving a balance between precision and energy consumption.

Inventive Principle:
Principle #16Partial or excessive action

Solution Approach 2:

The processing is segmented into motion-detecting regions and static regions, with motion compensation applied selectively. The system divides the image into blocks or regions, identifies those with significant motion, and applies computationally intensive motion-compensated temporal filtering only to those regions. Static regions receive simpler filtering, significantly reducing the overall computational load while maintaining accuracy where needed.

Inventive Principle:
Principle #1Segmentation

3Manufacturing precision

If multiple processing stages are applied, then image quality is improved, but processing time increases

Engineering Contradiction:
Improveimage qualityVSAvoidprocessing time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent employs periodic action by applying spatial and temporal filtering in alternating stages rather than simultaneously processing all filters. The processing pipeline is organized into periodic passes: one pass applies spatial filtering to remove noise within frames, another pass applies temporal filtering to reduce noise across frames. This periodic alternation allows the system to maintain high image quality through multiple processing stages while managing processing time through efficient staged execution.

Inventive Principle:
Principle #19Periodic action

Solution Approach 2:

The patent applies preliminary spatial filtering to image data before applying temporal filtering. By removing a portion of the noise through spatial filtering in advance, the subsequent temporal filtering operates on already-cleaned data, reducing its computational burden and processing time. This preliminary action approach ensures that multiple processing stages contribute to image quality without linearly increasing total processing time, as each stage builds upon the work of previous stages.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS10467496B2Temporal filtering of independent color channels in image data
Publication Date: 2019.11.05 APPLE INC
  • US10467496B2 patent drawing
  • US10467496B2 patent drawing
  • US10467496B2 patent drawing

AI summary

An image processing pipeline may perform temporal filtering on independent color channels in image data. A filter weight may be determined for a given pixel received at a temporal filter. The filter weight may be determined for blending a value of a channel in a full color encoding of the given pixel with a value of the same channel for a corresponding pixel in a previously filtered reference image frame. In some embodiments, the filtering strength for the channel may be determined independent from the filtering strength of another channel in the full color encoding of the given pixel. Spatial filtering may be applied to a filtered version of the given pixel prior to storing the given pixel as part of a new reference image frame.