Raw Image Scaler Power Reduction via Pseudo-RGB Conversion

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing raw domain image scaling methods face challenges in maintaining Bayer output pattern consistency and image quality while reducing power consumption, often requiring complex and resource-intensive logic circuitry, leading to inferior results and high power consumption.

Innovation Solution

The proposed system employs spatial and frequency domain filtering with a predetermined oversampling ratio to efficiently scale raw image data, maintaining Bayer consistency and reducing power consumption by optimizing spatial filter coefficient arrays and oversampling ratios.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Use of energy by moving object

If existing raw domain scaling methods are used, then power consumption is reduced, but image quality deteriorates and Bayer pattern consistency becomes difficult to maintain

Engineering Contradiction:
Improvepower consumptionVSAvoidimage quality
Core Design Contradiction:
Use of energy by moving objectVSManufacturing precision

Solution Approach 1:

The patent introduces an intermediary processing stage where raw image data is temporarily converted to a pseudo-RGB format before scaling. This intermediary representation allows standard RGB domain scaling algorithms to be applied, achieving high image quality while maintaining power efficiency. The pseudo-RGB conversion acts as a mediator that enables quality scaling without requiring full RGB domain processing power.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent changes the color space parameter from raw Bayer format to pseudo-RGB format, enabling the use of optimized RGB scaling algorithms. This parameter transformation allows the system to leverage well-established high-quality scaling methods while operating in a simplified domain that requires less processing power than full RGB processing.

Inventive Principle:
Principle #35Parameter changes

2Stability of the object's composition

If complex logic circuitry is used to maintain Bayer output pattern, then Bayer consistency is improved, but device complexity and power consumption increase

Engineering Contradiction:
ImproveBayer pattern consistencyVSAvoidlogic circuitry complexity
Core Design Contradiction:
Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The patent uses pseudo-RGB format as an intermediary that naturally preserves Bayer pattern information without requiring complex dedicated circuitry. By converting raw data to pseudo-RGB before scaling, the system maintains Bayer consistency through standard processing operations rather than specialized logic circuits.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces complex mechanical/logic circuitry for Bayer pattern maintenance with a software-based pseudo-RGB conversion and standard RGB scaling approach. This substitution eliminates the need for specialized hardware logic while achieving the same Bayer consistency through algorithmic processing.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Manufacturing precision

If full RGB domain scaling is used, then image quality is improved, but power consumption increases

Engineering Contradiction:
Improveimage qualityVSAvoidpower consumption
Core Design Contradiction:
Manufacturing precisionVSUse of energy by moving object

Solution Approach 1:

The patent transforms the data representation from raw Bayer to pseudo-RGB format, enabling the use of efficient RGB scaling algorithms that consume less power than full RGB domain processing. This parameter change allows the system to apply optimized scaling operations that achieve high image quality with reduced computational requirements.

Inventive Principle:
Principle #35Parameter changes

Solution Approach 2:

The patent applies partial RGB domain processing by working in pseudo-RGB space rather than full RGB space. This partial action provides sufficient image quality for the application while avoiding the excessive power consumption of complete RGB domain scaling, achieving an optimal balance between quality and energy usage.

Inventive Principle:
Principle #16Partial or excessive action

Data Source

PatentUS9654711B2Power reduction for image sensor with raw image scaler
Publication Date: 2017.05.16 SONY SEMICON SOLUTIONS CORP
  • US9654711B2 patent drawing
  • US9654711B2 patent drawing
  • US9654711B2 patent drawing

AI summary

A system and method for scaling an image includes receiving raw image data comprising input pixel values which correspond to pixels of an image sensor, filtering pixels in a spatial domain, and filtering pixels in a frequency domain according to an oversampling ratio. The system and method may also include outputting scaled image data as output pixel values, which correspond to subgroups of the input pixel values. The filtering may be done according to a Bayer-consistent ruleset which includes a set of filter weights and a series of scaling rules. The oversampling ratio is set to minimize an error after the filtering in the spatial domain and the filtering in the frequency domain.