Bayer to Four-Plane Image Data Conversion via Dual Memory Channels

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing pipelines face challenges in efficiently processing image data from various formats, such as Bayer and three sensor data, due to resource-intensive CPU execution and format compatibility issues, leading to increased bandwidth and power consumption.

Innovation Solution

An apparatus and method that convert image data from Bayer format to four-plane format using two memory channels (DMA channels) for improved compatibility and efficiency, allowing the same image signal processor to handle multiple data formats by separating pixel types into distinct memory channels for processing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If image processing is performed by executing software programs on CPU, then flexibility in processing algorithms is improved, but CPU bandwidth consumption and power consumption increase significantly

Engineering Contradiction:

Improveflexibility in processing algorithmsVSAvoidCPU bandwidth consumption and power consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent divides the image processing pipeline into separate hardware components, each dedicated to specific processing tasks. The image processing pipeline is segmented into distinct stages that can handle different image formats independently, reducing the burden on CPU and lowering power consumption while maintaining algorithmic flexibility through hardware implementation.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediate buffer memory that acts as a mediator between the image sensor and the processing components. This buffer allows data to be staged and processed efficiently, enabling the system to handle multiple image formats without requiring CPU intervention for data movement, thus reducing CPU bandwidth consumption.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Use of energy by stationary object

If image processing pipeline is implemented as hardware component separate from CPU, then power consumption and CPU bandwidth usage are reduced, but complexity of handling multiple image data formats increases

Engineering Contradiction:

Improvepower consumption and CPU bandwidth usageVSAvoidcomplexity of handling multiple image data formats

Core Design Contradiction:

Use of energy by stationary objectVSDevice complexity

Solution Approach 1:

The patent designs the image processing pipeline with universal components that can handle multiple image data formats (Bayer, three sensor format, etc.) through a unified architecture. The processing pipeline includes format detection and conversion capabilities that allow it to adapt to different input formats without requiring separate hardware paths, thereby reducing device complexity while maintaining multi-format support.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent implements preliminary format conversion and data normalization in the image processing pipeline before further processing. By converting all image formats to a standard internal format early in the pipeline, the system simplifies subsequent processing operations and reduces the complexity of handling multiple formats throughout the processing chain.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If different processing algorithms are used for different image formats, then processing accuracy for each format is improved, but complexity of image processing pipeline increases

Engineering Contradiction:

Improveprocessing accuracy for each formatVSAvoidcomplexity of image processing pipeline

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent applies local quality by implementing format-specific processing algorithms only where needed in the pipeline. Different demosaicing and processing algorithms are applied to specific data formats at appropriate stages, while maintaining a unified overall pipeline structure. This allows optimal processing for each format without requiring complete redesign of the entire pipeline for each format.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS10375368B2Image data conversion

Publication Date: 2019.08.06 APPLE INC

AI summary

Embodiments of the present disclosure relate to an apparatus for converting image data from a Bayer format image to a four-plane image format using two memory channels. An example apparatus includes an interface for receiving the Bayer image including repeating pixel groups, where each pixel group includes a first pixel type, a second pixel type, a third pixel type, and a fourth pixel type. The apparatus also includes a memory and a circuit to write the Bayer image to the memory as four-plane data. The four-plane data includes pixels of the first type and the third type in the Bayer image that are written via the first memory channel, and pixels of the second type and the fourth type in the Bayer image that are written via the second memory channel. Embodiments also relate to converting three sensor image data to a Bayer format image using the two memory channels.