NPU Demosaicing via Depth-Wise Convolution and MAC Utilization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing ML-based demosaicing techniques inefficiently utilize neural processing units (NPUs) due to memory-intensive operations, leading to poor performance compared to CPUs/GPUs.

Innovation Solution

Perform depth-wise convolution operations on image data using predetermined parameter values to extract pixel color values, followed by convolution operations and interpolation, optimizing the process for NPUs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If traditional ML-based demosaicing techniques are used, then demosaicing functionality is achieved, but processing efficiency on NPUs deteriorates due to numerous memory-intensive operations and poor utilization of MAC units

Engineering Contradiction:

Improvedemosaicing processing efficiencyVSAvoidmemory operation overhead

Core Design Contradiction:

ProductivityVSLoss of energy

Solution Approach 1:

The patent transforms the demosaicing algorithm by changing its operational parameters from memory-intensive random access patterns to convolution-friendly sequential access patterns with regular strides, enabling efficient NPU MAC unit utilization and reducing memory operation overhead

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent replaces traditional demosaicing mechanical operations (memory-intensive pixel manipulation and interpolation) with convolutional neural network operations that are natively optimized for NPU hardware, substituting inefficient memory access patterns with efficient MAC unit computations

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If traditional demosaicing algorithms are used, then color channel interpolation is performed, but NPU MAC unit utilization deteriorates leading to wasted processing cycles

Engineering Contradiction:

ImproveMAC unit utilizationVSAvoidprocessing cycle waste

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent modifies the algorithmic parameters to match NPU MAC unit operational characteristics, using convolution kernels with strides that fully utilize MAC units and eliminate idle processing cycles through optimized data flow and computation patterns

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12632924B2Efficient demosaicing on neural processing units

Publication Date: 2026.05.19 QUALCOMM INC

AI summary

Techniques and systems are provided for image demosaicing. For instance a process can include performing a depth-wise convolution operation on image data and a depth-wise convolutional filter with pre-determined parameter values to obtain a plurality of color channels for the image data; performing a convolution operation on the plurality of color channels to obtain a processed plurality of color channels; arranging the processed plurality of color channels into a demosaiced image; and outputting the demosaiced image.