Normalized Cross Correlation Circuit for Image Signal Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing image processing pipelines consume significant CPU bandwidth and power due to software execution of image processing algorithms, necessitating a hardware-based solution for efficient image processing.

Innovation Solution

A normalized cross correlation (NCC) circuit is implemented, comprising a filtering circuit for convolution with kernel data, a normalization circuit for computing normalization scores, and additional components like gain offset clipping, kernel summation, and peak finding, to efficiently process input patch data and kernel data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If image processing algorithms are executed as software programs on the CPU, then the processing can be performed with flexibility, but the CPU bandwidth consumption and power consumption increase significantly

Engineering Contradiction:

Improveprocessing flexibilityVSAvoidCPU bandwidth consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The patent replaces the software-based CPU execution mechanism with a dedicated hardware circuit implementation. The NCC circuit performs normalized cross correlation operations using specialized hardware components (filtering circuit, normalization circuit, peak finding circuit) instead of general-purpose CPU instructions, thereby reducing CPU bandwidth consumption and power usage while maintaining processing capability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces an intermediate hardware processing layer (the NCC circuit) between the image data source and the CPU. This intermediary hardware circuit handles the computationally intensive NCC operations, freeing the CPU from these tasks while still allowing the system to benefit from hardware-accelerated processing

Inventive Principle:

Principle #24Intermediary (Mediator)

2Adaptability or versatility

If image processing algorithms are executed as software programs on the CPU, then the processing can be performed with flexibility, but the power consumption increases significantly

Engineering Contradiction:

Improveprocessing flexibilityVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by stationary object

Solution Approach 1:

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent implements a self-sufficient hardware processing unit that can independently perform NCC operations without requiring continuous CPU intervention. The circuit contains all necessary functional units (filtering, normalization, peak finding) to complete the processing task autonomously, reducing overall system power consumption

Inventive Principle:

Principle #25Self-service

3Productivity

If a hardware-based NCC circuit is implemented, then CPU resource consumption is reduced, but the device complexity increases

Engineering Contradiction:

Improveimage processing efficiencyVSAvoidcircuit complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent divides the NCC processing function into three distinct hardware circuit modules: a filtering circuit for convolution operations, a normalization circuit for computing normalization scores, and a peak finding circuit for identifying feature locations. This segmentation allows each module to be optimized independently and simplifies the overall design by breaking down the complex NCC operation into manageable functional units

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent designs the NCC circuit with universal components that can handle various image processing tasks. The filtering circuit, normalization circuit, and peak finding circuit are designed to work together as a cohesive unit that can process different types of image data and kernel configurations, making the hardware implementation adaptable to multiple applications

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS10997736B2Circuit for performing normalized cross correlation

Publication Date: 2021.05.04 APPLE INC

AI summary

Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.