Image Sensor Memory Architecture for In-Memory Optical Flow Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing size of image data generated by image sensors requires higher bandwidth for smooth calculation processing, particularly in applications like advanced driver-assistance systems (ADAS) that need high-resolution image data for accurate object detection and real-time execution.
Innovation Solution
An image sensor module with a memory performing calculation processes using a processor-in-memory (PIM) circuit, which generates optical flow data and pattern density data to enhance image processing, enabling faster object detection and improved accuracy by distributing processing elements across multiple memory banks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image data size increases to improve image quality and resolution, then measurement precision is improved, but bandwidth requirement increases and calculation processing becomes slower
Solution Approach 1:
The patent merges the memory function and processing function into a single integrated structure where processing elements are directly coupled to memory banks. This allows image data to be processed in-place without external data movement, simultaneously handling high-resolution image data and maintaining high calculation speed through unified memory-processing architecture.
Solution Approach 2:
The memory is divided into multiple memory banks and processing elements are distributed across these banks. Each processing element can independently process data from its associated memory bank, enabling parallel processing of large image datasets and improving overall calculation throughput while maintaining access to high-resolution data.
2Productivity
If bandwidth is increased to improve calculation processing speed, then productivity is improved, but device complexity increases
Solution Approach 1:
By combining memory and processing units into an integrated PIM architecture, the patent eliminates the need for high-bandwidth external data transfer interfaces. The processing elements access memory directly through short internal connections, achieving high calculation speed without requiring complex high-bandwidth external communication infrastructure.
3Productivity
If processing is distributed across multiple memory banks to improve calculation speed, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent segments the processing system into multiple processing elements distributed across memory banks, with each element handling specific data locally. This segmentation enables parallel processing that improves calculation speed while keeping each individual processing unit simple and manageable, avoiding the complexity of centralized high-performance processing.
Data Source
AI summary
An image sensor module includes an image sensor configured to generate image data and memory including at least a memory bank storing the image data and a processor-in-memory (PIM) circuit, the PIM circuit including a plurality of processing elements. The memory is configured to read the image data from the memory bank; generate optical flow data and pattern density data using the plurality of processing elements, the optical flow data indicating time-sequential motion of at least one object included in the image data, and the pattern density data indicating a density of a pattern of the image data; and output the image data, the optical flow data, and the pattern density data.