Image Sensor Neural Network Circuitry Bandwidth Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current image sensors rely heavily on external processors for data processing, leading to increased bandwidth requirements and latency in object detection, as they transmit raw image data for extensive processing, which can be inefficient and slow.
Innovation Solution
The integration of multiple stacked integrated circuit layers within the image sensor, including pixel sensor groups, image processing circuitry groups, and neural network circuitry groups, allows for distributed processing and parallel processing of image data, reducing the need for external processing by transmitting only processed data and neural network outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image sensors transmit raw image data to external processors for object detection, then object detection can be performed, but bandwidth requirements increase and latency increases
Solution Approach 1:
The image sensor is divided into multiple pixel sensor groups, each with dedicated image processing circuitry and neural network circuitry. This segmentation allows distributed processing where each group handles its own data locally, reducing the total bandwidth required to transmit processed information from all groups to external processors.
Solution Approach 2:
The patent transitions from a two-dimensional planar architecture to a three-dimensional stacked architecture with multiple integrated circuit layers. This vertical stacking enables parallel processing paths and reduces data transmission distances, thereby reducing bandwidth requirements while maintaining detection accuracy.
2Measurement precision
If image sensors transmit raw image data to external processors for object detection, then object detection can be performed, but processing latency increases
Solution Approach 1:
Image processing circuitry and neural network circuitry are integrated directly into the image sensor before data leaves the sensor. This preliminary processing performs initial object detection and filtering on raw image data at the source, reducing latency by eliminating the need to transmit and process all raw data externally.
Solution Approach 2:
The patent merges pixel sensing, image processing, and neural network analysis functions into a single integrated sensor system. This consolidation eliminates data transmission bottlenecks between separate components, reducing processing latency while maintaining detection accuracy.
3Quantity of substance
If multiple stacked integrated circuit layers are integrated in the image sensor for distributed processing, then bandwidth usage reduces and latency reduces, but device complexity increases
Solution Approach 1:
The complex processing functions are segmented into distinct integrated circuit layers, each with specialized circuitry for specific tasks. This modular segmentation manages complexity by organizing functions into separate, manageable layers rather than a monolithic complex circuit.
Solution Approach 2:
The patent resolves complexity by moving from a two-dimensional layout to a three-dimensional stacked architecture. This vertical arrangement reduces interconnect complexity and allows parallel processing paths, managing device complexity while enabling distributed processing across multiple layers.
4Productivity
If multiple stacked integrated circuit layers are integrated in the image sensor for distributed processing, then processing speed increases and latency reduces, but device complexity increases
Solution Approach 1:
The patent merges pixel sensing, image processing, and neural network analysis into a single integrated multi-layer device. This merging enables parallel processing across layers, increasing processing speed while managing complexity through integrated design rather than separate components.
Solution Approach 2:
The transition to three-dimensional stacked architecture enables parallel processing paths that increase processing speed. The vertical stacking manages complexity by organizing processing stages in separate layers with controlled interconnects, avoiding the complexity of two-dimensional expansion.
Data Source
AI summary
A method includes obtaining multiple images captured by pixel sensors of an image sensor, analyzing, using neural network circuitry integrated in the image sensor, the multiple images for object detection, generating, for each of the multiple images using the neural network circuitry integrated in the image sensor, neural network output data related to results of the analysis of the multiple images for object detection, and transmitting, from the image sensor, the neural network output data for each of the multiple images and image data for a subset of the multiple images instead of image data of each of the multiple images.