In-Pixel Analog Image Processing for Low Power Vision Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multi-frame imaging systems face limitations in achieving high parallelism, reduced information flow, and low power consumption due to off-sensor digital processing, which is resource-intensive and power-hungry.
Innovation Solution
In-pixel embedded analog image processing is implemented, where each pixel has its own processor and analog circuitry for neighbor-in-space and neighbor-in-time processing, utilizing a four-substrate or single-substrate configuration with NEWS registers for data transfer between neighboring pixels, reducing the number of energized transistors and thus power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If off-sensor digital processing is used for multi-frame imaging, then image processing functionality is achieved, but power consumption and computational resource usage increase significantly
Solution Approach 1:
The imaging sensor is divided into multiple pixels, each containing its own dedicated processing elements including photodetectors, analog processing circuits, and registers. This segmentation allows each pixel to independently perform neighbor-in-time and neighbor-in-space processing operations locally, eliminating the need for data transfer to an external digital processor and significantly reducing overall power consumption.
Solution Approach 2:
The patent replaces digital processing mechanisms with analog processing mechanisms. Each pixel contains analog processing circuits that perform computations on analog signals directly at the pixel level, substituting the need for digital signal processing and associated high-power consumption with lower-power analog circuit operations.
2Reliability
If off-sensor digital processing is used for multi-frame imaging, then image processing functionality is achieved, but computational resources and processing complexity increase
Solution Approach 1:
The imaging sensor is divided into multiple pixels, each containing its own dedicated processing elements including photodetectors, analog processing circuits, and registers. This segmentation allows each pixel to independently perform neighbor-in-time and neighbor-in-space processing operations locally, eliminating the need for data transfer to an external digital processor and significantly reducing overall power consumption.
Solution Approach 2:
Each pixel is designed as a self-contained processing unit that performs its own image processing operations independently. The pixel includes integrated photodetectors, analog processing circuits, and registers that enable it to execute neighbor-in-time and neighbor-in-space processing without requiring external computational resources, thereby reducing overall system complexity.
3Productivity
If conventional digital image processing is used, then processing operations are performed, but the number of energized transistors and power consumption increase
Solution Approach 1:
The patent replaces digital processing mechanisms with analog processing mechanisms. Each pixel contains analog processing circuits that perform computations on analog signals directly at the pixel level, substituting the need for digital signal processing and associated high-power consumption with lower-power analog circuit operations.
Solution Approach 2:
The patent changes the fundamental parameter of signal representation from digital to analog. By processing images as analog signals throughout the pipeline, the system eliminates the need to switch transistors for digital logic operations, significantly reducing the number of energized transistors and associated power consumption while maintaining processing functionality.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enhances processing efficiency with increased MACs per second and MACs per Watt, enabling real-time vision stacks for mission-critical applications like ADAS and autonomous vehicles with reduced power usage.
Implementation Method 1
Each in-pixel processing element includes a photodetector, photodetector capture circuitry
Data Source
AI summary
An in-pixel embedded analog image processing system performs analog image computation within an image pixel. In embodiments, each in-pixel processing element includes a photodetector, photodetector control circuitry, analog circuitry configured to process both neighbor-in-space and neighbor-in-time functions for analog data representing an electrical current from the photodetector control circuitry, and a set of north-east-west-south (NEWS) registers, each register interconnected between a unique pair of neighboring in-pixel processing elements to transfer analog data between the pair of neighboring in-pixel processing elements. In embodiments, the in-pixel embedded analog image processing device takes advantage of high parallelism because each pixel has its own processor, and takes advantage of locality of data because all data is located within a pixel or within a neighboring pixel.