Computational Pixel Arrays With In-Pixel Event Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional imaging arrays, such as CCDs and CMOS devices, lack advanced signal-processing capabilities within pixels, limiting their ability to efficiently detect changes in radiation and perform complex image processing tasks.
Innovation Solution
The development of computational pixel imaging arrays with integrated circuits within each pixel, capable of digitizing signals and performing advanced signal-processing functions, including event-based imaging and inter-frame video compression, by using a plurality of pixels with signal converters, counters, and logic circuitry to identify changes in radiation and acquire key and delta frames at different frequencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional imaging arrays (CCD/CMOS) are used, then the structure is simple and manufacturing is easy, but advanced signal-processing capabilities within pixels are lacking
Solution Approach 1:
The patent merges the detector element with multiple integrated circuit components (signal converter, counters, memory, logic circuitry) within a single pixel structure. This integration enables advanced signal processing capabilities while maintaining a compact form factor that balances functionality with manufacturing feasibility.
Solution Approach 2:
The computational pixel is designed as a multi-functional unit that can perform detection, analog-to-digital conversion, event-based imaging, frame acquisition, and various signal processing operations within a single pixel structure. This multi-functionality addresses the versatility requirement while the standardized pixel design helps manage complexity.
2Productivity
If all pixels read out continuously at high frequency, then data rate and detection speed are improved, but bandwidth requirements and power consumption increase
Solution Approach 1:
The event-based imaging mode extracts and transmits only the relevant information (pixels that have detected events) rather than continuously reading out data from all pixels. This selective data extraction maintains high detection speed while significantly reducing the overall data volume and bandwidth requirements.
Solution Approach 2:
The system employs different readout frequencies for different operational modes: event-based imaging uses event-triggered readout, while frame-based imaging acquires key frames at a lower frequency and delta frames at higher frequency only when changes are detected. This periodic and conditional readout strategy reduces average data volume while maintaining detection responsiveness.
3Adaptability or versatility
If multiple functions are integrated within each pixel, then processing capability and dynamic range are improved, but manufacturing precision requirements increase
Solution Approach 1:
The complex pixel is segmented into distinct functional modules (detector, signal converter, counters, memory, logic circuitry) that can be designed and fabricated as separate components. This modular segmentation allows each component to be optimized independently and facilitates manufacturing processes while achieving high processing capability through integration.
4Loss of time
If event-based imaging with in-pixel processing is implemented, then latency and power consumption are reduced, but device complexity increases
Solution Approach 1:
Signal processing operations (conversion, counting, threshold comparison, event detection) are performed preliminarily within the pixel before data is transmitted off-chip. This preliminary processing reduces latency by eliminating the need to transfer raw data for processing and reduces power consumption by performing computations where the data is generated, despite the increased in-pixel circuitry complexity.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient detection of events and advanced image processing, reducing latency and power consumption while improving dynamic range and data rate, allowing for high-speed, low-bandwidth motion image video applications.
Implementation Method 1
Each pixel can include a detector configured to generate an analog signal when exposed to radiation
Data Source
AI summary
A computational pixel imaging device that includes an array of pixel integrated circuits for event-based detection and imaging. Each pixel may include a digital counter that accumulates a digital number, which indicates whether a change is detected by the pixel. The counter may count in one direction for a portion of an exposure and count in an opposite direction for another portion of the exposure. The imaging device may be configured to collect and transmit key frames at a lower rate, and collect and transmit delta or event frames at a higher rate. The key frames may include a full image of a scene, captured by the pixel array. The delta frames may include sparse data, captured by pixels that have detected meaningful changes in received light intensity. High speed, low transmission bandwidth motion image video can be reconstructed using the key frames and the delta frames.