AI Pixel Sensor Arrays With Neuromorphic In-Pixel Event Processing

The patent segments AI processing capabilities from centralized cloud systems and distributes them to individual pixel sensors. Each pixel contains its own AI processing unit that can independently perform neural network inference, pattern recognition, and event detection without requiring centralized processing. This segmentation enables real-time processing at the data source while eliminating data transmission delays.

The patent moves AI processing from the traditional centralized cloud dimension to a distributed edge dimension by embedding neural network processors directly within sensor pixels. This dimensional shift from centralized to distributed architecture enables parallel processing across millions of pixels simultaneously, achieving both high computational power and zero transmission latency.

The patent implements preliminary AI processing action directly at the pixel level before data leaves the sensor. Each pixel's embedded neural network performs preliminary filtering, feature extraction, and event detection on incoming photons, pre-processing data to identify only salient events. This preliminary action reduces downstream processing complexity while maintaining complete data collection capabilities.

The patent extracts only the most salient features and events from the complete photocurrent data stream using on-pixel AI processing. The neural networks identify and extract meaningful patterns such as moving objects, anomalies, or specific events of interest, discarding redundant background information. This extraction reduces data volume and processing complexity while preserving all essential information.

The patent merges photodetector functionality with AI processing capabilities into a single integrated pixel structure. The neural network processor, memory, and control logic are combined with the photodetector on the same substrate, creating a unified sensor element that maintains manufacturing simplicity while adding sophisticated event discrimination capability. This merging eliminates the need for separate processing components.

The patent creates universal pixels that perform multiple functions: photon detection, AI inference, pattern recognition, and event classification all within a single pixel element. This multi-functionality allows the same pixel structure to achieve both manufacturing simplicity and high measurement precision by consolidating diverse capabilities into a unified design.

The patent implements partial processing action by having each pixel perform only the minimum necessary AI computations to identify salient events. Rather than processing all possible features at maximum speed, the neural networks perform selective inference focused only on detecting and classifying events of interest. This partial action achieves real-time response while minimizing power consumption.

The patent enables self-service processing where each pixel independently performs its own AI inference and event detection without requiring external processing resources. The on-pixel neural networks use locally stored weights and biases to autonomously process incoming data, eliminating the need for power-hungry centralized processing and achieving both high speed and low power consumption.