Brain-Like In-Pixel Processing for Real-Time Object Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems face challenges in processing large volumes of image data for real-time object detection, classification, and tracking, similar to human visual systems, which efficiently process limited data for rapid and accurate object recognition.
Innovation Solution
A brain-like in-pixel intelligent processing system with an in-pixel processing array that mimics human visual pathways, utilizing a set of in-pixel processing units, a saccadic pixel selector, and circuits to process raw gray information at the pixel level, generating periphery, fovea, and lateral geniculate nucleus elements for object detection and identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If full color image data is processed by computer systems, then complete image information is obtained, but data processing time increases and real-time performance deteriorates
Solution Approach 1:
The patent extracts only the necessary gray scale information from full color images at the pixel level, discarding redundant color data. The in-pixel processing units selectively process luminance information while ignoring chrominance data, thereby reducing the data volume to be processed without compromising the essential image content needed for object detection and recognition.
Solution Approach 2:
The patent segments the image processing task into two distinct stages: (1) in-pixel processing that extracts and processes gray scale information directly at the sensor level, and (2) subsequent neural network processing that operates on the reduced data set. This segmentation allows the system to handle only essential information in the first stage, significantly reducing the computational burden in the second stage.
2Measurement precision
If detailed image information is processed at high speed, then processing accuracy improves, but computational power requirements increase
Solution Approach 1:
The patent performs preliminary processing of image data directly in the pixel layer before the data leaves the sensor. The in-pixel processing units compute gray scale values and perform initial feature extraction at the location where the data is generated, converting raw sensor output into processed gray scale information. This preliminary action reduces the complexity of subsequent processing tasks and lowers the computational power needed at later stages.
3Productivity
If large amount of image data is processed in real-time, then comprehensive object analysis is achieved, but system complexity increases
Solution Approach 1:
The patent merges the functions of data acquisition and initial data processing into a single integrated in-pixel processing unit. The same pixel structure that captures light also performs gray scale computation and data reduction, combining sensing and processing functions in one location. This merging eliminates the need for separate processing hardware and reduces overall system complexity while maintaining real-time processing capability.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables rapid and accurate object recognition by processing reduced data, facilitating real-time feedback and adaptive learning, emulating human visual system efficiency in a semiconductor chip.
Implementation Method 1
The photogate sensor captures a pixel of the image of an object corresponding to the in-pixel processing unit and produces an Iout current
Data Source
AI summary
Aspects of present disclosure relates to a brain-like in-pixel intelligent processing system having in-pixel processing array, feature vector generator, neural network processor, and region and object of interest identifier. Each in-pixel processing unit includes photogate sensor to acquire image, average circuit to generate P element, subtraction circuit to generate F element, and absolute circuit to generate LGN element. Feature vector generator generates P, F, and LGN feature vectors from certain selected in-pixel processing units of in-pixel processing array selected according to saccadic eye movement algorithm. Neural network processor processes P, F, and LGN feature vectors and detects object, recognizes object, and determines location of object. Region and object of interest identifier identifies region and object of interest from object, and provides feedback of identified region and object of interest and processed P, F, and LGN feature vectors to the in-pixel processing array to improve the object detection and identification.