Brain-Like In-Pixel Processing Array for Low-Power Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems face challenges in processing full-color images for object detection, classification, and tracking due to overwhelming data and high computational requirements, while human visual systems excel in rapid and accurate object recognition despite limitations.
Innovation Solution
A brain-like in-pixel intelligent processing system is developed, incorporating an in-pixel processing array with photogate sensors, average and subtraction circuits, and absolute circuits to emulate saccadic eye movements and generate feature vectors, processed by a neural network for object detection and identification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional computer systems process full-color images for object detection and classification, then comprehensive image information is captured, but computational requirements and power consumption increase significantly
Solution Approach 1:
The patent segments the image processing task into multiple stages: raw gray information extraction at pixel level, feature vector generation through in-pixel processing circuits, and neural network processing. This segmentation allows energy-intensive operations to be distributed and optimized, reducing overall power consumption while maintaining detection accuracy.
Solution Approach 2:
The in-pixel processing circuits perform preliminary processing of image data directly at the sensor array before data leaves the imaging device. Feature vectors are generated in advance through subtraction and absolute value circuits, reducing the computational burden on subsequent processing stages and lowering overall energy requirements.
2Measurement precision
If traditional computer systems process full-color images for object detection, classification and tracking, then accurate object identification is achieved, but data processing volume becomes overwhelming
Solution Approach 1:
The patent extracts only the essential gray information from full-color images at the pixel level, discarding color data that is not critical for object detection. The in-pixel processing circuits extract feature vectors that capture the most important visual information, significantly reducing data volume while preserving recognition accuracy.
Solution Approach 2:
The processing approach applies different operations to different regions of the image array. The in-pixel processing circuits perform local subtraction and absolute value operations on neighboring pixels, generating feature vectors that capture local patterns and edges. This local processing approach reduces overall data volume while maintaining comprehensive object detection capability.
3Speed
If human visual system processes detailed information at high speeds, then rapid object recognition is achieved, but the system has limited capacity to memorize large amounts of information
Solution Approach 1:
The system performs preliminary processing to generate compact feature vectors that capture essential visual information in a condensed format. This preliminary encoding allows rapid processing of visual data without requiring large memory capacity, mimicking the human visual system's ability to quickly recognize objects without memorizing every detail.
Solution Approach 2:
The patent transforms raw pixel data into feature vectors through mathematical operations (subtraction and absolute value calculations). This parameter transformation converts high-dimensional image data into lower-dimensional feature representations, enabling fast processing while reducing the information storage requirement.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The system enables efficient, pixel-level processing of raw gray information for rapid and accurate object detection and identification, mimicking human visual system capabilities on a semiconductor chip.
Implementation Method 1
The photogate sensor captures a pixel of an image of an object and produces an Iout current to the in-pixel processing array from the in-pixel processing unit
Data Source
AI summary
Aspects of present disclosure relates to a brain-like in-pixel intelligent processing system having an in-pixel processing array, a feature vector generator, a neural network processor, and a region and object of interest identifier. In-pixel processing array includes N×M in-pixel processing units. Each in-pixel processing unit includes photogate sensor to acquire image, average circuit to generate P element, subtraction circuit to generate F element, and absolute circuit to generate LGN element. Feature vector generator generates P, F, and LGN feature vectors of in-pixel processing array. Neural network processor processes P, F, and LGN feature vectors and detects object, recognizes object, and determines location of object. Region and object of interest identifier identifies region and object of interest from object, and provides feedback of identified region and object of interest and processed P, F, and LGN feature vectors to the in-pixel processing array to improve the object detection and identification.