Neural Network Image Processing With Event-Driven Pixel Activation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) for image processing require significant power consumption and resource-intensive operations, making them challenging for edge devices with constrained energy budgets, leading to latency and resource bottlenecks.
Innovation Solution
An Approximate Convolution-in-Pixel Scheme (AppCiP) architecture that integrates sensing and computing, utilizing parallel analog convolution and reconfigurable filtering modes, including low-precision quantized neural networks and event/object detection capabilities to reduce power consumption and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If conventional CNN architectures are used for image processing, then processing accuracy is maintained, but power consumption increases significantly
Solution Approach 1:
The patent segments the image processing task into two distinct modes: event detection mode using a subset of activated pixels for low-power operation, and object detection mode using all pixels for high-accuracy processing. This segmentation allows the system to optimize between power consumption and accuracy based on operational requirements.
Solution Approach 2:
The patent changes the operational parameters by switching between different pixel activation states (subset vs. all pixels) and processing modes (event detection vs. object detection). This parameter change enables the system to adapt power consumption levels while maintaining acceptable accuracy for different task types.
2Reliability
If all pixels are activated for high-accuracy image processing, then processing accuracy improves, but power consumption increases
Solution Approach 1:
The patent divides the pixel array into activated and non-activated subsets, allowing the system to process only the necessary portion of the image data. This segmentation maintains accuracy for event detection tasks while significantly reducing power consumption compared to activating all pixels.
Solution Approach 2:
The patent applies partial action by activating only a subset of pixels sufficient for event detection, rather than activating all pixels. This partial activation provides adequate accuracy for the detection task while reducing overall power consumption.
3Productivity
If data is transferred to cloud processing systems, then processing capability is enhanced, but latency and data transmission overhead increase
Solution Approach 1:
The patent introduces an intermediary processing layer at the edge device that performs initial event detection and filtering before data is transmitted to the cloud. This intermediary approach reduces the volume of data requiring transmission while maintaining the ability to perform complex processing when needed.
Solution Approach 2:
The patent performs preliminary event detection and data filtering at the edge device before cloud transmission. This preliminary action identifies and processes only the most critical data, reducing transmission overhead and latency while preserving processing capability for important tasks.
4Adaptability or versatility
If reconfigurable filtering modes with filter pruning are implemented, then adaptability to different tasks improves, but device complexity increases
Solution Approach 1:
The patent implements dynamic reconfigurability by allowing the system to switch between different filtering modes and pixel activation patterns based on task requirements. This dynamic adaptation provides versatility without requiring separate dedicated hardware for each function.
Solution Approach 2:
The patent creates a universal processing architecture that can handle multiple task types (event detection, object detection, filtering) through a single reconfigurable system. This multi-functionality reduces the need for separate specialized hardware components.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for improving image processing. One of the methods includes obtaining, from a first set of pixels, a first set of pixel values at a first time; obtaining, from a second set of pixels, a second set of pixel values at a second time; determining a number of changed pixel values by comparing the first and second sets of pixel values; comparing the number of changed pixel values to a threshold value; determining whether an event has occurred using the comparison of the number of changed pixel values to the threshold value; and in response to determining the event has occurred, activating a third set of pixels, wherein the third set of pixels includes one or more pixels adjacent to the first and second set of pixels.