Event-Based Object Recognition Using Spiking Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object recognition systems using asynchronous imaging devices face challenges in performing recognition processing at high speeds due to framing limitations, which slow down the overall processing time.
Innovation Solution
The integration of a spiking neural network with an event detection sensor that detects luminance changes exceeding a threshold, allowing for high-speed recognition processing without the need for framing, combined with additional sensors like image and ranging sensors for enhanced information processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If framing processing is applied to event information output from the event detection sensor, then the output is organized into standard image frames, but the recognition processing time increases and speed decreases
Solution Approach 1:
The patent extracts and removes the framing processing step from the event information processing pipeline. By directly feeding asynchronous event data from the DVS sensor into the neural network without framing, the system eliminates the time-consuming frame assembly operation while maintaining proper data organization through event timestamp and coordinate information.
Solution Approach 2:
The patent performs preliminary organization of event information by encoding temporal and spatial data (timestamps, pixel coordinates, luminance changes) into a format suitable for direct neural network processing. This preliminary encoding replaces the need for subsequent framing operations, allowing high-speed asynchronous processing while maintaining data structure integrity.
2Adaptability or versatility
If framing is performed on asynchronous event data, then compatibility with standard image processing pipelines is improved, but processing efficiency and speed are reduced
Solution Approach 1:
The patent changes the parameter representation of event data from framed image format to asynchronous event streams with explicit temporal parameters (timestamps) and spatial parameters (pixel coordinates). This parameter transformation allows the system to maintain compatibility with neural network input requirements while avoiding the productivity loss associated with framing operations.
3Device complexity
If synchronous imaging is used instead of asynchronous event detection, then framing processing is simplified, but detection speed and temporal resolution are reduced
Solution Approach 1:
The patent inverts the conventional imaging approach by using asynchronous event detection instead of synchronous frame capture. Rather than capturing complete frames at fixed intervals and then processing them, the system detects individual luminance change events as they occur and processes them in real-time, achieving both high detection speed and simplified processing by eliminating the need to wait for frame completion.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables faster recognition processing by eliminating the need for framing, thereby improving the speed and efficiency of object recognition systems.
Implementation Method 1
a pixel that photoelectrically converts incident light
Data Source
AI summary
An object recognition system according to an embodiment of the present disclosure includes an event detection sensor that detects, as an event, that a change in luminance of a pixel has exceeded a predetermined threshold, and a spiking neural network. The spiking neural network executes recognition processing on an object to be recognized on a basis of a result of the detection by the event detection sensor.