Hybrid Neural Processing for Event-Based Sensor Property Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional image processing methods, particularly neural networks, are not effective for processing data from event-based cameras due to their asynchronous and sparse nature, leading to inefficiencies and loss of temporal information.
Innovation Solution
A hybrid neural network architecture combining pulsed (spiking) and non-pulsed (artificial) neural networks is employed, where pulsed neurons integrate sensor data over time and non-pulsed neurons perform spatial analysis, allowing efficient processing of event-based sensor data without losing temporal resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional image processing methods are used for event-based camera data, then processing simplicity is maintained, but temporal information is lost and processing effectiveness deteriorates
Solution Approach 1:
The patent segments the processing into two distinct neural network components: a pulsed neural network for temporal processing of event streams, and a non-pulsed neural network for spatial processing. This segmentation allows each component to specialize in its strength, preserving temporal information while maintaining processing effectiveness.
Solution Approach 2:
The pulsed neural network acts as an intermediary between the event-based camera sensor and the non-pulsed neural network. It transforms the asynchronous event stream into a format that preserves temporal information while being compatible with conventional processing architectures.
2Loss of information
If pulsed neural networks are used to process event-based sensor data, then temporal information is preserved, but device complexity increases
Solution Approach 1:
The patent merges the advantages of pulsed and non-pulsed neural networks into a hybrid architecture. The pulsed network handles temporal processing where it excels, while the non-pulsed network handles spatial processing, combining their strengths while mitigating individual weaknesses.
Solution Approach 2:
The patent extracts the temporal processing function into a separate pulsed neural network component, isolating the complexity of temporal handling from the main processing pipeline. This allows temporal information to be preserved without burdening the entire system with pulsed network complexity.
3Quantity of substance
If sensor data is accumulated over time for processing, then more data is available for analysis, but temporal resolution is lost
Solution Approach 1:
The pulsed neural network processes the event stream continuously without accumulation, maintaining the continuous temporal flow of events. Each event is processed as it arrives, preserving temporal resolution while still enabling comprehensive analysis over time.
Solution Approach 2:
The system uses periodic subsequences of fixed length as input to the non-pulsed neural network, creating a structured rhythm that allows spatial processing while maintaining temporal information through the pulsed network's continuous processing.
Data Source
AI summary
A method for ascertaining a physical property of an object. The method includes detecting, for each input point in time of a sequence of input points in time, sensor data including information about a physical object, using an event-based sensor; for each subsequence of a breakdown of the sequence of input points in time into multiple subsequences including: feeding the sensor data detected for the input points in time of the subsequence to a pulsed neural network which generates a first processing result of the subsequence; feeding the processing result of the subsequence to a non-pulsed neural network; and processing the processing result of the subsequence by non-pulsed neurons of one or multiple first layer(s) of the non-pulsed neural network for generating a second processing result of the subsequence; and feeding the second processing results of the multiple subsequences to one or multiple second layer(s) of the non-pulsed neural network.