Spatiotemporal Spiking Neural Networks With Offset Kernel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural networks struggle to efficiently process event-based data, particularly spatiotemporal data, due to high computational requirements, inefficiencies in handling temporal data, and difficulties in implementing hardware solutions for edge devices, leading to power consumption and resource constraints.
Innovation Solution
A neural network architecture that processes event-based data using spiking neurons, employing kernels offset in spatial, temporal, or spatiotemporal dimensions to determine potentials based on event categories, facilitating efficient and low-complexity processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional neural networks (CNNs, RNNs) are used to process event-based spatiotemporal data, then processing capability is provided, but computational complexity and power consumption increase significantly
Solution Approach 1:
The patent segments the processing of event-based data by categorizing events into different types (e.g., positive polarity, negative polarity, zero polarity) and applying different kernel processing strategies for each category. This segmentation allows the system to avoid uniform complex processing for all events, thereby reducing overall computational complexity while maintaining processing capability.
Solution Approach 2:
The patent applies different kernel types (first kernel, second kernel, third kernel) based on the local characteristics of event categories. By matching specific kernel properties to specific event types, the system optimizes processing efficiency for each local case rather than using a single complex processing path for all events.
2Ease of operation
If conventional neural networks are deployed on edge devices, then data processing is enabled, but power consumption and resource constraints are exceeded
Solution Approach 1:
The patent implements partial processing by applying different levels of kernel processing based on event category. Not all events require the same level of processing - some events can be processed with simpler kernels or fewer operations, allowing the system to perform just enough computation needed for each event type, thereby reducing overall power consumption on resource-constrained edge devices.
3Adaptability or versatility
If temporal data processing is enhanced using RNNs or ConvLSTM, then temporal feature extraction is improved, but computational expense increases
Solution Approach 1:
The patent segments temporal processing by applying different kernel types for different event categories. Instead of using uniform complex RNN or ConvLSTM processing for all temporal data, the system divides temporal processing into categories (positive polarity events, negative polarity events, zero polarity events) and applies appropriate kernel processing to each segment, reducing overall computational expense while maintaining temporal feature extraction capability.
Data Source
AI summary
Disclosed is a method for processing event-based input data using a neural network. The neural network comprises a plurality of neurons and one or more connections associated with each of the plurality of neurons. Further, each of the plurality of neurons is configured to receive a corresponding portion of the event-based data. The method comprises receiving, at a neuron of the plurality of neurons, a plurality of events associated with the event-based data over the one or more connections associated with the neuron. Each of the one or more connections is associated with a kernel. The method further comprises determining a potential of the neuron over the period of time based on processing of the kernels. In order to determine the potential, the method further comprises offsetting the kernels in one of a spatial dimension, a temporal dimension, or a spatiotemporal dimension, and processing the offset kernels in order to determine the potential. The method further comprises generating, at the neuron, output based on the determined potential.