Real-Time Radar Action Recognition With Spiking Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional action recognition systems using vision sensors face challenges such as privacy concerns, high computational intensity, and the need for large training datasets, making them unsuitable for real-time edge device execution.
Innovation Solution
The proposed system employs a Spiking Neural Network (SNN) model for real-time radar-based action recognition, utilizing a data pre-processing layer, Convolutional Spiking neural network (CSNN) layers, and a Classifier layer to extract spatial and temporal features from radar data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional vision-based action recognition systems are used, then action recognition capability is achieved, but computational intensity increases and privacy concerns arise
Solution Approach 1:
The patent replaces vision-based mechanical/optical sensing systems with radar-based sensing systems. Radar sensors capture Doppler frequency data that directly encodes motion information, eliminating the need for complex image processing while maintaining action recognition accuracy. This substitution reduces computational intensity significantly.
Solution Approach 2:
The patent transforms the input data representation from visual images to Doppler frequency spectra. By changing the fundamental parameter domain from spatial visual information to frequency-domain motion information, the system achieves comparable recognition accuracy with substantially reduced computational requirements.
2Measurement precision
If conventional deep learning models are used for action recognition, then recognition accuracy is improved, but real-time execution on edge devices becomes difficult
Solution Approach 1:
The patent extracts and utilizes only the essential Doppler frequency features from radar signals that are directly relevant to action recognition. By focusing on these extracted features rather than processing complete raw radar data or visual images, the system achieves real-time processing capability on edge devices while maintaining accuracy.
Solution Approach 2:
The patent employs dynamic spike timing in spiking neural networks where neurons fire at specific times based on input strength and temporal patterns. This dynamic temporal coding allows the network to process information efficiently in real-time, capturing temporal dynamics of actions without requiring heavy computational resources.
3Measurement precision
If large training datasets are used for conventional models, then model accuracy improves, but data requirements and training time increase
Solution Approach 1:
The spiking neural network performs unsupervised learning by self-organizing to detect temporal patterns and features in radar Doppler data without requiring large labeled datasets. The network learns meaningful representations through its inherent temporal processing capabilities, reducing dependency on extensive training data.
Solution Approach 2:
The patent leverages the periodic nature of Doppler frequency signals and temporal patterns in human actions. By designing the spiking neural network to detect these periodic temporal patterns, the system achieves accurate action recognition with minimal training data, as the temporal structure itself provides strong supervisory signals.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach enables efficient action recognition on edge devices, reducing latency and computational costs while maintaining accurate classification of human actions from radar data.
Implementation Method 1
the radar data comprises a plurality of doppler frequencies reflected from the target upon motion of the target with respect to the one or more radar sensors
Data Source
AI summary
This disclosure relates generally to action recognition and more particularly to system and method for real-time radar-based action recognition. The classical machine learning techniques used for learning and inferring human actions from radar images are compute intensive, and require volumes of training data, making them unsuitable for deployment on network edge. The disclosed system utilizes neuromorphic computing and Spiking Neural Networks (SNN) to learn human actions from radar data captured by radar sensor(s). In an embodiment, the disclosed system includes a SNN model having a data pre-processing layer, Convolutional SNN layers and a Classifier layer. The preprocessing layer receives radar data including doppler frequencies reflected from the target and determines a binarized matrix. The CSNN layers extracts features (spatial and temporal) associated with the target's actions based on the binarized matrix. The classifier layer identifies a type of the action performed by the target based on the features.