Spiking Neural Network Object Detection Saliency Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing object detection models struggle to detect both salient and less salient objects in videos, especially when the camera is moving, as they either miss non-salient interesting objects or fail to handle background motion effectively.
Innovation Solution
A system using spiking neural networks generates intensity and color saliency maps by convolving spikes with Gaussian kernels, combining feature maps, and applying thresholding to detect objects of interest, while also normalizing spike activity based on image brightness for improved performance in varying lighting conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional motion models are used to detect moving objects in videos from a stationary camera, then moving objects can be detected effectively, but still objects or objects in moving camera videos cannot be detected because the background is moving
Solution Approach 1:
The patent segments object detection into two distinct models: motion models for stationary camera videos and saliency models for moving camera videos. This segmentation allows each model to be optimized for its specific condition, resolving the contradiction between reliability in specific conditions and adaptability across different conditions.
Solution Approach 2:
The patent implements dynamic model selection that adapts to camera motion conditions. The system dynamically switches between motion-based detection and saliency-based detection depending on whether the camera is stationary or moving, enabling reliable detection across varying conditions without compromising accuracy in either scenario.
2Measurement precision
If traditional saliency models are used to detect salient objects in images, then salient spots can be detected by building saliency maps from multiple channels, but non-salient yet interesting objects are missed
Solution Approach 1:
The patent merges motion detection capabilities with saliency detection capabilities into a unified object detection system. By combining these two approaches, the system can detect both salient objects (through saliency maps) and non-salient but interesting objects (through motion cues), achieving completeness without sacrificing precision.
Solution Approach 2:
The patent creates a universal object detection framework that performs both saliency-based detection and motion-based detection. This multi-functional system can handle various object types including salient objects, moving objects, and even still objects in moving camera videos, thereby improving detection completeness while maintaining precision through appropriate model selection.
3Productivity
If spiking neural networks are used to generate intensity and color saliency maps with Gaussian kernel convolution, then computational efficiency is improved, but the system must handle varying lighting conditions through normalization
Solution Approach 1:
The patent implements self-service normalization where the spiking neural network automatically adjusts its processing based on image brightness characteristics. The system performs intensity-based normalization that adapts to varying lighting conditions without requiring external control, maintaining computational efficiency while handling diverse lighting environments through inherent adaptive mechanisms.
Data Source
AI summary
Described is a system for object detection in images or videos using spiking neural networks. An intensity saliency map is generated from an intensity of an input image having color components using a spiking neural network. Additionally, a color saliency map is generated from a plurality of colors in the input image using a spiking neural network. An object detection model is generated by combining the intensity saliency map and multiple color saliency maps. The object detection model is used to detect multiple objects of interest in the input image.