Recurrent Mf-SSD Neural Network for Video Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object detection technologies are inefficient when processing video sequences, as they typically treat each frame independently, and existing multi-class detectors struggle to effectively utilize multiple frames for concurrent object localization and classification, with insufficient accuracy and speed in modern neural network architectures.
Innovation Solution
The Recurrent Multi-frame Single-Shot Detector (Recurrent Mf-SSD) neural network architecture incorporates a recurrent neural network layer after convolutional neural network layers to process higher-level image features across multiple frames, integrating temporal information through a data fusion layer to improve detection accuracy without sacrificing speed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple frames are processed independently using standard object detectors, then processing speed is maintained, but detection accuracy is insufficient
Solution Approach 1:
The patent merges multiple frame features by concatenating feature maps from different frames along the channel dimension, creating a combined feature representation that captures temporal information. This allows the network to leverage information from multiple frames to improve detection accuracy while maintaining processing efficiency through the merging operation.
Solution Approach 2:
The patent introduces a temporal dimension to the feature processing by stacking feature maps from multiple frames, transforming the problem from spatial-only feature analysis to spatio-temporal feature analysis. This dimensional extension enables the network to capture temporal patterns without significantly increasing computational complexity.
2Reliability
If raw pixel information from multiple frames is fed directly to a convolutional neural network, then temporal information is available, but it is too difficult for the network to learn relationships
Solution Approach 1:
The patent applies preliminary feature extraction using convolutional neural networks to each individual frame before combining them. This preprocessing step transforms raw pixel data into meaningful feature representations, making the subsequent temporal relationship learning much easier for the network. The preliminary feature extraction simplifies the input data structure while preserving essential information.
Solution Approach 2:
The patent introduces feature maps as an intermediary representation between raw pixel data and the final detection output. These feature maps serve as a bridge that captures essential visual patterns in a compressed, meaningful format, making temporal relationships more learnable. The intermediary feature representation reduces the complexity of direct pixel-to-output learning.
3Measurement precision
If feature maps from multiple frames are combined using concatenation, then temporal features are integrated, but computational complexity increases
Solution Approach 1:
The patent segments the feature processing into distinct stages: individual frame feature extraction, feature map concatenation, and final detection. This segmentation allows each stage to be optimized independently, managing computational complexity by breaking down the overall process into manageable components with clear boundaries.
Data Source
AI summary
An object detector includes an input interface to accept a sequence of video frames, a memory to store a neural network trained to detect objects in the video frames, a processor to process each video frame sequentially with the neural network to detect objects in the sequence of video frames, and an output interface to output the object detection information. The neural network includes a first subnetwork, a second subnetwork, and a third subnetwork. The first subnetwork receives as an input a video frame and outputs a feature map of the video frame. The second subnetwork is a recurrent neural network that takes the feature map as an input and outputs a temporal feature map. The third subnetwork takes the temporal feature map as an input and outputs object detection information.