Video Classification Using Dual-Stream CNNs with Different Architectures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The accuracy of video classification is compromised due to the use of dual-stream convolutional neural network models with the same network architecture for both original and optical flow images, which fail to effectively recognize motion in video files.
Innovation Solution
A method involving the extraction of original and optical flow images, which are then input into space-domain and time-domain convolutional neural networks of different architectures, with the results merged using weighted scores to improve classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If the same network architecture is used for both space-domain and time-domain convolutional neural network models, then the device complexity is reduced and ease of manufacture is improved, but the measurement precision of motion recognition deteriorates
Solution Approach 1:
The patent applies different network architectures to different functional streams: a deeper ResNeXt architecture for space-domain original images and a lighter Inception-v3 architecture for time-domain optical flow images. This local differentiation optimizes each stream for its specific processing needs, improving motion recognition accuracy without unnecessarily increasing overall system complexity.
Solution Approach 2:
The patent changes the architectural parameters of the convolutional neural network models based on the characteristics of different input types. The space-domain model uses deeper networks with more parameters to capture spatial features, while the time-domain model uses lighter networks optimized for temporal feature extraction from optical flow, thereby improving measurement precision.
2Measurement precision
If different network architectures are used for space-domain and time-domain convolutional neural network models, then the measurement precision of motion recognition is improved, but the device complexity increases
Solution Approach 1:
The patent segments the video classification task into two independent streams with different network architectures: space-domain processing using ResNeXt for original images and time-domain processing using Inception-v3 for optical flow images. This segmentation allows each stream to be optimized independently, improving motion recognition accuracy while managing complexity through modular design.
Solution Approach 2:
The patent dynamically selects different network architectures based on the input type characteristics. The system adapts by using deeper networks for spatial feature extraction from original images and lighter networks for temporal feature extraction from optical flow, optimizing performance for each domain while balancing overall system complexity.
Data Source
AI summary
A method for video classification includes: extracting an original image and an optical flow image corresponding to a to-be-classified video from the to-be-classified video; inputting the original image to a space-domain convolutional neural network model to obtain a space-domain classification result corresponding to the to-be-classified video; inputting the optical flow image to a time-domain convolutional neural network model to obtain a time-domain classification result corresponding to the to-be-categorized video, wherein the time-domain convolutional neural network model and the space-domain convolutional neural network model are convolutional neural network models of different network architectures; and merging the space-domain classification result and the time-domain classification result to obtain a classification result corresponding to the to-be-classified video.