Dynamic Channel Pruning for Video Processing Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large neural networks used for computer vision and video processing tasks require significant computational resources and memory, which are often not available on smaller systems like mobile devices or embedded systems.
Innovation Solution
The method involves obtaining a video with multiple frames, computing a global dependency value using a machine learning model, deactivating filters based on this value, and processing the video using the machine learning model with the deactivated filters. This approach dynamically prunes channels and filters, reducing computational overhead while maintaining accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large neural networks are used for video processing tasks, then the model can learn more difficult functions and achieve high-level capabilities, but the computational complexity, memory usage, and processing time increase significantly
Solution Approach 1:
The patent implements dynamic filter pruning where the machine learning model adaptively deactivates filters based on temporal redundancy analysis of video frames. The system dynamically adjusts the network architecture during video processing by computing global dependency values across multiple frames and selectively pruning filters that are redundant across temporal sequences, allowing the model to maintain high capability while reducing computational complexity adaptively
Solution Approach 2:
The system changes the operational parameters of the neural network by dynamically adjusting which filters are active or deactivated based on the computed global dependency values. This parameter change allows the model to process video more efficiently by activating only the necessary filters for each temporal context, reducing computational complexity while preserving model capability
2Measurement precision
If large neural networks are used for video processing, then high-level capabilities are achieved, but memory usage and storage space increase making distribution harder
Solution Approach 1:
The patent extracts and removes redundant filters from the neural network based on temporal redundancy analysis. By computing global dependency values across video frames and identifying filters that can be deactivated, the system extracts only the necessary computational elements, reducing memory usage while preserving model capability
3Device complexity
If dynamic filter pruning is applied based on temporal redundancy, then computational complexity and memory footprint are reduced, but the system must compute global dependency values across multiple frames
Solution Approach 1:
The system performs preliminary computation of global dependency values across video frames before final processing. By analyzing temporal redundancy in advance and determining which filters to deactivate beforehand, the system reduces the computational burden during actual video processing, achieving lower computational complexity without significant time loss
Data Source
AI summary
Aspects of the method, apparatus, non-transitory computer readable medium, and system include obtaining a video comprising a plurality of video frames; computing, using a machine learning model, a global dependency value based on the plurality of video frames; deactivating a filter of the machine learning model based on the global dependency value; and processing, using the machine learning model, at least a portion of the video based on the deactivated filter.