Tiny Video Processing Networks Through Constrained Architecture Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video understanding neural networks are computationally intensive, requiring large networks with slow runtimes that hinder their application in resource-constrained environments such as robotics, mobile devices, and edge devices, despite achieving state-of-the-art results.
Innovation Solution
A neural architecture search process that applies constraints on size and runtime to iteratively design 'tiny' video processing networks, using evolutionary and reinforcement learning to automatically discover architectures that are both fast and accurate, reducing computational requirements and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing large-scale neural networks are used for video understanding, then state-of-the-art accuracy is achieved, but computational intensity and runtime increase significantly
Solution Approach 1:
The patent applies parameter changes by systematically varying network architecture parameters (number of layers, filter sizes, convolution types) during the search process to identify configurations that achieve target accuracy with reduced computational cost. This enables finding optimal trade-offs between accuracy and processing speed.
Solution Approach 2:
The patent uses copying by creating simplified versions of existing large-scale video understanding networks through the search process. The system copies architectural patterns from successful large networks but reduces their scale and complexity to create smaller, faster models that maintain competitive performance.
2Measurement precision
If computationally intensive modules such as 3D convolutions and non-local blocks are used, then video understanding performance improves, but runtime increases to 500+ ms on GPU and 2000+ ms on CPU
Solution Approach 1:
The patent extracts and removes computationally intensive modules (such as 3D convolutions and non-local blocks) from the network architecture while maintaining video understanding performance through alternative lightweight operations. This extraction of heavy computational components directly reduces processing time.
Solution Approach 2:
The patent replaces expensive computationally intensive operations with cheaper, more efficient alternatives that can be discarded or simplified. The search process identifies and adopts lightweight operations that achieve the same functional goals with significantly lower computational cost and faster execution.
3Measurement precision
If extremely large networks with tens to hundreds of convolutional layers are used, then state-of-the-art results are achieved, but device complexity and resource requirements increase
Solution Approach 1:
The patent applies segmentation by dividing the network into smaller, manageable modules and layers during the search process. Instead of designing monolithic large networks, the system segments the architecture into reusable components that can be combined in optimized ways, reducing overall complexity while maintaining performance.
Solution Approach 2:
The patent uses dynamics by implementing adaptive architecture search that can adjust network complexity based on performance requirements. The system dynamically modifies architecture parameters during the search process to find the minimum necessary complexity for achieving target accuracy, avoiding fixed overly-complex designs.
Data Source
AI summary
Generally, the present disclosure is directed to a neural architecture search process for finding small and fast video processing networks for understanding of video data. The neural architecture search process can automatically design networks that provide comparable video processing performance at a fraction of the computational and storage cost of larger existing models, thereby conserving computing resources such as memory and processor usage.