Neural Network Feature Map Generation via Inter-Intra Block Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning algorithms for processing visual data, such as video, are computationally expensive and challenging to implement on battery-powered embedded devices with low energy or power budgets, particularly when trying to extract meaningful information or enhance video content.
Innovation Solution
A method and system that determine if a video frame block is inter or intra, performing specific processes based on differences or flatness to generate a feature map, using either neural network operations or inverse frequency transforms to reduce computational load by reusing reference feature maps or generating segments efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine learning algorithms are implemented on embedded devices to process video data, then meaningful information extraction and video enhancement capabilities are improved, but computational complexity and power consumption increase
Solution Approach 1:
The video frame is divided into multiple blocks, and each block is processed independently through classification into inter-blocks and intra-blocks. This segmentation allows selective application of processing methods, reducing overall computational load while maintaining video processing capability.
Solution Approach 2:
Different processing methods are applied to different blocks based on their characteristics. Inter-blocks use motion-compensated temporal prediction while intra-blocks use spatial prediction or neural network processing only when necessary. This local differentiation optimizes power consumption by applying computationally intensive operations only where needed.
2Measurement precision
If neural network operations are performed on all video blocks to generate feature maps, then processing accuracy is improved, but computational load and processing time increase
Solution Approach 1:
Neural network operations are applied partially rather than universally. Only intra-blocks that cannot be adequately processed by traditional methods undergo neural network transformation. This partial application maintains sufficient feature map accuracy while dramatically reducing computational load and improving processing speed.
Solution Approach 2:
The patent uses reference feature maps from previously processed frames or blocks as substitutes for full neural network processing. By copying and adapting existing feature map data, the system maintains accuracy for many blocks without repeating expensive neural network computations.
3Ease of operation
If traditional video processing methods are used without selective optimization, then processing simplicity is maintained, but energy efficiency and computational performance worsen
Solution Approach 1:
The processing method dynamically adapts to each block's characteristics by classifying blocks as inter or intra types and selecting appropriate processing paths. This dynamic approach automatically optimizes energy efficiency based on content complexity while maintaining operational simplicity through automated decision-making.
Solution Approach 2:
The system performs self-optimization by automatically classifying blocks and selecting processing methods without external intervention. The block classification and method selection are handled autonomously by the processing unit, maintaining simplicity while achieving energy efficiency through intelligent self-management.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach reduces computational complexity and power consumption by selectively performing neural network operations, allowing for efficient feature map generation in video processing, especially in scenarios with minimal frame changes, like security cameras or augmented reality headsets.
Implementation Method 1
performing an inverse frequency transform on a DC coefficient of the block to generate a DC offset
Data Source
AI summary
A method is described for operating on a frame of a video to generate a feature map of a neural network. The method determines if a block of the frame is an inter block or an intra block, and performs an inter block process in the event that the block is an inter block and/or an intra block process in the event that the block is an intra block. The inter block process determines a measure of differences between the block of the frame and a reference block of a reference frame of the video, and performs either a first process or a second process based on the measure to generate a segment of the feature map. The intra block process determines a measure of flatness of the block of the frame, and performs either a third process or a fourth process based on the measure to generate a segment of the feature map.