CNN Tensor Decoding With Split-Layer Metadata Across Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video compression technologies for machine vision tasks face challenges in efficiently encoding and decoding tensors from convolutional neural networks, particularly in identifying optimal starting layers for processing, which affects the performance and efficiency of video data transmission and processing.
Innovation Solution
A method and system for encoding and decoding tensors from convolutional neural networks by determining a starting layer based on decoded information, allowing for efficient splitting of the neural network into a backbone and head network, and using metadata to identify the appropriate CNN architecture and split points for seamless data processing across devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If video compression technology is used for machine vision tasks, then data transmission efficiency is improved, but the ability to efficiently encode and decode tensors from convolutional neural networks deteriorates due to lack of optimization for specific CNN operations
Solution Approach 1:
The patent applies parameter changes by modifying the encoding parameters specifically for tensor data from CNNs. It uses quantization parameters, precision control, and data format transformations (e.g., float32 to int16) to optimize the encoding process for machine vision workloads while maintaining compatibility with standard video compression frameworks like H.264/AVC and H.265/HEVC.
Solution Approach 2:
The patent segments the neural network into a backbone network and a head network, allowing independent optimization of each part. The backbone network processes the input data and generates feature maps that are then encoded, while the head network performs task-specific processing. This segmentation enables tailored encoding strategies for different functional blocks of the CNN.
2Ease of operation
If the neural network is processed as a whole, then processing simplicity is maintained, but the ability to optimize specific layers for different devices deteriorates
Solution Approach 1:
The patent divides the neural network into a backbone network and a head network. The backbone network is responsible for feature extraction and is optimized for encoding efficiency, while the head network is tailored for specific tasks and devices. This segmentation allows the system to maintain simplicity in the overall architecture while enabling device-specific optimizations in the head network portion.
Solution Approach 2:
The backbone network serves as a universal component that can be shared across different devices and applications. It performs general feature extraction that is applicable to various machine vision tasks, while the head network provides task-specific functionality. This multi-functionality approach allows a single encoded bitstream to serve multiple purposes across different devices.
3Quantity of substance
If compression is applied to reduce data size, then transmission bandwidth is improved, but the accuracy of decoded neural network features deteriorates
Solution Approach 1:
The patent employs parameter changes through adaptive quantization, where the quantization step size and precision are dynamically adjusted based on the importance of different feature maps and the capabilities of the target device. Less critical features use coarser quantization to reduce data size, while critical features maintain higher precision to preserve accuracy. This selective parameter adjustment optimizes the trade-off between compression ratio and decoding accuracy.
Solution Approach 2:
The patent applies local quality by differentiating the encoding quality across different spatial and channel dimensions of the feature maps. Important regions and channels are encoded with higher quality (finer quantization), while less important areas use lower quality encoding. This localized quality adjustment ensures that the most critical features maintain high accuracy while overall data size is reduced.
Data Source
AI summary
A system and method of decoding information for data generated by a first part of a neural network. The method comprises decoding information for determining at least a starting layer of a second part of the neural network, the neural network including at least the first part and the second part, the second part being different from the first part; and determining the starting layer of the second part of the neural network based on the decoded information.