Feature Map Reduction for Low-Latency Machine Vision Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding schemes are optimized for human vision and fail to meet the specific requirements of machine vision applications, particularly in terms of latency, scale, and computational load, necessitating a new approach for compressing and transmitting feature maps from machine task-specialized deep learning models.
Innovation Solution
A Video Coding for Machines (VCM) apparatus and method that utilizes sparsification and tensor decomposition to reduce the size of feature maps, incorporating a feature extractor, reducer, pre-quantizer, repacker, and encoder for encoding, and a decoder with inverse converter for reconstructing feature maps, optimizing for machine vision tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If feature maps are transmitted instead of images/videos, then computational load on cloud servers is reduced and privacy is protected, but feature map size is excessively larger than input images/videos
Solution Approach 1:
The patent segments the feature map transmission by separating it into two components: index information (compact representation) and optional residual feature maps. This segmentation allows the system to transmit only essential feature information in a compressed form, significantly reducing the quantity of data transmitted while maintaining the computational efficiency benefits of feature map-based processing.
Solution Approach 2:
The patent extracts only the most critical feature information from the full feature maps by generating index information that represents the essential characteristics. This extraction process removes redundant data while preserving the key features needed for machine vision tasks, thereby reducing feature map size without sacrificing the computational load reduction benefit.
2Object-affected harmful factors
If feature maps are transmitted instead of images/videos, then privacy protection is improved, but feature map size is excessively larger than input images/videos
Solution Approach 1:
The patent segments feature map transmission into index information and optional residuals, allowing privacy protection through feature-based processing while minimizing data transmission volume. The index information provides sufficient feature representation for privacy-preserving machine vision tasks without transmitting excessive data.
Solution Approach 2:
The patent extracts only the essential feature characteristics needed for privacy protection and machine vision tasks, discarding redundant information. This extraction maintains privacy benefits while significantly reducing the quantity of transmitted feature map data.
3Manufacturing precision
If existing video coding schemes are used, then video quality is optimized for human vision, but latency and computational requirements are not suitable for machine vision applications
Solution Approach 1:
The patent inverts the traditional approach by not compressing feature maps back to image space for quality assessment. Instead, it performs coding operations directly in the feature map domain, eliminating unnecessary inverse transformations and reducing latency while maintaining quality suitable for machine vision tasks.
Solution Approach 2:
The patent extracts and encodes only the essential feature information needed for machine vision tasks, removing redundant processing steps required for human vision optimization. This extraction approach reduces computational requirements and latency while maintaining sufficient quality for machine analysis.
Data Source
AI summary
An apparatus and method for coding machine vision data using a reduction of feature map are disclosed. To reduce the size of a feature map extracted by a machine task-specialized deep learning model, a Video Coding for Machines (VCM) coding apparatus and a method are provided. The VCM coding apparatus and the method utilize a sparsification method that reduces redundancy in terms of space and channels of the feature map, and the VCM coding apparatus and the method also utilize a feature map decomposition method based on tensor decomposition.