Video Coding Method Using Neural Network Feature Maps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine vision-oriented video coding frameworks face challenges with low coding efficiency and high decoding complexity, which hinder efficient transmission and processing of video data in applications like Internet of vehicles, unmanned driving, and smart cities.
Innovation Solution
A video encoding method that uses a neural network-based approach to segment and encode target objects and backgrounds separately, generating a bitstream that includes feature maps of the target object and background, allowing for efficient decoding and visual task processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If machine vision-oriented video coding framework is used, then visual task processing capability is improved, but decoding complexity increases
Solution Approach 1:
The video encoding process is segmented into two independent parts: (1) encoding the entire current picture to generate a first feature map, and (2) encoding only the target object region to generate a second feature map. This segmentation allows the decoder to choose between full-picture decoding and target-object-only decoding, thereby reducing decoding complexity for machine vision tasks while maintaining visual task processing capability
Solution Approach 2:
The target object feature map is extracted from the encoded bitstream as a separate component. The decoder can extract and process only the target object feature map for machine vision tasks without performing full picture decoding, thus reducing decoding complexity while preserving visual task processing capability
2Productivity
If target object is highlighted separately, then coding efficiency is improved, but bitrate overhead increases
Solution Approach 1:
Different encoding qualities are applied to different regions: the target object region is encoded with higher quality (second feature map) while the background is encoded with standard quality (first feature map). This local quality differentiation improves coding efficiency for machine vision tasks without requiring uniform high-quality encoding of the entire picture, thus controlling overall bitrate overhead
3Loss of information
If full picture is encoded, then complete visual information is preserved, but processing efficiency for machine tasks decreases
Solution Approach 1:
The encoder performs preliminary processing to generate both a first feature map (complete picture) and a second feature map (target object) during the encoding phase. This preliminary action allows the decoder to directly use the pre-extracted target object feature map for machine vision tasks, improving processing efficiency while maintaining visual information completeness through the first feature map when needed
Data Source
AI summary
A video coding method includes the following. A bitstream is decoded to obtain a feature map of a target object in a current picture. The feature map of the target object in the current picture is input to a visual task network and a prediction result output by the visual task network is obtained.