Scalable Video Coding With Base and Enhancement Layer Features
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies are inefficient in handling scalable and differential encoding of features for both computer vision and human vision tasks, leading to redundancy and the need for separate coding of features and input images, which hinders multi-task collaborative intelligence.
Innovation Solution
The proposed solution involves generating base layer features for computer vision processing and enhancement layer features for human vision reconstruction, encoding them into separate bitstreams, allowing scalable and efficient coding with differential encoding, enabling quick location and retrieval of features for computer vision tasks while ensuring privacy and security for human vision processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate coding of features and input images is performed, then computer vision tasks can be executed, but redundancy occurs and efficiency is reduced
Solution Approach 1:
The patent combines feature coding and image coding into a unified scalable coding framework where base layer features and enhancement layer features are integrated with corresponding image layers, eliminating redundant separate processing while maintaining both computer vision and human vision capabilities
Solution Approach 2:
The scalable coding structure serves multiple functions simultaneously: the base layer supports computer vision tasks with reduced data, while the enhancement layer enables high-quality image reconstruction for human vision, allowing a single system to handle both CV and HV requirements efficiently
2Reliability
If all features are encoded for human vision reconstruction, then picture quality is maintained, but data transmission and processing load increases
Solution Approach 1:
The patent segments the coding data into base layer features (sufficient for CV tasks) and enhancement layer features (needed for high-quality HV reconstruction), allowing selective transmission and processing based on the specific application requirements, thereby reducing unnecessary data load while maintaining quality where needed
Solution Approach 2:
Different quality levels are provided for different usage scenarios: base layer features provide adequate quality for computer vision algorithms, while enhancement layer features provide high quality for human vision, allowing each application to receive only the quality level it requires
3Speed
If base layer features are used for computer vision tasks, then processing speed is improved, but human vision reconstruction capability is lost
Solution Approach 1:
The patent creates a dynamic scalable coding system where the decoder can adaptively select between using only base layer features for fast computer vision processing or combining base and enhancement layers for high-quality human vision reconstruction, allowing the system to flexibly adjust to different operational requirements
Data Source
AI summary
The present disclosure relates to scalable encoding and decoding of pictures. In particular, a picture is processed by one or more network layers of a trained module to obtain base layer features. Then, enhancement layer features are obtained, e.g. by a trained network processing in sample domain. The base layer features are for use in computer vision processing. The base layer features together with enhancement layer features are for use in picture reconstruction, e.g. for human vision. The base layer features and the enhancement layer features are coded in a respective base layer bitstream and an enhancement layer bitstream. Accordingly, a scalable coding is provided which supports computer vision processing and/or picture reconstruction.