Scalable Video Coding With Base and Enhancement Feature Layers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video coding technologies inefficiently handle the joint coding of computer vision and human vision tasks, leading to redundancy and a need for scalable representation of features to support both processing operations in multi-task collaborative intelligence.
Innovation Solution
The apparatus and method encode video images into a base layer bitstream for computer vision tasks and an enhancement layer bitstream for human vision tasks, utilizing a trained network to generate latent space features, enabling efficient differential encoding and decoding of these layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If video images are encoded using conventional video coding technologies, then compression is achieved, but both computer vision and human vision tasks cannot be efficiently supported simultaneously
Solution Approach 1:
The patent divides the feature representation into two separate layers: a base layer for computer vision tasks and an enhancement layer for human vision tasks. This segmentation allows each layer to be optimized for its specific purpose, enabling efficient multi-task support without requiring the entire system to handle all tasks simultaneously, thus resolving the contradiction between adaptability and complexity.
Solution Approach 2:
The patent introduces a hierarchical dimension to feature representation by organizing features into base layer and enhancement layer. This dimensional organization allows the system to represent information at different levels of abstraction, enabling simultaneous support for both computer vision (base layer) and human vision (enhancement layer) tasks without increasing overall system complexity.
2Measurement precision
If high quality features are extracted for computer vision tasks, then processing accuracy is improved, but transmission bandwidth and storage requirements increase
Solution Approach 1:
The patent extracts only the essential features needed for computer vision tasks into the base layer, separating them from the additional details required for human vision tasks. This extraction allows the system to transmit and store only the critical information for CV tasks (base layer) while maintaining the option to add enhancement layer details when needed, thus reducing data volume while preserving processing accuracy.
Solution Approach 2:
The patent applies partial action by providing only the necessary base layer features for computer vision tasks, rather than encoding all possible detail levels. The enhancement layer can be added only when human vision quality is required, avoiding unnecessary data transmission and storage for CV-only applications.
3Loss of information
If video images are encoded for both computer vision and human vision tasks, then feature redundancy is reduced, but encoding complexity increases
Solution Approach 1:
The patent segments the encoding process into two independent stages: base layer encoding for computer vision and enhancement layer encoding for human vision. This segmentation allows each stage to process information independently, reducing the complexity of the overall encoding process while minimizing feature redundancy through targeted compression of each layer's specific requirements.
Solution Approach 2:
The patent changes the encoding parameters dynamically based on task requirements. The base layer uses parameters optimized for computer vision accuracy, while the enhancement layer uses parameters optimized for human vision quality. This parameter adaptation reduces information loss by tailoring compression characteristics to each layer's specific purpose, avoiding unnecessary redundancy.
Data Source
AI summary
The present disclosure relates to scalable encoding and decoding of pictures. In particular, a picture is processed by one or more network layers of a trained module to obtain base layer features. Then, enhancement layer features are obtained, e.g. by a trained network processing in sample domain. The base layer features are for use in computer vision processing. The base layer features together with enhancement layer features are for use in picture reconstruction, e.g. for human vision. The base layer features and the enhancement layer features are coded in a respective base layer bitstream and an enhancement layer bitstream. Accordingly, a scalable coding is provided which supports computer vision processing and/or picture reconstruction.