HDR Video Super-Resolution Neural Network Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The broadcast infrastructure often lacks support for transmitting high dynamic range (HDR) video content in full resolution, necessitating improved techniques for super-resolution of HDR images to enhance coding efficiency.
Innovation Solution
A method for super-resolution of HDR video involves a processor that receives current and auxiliary pictures at a first spatial resolution, generates neural network features, upscales the pictures using an upscaling neural network, and reconstructs the images to produce a super-resolution picture at a second spatial resolution, maintaining spatio-temporal consistency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If HDR video content is transmitted in full resolution, then picture quality is improved, but broadcast infrastructure compatibility deteriorates
Solution Approach 1:
The patent applies preliminary action by performing super-resolution processing on HDR video content before transmission. The system pre-enhances the spatial resolution of HDR pictures using neural network-based upscaling algorithms, converting lower-resolution input pictures into higher-resolution output pictures. This preliminary enhancement allows the content to be transmitted at reduced resolution while maintaining the ability to deliver high-quality images at the receiver end, thus resolving the contradiction between transmission efficiency and picture quality.
2Productivity
If super-resolution processing is applied to HDR images, then coding efficiency is improved, but processing complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the super-resolution processing into distinct functional modules: a picture scaler that performs initial downscaling, a neural network feature extraction module that processes the scaled pictures, and a reconstruction module that generates the final high-resolution output. This modular segmentation allows each component to be optimized independently, improving coding efficiency while managing processing complexity through specialized hardware or software implementations for each stage.
Solution Approach 2:
The patent uses an intermediary approach by introducing auxiliary pictures as intermediate elements in the super-resolution process. These auxiliary pictures serve as mediators that provide additional information to the neural network, enabling more accurate reconstruction of high-resolution details. The intermediary auxiliary pictures help bridge the gap between the scaled input and the desired high-resolution output, improving coding efficiency without requiring excessive processing complexity in the main processing path.
3Manufacturing precision
If neural network upscaling is used, then picture details are preserved, but computational requirements increase
Solution Approach 1:
The patent applies partial action by selectively processing only the most critical aspects of picture detail preservation. The neural network is designed to focus computational resources on reconstructing essential high-frequency details and edges while using more efficient downscaling algorithms for lower-priority regions. This partial processing approach maintains acceptable picture detail preservation while significantly reducing the computational energy consumption compared to full-resolution neural network processing of all picture content.
Data Source
AI summary
Methods and systems for the super resolution of high dynamic range (HDR) video are described. Given a sequence of video frames, a current frame and two or more neighboring frames are processed by a neural-network (NN) feature extraction module, followed by a NN upscaling module, and a NN reconstruction module. In parallel, the current frame is upscaled using traditional up-sampling to generate an intermediate up-sampled frame. The output of the reconstruction module is added to the intermediate up-sampled frame to generate an output frame. Additional traditional up-sampling may be performed on the output frame to match the desired up-scaling factor, beyond the up-scaling factor for which the neural network was trained.