Temporal Referencing Network for Flicker-Free Video Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing networks for video colorization suffer from temporal instability, leading to flickering artifacts that degrade the viewing experience.
Innovation Solution
A temporal referencing network (TRN) is configured to add temporal information to image processing networks, generating multiple temporal reference maps and determining temporal relations to convert the networks into automatic video processing networks with stability, using a recurrent convolutional encoder-decoder network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If an image processing network is applied to video processing, then colorization quality is improved, but temporal stability deteriorates causing flicker
Solution Approach 1:
The patent implements a feedback mechanism by generating temporal reference maps from previous frame outputs and feeding them back as additional input to the image processing network for the current frame. This feedback loop allows the network to leverage temporal information and maintain consistency across frames, thereby reducing flicker while preserving colorization quality.
Solution Approach 2:
The patent performs preliminary action by pre-computing temporal reference maps from previous frames before processing the current frame. These reference maps are generated in advance and stored, allowing the image processing network to access temporal context without adding significant computational delay to the real-time processing pipeline.
2Stability of the object's composition
If temporal reference maps are added to input and output, then temporal stability is improved, but device complexity increases
Solution Approach 1:
The patent applies universality by designing the temporal referencing network to perform multiple functions: generating temporal reference maps, determining temporal relations, and converting image processing networks into video processing networks. This multi-functional approach consolidates what would otherwise require separate components into a single integrated system, managing complexity while achieving temporal stability.
Solution Approach 2:
The temporal referencing network acts as an intermediary layer between the image processing network and the final video output. It processes temporal information separately and then integrates it back into the image processing pipeline through the temporal reference maps, allowing the original image processing network to remain largely unchanged while gaining temporal stability capabilities.
Data Source
AI summary
An image processing network for image colorization, image color enhancement, image super resolution, or any similar image-to-image processing is converted into an automatic video processing network with temporal stability by addition of a temporal referencing network (TRN). The implementation of the image processing network may remain unmodified, with the temporal information added based on the TRN. The TRN is configured to add temporal information to an input and to an output to an image processing network. The temporal information added to the input and the output includes multiple temporal reference maps generated for one or more input images and one or more output images of the image processing network. Temporal relations are determined based on application of the multiple temporal reference maps for the one or more input images to a recurrent network of the TRN.


