Temporal Referencing Network for Flicker-Free Video Processing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing image processing networks for video colorization suffer from temporal instability, leading to flickering artifacts that degrade the viewing experience.

Innovation Solution

A temporal referencing network (TRN) is configured to add temporal information to image processing networks, generating multiple temporal reference maps and determining temporal relations to convert the networks into automatic video processing networks with stability, using a recurrent convolutional encoder-decoder network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If an image processing network is applied to video processing, then colorization quality is improved, but temporal stability deteriorates causing flicker

Engineering Contradiction:
Improvecolorization qualityVSAvoidtemporal stability
Core Design Contradiction:
Manufacturing precisionVSStability of the object's composition

Solution Approach 1:

The patent implements a feedback mechanism by generating temporal reference maps from previous frame outputs and feeding them back as additional input to the image processing network for the current frame. This feedback loop allows the network to leverage temporal information and maintain consistency across frames, thereby reducing flicker while preserving colorization quality.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The patent performs preliminary action by pre-computing temporal reference maps from previous frames before processing the current frame. These reference maps are generated in advance and stored, allowing the image processing network to access temporal context without adding significant computational delay to the real-time processing pipeline.

Inventive Principle:
Principle #10Preliminary action

2Stability of the object's composition

If temporal reference maps are added to input and output, then temporal stability is improved, but device complexity increases

Engineering Contradiction:
Improvetemporal stabilityVSAvoidnetwork complexity
Core Design Contradiction:
Stability of the object's compositionVSDevice complexity

Solution Approach 1:

The patent applies universality by designing the temporal referencing network to perform multiple functions: generating temporal reference maps, determining temporal relations, and converting image processing networks into video processing networks. This multi-functional approach consolidates what would otherwise require separate components into a single integrated system, managing complexity while achieving temporal stability.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The temporal referencing network acts as an intermediary layer between the image processing network and the final video output. It processes temporal information separately and then integrates it back into the image processing pipeline through the temporal reference maps, allowing the original image processing network to remain largely unchanged while gaining temporal stability capabilities.

Inventive Principle:
Principle #24Intermediary (Mediator)

Data Source

PatentUS20250336040A1Temporal referencing network for video processing applications
Publication Date: 2025.10.30 SAMSUNG ELECTRONICS CO LTD
  • US20250336040A1 patent drawing
  • US20250336040A1 patent drawing
  • US20250336040A1 patent drawing

AI summary

An image processing network for image colorization, image color enhancement, image super resolution, or any similar image-to-image processing is converted into an automatic video processing network with temporal stability by addition of a temporal referencing network (TRN). The implementation of the image processing network may remain unmodified, with the temporal information added based on the TRN. The TRN is configured to add temporal information to an input and to an output to an image processing network. The temporal information added to the input and the output includes multiple temporal reference maps generated for one or more input images and one or more output images of the image processing network. Temporal relations are determined based on application of the multiple temporal reference maps for the one or more input images to a recurrent network of the TRN.